Anomaly detection system for mobile carrier based on Big Data concept

Serhii Bondarovets, Oksana Koval, Serhii Hnatiuk


The continuous growth of information technologies in the modern world has caused a gradual increase in data circulating in the information and telecommunication systems, which in turn generates a large number of new threats, that is not so easy to detect. Standard methods of detection based on the signature method, which is comparing the traffic coming into the network with databases of known threats. However, these methods are ineffective when the threat is new and it has not yet been added to the database. In this case, it is necessary to use a more intelligent methods, which are able to monitor any unusual activity for a particular system – the methods of anomaly detection. Particularly, this problem is actual for mobile operators that have recently often face different types of fraud (leakage international traffic, false billing), which is impossible to determine in real time. Therefore, it is appropriate to implement in carrier’s network intelligent system that is able to process large amounts of data in real time and warn about possible threats. However, known threats will be faster detected by signature module, so it is logical to include it in system. The performance of the system will be provided using the methods and tools of Big Data, concretely by using a distributed file system and parallel computing on multiple servers will dynamically process data. That anomaly detection system was developed in this paper.


Anomaly detection; Big Data concept; information security; data analysis; machine learning; cellular communication; signature detection.


B. Abraham, and A. Chuang, “Outlier detection and time series modeling”, Technometrics, vol. 31, iss. 2, pp. 241-248, May 1989. doi: 10.2307/1268821.

D. Barbara, Y. Li, J. Couto, J.-L. Lin, and S. Jajodia, “Bootstrapping a data mining intrusion detection system”, in Proc. of the 2003 ACM symposium on Applied computing (SAC '03), Melbourne, USA, pp. 421-425. doi: 10.1145/952532.952616.

H. Chen, R. Chiang, and V. Storey, “Business intelligence and analytics: From big data to big impact”, MIS Quarterly, vol. 36, iss. 4, pp. 1165-1188, December 2012.

P. Chan, M. Mahoney, and M. Arshad, “A machine learning approach to anomaly detection”, Florida Institute of Technology, Melbourne, USA, Tech. Rep. CS-2003-06, March 2003.

M. Mahoney, and P. Chan, “Learning nonstationary models of normal network traffic for detecting novel attacks”, in Proc. of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining (KDD '02), Edmonton, Canada, pp. 376-385. doi: 10.1145/775047.775102.

A. Nairac, T. Corbett-Clark, R. Ripley, N. Townsend, and L.Tarassenko, “Choosing an appropriate model for novelty detection”, in Proc. of the 5th IEEE International Conference on Artificial Neural Networks (Conf. Publ. No. 440), Cambridge, UK, pp. 117-122. doi: 10.1049/cp:19970712.

S. Papadimitriou, H. Kitagawa, P. Gibbons, and C. Faloutsos, “LOCI: Fast outlier detection using the local correlation integral”, Carnegie Mellon University, Pittsburgh, USA, Tech. Rep. CMU-CS-02-188, November 2002.

S. Ramaswamy, R. Rastogi, and K. Shim, “ Efficient algorithms for mining outliers from large data sets”, in Proc. of the 2000 ACM SIGMOD international conference on Management of data (SIGMOD '00), Dallas, USA, pp. 427-438. doi: 10.1145/335191.335437

R. Rehman, Intrusion Detection Systems with Snort: Advanced IDS Techniques Using Snort, Apache, MySQL, PHP, and ACID, New Jersey, USA, Pearson Education LTD, 2003.

A. Sebyala, T. Olukemi, and L. Sacks, “Active platform security through intrusion detection using naive bayesian network for anomaly detection”, in Proc. of the London communications symposium (2002), London, UK, pp. 1-5.

J. Zhang, and H. Wang, “Detecting outlying subspaces for high-dimensional data: the new task, algorithms, and performance”, Knowledge and Information Systems, vol. 10, iss. 3, pp. 333-355, October 2006. doi: 10.1007/s10115-006-0020-z.

ISSN 2411-1031 (Print), ISSN 2518-1033 (Online)