Affiliations: [a] Systems Engineering Institute, School of Electronic and Information Engineering, Xi’an Jiaotong University, Xianning Road, Xi’an, China
| [b] Department of Computer Science and Technology, School of Electronic and Information Engineering, Xi’an Jiaotong University, Xianning Road, Xi’an, China
Correspondence:
[*]
Corresponding author. Qinke Peng, Systems Engineering Institute, School of Electronic and Information
Engineering, Xi’an Jiaotong University, Xi’an, China. E-mail: qkpeng@mail.xjtu.edu.cn.
Abstract: In the era of data technology, data growth is occurring at an unprecedented scale. Business data and information are among the most valuable assets. Massive data analysis now drives nearly every aspect of society and can facilitate informed decision-making by businesses. Fully automated data flow detection of anomalies plays a crucial role in maintaining data service stability and preventing malicious attacks. This paper presents an extensible and generic real-time monitoring system framework (EGRTMS) for large-scale time-series data. EGRTMS employs a prediction module and an anomaly detection module within an anomaly filtering layer for the accurate identification of anomalies. Moreover, the alarm module and anomaly handling module within an anomaly trace processing layer enables the system to respond swiftly to the detected threats. Our solution does not rely on the labelling of anomalies; instead, a predictor module with a deep learning attention-based mechanism learns the normal behaviour of the data, and an anomaly handling module determines the dynamic alarm-threshold by utilizing a sliding window. The results of this study demonstrate that our framework significantly outperforms other anomaly detection systems on most real and synthetic datasets.
