Affiliations: [a] School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, Jilin, China | [b] Jilin Province Key Laboratory of Network and Information Security,Changchun University of Science and Technology, Changchun, Jilin, China | [c] Information Center, Changchun University of Science and Technology, Changchun, Jilin, China | [d] School of Computer Science and Technology, Shandong University, Qingdao, Shandong, China
Correspondence:
[*]
Corresponding author: Xiaoqiang Di, School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, Jilin, China. E-mail: dixiaoqiang@cust.edu.cn.
Abstract: Logs play an important role in anomaly detection, fault diagnosis, and trace checking of software and network systems. Log parsing, which converts each raw log line to a constant template and a variable parameter list, is a prerequisite for system security analysis. Traditional parsing methods utilizing specific rules can only parse logs of specific formats, and most parsing methods based on deep learning require labels. However, the existing parsing methods are not applicable to logs of inconsistent formats and insufficient labels. To address these issues, we propose a robust Log parsing method based on Self-supervised Learning (LogSL), which can extract templates from logs of different formats. The essential idea of LogSL is modeling log parsing as a multi-token prediction task, which makes the multi-token prediction model learn the distribution of tokens belonging to the template in raw log lines by self-supervision mode. Furthermore, to accurately predict the tokens of the template without labeled data, we construct a Multi-token Prediction Model (MPM) combining the pre-trained XLNet module, the n-layer stacked Long Short-Term Memory Net module, and the Self-attention module. We validate LogSL on 12 benchmark log datasets, resulting in the average parsing accuracy of our parser being 3.9% higher than that of the best baseline method. Experimental results show that LogSL has superiority in terms of robustness and accuracy. In addition, a case study of anomaly detection is conducted to demonstrate the support of the proposed MPM to system security tasks based on logs.
Keywords: System security, data analysis, log parsing, deep learning, self-supervised learning
