STFormer: A dual-stage transformer model utilizing spatio-temporal graph embedding for multivariate time series forecasting

Article type: Research Article

Authors: Xiao, Yuteng^{a; b; c} | Liu, Zhaoyang^d | Yin, Hongsheng^e | Wang, Xingang^{a; b; c; *} | Zhang, Yudong^{f; g; *}

Affiliations: [a] Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China | [b] Shandong Engineering Research Center of Big Data Applied Technology. Faculty of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China | [c] Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science, Jinan, China | [d] School of Information Engineering (School of Big Data), Xuzhou University of Technology, Xuzhou, China | [e] School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, China | [f] School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK | [g] School of Computer Science and Engineering, Southeast University, Nanjing, Jiangsu, China

Correspondence: [*] Corresponding author. Xingang Wang, E-mail: wxg@qlu.edu.cn. and Yudong Zhang, E-mail: yudongzhang@ieee.org.

Abstract: Multivariate Time Series (MTS) forecasting has gained significant importance in diverse domains. Although Recurrent Neural Network (RNN)-based approaches have made notable advancements in MTS forecasting, they do not effectively tackle the challenges posed by noise and unordered data. Drawing inspiration from advancing the Transformer model, we introduce a transformer-based method called STFormer to address this predicament. The STFormer utilizes a two-stage Transformer to capture spatio-temporal relationships and tackle the issue of noise. Furthermore, the MTS incorporates adaptive spatio-temporal graph structures to tackle the issue of unordered data specifically. The Transformer incorporates graph embedding to combine spatial position information with long-term temporal connections. Experimental results based on typical finance and environment datasets demonstrate that STFormer surpasses alternative baseline forecasting models and achieves state-of-the-art results for single-step horizon and multistep horizon forecasting.

Keywords: Multivariate time series forecasting, Spatio-temporal structure, transformer, graph embedding, recurrent neural network

DOI: 10.3233/JIFS-237250

Journal: Journal of Intelligent & Fuzzy Systems, vol. 46, no. 3, pp. 6951-6967, 2024