Spatio-temporal deep learning framework for pedestrian intention prediction in urban traffic scenes

Article type: Research Article

Authors: | Singh, Pardeep | Chand, Satish

Affiliations: School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi, India

Correspondence: [*] Corresponding author. E-mail: monikasingh314@gmail.com.

Abstract: Pedestrian intent prediction is an essential task for ensuring the safety of pedestrians and vehicles on the road. This task involves predicting whether a pedestrian intends to cross a road or not based on their behavior and surrounding environment. Previous studies have explored feature-based machine learning and vision-based deep learning models for this task but these methods have limitations in capturing the global spatio-temporal context and fusing different features of data effectively. To address these issues, we propose a novel hybrid framework HSTGCN for pedestrian intent prediction that combines spatio-temporal graph convolutional neural networks (STGCN) and long short-term memory (LSTM) networks. The proposed framework utilizes the strengths of both models by fusing multiple features, including skeleton pose, trajectory, height, orientation, and ego-vehicle speed, to predict their intentions accurately. The framework’s performance have been evaluated on the JAAD benchmark dataset and the results show that it outperforms the state-of-the-art methods. The proposed framework has potential applications in developing intelligent transportation systems, autonomous vehicles, and pedestrian safety technologies. The utilization of multiple features can significantly improve the performance of the pedestrian intent prediction task.

Keywords: STGCN, LSTM, skeleton, spatio temporal, pedestrian intent prediction

DOI: 10.3233/AIC-230053

Journal: AI Communications, vol. 37, no. 4, pp. 549-562, 2024