Affiliations: College of Civil Engineering, Nanjing Forestry University, Nanjing, Jiangsu, China
Correspondence:
[*]
Corresponding author: Qiang Li, College of Civil Engineering, Nanjing Forestry University, Nanjing 210042, Jiangsu, China. E-mail: liqiang2526@njfu.edu.cn.
Abstract: Traditional data storage models are inadequate in the face of the growing demand for big data in transportation and transportation management. Its poor horizontal scalability makes it difficult to deal with the growth of massive data; on the other hand, its complex management makes it challenging to achieve unified management and effective resource utilization due to the differences in equipment from different manufacturers. In order to effectively store and manage this huge amount of information, it is urgent to rely on advanced technical tools. In this context, while ensuring data security and simplifying data management, it is also necessary to meet the terminal’s demand for high real-time performance, and these factors jointly promote the continuous attention and improvement of data storage terminal performance. This paper proposes a Hadoop solution based on distributed computing. As a distributed system infrastructure, Hadoop allows users to develop distributed programs without a deep understanding of distributed details, fully using the high-speed computing and storage capabilities of Hadoop clusters, which is especially suitable for big data processing tasks on the Internet of Things (IoT) platform. The experimental results show that for a 10 GB data file, the traditional terminal (Terminal 1) can store 7.8 GB, while the Hadoop-based terminal (Terminal 2) can store 9.9 GB. For 50 GB of data files, Terminal 1 and Terminal 2 can store 40.4 GB and 49.8 GB of data respectively. These results show that Hadoop terminals have significant advantages in processing large-scale data, especially in terms of data storage efficiency, and can use storage resources more effectively to meet the high-performance requirements of traffic and transportation management for data storage terminals.
Keywords: Internet of Things, distributed computing, transportation management, data storage terminal, data transfer, data management
