Affiliations: School of Information Engineering, Shanghai Zhongqiao Vocational and Technical University, Shanghai 201514, China | E-mail: qifuli2008@163.com
Correspondence:
[*]
School of Information Engineering, Shanghai Zhongqiao Vocational and
Technical University, Shanghai 201514, China. E-mail: qifuli2008@163.com.
Abstract: The rapid development of social networks has facilitated the convenience of users to receive information. As a network communication platform for people’s daily use, microblog has countless information data. In view of the low efficiency and poor clustering effect of K-means algorithm, a parallel K-means clustering algorithm based on MapReduce model is studied; In order to alleviate the difficulty in calculating the similarity of microblog topic text, the space vector model and semantic similarity are used to calculate the similarity between texts to improve the quality of microblog text classification. The data expansion rate of corresponding nodes under different data sets shows that the average expansion rate of the parallel K-means algorithm reaches 0.89, and the running rate is the highest. The results show that the parallel K-means algorithm has good clustering stability and the highest clustering quality, reaching 1.24; The clustering time of the algorithm is the shortest, the average clustering time is 1.27 minutes, and the clustering effect and efficiency of the algorithm are the best. In the quality analysis of Weibo topic recommendation, the accuracy of P-K-means recommendation is 95.64%, user satisfaction is 98.64%, and the recommendation effect is also the best. It shows that the research on the parallel K-means clustering algorithm based on MapReduce has the best performance in microblogging topic mining and recommendation, which can efficiently recommend topics of interest to users and enhance users’ microblogging experience.
Keywords: Social network, big data, MapReduce, parallel K-means clustering algorithm, Weibo topic
