Affiliations: Department of Computer Engineering, IIST, Jeju National University, Jeju Special Self-Governing Province, Korea
Correspondence:
[*]
Corresponding author. Yung-Cheol Byun, Department of Computer Engineering, IIST, Jeju National University, Jejusi 63243, Jeju Special Self-Governing Province, Korea. E-mail: ycb@jejunu.ac.kr.
Abstract: Understanding the real-world short texts become an essential task in the recent research area. The document deduction analysis and latent coherent topic named as the important aspect of this process. Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic Analysis (PLSA) are suggested to model huge information and documents. This type of contexts’ main problem is the information limitation, words relationship, sparsity, and knowledge extraction. The knowledge discovery and machine learning techniques integrated with topic modeling were proposed to overcome this issue. The knowledge discovery was applied based on the hidden information extraction to increase the suitable dataset for further analysis. The integration of machine learning techniques, Artificial Neural Network (ANN) and Long Short-Term (LSTM) are applied to anticipate topic movements. LSTM layers are fed with latent topic distribution learned from the pre-trained Latent Dirichlet Allocation (LDA) model. We demonstrate general information from different techniques applied in short text topic modeling. We proposed three categories based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation using representative design and analysis of all categories’ performance in different tasks. Finally, the proposed system evaluates with state-of-art methods on real-world datasets, comprises them with long document topic modeling algorithms, and creates a classification framework that considers further knowledge and represents it in the machine learning pipeline.
Keywords: Machine Learning, knowledge discovery, Topic Modeling, Latent Dirichlet
Allocation, Short Text, Long Short Term Memory
