Affiliations: [a]
University of Minnesota Duluth, Duluth, MN, USA
| [b]
West Bengal State University, Kokata, West Bengal, India
| [c]
Jadavpur University, Kolkata, West Bengal, India
Correspondence:
[*]
Corresponding author. Rajat Pandit, West Bengal State University, Kolkata-700126, India. E-mail: rajatpandit123@gmail.com.
Note: [1] Work done while at Jadavpur University.
Abstract: One of the most challenging research problems in natural language processing (NLP) is that of word sense induction (WSI). It involves discovering senses of a word given its contexts of usage without the use of a sense inventory which differentiates it from traditional word sense disambiguation (WSD). This paper reports a work on sense induction in Bengali, a less-resourced language, based on distributional semantics and translation based context vectors learned from parallel corpora to improve the task performance. The performance of the proposed method of sense induction was compared with the k-means algorithm, which was considered as the baseline in our work. A dataset for sense induction was created for 15 Bengali words, encompassing a total of 111 contexts. The proposed model, in both mono and cross-lingual settings, outperformed k-means in precision (P), recall (R) and F-scores. K-means based sense induction produced average P, R and F-scores of 0.71, 0.73 and 0.66 respectively. The average P, R and F-scores produced by the mono-and cross-lingual settings of the proposed algorithm are 0.77, 0.73, 0.68 and 0.81, 0.77 and 0.72 respectively.
Keywords: Word sense induction (WSI), parallel corpora, translation, Word2Vec, context clustering
