WALAILAK JOURNAL OF SCIENCE AND TECHNOLOGY

Volume 16, No. 02, Month FEBRUARY, Year 2019, Pages 107 - 119

Short text document clustering using distributed word representation and document distance

Supavit KONGWUDHIKUNAKORN, Kitsana WAIYAMAI

Abstract Download PDF

This paper presents a method for clustering short text documents, such as instant messages, SMS, or news headlines. Vocabularies in the texts are expanded using external knowledge sources and represented by a Distributed Word Representation. Clustering is done using the K-means algorithm with Word Mover"es Distance as the distance metric. Experiments were done to compare the clustering quality of this method, and several leading methods, using large datasets from BBC headlines, SearchSnippets, StackExchange, and Twitter. For all datasets, the proposed algorithm produced document clusters with higher accuracy, precision, F1-score, and Adjusted Rand Index. We also observe that cluster description can be inferred from keywords represented in each cluster.

Keywords

Distributed word representation, document distance, short text documents, short text documents clustering

---