Abstract: Automatic knowledge discovery from texts (KDT) is proving to be a
promising method for businesses today to deal with the overload of textual
information. In this paper, we first explore the possibilities for KDT to
enhance communication in virtual communities, and then we present a practical
case study with real-life Internet data. The problem in the case study is to
manage the very successful virtual communities known as 'clubs' of the largest
Dutch Internet Service Provider. It is possible for anyone to start a club
about any subject, resulting in over 10,000 active clubs today. At the
beginning, the founder assigns the club to a predefined category. This often
results in illogical or inconsistent placements, which means that interesting
clubs may be hard to locate for potential new members. The ISP therefore is
looking for an automated way to categorize clubs in a logical and consistent
manner. The method used is the so-called bag-of-words approach, previously
applied mostly to scientific texts and structured documents. Each club is
described by a vector of word occurrences of all communications within that
club. Latent Semantic Indexing (LSI) is applied to reduce the dimensionality
problem prior to clustering. Clustering is done by the Within Groups Clustering
method using a cosine distance measure appropriate for texts. The results show
that KDT and the LSI method can successfully be applied for clustering the very
volatile and unstructured textual communication on the Internet.
Keywords: knowledge discovery in text, clustering, virtual community, latent semantic indexing, web portal
