An Effective Approach for Web Document Classification using the Concept of Association Analysis of Data Mining

Exponential growth of the web increased the importance of web document classification and data mining. To get the exact information, in the form of knowing what classes a web document belongs to, is expensive. Automatic classification of web document is of great use to search engines which provides this information at a low cost. In this paper, we propose an approach for classifying the web document using the frequent item word sets generated by the Frequent Pattern (FP) Growth which is an association analysis technique of data mining. These set of associated words act as feature set. The final classification obtained after Naive Bayes classifier used on the feature set. For the experimental work, we use Gensim package, as it is simple and robust. Results show that our approach can be effectively classifying the web document. Web document classification is the process of classifying documents into predefined categories based on their content. The classifiers used for this purpose should be trained from the web documents that are already classified. The task is to assign a document to one or more classes or categories. This may be done "manually" (or "intellectually") or algorithmically. Manual classification cost more. The intellectual classification of documents has mostly been the province of library science, while the algorithmic classification of documents is used mainly in information science and computer science. The problems are overlapping; however there is also interdisciplinary research on documents classification. The documents to be classified may be texts, images, music, etc. Each kind of document possesses its special classification problems. Documents may be classified according to their subjects or according to other attributes. Web document classification is the primary requirement for search engines, which retrieve documents in response to the user query. Documents classification or text categorization (as used in information retrieval context) is the process of assigning a document to a predefined set of categories based on the document content. Documents classification can be applied as an information filtering tool and can also be used to improve the retrieval results from a query process. Classification is one of the main data analysis techniques and deals with the categorizing a new data entry into one of the categories based on the values of different attributes. In general, classification algorithm needs to train a model based on pre-classified training documents. Once the model is ready, we can subject the test documents for evaluation through that model and that brings the classification process to an end. In this paper, we proposed an approach for automatically classifying web documents into a set of categories using FP-growth and Naive Bayes techniques. In our approach, we have given a set of example documents. We preprocess the documents by parsing and removing the stop words, doing stemming (10) and extracted noun as keywords. Then we apply FP-growth method (9) to find the frequent item word sets from each document.� The documents are treated as transactions and the set of frequently occurring words are viewed as a set of items in the transaction. The new documents are classified by applying Naive Bayes technique on these derived features sets.