DeepSearcher: A Chinese deep web classified search engine

Web search engines work well in finding crawlable pages, but not well in finding pages dynamically generated by back-end databases. These pages are often referred to as the Deep Web. According to many studies, the Deep Web reside in topic-specific back-end databases, and the size of the Deep Web increases rapidly. Moreover, contents provided by Deep Web are often of high quality and well-structured. Organizing such structured sources into a domain hierarchy will make users conveniently browse to find these valuable resources and this is one of the critical steps toward large-scale integration of heterogeneous Deep Web sources. We design a prototype of Chinese Deep Web classified search engine called DeepSearcher. We also propose a Deep Web crawling strategy and algorithm for Deep Web judgment and classification. Our experimental results indicate that this approach can achieve good results.