The Unreasonable Effectiveness of Data

Natural language processing problems are solved by the use of unreasonable effectiveness of data. The biggest successes in natural-language-related machine learning is statistical speech recognition and statistical machine translation. The first lesson of Web-scale learning is to use available large-scale data rather than hoping for annotated data that is not available. The statistical language models used in speech recognition and machine translation consist of a huge database of probabilities of short sequences of consecutive words. Natural language processing require choosing a representation language, encoding a model in that language, and performing inference on the model. Semantic interpretation deals with imprecise, ambiguous natural languages, and service interoperability deals with making data precise enough so that the programs operating on the data functions effectively.