PCA and PLS with very large data sets

Chemometrics was started around 30 years ago to cope with the rapidly increasing volumes of data produced in chemical laboratories. A multivariate approach based on projections-PCA and PLS-was developed that adequately solved many of the problems at hand. However, with the further increase in the size of our data sets seen today in all fields of science and technology, we start to see inadequacies in our multivariate methods, both in their efficiency and interpretability. Starting from a few examples of complicated problems seen in RD&P (research, development, and production), possible extensions and generalizations of the existing multivariate projection methods-PCA and PLS-will be discussed. Criteria such as scalability of methods to increasing size of problems and data, increasing sophistication in the handling of noise and non-linearities, interpretability of results, and relative simplicity of use, will be held as important. The discussion will be made from a perspective of the evolution of scientific methodology as (a) driven by new technology, e.g., computers and graphical displays, and the need to answer some always reoccurring and basic questions, and (b) constrained by the limitations of the human brain, i.e., our ability to understand and interpret scientific and data analytic results. (C) 2003 Elsevier B.V. All rights reserved.