Navigating Data-Centric Artificial Intelligence with DC-Check: Advances, Challenges, and Opportunities

Data-centric artificial intelligence (AI) is an emerging paradigm that emphasizes the critical role of data in real-world machine learning (ML) systems - as a complement to model development. However, data-centric AI is still in its infancy, lacking a standardized framework that outlines necessary data-centric considerations at various stages of the ML pipeline: Data, Training, Testing, and Deployment. This lack of guidance hampers effective communication and design of data-centric driven ML systems. To address this critical gap, we introduce the Data-Centric Checklist (DC-Check), an actionable checklist-style framework that encapsulates data-centric considerations for ML systems. DC-Check is aimed at both practitioners and researchers to serve as a reference guide to data-centric AI development. Around each question in DC-Check, we discuss the applicability of different approaches, survey the state of the art, and highlight specific data-centric AI challenges and research opportunities. While developing DC-Check, we also undertook an analysis of the current data-centric AI landscape. The insights obtained from this exploration support the DC-Check framework, reinforcing its utility and relevance in the rapidly evolving field. To make DC-Check and related resources easily accessible, we provide a DC-Check companion website (https://www.vanderschaar-lab.com/dc-check/), which will serve as a living resource, updated as methods and tools evolve. © 2020 IEEE.