Profiler: Integrated Statistical Analysis and Visualization for Data Quality Assessment

被引:121
|
作者
Kandel, Sean [1 ]
Parikh, Ravi [1 ]
Paepcke, Andreas [1 ]
Hellerstein, Joseph M.
Heer, Jeffrey [1 ]
机构
[1] Stanford Univ, Stanford, CA 94305 USA
来源
PROCEEDINGS OF THE INTERNATIONAL WORKING CONFERENCE ON ADVANCED VISUAL INTERFACES | 2012年
基金
美国国家科学基金会;
关键词
Data analysis; visualization; data quality; anomaly detection;
D O I
10.1145/2254556.2254659
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Data quality issues such as missing, erroneous, extreme and duplicate values undermine analysis and are time-consuming to find and fix. Automated methods can help identify anomalies, but determining what constitutes an error is context-dependent and so requires human judgment. While visualization tools can facilitate this process, analysts must often manually construct the necessary views, requiring significant expertise. We present Profiler, a visual analysis tool for assessing quality issues in tabular data. Profiler applies data mining methods to automatically flag problematic data and suggests coordinated summary visualizations for assessing the data in context. The system contributes novel methods for integrated statistical and visual analysis, automatic view suggestion, and scalable visual summaries that support real-time interaction with millions of data points. We present Profiler's architecture-including modular components for custom data types, anomaly detection routines and summary visualizations-and describe its application to motion picture, natural disaster and water quality data sets.
引用
收藏
页码:547 / 554
页数:8
相关论文
共 50 条
  • [1] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 1999, 40 (02): : J214 - J221
  • [2] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 1998, 39 (05): : J384 - J389
  • [3] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 2000, 41 (05): : J316 - J322
  • [4] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 2000, 41 (03): : J238 - J242
  • [5] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 1999, 40 (04): : J325 - J332
  • [6] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 1998, 39 (04): : J325 - J332
  • [7] Statistical data analysis in quality assessment
    Ohsumi, N
    JOURNAL OF THE FOOD HYGIENIC SOCIETY OF JAPAN, 1998, 39 (06): : J451 - J456
  • [8] Geo-spatial data analysis, quality assessment and visualization
    Ge, Yong
    Bai Hexiang
    Li, Sanping
    COMPUTATIONAL SCIENCE AND ITS APPLICATIONS - ICCSA 2008, PT 1, PROCEEDINGS, 2008, 5072 : 258 - 267
  • [9] Data Analysis in Multimedia Quality Assessment: Revisiting the Statistical Tests
    Narwaria, Manish
    Krasula, Lukas
    Le Callet, Patrick
    IEEE TRANSACTIONS ON MULTIMEDIA, 2018, 20 (08) : 2063 - 2072
  • [10] Data Visualization and Statistical Graphics in Big Data Analysis
    Cook, Dianne
    Lee, Eun-Kyung
    Majumder, Mahbubul
    ANNUAL REVIEW OF STATISTICS AND ITS APPLICATION, VOL 3, 2016, 3 : 133 - 159