Intelligent Automated Workload Analysis for Database Replatforming

被引:3
|
作者
Aleyasen, Amirhossein [1 ,2 ]
Morcos, Mark [1 ]
Antova, Lyublena [1 ]
Sugiyama, Marc [1 ]
Korablev, Dmitri [1 ]
Patvarczki, Jozsef [1 ]
Mutreja, Rima [1 ]
Duller, Michael [1 ]
Waas, Florian M. [1 ]
Winslett, Marianne [2 ]
机构
[1] Datometry Inc, San Francisco, CA 94105 USA
[2] Univ Illinois, San Francisco, CA 94105 USA
来源
PROCEEDINGS OF THE 2022 INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA (SIGMOD '22) | 2022年
关键词
workload analysis; data warehousing; porting complexity; database replatforming; adaptive data virtualization;
D O I
10.1145/3514221.3526050
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Performing a detailed workload analysis is a crucial step in determining the feasibility, timeline and cost of a major data warehouse replatforming project, i.e., migration from one platform to another. A large company's data warehouse applications may include millions of queries, some of which will use features that are unsupported or have different semantics in the new warehouse, or may have poor performance there. In this paper we present q Insight, a workload analyzer that Datometry has used in data warehouse replatforming efforts for dozens of major clients. qInsight leverages Datometry's Hyper-Q to obtain insights from a workload, including SQL features and workload structural information that could not be obtained without deep query analysis. qInsight uses the identified features and a weighting scheme based on human expert judgments to assess the difficulty of rewriting each application in the workload via traditional migration methods. Datometry's clients find this information useful in planning their projects, including the order in which to migrate applications. We present a q Insight-based data warehouse usage analysis of over 1.7 billion queries from real-world workloads.
引用
收藏
页码:2273 / 2285
页数:13
相关论文
共 50 条
  • [31] A suspect-oriented intelligent and automated computer forensic analysis
    Al Fahdi, M.
    Clarke, N. L.
    Li, F.
    Furnell, S. M.
    DIGITAL INVESTIGATION, 2016, 18 : 65 - 76
  • [32] Analysis: Automated evaluation of consistency within the PubChem Compound database
    Dashti, Hesam
    Wedell, Jonathan R.
    Westler, William M.
    Markley, John L.
    Eghbalnia, Hamid R.
    SCIENTIFIC DATA, 2019, 6 (1)
  • [33] A Comprehensive and Automated Approach to Intelligent Business Processes Execution Analysis
    Malu Castellanos
    Fabio Casati
    Umeshwar Dayal
    Ming-Chien Shan
    Distributed and Parallel Databases, 2004, 16 : 239 - 273
  • [34] NeSSI® platforms for analysis in manual, automated, and intelligent analyzer systems
    Simko, David M.
    CHIMICA OGGI-CHEMISTRY TODAY, 2008, 26 (01) : 25 - 27
  • [35] A comprehensive and automated approach to intelligent business processes execution analysis
    Castellanos, M
    Casati, F
    Dayal, U
    Shan, MC
    DISTRIBUTED AND PARALLEL DATABASES, 2004, 16 (03) : 239 - 273
  • [36] A survey of intelligent agent literature via automated document analysis
    Kline, DM
    Riggle, CG
    Kohers, G
    Madey, G
    DECISION SCIENCES INSTITUTE 1998 PROCEEDINGS, VOLS 1-3, 1998, : 702 - 702
  • [37] Guided programming and automated error analysis in an intelligent Prolog tutor
    Hong, J
    INTERNATIONAL JOURNAL OF HUMAN-COMPUTER STUDIES, 2004, 61 (04) : 505 - 534
  • [38] Intelligent Performance Analysis of Automated Steering Systems for Autonomous Vehicles
    Salih, Saif
    Olawoyin, Richard
    2020 IEEE INTERNATIONAL CONFERENCE ON ELECTRO INFORMATION TECHNOLOGY (EIT), 2020, : 200 - 205
  • [39] Intelligent signal processing in an automated measurement data analysis system
    Isernhagen, Henrik
    Neemann, Helmut
    Kuehn, Steffen
    Guehmann, Clemens
    2007 IEEE SYMPOSIUM ON COMPUTATIONAL INTELLIGENCE IN IMAGE AND SIGNAL PROCESSING, 2007, : 83 - +
  • [40] LS Footwear Database - Evaluating Automated Footwear Pattern Analysis
    Pavlou, Maria
    Allinson, Nigel M.
    ARTIFICIAL NEURAL NETWORKS - ICANN 2009, PT II, 2009, 5769 : 445 - 454