Data Integration Patterns for Data Warehouse Automation

被引：5

作者：

Tomingas, Kalle ^{[1
]}

Kliimask, Margus ^{[2
]}

Tammet, Tanel ^{[1
]}

机构：

[1] Tallinn Univ Technol, EE-19086 Tallinn, Estonia

[2] Eliko Competence Ctr, EE-12618 Tallinn, Estonia

来源：

NEW TRENDS IN DATABASE AND INFORMATION SYSTEMS II | 2015年 / 312卷

关键词：

data warehouse; etl; data mappings; template based sql generation; abstract syntax patterns; metadata management;

D O I：

10.1007/978-3-319-10518-5_4

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The paper presents a mapping-based and metadata-driven modular data transformation framework designed to solve extract-transform-load (ETL) automation, impact analysis, data quality and integration problems in data warehouse environments. We introduce a declarative mapping formalization technique, an abstract expression pattern concept and a related template engine technology for flexible ETL code generation and execution. The feasibility and efficiency of the approach is demonstrated on the pattern detection and data lineage analysis case studies using large real life SQL corpuses.

引用

页码：41 / 55

页数：15

共 50 条

[31] CardioVINEdb: a data warehouse approach for integration of life science data in cardiovascular diseases
Kormeier, Benjamin
Hippe, Klaus
Topel, Thoralf
Hofestadt, Ralf
JOURNAL OF INTEGRATIVE BIOINFORMATICS, 2010, 7 (01):
[32] Application of data warehouse and data mining in the steel enterprise information integration system
Pei, Shenglei
Jia, Guoqing
2014 SIXTH INTERNATIONAL CONFERENCE ON INTELLIGENT HUMAN-MACHINE SYSTEMS AND CYBERNETICS (IHMSC), VOL 2, 2014, : 181 - 184
[33] Analyzing Dimension Mappings and Properties in Data Warehouse Integration
Beneventano, Domenico
Olaru, Marius Octavian
Vincini, Maurizio
ON THE MOVE TO MEANINGFUL INTERNET SYSTEMS: OTM 2013 CONFERENCES, 2013, 8185 : 616 - 623
[34] A new process for healthcare big data warehouse integration
Arfaoui, Nouha
INTERNATIONAL JOURNAL OF DATA MINING MODELLING AND MANAGEMENT, 2023, 15 (03) : 240 - 254
[35] Maintenance automation: Data integration, a new recipe
Harrison, Brian
Control Engineering, 2020, 67 (11) : 37 - 39
[36] Automation, integration, and data management for the back end
Steiner, FP
Eshkar, U
Seibert, C
SOLID STATE TECHNOLOGY, 1998, 41 (11) : 69 - +
[37] Enhanced data warehouse platform and its application in the dispatching automation system
South China University of Technology, Guangzhou 510640, China
Dianli Xitong Zidonghue, 2008, 4 (81-84+102):
[38] Using Signifiers for Data Integration in Rail Automation
Wurl, Alexander
Falkner, Andreas
Haselbock, Alois
Mazak, Alexandra
PROCEEDINGS OF THE 6TH INTERNATIONAL CONFERENCE ON DATA SCIENCE, TECHNOLOGY AND APPLICATIONS (DATA), 2017, : 172 - 179
[39] Design patterns for data integration
Schwinn, Alexander
Schelp, Joachim
JOURNAL OF ENTERPRISE INFORMATION MANAGEMENT, 2005, 18 (04) : 471 - +
[40] Integration of Unstructured Data into a Clinical Data Warehouse for Kidney Transplant Screening -Challenges & Solutions
Zubke, Maximilian
Katzensteiner, Matthias
Bott, Oliver J.
DIGITAL PERSONALIZED HEALTH AND MEDICINE, 2020, 270 : 272 - 276

← 1 2 3 4 5 →