General information

• Duration: 2 1/2 years, January 2016 to June 2018
• Project funded by german Federal Ministry of Education and Research (BMBF)
• Project funding reference number (Förderkennzeichen): 01IS15053B
• Project overview (in german) at project executing organization DLR: project overview

Motivation

Both data as well as information derived from it has an increasing impact on companies, since many short- and longterm decisions are based on the interpretation of data. In addition to the long established relational databases, unstructured or semi-structured data sources are gaining importance. Examples of such unstructured data are Weblogs, data from social networks or RFID-data. Therefore, the relevance of data quality monitoring is increasing enormously for these kind of data sources, in order to prevent wrong decisions based on faulty analysis due to flawed data.

Goals and Course of Action

Main target of this joint project is the study of advanced concepts for data quality monitoring of heterogeneous data sources. The project will result in a prototypical tool for data quality analysis. This tool mainly consists of components for semi-automated detection of data quality rules, as well as the automated monitoring of data quality rules.

Focus is laid on the area of detection and derivation of data quality rules. One goal is to automatically detect complex statistical dependencies and suggest rules based on the results. The suggested rules will be presented using a data quality rule language, which will be constructed in the scope of this project. The data quality manager supervises these automatically created rules and may adjust them. This is introduced in order to improve the rule detection continuously.

A central data quality repository will be introduced, which contains data quality rules as well as metadata on rule checking executions. Besides the semi-automatically detected rules, the data quality manager may insert manually created data quality rules.

The permanent data quality monitoring uses the rules from the central data quality repository and regularly checks those against current database content. Deviations will be logged and reported.

Details

You can find more details in the published papers (see publications; a longer article is available here). Further details can be found in the concepts document.

Consortium

• Hochschule Hannover, Faculty IV - Department for Computer Science
• mVise AG
• CTS Eventim (associated partner)