Holistic data cleaning: Putting violations into context

Xu Chu, Ihab F. Ilyas, Paolo Papotti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

119 Citations (Scopus)

Abstract

Data cleaning is an important problem and data quality rules are the most promising way to face it with a declarative approach. Previous work has focused on specific formalisms, such as functional dependencies (FDs), conditional functional dependencies (CFDs), and matching dependencies (MDs), and those have always been studied in isolation. Moreover, such techniques are usually applied in a pipeline or interleaved. In this work we tackle the problem in a novel, unified framework. First, we let users specify quality rules using denial constraints with ad-hoc predicates. This language subsumes existing formalisms and can express rules involving numerical values, with predicates such as "greater than" and "less than". More importantly, we exploit the interaction of the heterogeneous constraints by encoding them in a conflict hypergraph. Such holistic view of the conflicts is the starting point for a novel definition of repair context which allows us to compute automatically repairs of better quality w.r.t. previous approaches in the literature. Experimental results on real datasets show that the holistic approach outperforms previous algorithms in terms of quality and efficiency of the repair.

Original languageEnglish
Title of host publicationICDE 2013 - 29th International Conference on Data Engineering
Pages458-469
Number of pages12
DOIs
Publication statusPublished - 15 Aug 2013
Event29th International Conference on Data Engineering, ICDE 2013 - Brisbane, QLD, Australia
Duration: 8 Apr 201311 Apr 2013

Publication series

NameProceedings - International Conference on Data Engineering
ISSN (Print)1084-4627

Other

Other29th International Conference on Data Engineering, ICDE 2013
CountryAustralia
CityBrisbane, QLD
Period8/4/1311/4/13

    Fingerprint

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Cite this

Chu, X., Ilyas, I. F., & Papotti, P. (2013). Holistic data cleaning: Putting violations into context. In ICDE 2013 - 29th International Conference on Data Engineering (pp. 458-469). [6544847] (Proceedings - International Conference on Data Engineering). https://doi.org/10.1109/ICDE.2013.6544847