A Novel Cost-Based Model for Data Repairing

Shuang Hao, Nan Tang, Guoliang Li, Jian He, Na Ta, Jianhua Feng

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Integrity constraint based data repairing is an iterative process consisting of two parts: detect and group errors that violate given integrity constraints (ICs); and modify values inside each group such that the modified database satisfies those ICs. However, most existing automatic solutions treat the process of detecting and grouping errors straightforwardly (e.g., violations of functional dependencies using string equality), while putting more attention on heuristics of modifying values within each group. In this paper, we propose a revised semantics of violations and data consistency w.r.t. a set of ICs. The revised semantics relies on string similarities, in contrast to traditional methods that use syntactic error detection using string equality. Along with the revised semantics, we also propose a new cost model to quantify the cost of data repair by considering distances between strings. We show that the revised semantics provides a significant change for better detecting and grouping errors, which in turn improves both precision and recall of the following data repairing step. We prove that finding minimum-cost repairs in the new model is NP-hard, even for a single FD. We devise efficient algorithms to find approximate repairs. In addition, we develop indices and optimization techniques to improve the efficiency. Experiments show that our approach significantly outperforms existing automatic repair algorithms in both precision and recall.

Original languageEnglish
Article number7779087
Pages (from-to)727-742
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume29
Issue number4
DOIs
Publication statusPublished - 1 Apr 2017

    Fingerprint

Keywords

  • Data repairing
  • fault-tolerant violation
  • functional dependencies
  • graph model
  • maximal independent set

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Cite this