Raha: A configuration-free error detection system

Mohammad Mahdavi, Samuel Madden, Ziawasch Abedjan, Mourad Ouzzani, Nan Tang, Raul Castro Fernandez, Michael Stonebraker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Detecting erroneous values is a key step in data cleaning. Error detection algorithms usually require a user to provide input configurations in the form of rules or statistical parameters. However, providing a complete, yet correct, set of configurations for each new dataset is not trivial, as the user has to know about both the dataset and the error detection algorithms upfront. In this paper, we present Raha, a new configuration-free error detection system. By generating a limited number of configurations for error detection algorithms that cover various types of data errors, we can generate an expressive feature vector for each tuple value. Leveraging these feature vectors, we propose a novel sampling and classification scheme that effectively chooses the most representative values for training. Furthermore, our system can exploit historical data to filter out irrelevant error detection algorithms and configurations. In our experiments, Raha outperforms the state-of-the-art error detection techniques with no more than 20 labeled tuples on each dataset.

Original languageEnglish
Title of host publicationSIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data
PublisherAssociation for Computing Machinery
Pages865-882
Number of pages18
ISBN (Electronic)9781450356435
DOIs
Publication statusPublished - 25 Jun 2019
Event2019 International Conference on Management of Data, SIGMOD 2019 - Amsterdam, Netherlands
Duration: 30 Jun 20195 Jul 2019

Publication series

NameProceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)0730-8078

Conference

Conference2019 International Conference on Management of Data, SIGMOD 2019
CountryNetherlands
CityAmsterdam
Period30/6/195/7/19

Fingerprint

Error detection
Cleaning
Sampling
Experiments

ASJC Scopus subject areas

  • Software
  • Information Systems

Cite this

Mahdavi, M., Madden, S., Abedjan, Z., Ouzzani, M., Tang, N., Fernandez, R. C., & Stonebraker, M. (2019). Raha: A configuration-free error detection system. In SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data (pp. 865-882). (Proceedings of the ACM SIGMOD International Conference on Management of Data). Association for Computing Machinery. https://doi.org/10.1145/3299869.3324956

Raha : A configuration-free error detection system. / Mahdavi, Mohammad; Madden, Samuel; Abedjan, Ziawasch; Ouzzani, Mourad; Tang, Nan; Fernandez, Raul Castro; Stonebraker, Michael.

SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data. Association for Computing Machinery, 2019. p. 865-882 (Proceedings of the ACM SIGMOD International Conference on Management of Data).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mahdavi, M, Madden, S, Abedjan, Z, Ouzzani, M, Tang, N, Fernandez, RC & Stonebraker, M 2019, Raha: A configuration-free error detection system. in SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data. Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, pp. 865-882, 2019 International Conference on Management of Data, SIGMOD 2019, Amsterdam, Netherlands, 30/6/19. https://doi.org/10.1145/3299869.3324956
Mahdavi M, Madden S, Abedjan Z, Ouzzani M, Tang N, Fernandez RC et al. Raha: A configuration-free error detection system. In SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data. Association for Computing Machinery. 2019. p. 865-882. (Proceedings of the ACM SIGMOD International Conference on Management of Data). https://doi.org/10.1145/3299869.3324956
Mahdavi, Mohammad ; Madden, Samuel ; Abedjan, Ziawasch ; Ouzzani, Mourad ; Tang, Nan ; Fernandez, Raul Castro ; Stonebraker, Michael. / Raha : A configuration-free error detection system. SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data. Association for Computing Machinery, 2019. pp. 865-882 (Proceedings of the ACM SIGMOD International Conference on Management of Data).
@inproceedings{b5c0af4456734d949d3ef7cdfe4e3ab0,
title = "Raha: A configuration-free error detection system",
abstract = "Detecting erroneous values is a key step in data cleaning. Error detection algorithms usually require a user to provide input configurations in the form of rules or statistical parameters. However, providing a complete, yet correct, set of configurations for each new dataset is not trivial, as the user has to know about both the dataset and the error detection algorithms upfront. In this paper, we present Raha, a new configuration-free error detection system. By generating a limited number of configurations for error detection algorithms that cover various types of data errors, we can generate an expressive feature vector for each tuple value. Leveraging these feature vectors, we propose a novel sampling and classification scheme that effectively chooses the most representative values for training. Furthermore, our system can exploit historical data to filter out irrelevant error detection algorithms and configurations. In our experiments, Raha outperforms the state-of-the-art error detection techniques with no more than 20 labeled tuples on each dataset.",
author = "Mohammad Mahdavi and Samuel Madden and Ziawasch Abedjan and Mourad Ouzzani and Nan Tang and Fernandez, {Raul Castro} and Michael Stonebraker",
year = "2019",
month = "6",
day = "25",
doi = "10.1145/3299869.3324956",
language = "English",
series = "Proceedings of the ACM SIGMOD International Conference on Management of Data",
publisher = "Association for Computing Machinery",
pages = "865--882",
booktitle = "SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data",

}

TY - GEN

T1 - Raha

T2 - A configuration-free error detection system

AU - Mahdavi, Mohammad

AU - Madden, Samuel

AU - Abedjan, Ziawasch

AU - Ouzzani, Mourad

AU - Tang, Nan

AU - Fernandez, Raul Castro

AU - Stonebraker, Michael

PY - 2019/6/25

Y1 - 2019/6/25

N2 - Detecting erroneous values is a key step in data cleaning. Error detection algorithms usually require a user to provide input configurations in the form of rules or statistical parameters. However, providing a complete, yet correct, set of configurations for each new dataset is not trivial, as the user has to know about both the dataset and the error detection algorithms upfront. In this paper, we present Raha, a new configuration-free error detection system. By generating a limited number of configurations for error detection algorithms that cover various types of data errors, we can generate an expressive feature vector for each tuple value. Leveraging these feature vectors, we propose a novel sampling and classification scheme that effectively chooses the most representative values for training. Furthermore, our system can exploit historical data to filter out irrelevant error detection algorithms and configurations. In our experiments, Raha outperforms the state-of-the-art error detection techniques with no more than 20 labeled tuples on each dataset.

AB - Detecting erroneous values is a key step in data cleaning. Error detection algorithms usually require a user to provide input configurations in the form of rules or statistical parameters. However, providing a complete, yet correct, set of configurations for each new dataset is not trivial, as the user has to know about both the dataset and the error detection algorithms upfront. In this paper, we present Raha, a new configuration-free error detection system. By generating a limited number of configurations for error detection algorithms that cover various types of data errors, we can generate an expressive feature vector for each tuple value. Leveraging these feature vectors, we propose a novel sampling and classification scheme that effectively chooses the most representative values for training. Furthermore, our system can exploit historical data to filter out irrelevant error detection algorithms and configurations. In our experiments, Raha outperforms the state-of-the-art error detection techniques with no more than 20 labeled tuples on each dataset.

UR - http://www.scopus.com/inward/record.url?scp=85069437614&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069437614&partnerID=8YFLogxK

U2 - 10.1145/3299869.3324956

DO - 10.1145/3299869.3324956

M3 - Conference contribution

AN - SCOPUS:85069437614

T3 - Proceedings of the ACM SIGMOD International Conference on Management of Data

SP - 865

EP - 882

BT - SIGMOD 2019 - Proceedings of the 2019 International Conference on Management of Data

PB - Association for Computing Machinery

ER -