Cross-platform data processing: Use cases and challenges

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

There is a zoo of data processing platforms which help users and organizations to extract value out of their data. Although each of these platforms excels in specific aspects, users typically end up running their data analytics on suboptimal platforms. This is not only because choosing the right platform among the myriad of big data platforms is a daunting task, but also due to the fact that today's data analytics are moving beyond the limits of a single platform. Thus, there is an urgent need for cross-platform data processing, i.e., using more than one data processing platform to perform a data analytics task. Despite the need, achieving this is still a dreadful process where developers have to get intimate with many systems and write ad hoc scripts for integrating them. This tutorial is motivated by this need. We will discuss the importance of supporting cross-platform data processing in a systematic way as well as the current efforts to achieve that. In particular, we will introduce a classification of the different cases where an application needs or benefits from cross-platform data processing and the challenges of each case. Along with this classification, we will also present the efforts known up to date to support cross-platform data processing. We will conclude this tutorial with a discussion of several important open problems.

Original languageEnglish
Title of host publicationProceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1723-1726
Number of pages4
ISBN (Electronic)9781538655207
DOIs
Publication statusPublished - 24 Oct 2018
Event34th IEEE International Conference on Data Engineering, ICDE 2018 - Paris, France
Duration: 16 Apr 201819 Apr 2018

Other

Other34th IEEE International Conference on Data Engineering, ICDE 2018
CountryFrance
CityParis
Period16/4/1819/4/18

Fingerprint

Tutorial
Ad hoc
Excel
End users
Zoo
Developer
Big data

Keywords

  • Cross platform
  • Data processing
  • Polystores
  • Query processing

ASJC Scopus subject areas

  • Information Systems
  • Information Systems and Management
  • Hardware and Architecture

Cite this

Kaoudi, Z., & Quiane Ruiz, J. A. (2018). Cross-platform data processing: Use cases and challenges. In Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018 (pp. 1723-1726). [8509444] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDE.2018.00223

Cross-platform data processing : Use cases and challenges. / Kaoudi, Zoi; Quiane Ruiz, Jorge Arnulfo.

Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1723-1726 8509444.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kaoudi, Z & Quiane Ruiz, JA 2018, Cross-platform data processing: Use cases and challenges. in Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018., 8509444, Institute of Electrical and Electronics Engineers Inc., pp. 1723-1726, 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, 16/4/18. https://doi.org/10.1109/ICDE.2018.00223
Kaoudi Z, Quiane Ruiz JA. Cross-platform data processing: Use cases and challenges. In Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1723-1726. 8509444 https://doi.org/10.1109/ICDE.2018.00223
Kaoudi, Zoi ; Quiane Ruiz, Jorge Arnulfo. / Cross-platform data processing : Use cases and challenges. Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1723-1726
@inproceedings{b2dce3017e1b4528a255ffae5d4e93cf,
title = "Cross-platform data processing: Use cases and challenges",
abstract = "There is a zoo of data processing platforms which help users and organizations to extract value out of their data. Although each of these platforms excels in specific aspects, users typically end up running their data analytics on suboptimal platforms. This is not only because choosing the right platform among the myriad of big data platforms is a daunting task, but also due to the fact that today's data analytics are moving beyond the limits of a single platform. Thus, there is an urgent need for cross-platform data processing, i.e., using more than one data processing platform to perform a data analytics task. Despite the need, achieving this is still a dreadful process where developers have to get intimate with many systems and write ad hoc scripts for integrating them. This tutorial is motivated by this need. We will discuss the importance of supporting cross-platform data processing in a systematic way as well as the current efforts to achieve that. In particular, we will introduce a classification of the different cases where an application needs or benefits from cross-platform data processing and the challenges of each case. Along with this classification, we will also present the efforts known up to date to support cross-platform data processing. We will conclude this tutorial with a discussion of several important open problems.",
keywords = "Cross platform, Data processing, Polystores, Query processing",
author = "Zoi Kaoudi and {Quiane Ruiz}, {Jorge Arnulfo}",
year = "2018",
month = "10",
day = "24",
doi = "10.1109/ICDE.2018.00223",
language = "English",
pages = "1723--1726",
booktitle = "Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Cross-platform data processing

T2 - Use cases and challenges

AU - Kaoudi, Zoi

AU - Quiane Ruiz, Jorge Arnulfo

PY - 2018/10/24

Y1 - 2018/10/24

N2 - There is a zoo of data processing platforms which help users and organizations to extract value out of their data. Although each of these platforms excels in specific aspects, users typically end up running their data analytics on suboptimal platforms. This is not only because choosing the right platform among the myriad of big data platforms is a daunting task, but also due to the fact that today's data analytics are moving beyond the limits of a single platform. Thus, there is an urgent need for cross-platform data processing, i.e., using more than one data processing platform to perform a data analytics task. Despite the need, achieving this is still a dreadful process where developers have to get intimate with many systems and write ad hoc scripts for integrating them. This tutorial is motivated by this need. We will discuss the importance of supporting cross-platform data processing in a systematic way as well as the current efforts to achieve that. In particular, we will introduce a classification of the different cases where an application needs or benefits from cross-platform data processing and the challenges of each case. Along with this classification, we will also present the efforts known up to date to support cross-platform data processing. We will conclude this tutorial with a discussion of several important open problems.

AB - There is a zoo of data processing platforms which help users and organizations to extract value out of their data. Although each of these platforms excels in specific aspects, users typically end up running their data analytics on suboptimal platforms. This is not only because choosing the right platform among the myriad of big data platforms is a daunting task, but also due to the fact that today's data analytics are moving beyond the limits of a single platform. Thus, there is an urgent need for cross-platform data processing, i.e., using more than one data processing platform to perform a data analytics task. Despite the need, achieving this is still a dreadful process where developers have to get intimate with many systems and write ad hoc scripts for integrating them. This tutorial is motivated by this need. We will discuss the importance of supporting cross-platform data processing in a systematic way as well as the current efforts to achieve that. In particular, we will introduce a classification of the different cases where an application needs or benefits from cross-platform data processing and the challenges of each case. Along with this classification, we will also present the efforts known up to date to support cross-platform data processing. We will conclude this tutorial with a discussion of several important open problems.

KW - Cross platform

KW - Data processing

KW - Polystores

KW - Query processing

UR - http://www.scopus.com/inward/record.url?scp=85057070441&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057070441&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2018.00223

DO - 10.1109/ICDE.2018.00223

M3 - Conference contribution

AN - SCOPUS:85057070441

SP - 1723

EP - 1726

BT - Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -