Distributed computing for smart meter data management for electrical utility applications

Ameema Zainab, Shady S. Refaat, Haitham Abu-Rub, Othmane Bouhali

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the advent of Internet-of-Things (IoT) devices, including smart meters and sensors in the smart grid, there has been immense research interest in big data management, analytics, and parallel processing of data. However, complex hardware and software parameters configurations and in-depth understanding of the data processing design are essential for efficient utilization of big data analytics platforms. In this work, we analyze the parallelization of load prediction by utilizing spark regression python library to assess the performance with workloads of up to 8 nodes. The results of different configurations have been studied and analyzed against the performance of Apache Spark. It was found that a trade-off between the number of nodes and cores is necessary to perform efficient parallel computing. Multiple sets of combinations of nodes and cores are considered in this paper to evaluate the performance. The work also signifies the importance of high-performance computing capability for smart meters big data management. The obtained results indicate that the computational time is not only dependent on the data size but also on the number of compute nodes and the number of cores assigned to run the job.

Original languageEnglish
Title of host publicationProceedings of the 30th International Conference on Cybernetics and Informatics, K and I 2020
EditorsJan Ciganek, Stefan Kozak, Alena Kozakova
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728143811
DOIs
Publication statusPublished - Jan 2020
Event30th International Conference on Cybernetics and Informatics, K and I 2020 - Velke Karlovice, Czech Republic
Duration: 29 Jan 20201 Feb 2020

Publication series

NameProceedings of the 30th International Conference on Cybernetics and Informatics, K and I 2020

Conference

Conference30th International Conference on Cybernetics and Informatics, K and I 2020
CountryCzech Republic
CityVelke Karlovice
Period29/1/201/2/20

    Fingerprint

Keywords

  • Apache-Spark
  • Big data
  • High performance computing
  • Load forecast
  • Parallel computing
  • Smart grid

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems
  • Mechanical Engineering
  • Control and Optimization

Cite this

Zainab, A., Refaat, S. S., Abu-Rub, H., & Bouhali, O. (2020). Distributed computing for smart meter data management for electrical utility applications. In J. Ciganek, S. Kozak, & A. Kozakova (Eds.), Proceedings of the 30th International Conference on Cybernetics and Informatics, K and I 2020 [9039899] (Proceedings of the 30th International Conference on Cybernetics and Informatics, K and I 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/KI48306.2020.9039899