Scalable algorithms for signal reconstruction by leveraging similarity joins

Abolfazl Asudeh, Jees Augustine, Azade Nazi, Saravanan Thirumuruganathan, Nan Zhang, Gautam Das, Divesh Srivastava

Research output: Contribution to journalArticle

Abstract

Signal reconstruction problem (SRP) is an important optimization problem where the objective is to identify a solution to an underdetermined system of linear equations that is closest to a given prior. It has a substantial number of applications in diverse areas including network traffic engineering, medical image reconstruction, acoustics, astronomy and many more. Most common approaches for SRP do not scale to large problem sizes. In this paper, we propose multiple optimization steps, developing scalable algorithms for the problem. We first propose a dual formulation of the problem and develop the Direct algorithm that is significantly more efficient than the state of the art. Second, we show how adapting database techniques developed for scalable similarity joins provides a significant speedup over Direct, scaling our proposal up to large-scale settings. Third, we describe a number of practical techniques that allow our algorithm to scale to settings of size in the order of a million by a billion. We also adapt our proposal to identify the top-k components of the solved system of linear equations. Finally, we consider the dynamic setting where the inputs to the linear system change and propose efficient algorithms inspired by the database techniques of materialization and reuse. Extensive experiments on real-world and synthetic data confirm the efficiency, effectiveness and scalability of our proposal.

Original languageEnglish
JournalVLDB Journal
DOIs
Publication statusPublished - 1 Jan 2019

Fingerprint

Signal reconstruction
Linear equations
Astronomy
Image reconstruction
Linear systems
Scalability
Acoustics
Experiments

Keywords

  • Scalable algorithm
  • Signal reconstruction
  • Traffic reconstruction
  • Underdetermined systems

ASJC Scopus subject areas

  • Information Systems
  • Hardware and Architecture

Cite this

Scalable algorithms for signal reconstruction by leveraging similarity joins. / Asudeh, Abolfazl; Augustine, Jees; Nazi, Azade; Thirumuruganathan, Saravanan; Zhang, Nan; Das, Gautam; Srivastava, Divesh.

In: VLDB Journal, 01.01.2019.

Research output: Contribution to journalArticle

Asudeh, Abolfazl ; Augustine, Jees ; Nazi, Azade ; Thirumuruganathan, Saravanan ; Zhang, Nan ; Das, Gautam ; Srivastava, Divesh. / Scalable algorithms for signal reconstruction by leveraging similarity joins. In: VLDB Journal. 2019.
@article{d97d2b72e6774ea2801d7d27b5c0a8b9,
title = "Scalable algorithms for signal reconstruction by leveraging similarity joins",
abstract = "Signal reconstruction problem (SRP) is an important optimization problem where the objective is to identify a solution to an underdetermined system of linear equations that is closest to a given prior. It has a substantial number of applications in diverse areas including network traffic engineering, medical image reconstruction, acoustics, astronomy and many more. Most common approaches for SRP do not scale to large problem sizes. In this paper, we propose multiple optimization steps, developing scalable algorithms for the problem. We first propose a dual formulation of the problem and develop the Direct algorithm that is significantly more efficient than the state of the art. Second, we show how adapting database techniques developed for scalable similarity joins provides a significant speedup over Direct, scaling our proposal up to large-scale settings. Third, we describe a number of practical techniques that allow our algorithm to scale to settings of size in the order of a million by a billion. We also adapt our proposal to identify the top-k components of the solved system of linear equations. Finally, we consider the dynamic setting where the inputs to the linear system change and propose efficient algorithms inspired by the database techniques of materialization and reuse. Extensive experiments on real-world and synthetic data confirm the efficiency, effectiveness and scalability of our proposal.",
keywords = "Scalable algorithm, Signal reconstruction, Traffic reconstruction, Underdetermined systems",
author = "Abolfazl Asudeh and Jees Augustine and Azade Nazi and Saravanan Thirumuruganathan and Nan Zhang and Gautam Das and Divesh Srivastava",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s00778-019-00562-z",
language = "English",
journal = "VLDB Journal",
issn = "1066-8888",
publisher = "Springer New York",

}

TY - JOUR

T1 - Scalable algorithms for signal reconstruction by leveraging similarity joins

AU - Asudeh, Abolfazl

AU - Augustine, Jees

AU - Nazi, Azade

AU - Thirumuruganathan, Saravanan

AU - Zhang, Nan

AU - Das, Gautam

AU - Srivastava, Divesh

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Signal reconstruction problem (SRP) is an important optimization problem where the objective is to identify a solution to an underdetermined system of linear equations that is closest to a given prior. It has a substantial number of applications in diverse areas including network traffic engineering, medical image reconstruction, acoustics, astronomy and many more. Most common approaches for SRP do not scale to large problem sizes. In this paper, we propose multiple optimization steps, developing scalable algorithms for the problem. We first propose a dual formulation of the problem and develop the Direct algorithm that is significantly more efficient than the state of the art. Second, we show how adapting database techniques developed for scalable similarity joins provides a significant speedup over Direct, scaling our proposal up to large-scale settings. Third, we describe a number of practical techniques that allow our algorithm to scale to settings of size in the order of a million by a billion. We also adapt our proposal to identify the top-k components of the solved system of linear equations. Finally, we consider the dynamic setting where the inputs to the linear system change and propose efficient algorithms inspired by the database techniques of materialization and reuse. Extensive experiments on real-world and synthetic data confirm the efficiency, effectiveness and scalability of our proposal.

AB - Signal reconstruction problem (SRP) is an important optimization problem where the objective is to identify a solution to an underdetermined system of linear equations that is closest to a given prior. It has a substantial number of applications in diverse areas including network traffic engineering, medical image reconstruction, acoustics, astronomy and many more. Most common approaches for SRP do not scale to large problem sizes. In this paper, we propose multiple optimization steps, developing scalable algorithms for the problem. We first propose a dual formulation of the problem and develop the Direct algorithm that is significantly more efficient than the state of the art. Second, we show how adapting database techniques developed for scalable similarity joins provides a significant speedup over Direct, scaling our proposal up to large-scale settings. Third, we describe a number of practical techniques that allow our algorithm to scale to settings of size in the order of a million by a billion. We also adapt our proposal to identify the top-k components of the solved system of linear equations. Finally, we consider the dynamic setting where the inputs to the linear system change and propose efficient algorithms inspired by the database techniques of materialization and reuse. Extensive experiments on real-world and synthetic data confirm the efficiency, effectiveness and scalability of our proposal.

KW - Scalable algorithm

KW - Signal reconstruction

KW - Traffic reconstruction

KW - Underdetermined systems

UR - http://www.scopus.com/inward/record.url?scp=85071050070&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071050070&partnerID=8YFLogxK

U2 - 10.1007/s00778-019-00562-z

DO - 10.1007/s00778-019-00562-z

M3 - Article

AN - SCOPUS:85071050070

JO - VLDB Journal

JF - VLDB Journal

SN - 1066-8888

ER -