FreeLoader

Scavenging desktop storage resources for scientific data

Sudharshan S. Vazhkudai, Xiaosong Ma, Vincent W. Frech, Jonathan W. Strickland, Nandan Tammineedi, Stephen L. Scott

Research output: Chapter in Book/Report/Conference proceedingConference contribution

41 Citations (Scopus)

Abstract

High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.

Original languageEnglish
Title of host publicationProceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05
Volume2005
DOIs
Publication statusPublished - 1 Dec 2005
Externally publishedYes
EventACM/IEEE 2005 Supercomputing Conference, SC'05 - Seatle, WA, United States
Duration: 12 Nov 200518 Nov 2005

Other

OtherACM/IEEE 2005 Supercomputing Conference, SC'05
CountryUnited States
CitySeatle, WA
Period12/11/0518/11/05

Fingerprint

Scavenging
Bandwidth
Telecommunication networks
Experiments
Processing
Costs

Keywords

  • Distributed storage
  • Parallel I/O
  • Scientific data management
  • Serverless storage system
  • Storage cache
  • Storage scavenging
  • Striped storage

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Vazhkudai, S. S., Ma, X., Frech, V. W., Strickland, J. W., Tammineedi, N., & Scott, S. L. (2005). FreeLoader: Scavenging desktop storage resources for scientific data. In Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05 (Vol. 2005). [1560008] https://doi.org/10.1109/SC.2005.27

FreeLoader : Scavenging desktop storage resources for scientific data. / Vazhkudai, Sudharshan S.; Ma, Xiaosong; Frech, Vincent W.; Strickland, Jonathan W.; Tammineedi, Nandan; Scott, Stephen L.

Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05. Vol. 2005 2005. 1560008.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Vazhkudai, SS, Ma, X, Frech, VW, Strickland, JW, Tammineedi, N & Scott, SL 2005, FreeLoader: Scavenging desktop storage resources for scientific data. in Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05. vol. 2005, 1560008, ACM/IEEE 2005 Supercomputing Conference, SC'05, Seatle, WA, United States, 12/11/05. https://doi.org/10.1109/SC.2005.27
Vazhkudai SS, Ma X, Frech VW, Strickland JW, Tammineedi N, Scott SL. FreeLoader: Scavenging desktop storage resources for scientific data. In Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05. Vol. 2005. 2005. 1560008 https://doi.org/10.1109/SC.2005.27
Vazhkudai, Sudharshan S. ; Ma, Xiaosong ; Frech, Vincent W. ; Strickland, Jonathan W. ; Tammineedi, Nandan ; Scott, Stephen L. / FreeLoader : Scavenging desktop storage resources for scientific data. Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05. Vol. 2005 2005.
@inproceedings{7f21c53ea2894cb5bff21a818b5942e2,
title = "FreeLoader: Scavenging desktop storage resources for scientific data",
abstract = "High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.",
keywords = "Distributed storage, Parallel I/O, Scientific data management, Serverless storage system, Storage cache, Storage scavenging, Striped storage",
author = "Vazhkudai, {Sudharshan S.} and Xiaosong Ma and Frech, {Vincent W.} and Strickland, {Jonathan W.} and Nandan Tammineedi and Scott, {Stephen L.}",
year = "2005",
month = "12",
day = "1",
doi = "10.1109/SC.2005.27",
language = "English",
isbn = "1595930612",
volume = "2005",
booktitle = "Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05",

}

TY - GEN

T1 - FreeLoader

T2 - Scavenging desktop storage resources for scientific data

AU - Vazhkudai, Sudharshan S.

AU - Ma, Xiaosong

AU - Frech, Vincent W.

AU - Strickland, Jonathan W.

AU - Tammineedi, Nandan

AU - Scott, Stephen L.

PY - 2005/12/1

Y1 - 2005/12/1

N2 - High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.

AB - High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.

KW - Distributed storage

KW - Parallel I/O

KW - Scientific data management

KW - Serverless storage system

KW - Storage cache

KW - Storage scavenging

KW - Striped storage

UR - http://www.scopus.com/inward/record.url?scp=33845399711&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33845399711&partnerID=8YFLogxK

U2 - 10.1109/SC.2005.27

DO - 10.1109/SC.2005.27

M3 - Conference contribution

SN - 1595930612

SN - 9781595930613

VL - 2005

BT - Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05

ER -