Delay-tolerant bulk data transfers on the internet

Nikolaos Laoutaris, Georgios Smaragdakis, Rade Stanojevic, Pablo Rodriguez, Ravi Sundaram

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Many emerging scientific and industrial applications require transferring multiple terabytes of data on a daily basis. Examples include pushing scientific data from particle accelerators/colliders to laboratories around the world, synchronizing datacenters across continents, and replicating collections of high-definition videos from events taking place at different time-zones. A key property of all above applications is their ability to tolerate delivery delays ranging from a few hours to a few days. Such delay-tolerant bulk (DTB) data are currently being serviced mostly by the postal system using hard drives and DVDs, or by expensive dedicated networks. In this paper, we propose transmitting such data through commercial ISPs by taking advantage of already-paid-for off-peak bandwidth resulting from diurnal traffic patterns and percentile pricing. We show that between sender-receiver pairs with small time-zone difference, simple source scheduling policies are able to take advantage of most of the existing off-peak capacity. When the time-zone difference increases, taking advantage of the full capacity requires performing store-and-forward through intermediate storage nodes. We present an extensive evaluation of the two options based on traffic data from 200+ links of a large transit provider with points of presence (PoPs) at three continents. Our results indicate that there exists huge potential for performing multiterabyte transfers on a daily basis at little or no additional cost.

Original languageEnglish
Article number6423829
Pages (from-to)1852-1865
Number of pages14
JournalIEEE/ACM Transactions on Networking
Volume21
Issue number6
DOIs
Publication statusPublished - Dec 2013
Externally publishedYes

Fingerprint

Data transfer
Internet
Videodisks
Colliding beam accelerators
Telecommunication traffic
Industrial applications
Particle accelerators
Costs
Scheduling
Bandwidth

Keywords

  • Bandwidth pricing
  • bulk data transfers
  • content distribution
  • delay-tolerant networks (DTNs)

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Cite this

Laoutaris, N., Smaragdakis, G., Stanojevic, R., Rodriguez, P., & Sundaram, R. (2013). Delay-tolerant bulk data transfers on the internet. IEEE/ACM Transactions on Networking, 21(6), 1852-1865. [6423829]. https://doi.org/10.1109/TNET.2012.2237555

Delay-tolerant bulk data transfers on the internet. / Laoutaris, Nikolaos; Smaragdakis, Georgios; Stanojevic, Rade; Rodriguez, Pablo; Sundaram, Ravi.

In: IEEE/ACM Transactions on Networking, Vol. 21, No. 6, 6423829, 12.2013, p. 1852-1865.

Research output: Contribution to journalArticle

Laoutaris, N, Smaragdakis, G, Stanojevic, R, Rodriguez, P & Sundaram, R 2013, 'Delay-tolerant bulk data transfers on the internet', IEEE/ACM Transactions on Networking, vol. 21, no. 6, 6423829, pp. 1852-1865. https://doi.org/10.1109/TNET.2012.2237555
Laoutaris, Nikolaos ; Smaragdakis, Georgios ; Stanojevic, Rade ; Rodriguez, Pablo ; Sundaram, Ravi. / Delay-tolerant bulk data transfers on the internet. In: IEEE/ACM Transactions on Networking. 2013 ; Vol. 21, No. 6. pp. 1852-1865.
@article{e9fe5888e2f047a389bebf32bcf50580,
title = "Delay-tolerant bulk data transfers on the internet",
abstract = "Many emerging scientific and industrial applications require transferring multiple terabytes of data on a daily basis. Examples include pushing scientific data from particle accelerators/colliders to laboratories around the world, synchronizing datacenters across continents, and replicating collections of high-definition videos from events taking place at different time-zones. A key property of all above applications is their ability to tolerate delivery delays ranging from a few hours to a few days. Such delay-tolerant bulk (DTB) data are currently being serviced mostly by the postal system using hard drives and DVDs, or by expensive dedicated networks. In this paper, we propose transmitting such data through commercial ISPs by taking advantage of already-paid-for off-peak bandwidth resulting from diurnal traffic patterns and percentile pricing. We show that between sender-receiver pairs with small time-zone difference, simple source scheduling policies are able to take advantage of most of the existing off-peak capacity. When the time-zone difference increases, taking advantage of the full capacity requires performing store-and-forward through intermediate storage nodes. We present an extensive evaluation of the two options based on traffic data from 200+ links of a large transit provider with points of presence (PoPs) at three continents. Our results indicate that there exists huge potential for performing multiterabyte transfers on a daily basis at little or no additional cost.",
keywords = "Bandwidth pricing, bulk data transfers, content distribution, delay-tolerant networks (DTNs)",
author = "Nikolaos Laoutaris and Georgios Smaragdakis and Rade Stanojevic and Pablo Rodriguez and Ravi Sundaram",
year = "2013",
month = "12",
doi = "10.1109/TNET.2012.2237555",
language = "English",
volume = "21",
pages = "1852--1865",
journal = "IEEE/ACM Transactions on Networking",
issn = "1063-6692",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

TY - JOUR

T1 - Delay-tolerant bulk data transfers on the internet

AU - Laoutaris, Nikolaos

AU - Smaragdakis, Georgios

AU - Stanojevic, Rade

AU - Rodriguez, Pablo

AU - Sundaram, Ravi

PY - 2013/12

Y1 - 2013/12

N2 - Many emerging scientific and industrial applications require transferring multiple terabytes of data on a daily basis. Examples include pushing scientific data from particle accelerators/colliders to laboratories around the world, synchronizing datacenters across continents, and replicating collections of high-definition videos from events taking place at different time-zones. A key property of all above applications is their ability to tolerate delivery delays ranging from a few hours to a few days. Such delay-tolerant bulk (DTB) data are currently being serviced mostly by the postal system using hard drives and DVDs, or by expensive dedicated networks. In this paper, we propose transmitting such data through commercial ISPs by taking advantage of already-paid-for off-peak bandwidth resulting from diurnal traffic patterns and percentile pricing. We show that between sender-receiver pairs with small time-zone difference, simple source scheduling policies are able to take advantage of most of the existing off-peak capacity. When the time-zone difference increases, taking advantage of the full capacity requires performing store-and-forward through intermediate storage nodes. We present an extensive evaluation of the two options based on traffic data from 200+ links of a large transit provider with points of presence (PoPs) at three continents. Our results indicate that there exists huge potential for performing multiterabyte transfers on a daily basis at little or no additional cost.

AB - Many emerging scientific and industrial applications require transferring multiple terabytes of data on a daily basis. Examples include pushing scientific data from particle accelerators/colliders to laboratories around the world, synchronizing datacenters across continents, and replicating collections of high-definition videos from events taking place at different time-zones. A key property of all above applications is their ability to tolerate delivery delays ranging from a few hours to a few days. Such delay-tolerant bulk (DTB) data are currently being serviced mostly by the postal system using hard drives and DVDs, or by expensive dedicated networks. In this paper, we propose transmitting such data through commercial ISPs by taking advantage of already-paid-for off-peak bandwidth resulting from diurnal traffic patterns and percentile pricing. We show that between sender-receiver pairs with small time-zone difference, simple source scheduling policies are able to take advantage of most of the existing off-peak capacity. When the time-zone difference increases, taking advantage of the full capacity requires performing store-and-forward through intermediate storage nodes. We present an extensive evaluation of the two options based on traffic data from 200+ links of a large transit provider with points of presence (PoPs) at three continents. Our results indicate that there exists huge potential for performing multiterabyte transfers on a daily basis at little or no additional cost.

KW - Bandwidth pricing

KW - bulk data transfers

KW - content distribution

KW - delay-tolerant networks (DTNs)

UR - http://www.scopus.com/inward/record.url?scp=84891626326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891626326&partnerID=8YFLogxK

U2 - 10.1109/TNET.2012.2237555

DO - 10.1109/TNET.2012.2237555

M3 - Article

VL - 21

SP - 1852

EP - 1865

JO - IEEE/ACM Transactions on Networking

JF - IEEE/ACM Transactions on Networking

SN - 1063-6692

IS - 6

M1 - 6423829

ER -