On blind mice and the elephant

Understanding the network impact of a large distributed system

John S. Otto, Mario A. Sánchez, David R. Choffnes, Fabián E. Bustamante, Georgos Siganos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

48 Citations (Scopus)

Abstract

A thorough understanding of the network impact of emerging large-scale distributed systems - where traffic flows and what it costs -must encompass users' behavior, the traffic they generate and the topology over which that traffic flows. In the case of BitTorrent, however, previous studies have been limited by narrow perspectives that restrict such analysis. This paper presents a comprehensive view of BitTorrent, using data from a representative set of 500,000 users sampled over a two year period, located in 169 countries and 3,150 networks. This unique perspective captures unseen trends and reveals several unexpected features of the largest peer-to-peer system. For instance, over the past year total BitTorrent traffic has increased by 12%, driven by 25% increases in per-peer hourly download volume despite a 10% decrease in the average number of online peers. We also observe stronger diurnal usage patterns and, surprisingly given the bandwidth-intensive nature of the application, a close alignment between these patterns and overall traffic. Considering the aggregated traffic across access links, this has potential implications on BitTorrent-associated costs for Internet Service Providers (ISPs). Using data from a transit ISP, we find a disproportionately large impact under a commonly used burstable (95th-percentile) billing model. Last, when examining BitTorrent traffic's paths, we find that for over half its users, most network traffic never reaches large transit networks, but is instead carried by small transit ISPs. This raises questions on the effectiveness of most in-network monitoring systems to capture trends on peer-to-peer traffic and further motivates our approach.

Original languageEnglish
Title of host publicationProceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11
Pages110-121
Number of pages12
DOIs
Publication statusPublished - 29 Sep 2011
Externally publishedYes
EventACM SIGCOMM 2011 Conference, SIGCOMM'11 - Toronto, ON, Canada
Duration: 15 Aug 201119 Aug 2011

Other

OtherACM SIGCOMM 2011 Conference, SIGCOMM'11
CountryCanada
CityToronto, ON
Period15/8/1119/8/11

Fingerprint

Internet service providers
Costs
Topology
Bandwidth
Monitoring

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Cite this

Otto, J. S., Sánchez, M. A., Choffnes, D. R., Bustamante, F. E., & Siganos, G. (2011). On blind mice and the elephant: Understanding the network impact of a large distributed system. In Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11 (pp. 110-121) https://doi.org/10.1145/2018436.2018450

On blind mice and the elephant : Understanding the network impact of a large distributed system. / Otto, John S.; Sánchez, Mario A.; Choffnes, David R.; Bustamante, Fabián E.; Siganos, Georgos.

Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11. 2011. p. 110-121.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Otto, JS, Sánchez, MA, Choffnes, DR, Bustamante, FE & Siganos, G 2011, On blind mice and the elephant: Understanding the network impact of a large distributed system. in Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11. pp. 110-121, ACM SIGCOMM 2011 Conference, SIGCOMM'11, Toronto, ON, Canada, 15/8/11. https://doi.org/10.1145/2018436.2018450
Otto JS, Sánchez MA, Choffnes DR, Bustamante FE, Siganos G. On blind mice and the elephant: Understanding the network impact of a large distributed system. In Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11. 2011. p. 110-121 https://doi.org/10.1145/2018436.2018450
Otto, John S. ; Sánchez, Mario A. ; Choffnes, David R. ; Bustamante, Fabián E. ; Siganos, Georgos. / On blind mice and the elephant : Understanding the network impact of a large distributed system. Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11. 2011. pp. 110-121
@inproceedings{cf317798833342158c05dba3cba739d2,
title = "On blind mice and the elephant: Understanding the network impact of a large distributed system",
abstract = "A thorough understanding of the network impact of emerging large-scale distributed systems - where traffic flows and what it costs -must encompass users' behavior, the traffic they generate and the topology over which that traffic flows. In the case of BitTorrent, however, previous studies have been limited by narrow perspectives that restrict such analysis. This paper presents a comprehensive view of BitTorrent, using data from a representative set of 500,000 users sampled over a two year period, located in 169 countries and 3,150 networks. This unique perspective captures unseen trends and reveals several unexpected features of the largest peer-to-peer system. For instance, over the past year total BitTorrent traffic has increased by 12{\%}, driven by 25{\%} increases in per-peer hourly download volume despite a 10{\%} decrease in the average number of online peers. We also observe stronger diurnal usage patterns and, surprisingly given the bandwidth-intensive nature of the application, a close alignment between these patterns and overall traffic. Considering the aggregated traffic across access links, this has potential implications on BitTorrent-associated costs for Internet Service Providers (ISPs). Using data from a transit ISP, we find a disproportionately large impact under a commonly used burstable (95th-percentile) billing model. Last, when examining BitTorrent traffic's paths, we find that for over half its users, most network traffic never reaches large transit networks, but is instead carried by small transit ISPs. This raises questions on the effectiveness of most in-network monitoring systems to capture trends on peer-to-peer traffic and further motivates our approach.",
author = "Otto, {John S.} and S{\'a}nchez, {Mario A.} and Choffnes, {David R.} and Bustamante, {Fabi{\'a}n E.} and Georgos Siganos",
year = "2011",
month = "9",
day = "29",
doi = "10.1145/2018436.2018450",
language = "English",
isbn = "9781450307970",
pages = "110--121",
booktitle = "Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11",

}

TY - GEN

T1 - On blind mice and the elephant

T2 - Understanding the network impact of a large distributed system

AU - Otto, John S.

AU - Sánchez, Mario A.

AU - Choffnes, David R.

AU - Bustamante, Fabián E.

AU - Siganos, Georgos

PY - 2011/9/29

Y1 - 2011/9/29

N2 - A thorough understanding of the network impact of emerging large-scale distributed systems - where traffic flows and what it costs -must encompass users' behavior, the traffic they generate and the topology over which that traffic flows. In the case of BitTorrent, however, previous studies have been limited by narrow perspectives that restrict such analysis. This paper presents a comprehensive view of BitTorrent, using data from a representative set of 500,000 users sampled over a two year period, located in 169 countries and 3,150 networks. This unique perspective captures unseen trends and reveals several unexpected features of the largest peer-to-peer system. For instance, over the past year total BitTorrent traffic has increased by 12%, driven by 25% increases in per-peer hourly download volume despite a 10% decrease in the average number of online peers. We also observe stronger diurnal usage patterns and, surprisingly given the bandwidth-intensive nature of the application, a close alignment between these patterns and overall traffic. Considering the aggregated traffic across access links, this has potential implications on BitTorrent-associated costs for Internet Service Providers (ISPs). Using data from a transit ISP, we find a disproportionately large impact under a commonly used burstable (95th-percentile) billing model. Last, when examining BitTorrent traffic's paths, we find that for over half its users, most network traffic never reaches large transit networks, but is instead carried by small transit ISPs. This raises questions on the effectiveness of most in-network monitoring systems to capture trends on peer-to-peer traffic and further motivates our approach.

AB - A thorough understanding of the network impact of emerging large-scale distributed systems - where traffic flows and what it costs -must encompass users' behavior, the traffic they generate and the topology over which that traffic flows. In the case of BitTorrent, however, previous studies have been limited by narrow perspectives that restrict such analysis. This paper presents a comprehensive view of BitTorrent, using data from a representative set of 500,000 users sampled over a two year period, located in 169 countries and 3,150 networks. This unique perspective captures unseen trends and reveals several unexpected features of the largest peer-to-peer system. For instance, over the past year total BitTorrent traffic has increased by 12%, driven by 25% increases in per-peer hourly download volume despite a 10% decrease in the average number of online peers. We also observe stronger diurnal usage patterns and, surprisingly given the bandwidth-intensive nature of the application, a close alignment between these patterns and overall traffic. Considering the aggregated traffic across access links, this has potential implications on BitTorrent-associated costs for Internet Service Providers (ISPs). Using data from a transit ISP, we find a disproportionately large impact under a commonly used burstable (95th-percentile) billing model. Last, when examining BitTorrent traffic's paths, we find that for over half its users, most network traffic never reaches large transit networks, but is instead carried by small transit ISPs. This raises questions on the effectiveness of most in-network monitoring systems to capture trends on peer-to-peer traffic and further motivates our approach.

UR - http://www.scopus.com/inward/record.url?scp=80053148796&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053148796&partnerID=8YFLogxK

U2 - 10.1145/2018436.2018450

DO - 10.1145/2018436.2018450

M3 - Conference contribution

SN - 9781450307970

SP - 110

EP - 121

BT - Proceedings of the ACM SIGCOMM 2011 Conference, SIGCOMM'11

ER -