Datacenters as computers

Google engineering & database research perspectives

Shivakumar Venkataraman, Divyakant Agrawal

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

In this collaborative keynote address, we will share Google's experience in building a scalable data infrastructure that leverages datacenters for managing Google's advertising data over the last decade. In order to support the massive online advertising platform at Google, the data infrastructure must simultaneously support both transactional and analytical workloads. The focus of this talk will be to highlight how the datacenter architecture and the cloud computing paradigm has enabled us to manage the exponential growth in data volumes and user queries, make our services highly available and fault tolerant to massive datacenter outages, and deliver results with very low latencies. We note that other Internet companies have also undergone similar growth in data volumes and user queries. In fact, this phenomenon has resulted in at least two new terms in the technology lexicon: big data and cloud computing. Cloud computing (and datacenters) have been largely responsible for scaling the data volumes from terabytes range just a few years ago to now reaching in the exabyte range over the next couple of years. Delivering solutions at this scale that are fault-tolerant, latency sensitive, and highly available requires a combination of research advances with engineering ingenuity at Google and elsewhere. Next, we will try to answer the following question: is a datacenter just another (very large) computer? Or, does it fundamentally change the design principles for data-centric applications and systems. We will conclude with some of the unique research challenges that need to be addressed in order to sustain continuous growth in data volumes while supporting high throughput and low latencies.

Original languageEnglish
Title of host publicationProceedings of the VLDB Endowment
PublisherAssociation for Computing Machinery
Pages1720-1721
Number of pages2
Volume7
Edition13
Publication statusPublished - 2014
Externally publishedYes

Fingerprint

Cloud computing
Marketing
Outages
Throughput
Internet
Industry

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Venkataraman, S., & Agrawal, D. (2014). Datacenters as computers: Google engineering & database research perspectives. In Proceedings of the VLDB Endowment (13 ed., Vol. 7, pp. 1720-1721). Association for Computing Machinery.

Datacenters as computers : Google engineering & database research perspectives. / Venkataraman, Shivakumar; Agrawal, Divyakant.

Proceedings of the VLDB Endowment. Vol. 7 13. ed. Association for Computing Machinery, 2014. p. 1720-1721.

Research output: Chapter in Book/Report/Conference proceedingChapter

Venkataraman, S & Agrawal, D 2014, Datacenters as computers: Google engineering & database research perspectives. in Proceedings of the VLDB Endowment. 13 edn, vol. 7, Association for Computing Machinery, pp. 1720-1721.
Venkataraman S, Agrawal D. Datacenters as computers: Google engineering & database research perspectives. In Proceedings of the VLDB Endowment. 13 ed. Vol. 7. Association for Computing Machinery. 2014. p. 1720-1721
Venkataraman, Shivakumar ; Agrawal, Divyakant. / Datacenters as computers : Google engineering & database research perspectives. Proceedings of the VLDB Endowment. Vol. 7 13. ed. Association for Computing Machinery, 2014. pp. 1720-1721
@inbook{6da4652d46f149f68fa8f520d8de11e2,
title = "Datacenters as computers: Google engineering & database research perspectives",
abstract = "In this collaborative keynote address, we will share Google's experience in building a scalable data infrastructure that leverages datacenters for managing Google's advertising data over the last decade. In order to support the massive online advertising platform at Google, the data infrastructure must simultaneously support both transactional and analytical workloads. The focus of this talk will be to highlight how the datacenter architecture and the cloud computing paradigm has enabled us to manage the exponential growth in data volumes and user queries, make our services highly available and fault tolerant to massive datacenter outages, and deliver results with very low latencies. We note that other Internet companies have also undergone similar growth in data volumes and user queries. In fact, this phenomenon has resulted in at least two new terms in the technology lexicon: big data and cloud computing. Cloud computing (and datacenters) have been largely responsible for scaling the data volumes from terabytes range just a few years ago to now reaching in the exabyte range over the next couple of years. Delivering solutions at this scale that are fault-tolerant, latency sensitive, and highly available requires a combination of research advances with engineering ingenuity at Google and elsewhere. Next, we will try to answer the following question: is a datacenter just another (very large) computer? Or, does it fundamentally change the design principles for data-centric applications and systems. We will conclude with some of the unique research challenges that need to be addressed in order to sustain continuous growth in data volumes while supporting high throughput and low latencies.",
author = "Shivakumar Venkataraman and Divyakant Agrawal",
year = "2014",
language = "English",
volume = "7",
pages = "1720--1721",
booktitle = "Proceedings of the VLDB Endowment",
publisher = "Association for Computing Machinery",
edition = "13",

}

TY - CHAP

T1 - Datacenters as computers

T2 - Google engineering & database research perspectives

AU - Venkataraman, Shivakumar

AU - Agrawal, Divyakant

PY - 2014

Y1 - 2014

N2 - In this collaborative keynote address, we will share Google's experience in building a scalable data infrastructure that leverages datacenters for managing Google's advertising data over the last decade. In order to support the massive online advertising platform at Google, the data infrastructure must simultaneously support both transactional and analytical workloads. The focus of this talk will be to highlight how the datacenter architecture and the cloud computing paradigm has enabled us to manage the exponential growth in data volumes and user queries, make our services highly available and fault tolerant to massive datacenter outages, and deliver results with very low latencies. We note that other Internet companies have also undergone similar growth in data volumes and user queries. In fact, this phenomenon has resulted in at least two new terms in the technology lexicon: big data and cloud computing. Cloud computing (and datacenters) have been largely responsible for scaling the data volumes from terabytes range just a few years ago to now reaching in the exabyte range over the next couple of years. Delivering solutions at this scale that are fault-tolerant, latency sensitive, and highly available requires a combination of research advances with engineering ingenuity at Google and elsewhere. Next, we will try to answer the following question: is a datacenter just another (very large) computer? Or, does it fundamentally change the design principles for data-centric applications and systems. We will conclude with some of the unique research challenges that need to be addressed in order to sustain continuous growth in data volumes while supporting high throughput and low latencies.

AB - In this collaborative keynote address, we will share Google's experience in building a scalable data infrastructure that leverages datacenters for managing Google's advertising data over the last decade. In order to support the massive online advertising platform at Google, the data infrastructure must simultaneously support both transactional and analytical workloads. The focus of this talk will be to highlight how the datacenter architecture and the cloud computing paradigm has enabled us to manage the exponential growth in data volumes and user queries, make our services highly available and fault tolerant to massive datacenter outages, and deliver results with very low latencies. We note that other Internet companies have also undergone similar growth in data volumes and user queries. In fact, this phenomenon has resulted in at least two new terms in the technology lexicon: big data and cloud computing. Cloud computing (and datacenters) have been largely responsible for scaling the data volumes from terabytes range just a few years ago to now reaching in the exabyte range over the next couple of years. Delivering solutions at this scale that are fault-tolerant, latency sensitive, and highly available requires a combination of research advances with engineering ingenuity at Google and elsewhere. Next, we will try to answer the following question: is a datacenter just another (very large) computer? Or, does it fundamentally change the design principles for data-centric applications and systems. We will conclude with some of the unique research challenges that need to be addressed in order to sustain continuous growth in data volumes while supporting high throughput and low latencies.

UR - http://www.scopus.com/inward/record.url?scp=84905841354&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84905841354&partnerID=8YFLogxK

M3 - Chapter

VL - 7

SP - 1720

EP - 1721

BT - Proceedings of the VLDB Endowment

PB - Association for Computing Machinery

ER -