Borg: the next generation

Muhammad Tirmazi, Adam Barker, Nan Deng, Md E. Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Balter, John Wilkes

Research output: Chapter in Book/Report/Conference proceedingConference contribution

168 Citations (Scopus)
7 Downloads (Pure)

Abstract

This paper analyzes a newly-published trace that covers 8 different Borg [35] clusters for the month of May 2019. The trace enables researchers to explore how scheduling works in large-scale production compute clusters. We highlight how Borg has evolved and perform a longitudinal comparison of the newly-published 2019 trace against the 2011 trace, which has been highly cited within the research community.

Our findings show that Borg features such as alloc sets are used for resource-heavy workloads; automatic vertical scaling is effective; job-dependencies account for much of the high failure rates reported by prior studies; the workload arrival rate has increased, as has the use of resource over-commitment; the workload mix has changed, jobs have migrated from the free tier into the best-effort batch tier; the workload exhibits an extremely heavy-tailed distribution where the top 1% of jobs consume over 99% of resources; and there is a great deal of variation between different clusters.
Original languageEnglish
Title of host publicationProceedings of the Fifteenth European Conference on Computer Systems (EuroSys '20)
Place of PublicationNew York
PublisherACM
Pages1-14
ISBN (Print)9781450368827
DOIs
Publication statusPublished - 15 Apr 2020
EventFifteenth European Conference on Computer Systems (EuroSys ’20) - Heraklion, Greece
Duration: 27 Apr 202030 Apr 2020
Conference number: 15
https://www.eurosys2020.org/

Conference

ConferenceFifteenth European Conference on Computer Systems (EuroSys ’20)
Abbreviated titleEuroSys '20
Country/TerritoryGreece
CityHeraklion
Period27/04/2030/04/20
Internet address

Keywords

  • Data centers
  • Cloud computing

Fingerprint

Dive into the research topics of 'Borg: the next generation'. Together they form a unique fingerprint.

Cite this