Container-based Cloud Virtual Machine benchmarking

Blesson Varghese, Lawan Thamsuhang Subba, Long Thanh Thai, Adam David Barker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Citations (Scopus)


With the availability of a wide range of cloud Virtual Machines (VMs) it is difficult to determine which VMs can maximise the performance of an application. Benchmarking is commonly used to this end for capturing the performance of VMs. Most cloud benchmarking techniques are typically heavyweight - time consuming processes which have to benchmark the entire VM in order to obtain accurate benchmark data. Such benchmarks cannot be used in real-time on the cloud and incur extra costs even before an application is deployed. In this paper, we present lightweight cloud benchmarking techniques that execute quickly and can be used in near real-time on the cloud. The exploration of lightweight benchmarking techniques are facilitated by the development of DocLite - Docker Container-based Lightweight Benchmarking. DocLite is built on the Docker container technology which allows a user-definedportion (such as memory size and the number of CPU cores) of the VM to be benchmarked. DocLite operates in two modes, in the first mode, containers are used to benchmark a small portion of the VM to generate performance ranks. In the second mode, historic benchmark data is used along with the first modeas a hybrid to generate VM ranks. The generated ranks are evaluated against three scientific high-performance computing applications. The proposed techniques are up to 91 times faster than a heavyweight technique which benchmarks the entire VM. It is observed that the first mode can generate ranks with over 90% and 86% accuracy for sequential and parallel execution of an application. The hybrid mode improves the correlation slightly but the first mode is sufficient for benchmarking cloud VMs.
Original languageEnglish
Title of host publication2016 IEEE International Conference on Cloud Engineering (IC2E)
PublisherIEEE Computer Society
ISBN (Electronic)9781509019618
Publication statusPublished - 4 Apr 2016
EventIEEE International Conference on Cloud Engineering (IC2E) 2016 - Berlin, Germany
Duration: 4 Apr 20168 Apr 2016


ConferenceIEEE International Conference on Cloud Engineering (IC2E) 2016


  • Benchmarking
  • Docker
  • Containers
  • Lightweight benchmark
  • Hybrid benchmark


Dive into the research topics of 'Container-based Cloud Virtual Machine benchmarking'. Together they form a unique fingerprint.

Cite this