CoMPI: Enhancing MPI based applications performance and scalability using run-time compression

Rosa Filgueira; David E. Singh; Alejandro Calderón; Jesús Carretero

doi:10.1007/978-3-642-03770-2_27

CoMPI: Enhancing MPI based applications performance and scalability using run-time compression

Rosa Filgueira^*, David E. Singh, Alejandro Calderón, Jesús Carretero

^*Corresponding author for this work

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper presents an optimization of MPI communications, called CoMPI, based on run-time compression of MPI messages exchanged by applications. A broad number of compression algorithms have been fully implemented and tested for both MPI collective and point to point primitives. In addition, this paper presents a study of several compression algorithms that can be used for run-time compression, based on the datatype used by applications. This study has been validated by using several MPI benchmarks and real HPC applications. Show that, in most of the cases, using compression reduces the application communication time enhancing application performance and scalability. In this way, CoMPI obtains important improvements in the overall execution time for many of the considered scenarios.

Original language	English
Title of host publication	Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings
Publisher	Springer-Verlag
Pages	207-218
Number of pages	12
ISBN (Print)	3642037690, 9783642037696
DOIs	https://doi.org/10.1007/978-3-642-03770-2_27
Publication status	Published - 2009
Event	16th European Parallel Virtual Machine and Message Passing Interface Users' Group Meeting, EuroPVM/MPI - Espoo, Finland Duration: 7 Sept 2009 → 10 Sept 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5759 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th European Parallel Virtual Machine and Message Passing Interface Users' Group Meeting, EuroPVM/MPI
Country/Territory	Finland
City	Espoo
Period	7/09/09 → 10/09/09

Access to Document

10.1007/978-3-642-03770-2_27

Cite this

Filgueira, R., Singh, D. E., Calderón, A., & Carretero, J. (2009). CoMPI: Enhancing MPI based applications performance and scalability using run-time compression. In Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings (pp. 207-218). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5759 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-642-03770-2_27

Filgueira, Rosa ; Singh, David E. ; Calderón, Alejandro et al. / CoMPI : Enhancing MPI based applications performance and scalability using run-time compression. Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings. Springer-Verlag, 2009. pp. 207-218 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{304b14a306844a30b29c815f8f2ba634,

title = "CoMPI: Enhancing MPI based applications performance and scalability using run-time compression",

abstract = "This paper presents an optimization of MPI communications, called CoMPI, based on run-time compression of MPI messages exchanged by applications. A broad number of compression algorithms have been fully implemented and tested for both MPI collective and point to point primitives. In addition, this paper presents a study of several compression algorithms that can be used for run-time compression, based on the datatype used by applications. This study has been validated by using several MPI benchmarks and real HPC applications. Show that, in most of the cases, using compression reduces the application communication time enhancing application performance and scalability. In this way, CoMPI obtains important improvements in the overall execution time for many of the considered scenarios.",

author = "Rosa Filgueira and Singh, {David E.} and Alejandro Calder{\'o}n and Jes{\'u}s Carretero",

year = "2009",

doi = "10.1007/978-3-642-03770-2_27",

language = "English",

isbn = "3642037690",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "207--218",

booktitle = "Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings",

address = "Germany",

note = "16th European Parallel Virtual Machine and Message Passing Interface Users' Group Meeting, EuroPVM/MPI ; Conference date: 07-09-2009 Through 10-09-2009",

}

Filgueira, R, Singh, DE, Calderón, A & Carretero, J 2009, CoMPI: Enhancing MPI based applications performance and scalability using run-time compression. in Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5759 LNCS, Springer-Verlag, pp. 207-218, 16th European Parallel Virtual Machine and Message Passing Interface Users' Group Meeting, EuroPVM/MPI, Espoo, Finland, 7/09/09. https://doi.org/10.1007/978-3-642-03770-2_27

CoMPI: Enhancing MPI based applications performance and scalability using run-time compression. / Filgueira, Rosa; Singh, David E.; Calderón, Alejandro et al.
Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings. Springer-Verlag, 2009. p. 207-218 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5759 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - CoMPI

T2 - 16th European Parallel Virtual Machine and Message Passing Interface Users' Group Meeting, EuroPVM/MPI

AU - Filgueira, Rosa

AU - Singh, David E.

AU - Calderón, Alejandro

AU - Carretero, Jesús

PY - 2009

Y1 - 2009

N2 - This paper presents an optimization of MPI communications, called CoMPI, based on run-time compression of MPI messages exchanged by applications. A broad number of compression algorithms have been fully implemented and tested for both MPI collective and point to point primitives. In addition, this paper presents a study of several compression algorithms that can be used for run-time compression, based on the datatype used by applications. This study has been validated by using several MPI benchmarks and real HPC applications. Show that, in most of the cases, using compression reduces the application communication time enhancing application performance and scalability. In this way, CoMPI obtains important improvements in the overall execution time for many of the considered scenarios.

AB - This paper presents an optimization of MPI communications, called CoMPI, based on run-time compression of MPI messages exchanged by applications. A broad number of compression algorithms have been fully implemented and tested for both MPI collective and point to point primitives. In addition, this paper presents a study of several compression algorithms that can be used for run-time compression, based on the datatype used by applications. This study has been validated by using several MPI benchmarks and real HPC applications. Show that, in most of the cases, using compression reduces the application communication time enhancing application performance and scalability. In this way, CoMPI obtains important improvements in the overall execution time for many of the considered scenarios.

UR - http://www.scopus.com/inward/record.url?scp=70350465380&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-03770-2_27

DO - 10.1007/978-3-642-03770-2_27

M3 - Conference contribution

AN - SCOPUS:70350465380

SN - 3642037690

SN - 9783642037696

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 207

EP - 218

BT - Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings

PB - Springer-Verlag

Y2 - 7 September 2009 through 10 September 2009

ER -

Filgueira R, Singh DE, Calderón A, Carretero J. CoMPI: Enhancing MPI based applications performance and scalability using run-time compression. In Recent Advances in Parallel Virtual Machine and Message Passing Interface - 16th European PVM/MPI Users' Group Meeting, Proceedings. Springer-Verlag. 2009. p. 207-218. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-03770-2_27

CoMPI: Enhancing MPI based applications performance and scalability using run-time compression

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this