Parallel mutation testing for large scale systems

Pablo Cerro Cañizares; Alberto Nunez; Rosa Filgueira; Juan de Lara

doi:10.1007/s10586-023-04074-y

Parallel mutation testing for large scale systems

Pablo Cerro Cañizares, Alberto Nunez, Rosa Filgueira, Juan de Lara

School of Computer Science

Research output: Contribution to journal › Article › peer-review

Abstract

Mutation testing is a valuable technique for measuring the quality of test suites in terms of detecting faults. However, one of its main drawbacks is its high computational cost. For this purpose, several approaches have been recently proposed to speed-up the mutation testing process by exploiting computational resources in distributed systems. However, bottlenecks have been detected when those techniques are applied in large-scale systems. This work improves the performance of mutation testing using large-scale systems by proposing a new load distribution algorithm, and parallelising different steps of the process. To demonstrate the benefits of our approach, we report on a thorough empirical evaluation, which analyses and compares our proposal with existing solutions executed in large-scale systems. The results show that our proposal outperforms the state-of-the-art distribution algorithms up to 35% in three different scenarios, reaching a reduction of the execution time of—at best—up to 99.66%.

Original language	English
Number of pages	27
Journal	Cluster Computing
Volume	First online
Early online date	20 Jun 2023
DOIs	https://doi.org/10.1007/s10586-023-04074-y
Publication status	E-pub ahead of print - 20 Jun 2023

Keywords

Mutation testing
Parallel mutation testing
Large scale systems
High performance computing
Distributed systems
Testing

Access to Document

10.1007/s10586-023-04074-yLicence: CC BY

Cañizares-2023-Parallel-mutation-testing-ClusterComputing-CCBY
Copyright © The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4.0/
Final published version, 2.43 MBLicence: CC BY

Cite this

@article{b77fff543a54499ebdf7c6e00e926557,

title = "Parallel mutation testing for large scale systems",

abstract = "Mutation testing is a valuable technique for measuring the quality of test suites in terms of detecting faults. However, one of its main drawbacks is its high computational cost. For this purpose, several approaches have been recently proposed to speed-up the mutation testing process by exploiting computational resources in distributed systems. However, bottlenecks have been detected when those techniques are applied in large-scale systems. This work improves the performance of mutation testing using large-scale systems by proposing a new load distribution algorithm, and parallelising different steps of the process. To demonstrate the benefits of our approach, we report on a thorough empirical evaluation, which analyses and compares our proposal with existing solutions executed in large-scale systems. The results show that our proposal outperforms the state-of-the-art distribution algorithms up to 35% in three different scenarios, reaching a reduction of the execution time of—at best—up to 99.66%.",

keywords = "Mutation testing, Parallel mutation testing, Large scale systems, High performance computing, Distributed systems, Testing",

author = "{Cerro Ca{\~n}izares}, Pablo and Alberto Nunez and Rosa Filgueira and {de Lara}, Juan",

note = "Funding: Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by the Spanish MINECO/FEDER project under Grants PID2021-122270OB-I00, TED2021-129381B-C21 and PID2019-108528RB-C22, the Comunidad de Madrid project FORTE-CM under Grant S2018/TCS-4314, Project S2018/TCS-4339 (BLOQUES-CM) co-funded by EIE Funds of the European Union and Comunidad de Madrid and the Project HPC-EUROPA3 (INFRAIA-2016-1-730897), with the support of the EC Research Innovation Action under the H2020 Programme.",

year = "2023",

month = jun,

day = "20",

doi = "10.1007/s10586-023-04074-y",

language = "English",

volume = "First online",

journal = "Cluster Computing",

issn = "1386-7857",

publisher = "Kluwer Academic Publishers",

}

TY - JOUR

T1 - Parallel mutation testing for large scale systems

AU - Cerro Cañizares, Pablo

AU - Nunez, Alberto

AU - Filgueira, Rosa

AU - de Lara, Juan

N1 - Funding: Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by the Spanish MINECO/FEDER project under Grants PID2021-122270OB-I00, TED2021-129381B-C21 and PID2019-108528RB-C22, the Comunidad de Madrid project FORTE-CM under Grant S2018/TCS-4314, Project S2018/TCS-4339 (BLOQUES-CM) co-funded by EIE Funds of the European Union and Comunidad de Madrid and the Project HPC-EUROPA3 (INFRAIA-2016-1-730897), with the support of the EC Research Innovation Action under the H2020 Programme.

PY - 2023/6/20

Y1 - 2023/6/20

N2 - Mutation testing is a valuable technique for measuring the quality of test suites in terms of detecting faults. However, one of its main drawbacks is its high computational cost. For this purpose, several approaches have been recently proposed to speed-up the mutation testing process by exploiting computational resources in distributed systems. However, bottlenecks have been detected when those techniques are applied in large-scale systems. This work improves the performance of mutation testing using large-scale systems by proposing a new load distribution algorithm, and parallelising different steps of the process. To demonstrate the benefits of our approach, we report on a thorough empirical evaluation, which analyses and compares our proposal with existing solutions executed in large-scale systems. The results show that our proposal outperforms the state-of-the-art distribution algorithms up to 35% in three different scenarios, reaching a reduction of the execution time of—at best—up to 99.66%.

AB - Mutation testing is a valuable technique for measuring the quality of test suites in terms of detecting faults. However, one of its main drawbacks is its high computational cost. For this purpose, several approaches have been recently proposed to speed-up the mutation testing process by exploiting computational resources in distributed systems. However, bottlenecks have been detected when those techniques are applied in large-scale systems. This work improves the performance of mutation testing using large-scale systems by proposing a new load distribution algorithm, and parallelising different steps of the process. To demonstrate the benefits of our approach, we report on a thorough empirical evaluation, which analyses and compares our proposal with existing solutions executed in large-scale systems. The results show that our proposal outperforms the state-of-the-art distribution algorithms up to 35% in three different scenarios, reaching a reduction of the execution time of—at best—up to 99.66%.

KW - Mutation testing

KW - Parallel mutation testing

KW - Large scale systems

KW - High performance computing

KW - Distributed systems

KW - Testing

U2 - 10.1007/s10586-023-04074-y

DO - 10.1007/s10586-023-04074-y

M3 - Article

SN - 1386-7857

VL - First online

JO - Cluster Computing

JF - Cluster Computing

ER -

Parallel mutation testing for large scale systems

Abstract

Keywords

Access to Document

Fingerprint

Cite this