Analysis of algorithm components and parameters: some case studies

Nguyen Dang; Patrick De Causmaecker

doi:10.1007/978-3-030-05348-2_25

Analysis of algorithm components and parameters: some case studies

Nguyen Dang, Patrick De Causmaecker

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

2 Downloads (Pure)

Abstract

Modern high-performing algorithms are usually highly parameterised, and can be configured either manually or by an automatic algorithm configurator. The algorithm performance dataset obtained after the configuration step can be used to gain insights into how different algorithm parameters influence algorithm performance. This can be done by a number of analysis methods that exploit the idea of learning prediction models from an algorithm performance dataset and then using them for the data analysis on the importance of variables. In this paper, we demonstrate the complementary usage of three methods along this line, namely forward selection, fANOVA and ablation analysis with surrogates on three case studies, each of which represents some special situations that the analyses can fall into. By these examples, we illustrate how to interpret analysis results and discuss the advantage of combining different analysis methods.

Original language	English
Title of host publication	Learning and Intelligent Optimization
Subtitle of host publication	12th International Conference, LION 12, Kalamata, Greece, June 10–15, 2018, Revised Selected Papers
Editors	Roberto Battiti, Mauro Brunato, Ilias Kotsireas, Panos M. Pardalos
Place of Publication	Cham
Publisher	Springer
Pages	288-303
ISBN (Electronic)	9783030053482
ISBN (Print)	9783030053475
DOIs	https://doi.org/10.1007/978-3-030-05348-2_25
Publication status	Published - 2018
Event	Learning and Intelligent Optimization Conference (LION 12) - Elite City Resort, Kalamata, Greece Duration: 10 Jun 2018 → 15 Jun 2018 Conference number: 12 http://www.caopt.com/LION12/

Publication series

Name	Lecture Notes in Computer Science (Theoretical Computer Science and General Issues)
Publisher	Springer
Volume	11353
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Learning and Intelligent Optimization Conference (LION 12)
Abbreviated title	LION
Country/Territory	Greece
City	Kalamata
Period	10/06/18 → 15/06/18
Internet address	http://www.caopt.com/LION12/

Keywords

Forward selection
fANOVA
Ablation analysis with surrogates
Parameter analysis

Access to Document

10.1007/978-3-030-05348-2_25

typeinst
© 2019, Springer Nature Switzerland AG. This work has been made available online in accordance with the publisher's policies. This is the author created accepted version manuscript following peer review and as such may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1007/978-3-030-05348-2_25
Accepted author manuscript, 679 KB

Cite this

Dang, N., & De Causmaecker, P. (2018). Analysis of algorithm components and parameters: some case studies. In R. Battiti, M. Brunato, I. Kotsireas, & P. M. Pardalos (Eds.), Learning and Intelligent Optimization : 12th International Conference, LION 12, Kalamata, Greece, June 10–15, 2018, Revised Selected Papers (pp. 288-303). (Lecture Notes in Computer Science (Theoretical Computer Science and General Issues); Vol. 11353). Springer. https://doi.org/10.1007/978-3-030-05348-2_25

Dang, Nguyen ; De Causmaecker, Patrick. / Analysis of algorithm components and parameters : some case studies. Learning and Intelligent Optimization : 12th International Conference, LION 12, Kalamata, Greece, June 10–15, 2018, Revised Selected Papers. editor / Roberto Battiti ; Mauro Brunato ; Ilias Kotsireas ; Panos M. Pardalos. Cham : Springer, 2018. pp. 288-303 (Lecture Notes in Computer Science (Theoretical Computer Science and General Issues)).

@inproceedings{59b16b0671ce4e6ab935fa9975d71c9c,

title = "Analysis of algorithm components and parameters: some case studies",

abstract = "Modern high-performing algorithms are usually highly parameterised, and can be configured either manually or by an automatic algorithm configurator. The algorithm performance dataset obtained after the configuration step can be used to gain insights into how different algorithm parameters influence algorithm performance. This can be done by a number of analysis methods that exploit the idea of learning prediction models from an algorithm performance dataset and then using them for the data analysis on the importance of variables. In this paper, we demonstrate the complementary usage of three methods along this line, namely forward selection, fANOVA and ablation analysis with surrogates on three case studies, each of which represents some special situations that the analyses can fall into. By these examples, we illustrate how to interpret analysis results and discuss the advantage of combining different analysis methods.",

keywords = "Forward selection, fANOVA, Ablation analysis with surrogates, Parameter analysis",

author = "Nguyen Dang and {De Causmaecker}, Patrick",

year = "2018",

doi = "10.1007/978-3-030-05348-2_25",

language = "English",

isbn = "9783030053475",

series = "Lecture Notes in Computer Science (Theoretical Computer Science and General Issues)",

publisher = "Springer",

pages = "288--303",

editor = "Roberto Battiti and Mauro Brunato and Ilias Kotsireas and Pardalos, {Panos M.}",

booktitle = "Learning and Intelligent Optimization",

address = "Netherlands",

note = "Learning and Intelligent Optimization Conference (LION 12), LION ; Conference date: 10-06-2018 Through 15-06-2018",

url = "http://www.caopt.com/LION12/",

}

Dang, N & De Causmaecker, P 2018, Analysis of algorithm components and parameters: some case studies. in R Battiti, M Brunato, I Kotsireas & PM Pardalos (eds), Learning and Intelligent Optimization : 12th International Conference, LION 12, Kalamata, Greece, June 10–15, 2018, Revised Selected Papers. Lecture Notes in Computer Science (Theoretical Computer Science and General Issues), vol. 11353, Springer, Cham, pp. 288-303, Learning and Intelligent Optimization Conference (LION 12), Kalamata, Greece, 10/06/18. https://doi.org/10.1007/978-3-030-05348-2_25

Analysis of algorithm components and parameters: some case studies. / Dang, Nguyen; De Causmaecker, Patrick.
Learning and Intelligent Optimization : 12th International Conference, LION 12, Kalamata, Greece, June 10–15, 2018, Revised Selected Papers. ed. / Roberto Battiti; Mauro Brunato; Ilias Kotsireas; Panos M. Pardalos. Cham: Springer, 2018. p. 288-303 (Lecture Notes in Computer Science (Theoretical Computer Science and General Issues); Vol. 11353).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Analysis of algorithm components and parameters

T2 - Learning and Intelligent Optimization Conference (LION 12)

AU - Dang, Nguyen

AU - De Causmaecker, Patrick

N1 - Conference code: 12

PY - 2018

Y1 - 2018

N2 - Modern high-performing algorithms are usually highly parameterised, and can be configured either manually or by an automatic algorithm configurator. The algorithm performance dataset obtained after the configuration step can be used to gain insights into how different algorithm parameters influence algorithm performance. This can be done by a number of analysis methods that exploit the idea of learning prediction models from an algorithm performance dataset and then using them for the data analysis on the importance of variables. In this paper, we demonstrate the complementary usage of three methods along this line, namely forward selection, fANOVA and ablation analysis with surrogates on three case studies, each of which represents some special situations that the analyses can fall into. By these examples, we illustrate how to interpret analysis results and discuss the advantage of combining different analysis methods.

AB - Modern high-performing algorithms are usually highly parameterised, and can be configured either manually or by an automatic algorithm configurator. The algorithm performance dataset obtained after the configuration step can be used to gain insights into how different algorithm parameters influence algorithm performance. This can be done by a number of analysis methods that exploit the idea of learning prediction models from an algorithm performance dataset and then using them for the data analysis on the importance of variables. In this paper, we demonstrate the complementary usage of three methods along this line, namely forward selection, fANOVA and ablation analysis with surrogates on three case studies, each of which represents some special situations that the analyses can fall into. By these examples, we illustrate how to interpret analysis results and discuss the advantage of combining different analysis methods.

KW - Forward selection

KW - fANOVA

KW - Ablation analysis with surrogates

KW - Parameter analysis

U2 - 10.1007/978-3-030-05348-2_25

DO - 10.1007/978-3-030-05348-2_25

M3 - Conference contribution

SN - 9783030053475

T3 - Lecture Notes in Computer Science (Theoretical Computer Science and General Issues)

SP - 288

EP - 303

BT - Learning and Intelligent Optimization

A2 - Battiti, Roberto

A2 - Brunato, Mauro

A2 - Kotsireas, Ilias

A2 - Pardalos, Panos M.

PB - Springer

CY - Cham

Y2 - 10 June 2018 through 15 June 2018

ER -

Dang N, De Causmaecker P. Analysis of algorithm components and parameters: some case studies. In Battiti R, Brunato M, Kotsireas I, Pardalos PM, editors, Learning and Intelligent Optimization : 12th International Conference, LION 12, Kalamata, Greece, June 10–15, 2018, Revised Selected Papers. Cham: Springer. 2018. p. 288-303. (Lecture Notes in Computer Science (Theoretical Computer Science and General Issues)). Epub 2018 Dec 31. doi: 10.1007/978-3-030-05348-2_25

Analysis of algorithm components and parameters: some case studies

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this