A combined approach for analysing heuristic algorithms

Jeroen Corstjens, Nguyen Dang, Benoît Depaire, An Caris, Patrick De Causmaeckers

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)
4 Downloads (Pure)


When developing optimisation algorithms, the focus often lies on obtaining an algorithm that is able to outperform other existing algorithms for some performance measure. It is not common practice to question the reasons for possible performance differences observed. These types of questions relate to evaluating the impact of the various heuristic parameters and often remain unanswered. In this paper, the focus is on gaining insight in the behaviour of a heuristic algorithm by investigating how the various elements operating within the algorithm correlate with performance, obtaining indications of which combinations work well and which do not, and how all these effects are influenced by the specific problem instance the algorithm is solving. We consider two approaches for analysing algorithm parameters and components—functional analysis of variance and multilevel regression analysis—and study the benefits of using both approaches jointly. We present the results of a combined methodology that is able to provide more insights than when the two approaches are used separately. The illustrative case studies in this paper analyse a large neighbourhood search algorithm applied to the vehicle routing problem with time windows and an iterated local search algorithm for the unrelated parallel machine scheduling problem with sequence-dependent setup times.
Original languageEnglish
Pages (from-to)591–628
Number of pages38
JournalJournal of Heuristics
Issue number10
Early online date10 Aug 2018
Publication statusPublished - 1 Oct 2019


  • Functional analysis of variance
  • fANOVA
  • Multilevel regression
  • Algorithm performance
  • Vehicle routing problem with time windows
  • Large neighbourhood search
  • Iterated local search
  • Unrelated parallel machine scheduling problem


Dive into the research topics of 'A combined approach for analysing heuristic algorithms'. Together they form a unique fingerprint.

Cite this