Data for a meta-analysis of the adaptive layer in Adaptive Large Neighborhood Search

Published: 23 Jun 2020 | Version 3 | DOI: 10.17632/h4smx32r4t.3

Description of this data

Research on metaheuristics has focused almost exclusively on (novel) algorithmic development and on competitive testing, both of which have been frequently argued to yield very little generalizable knowledge. One way to obtain problem- and implementation-independent insights about metaheuristic algorithms is meta-analysis, a systematic statistical examination that combines the results of several independent studies. Meta-analysis is widely used in several scientific domains, most notably the medical sciences (e.g., to establish the efficacy of a certain treatment), but has not yet been applied in operations research.

In order to demonstrate its potential in learning about algorithms, we carried out a meta-analysis of the adaptive layer in adaptive large neighborhood search (ALNS). Although ALNS has been widely used to solve a broad range of problems, it has not yet been established whether or not adaptiveness actually contributes to the performance of an ALNS algorithm. A total of 134 studies were identified through Google Scholar or personal e-mail correspondence with researchers in the domain, 63 of which fit a set of predefined eligibility criteria. After sending requests for data to the authors of the eligible studies, results for 25 different implementations of ALNS were collected and analyzed using a random-effects model.

The dataset contains a detailed comparison of ALNS with the non-adaptive variant per study and per instance, together with the meta-analysis summary results. The data allows to replicate the analysis, to evaluate the algorithms using other metrics, to revisit the importance of ALNS adaptive layer if results from more studies become available, or to simply consult the ready-to-use formulas in data_analyzed.xls to carry out a meta-analysis of any research question.

On average, the addition of an adaptive layer in an ALNS algorithm improves the objective function value by 0.14% (95% confidence interval 0.06 to 0.21%). Although the adaptive layer can (and in a limited number of studies does) have an added value, it also adds considerable complexity and can therefore only be recommended in some very specific situations.

The findings of this meta-analysis underline the importance of evaluating the contribution of metaheuristic components, and of knowledge over competitive testing. Our goal is to promote meta-analysis as a methodology of obtaining knowledge and understanding of metahueristics frameworks, and we hope to see an increase in its popularity in the domain of operations research.

  • Compared to Version 1, the description of data tables' rows and columns has been improved, in order to improve clarity and better match the notation introduced in the corresponding article (Turkeš, Sörensen, Hvattum, Meta-analysis of metaheuristics: Quantifying the effect of adaptiveness in Adaptive Large Neighborhood Search). In addition, some errors have been corrected, so please refer to this version.

Experiment data files

Latest version

  • Version 3


    Published: 2020-06-23

    DOI: 10.17632/h4smx32r4t.3

    Cite this dataset

    Turkes, Renata (2020), “Data for a meta-analysis of the adaptive layer in Adaptive Large Neighborhood Search”, Mendeley Data, v3


Views: 266
Downloads: 40

Previous versions

Compare to version


Universiteit Antwerpen


Operations Research


CC BY 4.0 Learn more

The files associated with this dataset are licensed under a Creative Commons Attribution 4.0 International licence.

What does this mean?
You can share, copy and modify this dataset so long as you give appropriate credit, provide a link to the CC BY license, and indicate if changes were made, but you may not do so in a way that suggests the rights holder has endorsed you or your use of the dataset. Note that further permission may be required for any content within the dataset that is identified as belonging to a third party.