IRIDIA BibTeX Repository
What is this?
This list of references in automatically generated from a collection of BibTeX files organized in a way that tries to avoid redundancy, minimise mistakes and facilitate customization.
You only need to fork (or link) the git repository in your papers and sync with the main copy to send/receive updates.
Most customisations, such as shorter journal or conference names, do not require changing the existing .bib
files.
You should not need to edit the entries directly unless you find mistakes. See the README for more details.
References

[1]

AAAI.
35th AAAI Conference on Artificial Intelligence: Reproducibility
Checklist.
https://aaai.org/Conferences/AAAI21/reproducibilitychecklist/, 2021.
Last accessed: June 6th, 2021.
[ bib ]

[2]

ACM.
Artifact Review and Badging Version 1.1.
https://www.acm.org/publications/policies/artifactreviewandbadgingcurrent,
August 2020.
[ bib ]

[3]

Emile H. L. Aarts, Jan H. M. Korst, and Wil Michiels.
Simulated Annealing.
In E. K. Burke and G. Kendall, editors, Search Methodologies,
pages 187–210. Springer, Boston, MA, 2005.
[ bib 
DOI ]

[4]

Hussein A. Abbass.
The selfadaptive Pareto differential evolution algorithm.
In Proceedings of the 2002 Congress on Evolutionary Computation
(CEC'02), pages 831–836, Piscataway, NJ, 2002. IEEE Press.
[ bib ]

[5]

Hussein A. Abbass, Ruhul Sarker, and Charles Newton.
PDE: a Paretofrontier differential evolution approach for
multiobjective optimization problems.
In Proceedings of the 2001 Congress on Evolutionary Computation
(CEC'01), pages 971–978, Piscataway, NJ, 2001. IEEE Press.
[ bib ]

[6]

F. Ben Abdelaziz, S. Krichen, and J. Chaouachi.
A hybrid heuristic for multiobjective knapsack problems.
In M. G. C. Resende and J. Pinho de Souza, editors, Proceedings
of MIC 1997, the 2nd Metaheuristics International Conference, pages
205–212, 1997.
[ bib 
DOI ]

[7]

David Abramson, Mohan Krishna Amoorthy, and Henry Dang.
Simulated annealing cooling schedules for the school timetabling
problem.
AsiaPacific Journal of Operational Research, 16(1):1–22,
1999.
[ bib ]

[8]

David Abramson.
Constructing School Timetables Using Simulated Annealing:
Sequential and Parallel Algorithms.
Management Science, 37(1):98–113, 1991.
[ bib ]

[9]

A. Acan.
An external memory implementation in ant colony optimization.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 73–84. Springer, Heidelberg, 2004.
[ bib ]
Keywords: memorybased ACO

[10]

A. Acan.
An external partial permutations memory for ant colony
optimization.
In G. R. Raidl and J. Gottlieb, editors, Proceedings of EvoCOP
2005 – 5th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 3448 of Lecture Notes in Computer Science, pages
1–11. Springer, Heidelberg, 2005.
[ bib ]
Keywords: memorybased ACO

[11]

Tobias Achterberg.
SCIP: Solving constraint integer programs.
Mathematical Programming Computation, 1(1):1–41, July 2009.
[ bib ]
http://mpc.zib.de/archive/2009/1/Achterberg2009_Article_SCIPSolvingConstraintIntegerPr.pdf

[12]

Tobias Achterberg and Timo Berthold.
Improving the feasibility pump.
Discrete Optimization, 4(1):77–86, 2007.
[ bib ]

[13]

HéctorGabriel AcostaMesa, Fernando RechyRamírez, Efrén
MezuraMontes, Nicandro CruzRamírez, and Rodolfo Hernández
Jiménez.
Application of time series discretization using evolutionary
programming for classification of precancerous cervical lesions.
Journal of Biomedical Informatics, 49:73–83, 2014.
[ bib 
DOI ]
Keywords: irace

[14]

Bernardetta Addis, Marco Locatelli, and Fabio Schoen.
Disk Packing in a Square: A New Global Optimization Approach.
INFORMS Journal on Computing, 20(4):516–524, 2008.
[ bib 
DOI ]

[15]

B. AdensoDíaz.
Restricted Neighborhood in the Tabu Search for the Flowshop
Problem.
European Journal of Operational Research, 62(1):27–37, 1992.
[ bib ]

[16]

B. AdensoDíaz and Manuel Laguna.
FineTuning of Algorithms Using Fractional Experimental Design
and Local Search.
Operations Research, 54(1):99–114, 2006.
[ bib ]
Keywords: Calibra

[17]

Bekir Afsar, Kaisa Miettinen, and Francisco Ruiz.
Assessing the Performance of Interactive Multiobjective
Optimization Methods: A Survey.
ACM Computing Surveys, 54(4), 2021.
[ bib 
DOI ]
Interactive methods are useful decisionmaking tools for
multiobjective optimization problems, because they allow a
decisionmaker to provide her/his preference information
iteratively in a comfortable way at the same time as (s)he
learns about all different aspects of the problem. A wide
variety of interactive methods is nowadays available, and
they differ from each other in both technical aspects and
type of preference information employed. Therefore, assessing
the performance of interactive methods can help users to
choose the most appropriate one for a given problem. This is
a challenging task, which has been tackled from different
perspectives in the published literature. We present a
bibliographic survey of papers where interactive
multiobjective optimization methods have been assessed
(either individually or compared to other methods). Besides
other features, we collect information about the type of
decisionmaker involved (utility or value functions,
artificial or human decisionmaker), the type of preference
information provided, and aspects of interactive methods that
were somehow measured. Based on the survey and on our own
experiences, we identify a series of desirable properties of
interactive methods that we believe should be assessed.
Keywords: decisionmakers, Interactive methods, performance assessment,
preference information, multiobjective optimization problems

[18]

Hernán E. Aguirre and Kiyoshi Tanaka.
Working principles, behavior, and performance of MOEAs on
MNKlandscapes.
European Journal of Operational Research, 181(3):1670–1690,
2007.
[ bib 
DOI ]

[19]

Hernán E. Aguirre and Kiyoshi Tanaka.
ManyObjective Optimization by Space Partitioning and Adaptive
εRanking on MNKLandscapes.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pages
407–422. Springer, Heidelberg, 2009.
[ bib ]

[20]

Hernán E. Aguirre.
Advances on Manyobjective Evolutionary Optimization.
In C. Blum and E. Alba, editors, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2013, pages
641–666, New York, NY, 2013. ACM Press.
[ bib ]
Keywords: manyobjective evolutionary optimization

[21]

Samad Ahmadi and Ibrahim H. Osman.
Density Based Problem Space Search for the Capacitated
Clustering pMedian Problem.
Annals of Operations Research, 131:21–43, 2004.
[ bib ]

[22]

A. Aho, J. Hopcroft, and J. Ullman.
Data structures and algorithms.
AddisonWesley, Reading, MA, 1983.
[ bib ]

[23]

Ali Ahrari, Saber Elsayed, Ruhul Sarker, Daryl Essam, and Carlos A. Coello
Coello.
Weighted pointwise prediction method for dynamic multiobjective
optimization.
Information Sciences, 546:349–367, 2021.
[ bib ]

[24]

R. K. Ahuja, O. Ergun, and A. P. Punnen.
A Survey of Very Largescale Neighborhood Search Techniques.
Discrete Applied Mathematics, 123(1–3):75–102, 2002.
[ bib ]

[25]

R. K. Ahuja, T. Magnanti, and J. B. Orlin.
Network Flows: Theory, Algorithms and Applications.
PrenticeHall, 1993.
[ bib ]

[26]

Uwe Aickelin, Edmund K. Burke, and Jingpeng Li.
Improved Squeaky Wheel Optimisation for Driver Scheduling.
In T. P. Runarsson, H.G. Beyer, E. K. Burke, J.J. Merelo,
D. Whitley, and X. Yao, editors, Parallel Problem Solving from Nature –
PPSN IX, volume 4193 of Lecture Notes in Computer Science, pages
182–191. Springer, Heidelberg, 2006.
[ bib ]

[27]

Sandip Aine, Rajeev Kumar, and P. P. Chakrabarti.
Adaptive parameter control of evolutionary algorithms to improve
qualitytime tradeoff.
Applied Soft Computing, 9(2):527–540, 2009.
[ bib 
DOI ]
Keywords: anytime

[28]

Hassene Aissi and Bernard Roy.
Robustness in Multicriteria Decision Aiding.
In M. Ehrgott, J. R. Figueira, and S. Greco, editors, Trends in
Multiple Criteria Decision Analysis, volume 142 of International Series
in Operations Research & Management Science, chapter 4, pages 87–121.
Springer, US, 2010.
[ bib ]

[29]

S. M. Aktürk, Alper Atamtürk, and S. Gürel.
A Strong Conic Quadratic Reformulation for MachineJob
Assignment with Controllable Processing Times.
Research Report BCOL.07.01, University of CaliforniaBerkeley, 2007.
[ bib ]

[30]

I. Alaya, Christine Solnon, and Khaled Ghédira.
Ant Colony Optimization for MultiObjective Optimization
Problems.
In 19th IEEE International Conference on Tools with Artificial
Intelligence (ICTAI 2007), volume 1, pages 450–457. IEEE Computer Society
Press, Los Alamitos, CA, 2007.
[ bib ]

[31]

I. Alaya, Christine Solnon, and Khaled Ghédira.
Ant algorithm for the multidimensional knapsack problem.
In B. Filipič and J. Šilc, editors, International
Conference on Bioinspired Optimization Methods and their Applications (BIOMA
2004), pages 63–72, 2004.
[ bib 
http ]

[32]

Enrique Alba and Francisco Chicano.
ACOhg: dealing with huge graphs.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pages 10–17. ACM Press,
New York, NY, 2007.
[ bib 
DOI ]

[33]

A. A. Albrecht, P. C. R. Lane, and K. Steinhöfel.
Analysis of Local Search Landscapes for kSAT Instances.
Mathematics in Computer Science, 3(4):465–488, 2010.
[ bib 
DOI ]

[34]

Aldeida Aleti and Irene Moser.
A systematic literature review of adaptive parameter control
methods for evolutionary algorithms.
ACM Computing Surveys, 49(3, Article 56):35, October 2016.
[ bib 
DOI ]

[35]

Alnur Ali and Marina Meilă.
Experiments with Kemeny ranking: What Works When?
Mathematical Social Science, 64(1):28–40, July 2012.
[ bib 
DOI ]
Computational Foundations of Social Choice
Keywords: Borda ranking, Kemeny ranking

[36]

Mohamad Alissa, Kevin Sim, and Emma Hart.
Algorithm Selection Using Deep Learning without Feature
Extraction.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pages 198–206, New York, NY, 2019. ACM Press.
[ bib 
DOI ]

[37]

Ali Allahverdi and Harun Aydilek.
Algorithms for nowait flowshops with total completion time
subject to makespan.
International Journal of Advanced Manufacturing Technology,
pages 1–15, 2013.
[ bib ]

[38]

Richard Allmendinger and Joshua D. Knowles.
Evolutionary Optimization on Problems Subject to Changes of
Variables.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of
Lecture Notes in Computer Science, pages 151–160. Springer, Heidelberg,
2010.
[ bib ]
Motivated by an experimental problem involving the
identification of effective drug combinations drawn from a
nonstatic drug library, this paper examines evolutionary
algorithm strategies for dealing with changes of
variables. We consider four standard techniques from dynamic
optimization, and propose one new technique. The results show
that only little additional diversity needs to be introduced
into the population when changing a small number of
variables, while changing many variables or optimizing a
rugged landscape requires often a restart of the optimization
process

[39]

Richard Allmendinger and Joshua D. Knowles.
Evolutionary Search in Lethal Environments.
In International Conference on Evolutionary Computation Theory
and Applications, pages 63–72. SciTePress, 2011.
[ bib 
DOI 
eprint ]

[40]

Richard Allmendinger.
Tuning evolutionary search for closedloop optimization.
PhD thesis, The University of Manchester, UK, 2012.
[ bib ]

[41]

Christian Almeder.
A hybrid optimization approach for multilevel capacitated
lotsizing problems.
European Journal of Operational Research, 200(2):599–606,
2010.
[ bib 
DOI ]
Solving multilevel capacitated lotsizing problems
is still a challenging task, in spite of increasing
computational power and faster algorithms. In this
paper a new approach combining an antbased
algorithm with an exact solver for (mixedinteger)
linear programs is presented. A MAXMIN ant system
is developed to determine the principal production
decisions, a LP/MIP solver is used to calculate
the corresponding production quantities and
inventory levels. Two different local search methods
and an improvement strategy based on reduced
mixedinteger problems are developed and integrated
into the ant algorithm. This hybrid approach
provides superior results for small and mediumsized
problems in comparison to the existing approaches in
the literature. For largescale problems the
performance of this method is among the best
Keywords: Ant colony optimization, Manufacturing, Material
requirements planning, Mixedinteger programming

[42]

A. Alsheddy and E. Tsang.
Guided Pareto local search and its application to the 0/1
multiobjective knapsack problems.
In M. Caserta and S. Voß, editors, Proceedings of MIC 2009,
the 8th Metaheuristics International Conference, Hamburg, Germany, 2010.
University of Hamburg.
[ bib ]

[43]

S. Alupoaei and S. Katkoori.
Ant Colony System Application to Marcocell Overlap Removal.
IEEE Transactions on Very Large Scale Integration (VLSI)
Systems, 12(10):1118–1122, 2004.
[ bib ]

[44]

Sanae Amani, Mahnoosh Alizadeh, and Christos Thrampoulidis.
Linear Stochastic Bandits Under Safety Constraints.
In H. M. Wallach, H. Larochelle, A. Beygelzimer,
F. d'AlchéBuc, E. B. Fox, and R. Garnett, editors, Advances in
Neural Information Processing Systems (NeurIPS 32), pages 9256–9266, 2019.
[ bib 
eprint ]

[45]

C. Amir, A. Badr, and I Farag.
A Fuzzy Logic Controller for Ant Algorithms.
Computing and Information Systems, 11(2):26–34, 2007.
[ bib ]

[46]

Christophe Andrieu, Nando de Freitas, Arnaud Doucet, and Michael I. Jordan.
An Introduction to MCMC for Machine Learning.
Machine Learning, 50(12):5–43, 2003.
[ bib ]

[47]

K. A. Andersen, K. Jörnsten, and M. Lind.
On bicriterion minimal spanning trees: An approximation.
Computers & Operations Research, 23(12):1171–1182, 1996.
[ bib ]

[48]

Klaus Andersen, René Victor Valqui Vidal, and Villy Bæk Iversen.
Design of a Teleprocessing Communication Network Using Simulated
Annealing.
In R. V. V. Vidal, editor, Applied Simulated Annealing, pages
201–215. Springer, 1993.
[ bib ]

[49]

J. H. Andersen and R. S. Powell.
The Use of Continuous Decision Variables in an Optimising Fixed
Speed Pump Scheduling Algorithm.
In R. S. Powell and K. S. Hindi, editors, Computing and Control
for the Water Industry, pages 119–128. Research Studies Press Ltd., 1999.
[ bib ]

[50]

Y. P. Aneja and K. P. K. Nair.
Bicriteria Transportation Problem.
Management Science, 25(1):73–78, 1979.
[ bib ]

[51]

Eric Angel, Evripidis Bampis, and Laurent Gourvés.
Approximating the Pareto curve with local search for the
bicriteria TSP(1,2) problem.
Theoretical Computer Science, 310(13):135–146, 2004.
[ bib 
DOI ]
Keywords: Archiving, Local search, Multicriteria TSP,
Approximation algorithms

[52]

D. Anghinolfi, A. Boccalatte, M. Paolucci, and C. Vecchiola.
Performance Evaluation of an Adaptive Ant Colony Optimization
Applied to Single Machine Scheduling.
In X. Li et al., editors, Simulated Evolution and Learning, 7th
International Conference, SEAL 2008, volume 5361 of Lecture Notes in
Computer Science, pages 411–420. Springer, Heidelberg, 2008.
[ bib ]

[53]

Daniel Angus and Clinton Woodward.
Multiple Objective Ant Colony Optimisation.
Swarm Intelligence, 3(1):69–85, 2009.
[ bib 
DOI ]

[54]

Daniel Angus.
PopulationBased Ant Colony Optimisation for Multiobjective
Function Optimisation.
In M. Randall, H. A. Abbass, and J. Wiles, editors, Progress in
Artificial Life (ACAL), volume 4828 of Lecture Notes in Computer
Science, pages 232–244. Springer, Heidelberg, 2007.
[ bib 
DOI ]

[55]

Kurt Anstreicher, Nathan Brixius, JeanPierre Goux, and Jeff Linderoth.
Solving large quadratic assignment problems on computational
grids.
Mathematical Programming Series B, 91(3):563–588, February
2002.
[ bib 
DOI ]
The quadratic assignment problem (QAP) is among the hardest
combinatorial optimization problems. Some instances of size
n = 30 have remained unsolved for decades. The solution of
these problems requires both improvements in mathematical
programming algorithms and the utilization of powerful
computational platforms. In this article we describe a novel
approach to solve QAPs using a stateoftheart
branchandbound algorithm running on a federation of
geographically distributed resources known as a computational
grid. Solution of QAPs of unprecedented complexity, including
the nug30, kra30b, and tho30 instances, is reported.

[56]

J. Ansel, S. Kamil, K. Veeramachaneni, J. RaganKelley, J. Bosboom, UnaMay
O'Reilly, and S. Amarasinghe.
OpenTuner: An extensible framework for program autotuning.
In Proceedings of the 23rd International Conference on Parallel
Architectures and Compilation, pages 303–315, New York, NY, 2014. ACM
Press.
[ bib 
DOI ]

[57]

Carlos Ansótegui, Yuri Malitsky, Horst Samulowitz, Meinolf Sellmann, and
Kevin Tierney.
ModelBased Genetic Algorithms for Algorithm Configuration.
In Q. Yang and M. Wooldridge, editors, Proceedings of the
TwentyFourth International Joint Conference on Artificial Intelligence
(IJCAI15), pages 733–739. IJCAI/AAAI Press, Menlo Park, CA, 2015.
[ bib 
DOI ]
Keywords: GGA++

[58]

Carlos Ansótegui, Yuri Malitsky, and Meinolf Sellmann.
MaxSAT by Improved InstanceSpecific Algorithm Configuration.
In D. Stracuzzi et al., editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pages 2594–2600. AAAI Press, 2014.
[ bib ]

[59]

Carlos Ansótegui, Meinolf Sellmann, and Kevin Tierney.
A GenderBased Genetic Algorithm for the Automatic Configuration
of Algorithms.
In I. P. Gent, editor, Principles and Practice of Constraint
Programming, CP 2009, volume 5732 of Lecture Notes in Computer
Science, pages 142–157. Springer, Heidelberg, 2009.
[ bib 
DOI ]
Keywords: GGA

[60]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Implementing the DantzigFulkersonJohnson Algorithm for
Large Traveling Salesman Problems.
Mathematical Programming Series B, 97(1–2):91–153, 2003.
[ bib 
DOI ]

[61]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Finding Cuts in the TSP.
Technical Report 95–05, DIMACS Center, Rutgers University,
Piscataway, NJ, USA, March 1995.
[ bib ]

[62]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
On the Solution of Traveling Salesman Problems.
Documenta Mathematica, Extra Volume ICM III:645–656, 1998.
[ bib ]

[63]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Finding Tours in the TSP.
Technical Report 99885, Forschungsinstitut für Diskrete
Mathematik, University of Bonn, Germany, 1999.
[ bib ]

[64]

J. S. Appleby, D. V. Blake, and E. A. Newman.
Techniques for producing school timetables on a computer and
their application to other scheduling problems.
The Computer Journal, 3(4):237–245, 1961.
[ bib 
DOI ]

[65]

David Applegate and William J. Cook.
A Computational Study of the JobShop Scheduling Problem.
ORSA Journal on Computing, 3(2):149–156, 1991.
[ bib ]

[66]

David Applegate, William J. Cook, and André Rohe.
Chained LinKernighan for Large Traveling Salesman
Problems.
INFORMS Journal on Computing, 15(1):82–92, 2003.
[ bib 
DOI ]

[67]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
The Traveling Salesman Problem: A Computational Study.
Princeton University Press, Princeton, NJ, 2006.
[ bib ]

[68]

David Applegate, Robert E. Bixby, Vašek Chvátal, William J. Cook,
D. Espinoza, M. Goycoolea, and Keld Helsgaun.
Certification of an Optimal TSP Tour Through 85,900 Cities.
Operations Research Letters, 37(1):11–15, 2009.
[ bib ]

[69]

Jay April, Fred Glover, James P. Kelly, and Manuel Laguna.
Simulationbased optimization: Practical introduction to
simulation optimization.
In S. E. Chick, P. J. Sanchez, D. M. Ferrin, and D. J. Morrice,
editors, Proceedings of the 35th Winter Simulation Conference: Driving
Innovation, volume 1, pages 71–78, New York, NY, December 2003. ACM Press.
[ bib 
DOI ]

[70]

Florian Arnold, Ítalo Santana, Kenneth Sörensen, and Thibaut Vidal.
PILS: Exploring highorder neighborhoods by pattern mining and
injection.
Arxiv preprint arXiv:1912.11462, 2019.
[ bib 
http ]

[71]

Florian Arnold and Kenneth Sörensen.
Knowledgeguided local search for the vehicle routing problem.
Computers & Operations Research, 105:32–46, 2019.
[ bib 
DOI ]

[72]

Florian Arnold and Kenneth Sörensen.
What makes a VRP solution good? The generation of
problemspecific knowledge for heuristics.
Computers & Operations Research, 106:280–288, 2019.
[ bib 
DOI ]

[73]

Sanjeev Arora and Boaz Barak.
Computational complexity: a modern approach.
Cambridge University Press, 2009.
[ bib ]

[74]

Marvin A. Arostegui Jr, Sukran N. Kadipasaoglu, and Basheer M. Khumawala.
An empirical comparison of tabu search, simulated annealing, and
genetic algorithms for facilities location problems.
International Journal of Production Economics, 103(2):742–754,
2006.
[ bib ]

[75]

José Elias C. Arroyo and V. A. Armentano.
A partial enumeration heuristic for multiobjective flowshop
scheduling problems.
Journal of the Operational Research Society, 55(9):1000–1007,
2004.
[ bib ]

[76]

José Elias C. Arroyo and V. A. Armentano.
Genetic local search for multiobjective flowshop scheduling
problems.
European Journal of Operational Research, 167(3):717–738,
2005.
[ bib ]
Keywords: Multicriteria Scheduling

[77]

José Elias C. Arroyo and Joseph Y.T. Leung.
An Effective Iterated Greedy Algorithm for Scheduling Unrelated
Parallel Batch Machines with Nonidentical Capacities and Unequal Ready
Times.
Computers and Industrial Engineering, 105:84–100, 2017.
[ bib ]

[78]

Etor Arza, Josu Ceberio, Aritz Pérez, and Ekhine Irurozki.
Approaching the quadratic assignment problem with kernels of
mallows models under the hamming distance.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference Companion, GECCO Companion 2019. ACM Press, New York, NY, 2019.
[ bib 
DOI ]
Keywords: QAP, EDA, Mallows

[79]

Y. Asahiro, K. Iwama, and E. Miyano.
Random Generation of Test Instances with Controlled Attributes.
In D. S. Johnson and M. A. Trick, editors, Cliques, Coloring,
and Satisfiability: Second DIMACS Implementation Challenge, volume 26 of
DIMACS Series on Discrete Mathematics and Theoretical Computer
Science, pages 377–393. American Mathematical Society, Providence, RI,
1996.
[ bib ]

[80]

N. Ascheuer, Matteo Fischetti, and M. Grötschel.
Solving asymmetric travelling salesman problem with time windows
by branchandcut.
Mathematical Programming, 90:475–506, 2001.
[ bib ]

[81]

N. Ascheuer.
Hamiltonian Path Problems in the Online Optimization of
Flexible Manufacturing Systems.
PhD thesis, Technische Universität Berlin, Berlin, Germany, 1995.
[ bib ]

[82]

Alper Atamtürk.
On the facets of the mixed–integer knapsack polyhedron.
Mathematical Programming, 98(1):145–175, 2003.
[ bib 
DOI ]

[83]

R. Atkinson, Jakobus E. van Zyl, Godfrey A. Walters, and Dragan A. Savic.
Genetic algorithm optimisation of levelcontrolled pumping
station operation.
In Water network modelling for optimal design and management,
pages 79–90. Centre for Water Systems, Exeter, UK, 2000.
[ bib ]

[84]

Charles Audet, CongKien Dang, and Dominique Orban.
Algorithmic Parameter Optimization of the DFO Method with the
OPAL Framework.
In K. Naono, K. Teranishi, J. Cavazos, and R. Suda, editors,
Software Automatic Tuning: From Concepts to StateoftheArt Results, pages
255–274. Springer, 2010.
[ bib ]

[85]

Charles Audet, CongKien Dang, and Dominique Orban.
Optimization of Algorithms with OPAL.
Mathematical Programming Computation, 6(3):233–254, 2014.
[ bib ]

[86]

P. Audze and Vilnis Eglãjs.
New approach to the design of multifactor experiments.
Problems of Dynamics and Strengths, 35:104–107, 1977.
(in Russian).
[ bib ]

[87]

Charles Audet and Dominique Orban.
Finding Optimal Algorithmic Parameters Using DerivativeFree
Optimization.
SIAM Journal on Optimization, 17(3):642–664, 2006.
[ bib ]

[88]

Peter Auer.
Using Confidence Bounds for ExploitationExploration
Tradeoffs.
Journal of Machine Learning Research, 3:397–422, November
2002.
[ bib ]
We show how a standard tool from statistics — namely
confidence bounds — can be used to elegantly deal with
situations which exhibit an exploitationexploration
tradeoff. Our technique for designing and analyzing
algorithms for such situations is general and can be applied
when an algorithm has to make exploitationversusexploration
decisions based on uncertain information provided by a random
process. We apply our technique to two models with such an
exploitationexploration tradeoff. For the adversarial
bandit problem with shifting our new algorithm suffers only
O((ST)^{1/2}) regret with high probability over T trials
with S shifts. Such a regret bound was previously known
only in expectation. The second model we consider is
associative reinforcement learning with linear value
functions. For this model our technique improves the regret
from O(T^{3/4}) to O(T^{1/2}).

[89]

Peter Auer, Nicolo CesaBianchi, and Paul Fischer.
Finitetime analysis of the multiarmed bandit problem.
Machine Learning, 47(23):235–256, 2002.
[ bib ]

[90]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Articulating User Preferences in ManyObjective Problems by
Sampling the Weighted Hypervolume.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pages 555–562. ACM Press,
New York, NY, 2009.
[ bib ]

[91]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Investigating and Exploiting the Bias of the Weighted
Hypervolume to Articulate User Preferences.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pages 563–570. ACM Press,
New York, NY, 2009.
[ bib ]

[92]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Theory of the hypervolume indicator: optimal μdistributions
and the choice of the reference point.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pages 87–102. ACM Press,
New York, NY, 2009.
[ bib ]

[93]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Hypervolumebased multiobjective optimization: Theoretical
foundations and practical implications.
Theoretical Computer Science, 425:75–103, 2012.
[ bib 
DOI ]

[94]

Anne Auger, Dimo Brockhoff, Manuel LópezIbáñez, Kaisa Miettinen,
Boris Naujoks, and Günther Rudolph.
Which questions should be asked to find the most appropriate
method for decision making and problem solving? (Working Group
“Algorithm Design Methods”).
In S. Greco, J. D. Knowles, K. Miettinen, and E. Zitzler, editors,
Learning in Multiobjective Optimization (Dagstuhl Seminar 12041),
volume 2(1) of Dagstuhl Reports, pages 92–93. Schloss
Dagstuhl–LeibnizZentrum für Informatik, Germany, 2012.
[ bib 
DOI ]

[95]

A. Auger and B. Doerr, editors.
Theory of Randomized Search Heuristics: Foundations and Recent
Developments, volume 1 of Series on Theoretical Computer Science.
World Scientific Publishing Co., Singapore, 2011.
[ bib ]

[96]

Anne Auger and Nikolaus Hansen.
A restart CMA evolution strategy with increasing population
size.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pages 1769–1776. IEEE Press, Piscataway, NJ, September 2005.
[ bib 
DOI ]
Keywords: IPOPCMAES

[97]

Anne Auger and Nikolaus Hansen.
Performance evaluation of an advanced local search evolutionary
algorithm.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pages 1777–1784. IEEE Press, Piscataway, NJ, September 2005.
[ bib ]
Keywords: LRCMAES

[98]

Mustafa Avci and Seyda Topaloglu.
A Multistart Iterated Local Search Algorithm for the
Generalized Quadratic Multiple Knapsack Problem.
Computers & Operations Research, 83:54–65, 2017.
[ bib ]

[99]

Andreea Avramescu, Richard Allmendinger, and Manuel LópezIbáñez.
Managing Manufacturing and Delivery of Personalised Medicine:
Current and Future Models.
Arxiv preprint arXiv:2105.12699 [econ.GN], 2021.
[ bib 
http ]

[100]

Andreea Avramescu, Richard Allmendinger, and Manuel LópezIbáñez.
A MultiObjective MultiType Facility Location Problem for the
Delivery of Personalised Medicine.
In P. Castillo and J. L. Jiménez Laredo, editors,
Applications of Evolutionary Computation, volume 12694 of Lecture Notes
in Computer Science, pages 388–403. Springer, Cham, Switzerland, 2021.
[ bib 
DOI 
supplementary material ]
Advances in personalised medicine targeting specific
subpopulations and individuals pose a challenge to the
traditional pharmaceutical industry. With a higher level of
personalisation, an already critical supply chain is facing
additional demands added by the very sensitive nature of its
products. Nevertheless, studies concerned with the efficient
development and delivery of these products are scarce. Thus,
this paper presents the case of personalised medicine and the
challenges imposed by its mass delivery. We propose a
multiobjective mathematical model for the
locationallocation problem with two interdependent facility
types in the case of personalised medicine products. We show
its practical application through a cell and gene therapy
case study. A multiobjective genetic algorithm with a novel
population initialisation procedure is used as solution
method.
Keywords: Personalised medicine, Biopharmaceuticals Supply chain,
Facility locationallocation, Evolutionary multiobjective
optimisation

[101]

Dogan Aydin, Gürcan Yavuz, Serdar Özyön, Celal Yasar, and
Thomas Stützle.
Artificial Bee Colony Framework to Nonconvex Economic Dispatch
Problem with Valve Point Effects: A Case Study.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2017, pages
1311–1318, New York, NY, 2017. ACM Press.
[ bib ]

[102]

Dogan Aydin, Gürcan Yavuz, and Thomas Stützle.
ABCX: A Generalized, Automatically Configurable Artificial
Bee Colony Framework.
Swarm Intelligence, 11(1):1–38, 2017.
[ bib ]

[103]

Amine AzizAlaoui, Carola Doerr, and Johann Dréo.
Towards Large Scale Automated Algorithm Design by Integrating
Modular Benchmarking Frameworks.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference Companion, GECCO Companion
2021, pages 1365–1374, New York, NY, 2021. ACM Press.
[ bib 
DOI ]

[104]

Mahdi Aziz and MohammadH. TayaraniN.
An adaptive memetic Particle Swarm Optimization algorithm for
finding largescale Latin hypercube designs.
Engineering Applications of Artificial Intelligence,
36:222–237, 2014.
[ bib 
DOI ]
Keywords: Frace

[105]

Ilya Loshchilov and T. Glasmachers.
Black Box Optimization Competition, 2017.
[ bib 
http ]

[106]

Anne Auger, Dimo Brockhoff, Nikolaus Hansen, Dejan Tusar, Tea Tušar, and
Tobias Wagner.
GECCO Workshop on RealParameter BlackBox Optimization
Benchmarking (BBOB 2016): Focus on multiobjective problems.
https://numbbo.github.io/workshops/BBOB2016/, 2016.
[ bib ]

[107]

S. Bleuler, Marco Laumanns, Lothar Thiele, and Eckart Zitzler.
PISA – A Platform and Programming Language Independent
Interface for Search Algorithms.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 494–508. Springer,
Heidelberg, 2003.
[ bib ]

[108]

Domagoj Babić.
Spear theorem prover.
https://www.domagojbabic.com/index.php/ResearchProjects/Spear,
2008.
[ bib ]

[109]

Domagoj Babić and Alan J. Hu.
Structural Abstraction of Software Verification Conditions.
In Computer Aided Verification: 19th International Conference,
CAV 2007, pages 366–378, 2007.
[ bib ]
Spearswv instances,
http://www.cs.ubc.ca/labs/beta/Projects/ParamILS/benchmark_instances/SpearSWV/SWVscrambledfirst302.tar.gz,
http://www.cs.ubc.ca/labs/beta/Projects/ParamILS/benchmark_instances/SpearSWV/SWVscrambledlast302.tar.gz

[110]

Domagoj Babić and Frank Hutter.
Spear Theorem Prover.
In SAT'08: Proceedings of the SAT 2008 Race, 2008.
[ bib 
eprint 
supplementary material ]
Unreviewed paper

[111]

Thomas Bäck, David B. Fogel, and Zbigniew Michalewicz.
Handbook of evolutionary computation.
IOP Publishing, 1997.
[ bib ]

[112]

François Bachoc, Céline Helbert, and Victor Picheny.
Gaussian process optimization with failures: Classification and
convergence proof.
Journal of Global Optimization, 2020.
[ bib 
DOI 
eprint ]
We consider the optimization of a computer model where each
simulation either fails or returns a valid output
performance. We first propose a new joint Gaussian process
model for classification of the inputs (computation failure
or success) and for regression of the performance
function. We provide results that allow for a computationally
efficient maximum likelihood estimation of the covariance
parameters, with a stochastic approximation of the likelihood
gradient. We then extend the classical improvement criterion
to our setting of joint classification and regression. We
provide an efficient computation procedure for the extended
criterion and its gradient. We prove the almost sure
convergence of the global optimization algorithm following
from this extended criterion. We also study the practical
performances of this algorithm, both on simulated data and on
a real computer model in the context of automotive fan
design.
Keywords: crashed simulation; latent gaussian process; automotive fan
design; industrial application; GP classification; Expected
Feasible Improvement with Gaussian Process Classification
with signs; EFI GPC sign

[113]

Achim Bachem, Barthel Steckemetz, and Michael Wottawa.
An efficient parallel clusterheuristic for large Traveling
Salesman Problems.
Technical Report 94150, University of Koln, Germany, 1994.
[ bib ]
Keywords: Genetic Edge Recombination (ERX)

[114]

Thomas Bäck.
Evolutionary algorithms in theory and practice: evolution
strategies, evolutionary programming, genetic algorithms.
Oxford University Press, 1996.
[ bib ]

[115]

Johannes Bader and Eckart Zitzler.
HypE: An Algorithm for Fast HypervolumeBased ManyObjective
Optimization.
Evolutionary Computation, 19(1):45–76, 2011.
[ bib 
DOI ]

[116]

Hossein Baharmand, Tina Comes, and Matthieu Lauras.
Biobjective multilayer location– allocation model
for the immediate aftermath of suddenonset disasters.
Transportation Research Part E: Logistics and Transportation
Review, 127:86–110, 2019.
[ bib 
DOI ]
Locating distribution centers is critical for humanitarians
in the immediate aftermath of a suddenonset disaster. A
major challenge lies in balancing the complexity and
uncertainty of the problem with time and resource
constraints. To address this problem, we propose a
locationallocation model that divides the topography of
affected areas into multiple layers; considers constrained
number and capacity of facilities and fleets; and allows
decisionmakers to explore tradeoffs between response time
and logistics costs. To illustrate our theoretical work, we
apply the model to a real dataset from the 2015 Nepal
earthquake response. For this case, our method results in a
considerable reduction of logistics costs.

[117]

Monya Baker.
Is there a reproducibility crisis?
Nature, 533:452–454, 2016.
[ bib ]

[118]

Edward K. Baker.
An Exact Algorithm for the TimeConstrained Traveling Salesman
Problem.
Operations Research, 31(5):938–945, 1983.
[ bib 
DOI ]

[119]

Burcu Balcik and Benita M. Beamon.
Facility location in humanitarian relief.
International Journal of Logistics, 11(2):101–121, 2008.
[ bib ]

[120]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Incremental local search in ant colony optimization: Why it
fails for the quadratic assignment problem.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 5th International Workshop, ANTS 2006, volume 4150 of
Lecture Notes in Computer Science, pages 156–166. Springer, Heidelberg,
2006.
[ bib ]

[121]

Prasanna Balaprakash, Mauro Birattari, and Thomas Stützle.
Improvement Strategies for the FRace Algorithm: Sampling
Design and Iterative Refinement.
In T. BartzBeielstein, M. J. Blesa, C. Blum, B. Naujoks, A. Roli,
G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics, volume 4771
of Lecture Notes in Computer Science, pages 108–122. Springer,
Heidelberg, 2007.
[ bib 
DOI ]
Keywords: Iterated Race

[122]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Adaptive Sampling Size and Importance Sampling in
Estimationbased Local Search for the Probabilistic Traveling Salesman
Problem.
European Journal of Operational Research, 199(1):98–110, 2009.
[ bib ]

[123]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Estimationbased Metaheuristics for the Probabilistic Travelling
Salesman Problem.
Computers & Operations Research, 37(11):1939–1951, 2010.
[ bib 
DOI ]

[124]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Estimationbased Metaheuristics for the Single Vehicle Routing
Problem with Stochastic Demands and Customers.
Computational Optimization and Applications, 61(2):463–487,
2015.
[ bib 
DOI ]

[125]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, Zhi Yuan, and Marco
Dorigo.
Estimationbased Ant Colony Optimization Algorithms for the
Probabilistic Travelling Salesman Problem.
Swarm Intelligence, 3(3):223–242, 2009.
[ bib ]

[126]

Egon Balas and M. C. Carrera.
A Dynamic Subgradientbased Branch and Bound Procedure for Set
Covering.
Operations Research, 44(6):875–890, 1996.
[ bib ]

[127]

Egon Balas and A. Ho.
Set Covering Algorithms Using Cutting Planes, Heuristics, and
Subgradient Optimization: A Computational Study.
Mathematical Programming Study, 12:37–60, 1980.
[ bib ]

[128]

Egon Balas and C. Martin.
Pivot and Complement–A Heuristic for 0–1 Programming.
Management Science, 26(1):86–96, 1980.
[ bib ]

[129]

Egon Balas and M. W. Padberg.
Set Partitioning: A Survey.
SIAM Review, 18:710–760, 1976.
[ bib ]

[130]

Egon Balas and Neil Simonetti.
Linear Time DynamicProgramming Algorithms for New Classes of
Restricted TSPs: A Computational Study.
INFORMS Journal on Computing, 13(1):56–75, 2001.
[ bib 
DOI 
eprint ]
Consider the following restricted (symmetric or
asymmetric) travelingsalesman problem (TSP):
given an initial ordering of the n cities and an
integer k > 0, find a minimumcost
feasible tour, where a feasible tour is one in which
city i precedes city j whenever j >= i + k in the
initial ordering. Balas (1996) has proposed a
dynamicprogramming algorithm that solves this
problem in time linear in n, though exponential in
k. Some important realworld problems are amenable
to this model or some of its close relatives. The
algorithm of Balas (1996) constructs a layered
network with a layer of nodes for each position in
the tour, such that sourcesink paths in this
network are in onetoone correspondence with tours
that satisfy the postulated precedence
constraints. In this paper we discuss an
implementation of the dynamicprogramming algorithm
for the general case when the integer k is replaced
with cityspecific integers k(j), j = 1, . . .,
n. We discuss applications to, and computational
experience with, TSPs with time windows, a model
frequently used in vehicle routing as well as in
scheduling with setup, release and delivery
times. We also introduce a new model, the TSP with
target times, applicable to JustinTime
scheduling problems. Finally for TSPs that have no
precedence restrictions, we use the algorithm as a
heuristic that finds in linear time a local optimum
over an exponentialsize neighborhood. For this
case, we implement an iterated version of our
procedure, based on contracting some arcs of the
tour produced by a first application of the
algorithm, then reapplying the algorithm to the
shrunken graph with the same k.
Keywords: tsptw

[131]

Egon Balas and A. Vazacopoulos.
Guided Local Search with Shifting Bottleneck for Job Shop
Scheduling.
Management Science, 44(2):262–275, 1998.
[ bib ]

[132]

Steven C. Bankes.
Tools and techniques for developing policies for complex and
uncertain systems.
Proceedings of the National Academy of Sciences, 99(suppl
3):7263–7266, 2002.
[ bib 
DOI ]
Agentbased models (ABM) are examples of complex adaptive
systems, which can be characterized as those systems for
which no model less complex than the system itself can
accurately predict in detail how the system will behave at
future times. Consequently, the standard tools of policy
analysis, based as they are on devising policies that perform
well on some best estimate model of the system, cannot be
reliably used for ABM. This paper argues that policy analysis
by using ABM requires an alternative approach to decision
theory. The general characteristics of such an approach are
described, and examples are provided of its application to
policy analysis.ABM, agentbased model

[133]

P. Baptiste and L. K. Hguny.
A branch and bound algorithm for the
F/no_idle/C_{}max.
In Proceedings of the international conference on industrial
engineering and production management, IEPM'97, pages 429–438, Lyon, 1997.
[ bib ]

[134]

Thomas BartzBeielstein.
Experimental Research in Evolutionary Computation: The New
Experimentalism.
Springer, Berlin, Germany, 2006.
[ bib ]
Keywords: SPO

[135]

Thomas BartzBeielstein.
How to Create Generalizable Results.
In J. Kacprzyk and W. Pedrycz, editors, Springer Handbook of
Computational Intelligence, pages 1127–1142. Springer, Berlin/
Heidelberg, 2015.
[ bib ]
Keywords: Mixedeffects models, randomeffects model, problem instance
generation

[136]

Eduardo Batista de Moraes Barbosa, Edson Luiz Francça Senne, and
Messias Borges Silva.
Improving the Performance of Metaheuristics: An Approach
Combining Response Surface Methodology and Racing Algorithms.
International Journal of Engineering Mathematics, 2015:Article
ID 167031, 2015.
[ bib 
DOI ]
Keywords: Frace

[137]

Alejandro Barredo Arrieta, Natalia DíazRodríguez, Javier Del Ser,
Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio
GilLopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco
Herrera.
Explainable Artificial Intelligence (XAI): Concepts,
taxonomies, opportunities and challenges toward responsible AI.
Information Fusion, 58:82–115, June 2020.
[ bib 
DOI ]

[138]

Thomas BartzBeielstein, Carola Doerr, Daan van den Berg, Jakob Bossek, Sowmya
Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, William La
Cava, Manuel LópezIbáñez, Katherine M. Malan, Jason H. Moore,
Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas
Weise.
Benchmarking in Optimization: Best Practice and Open Issues.
Arxiv preprint arXiv:2007.03488 [cs.NE], 2020.
[ bib 
http ]

[139]

Thomas BartzBeielstein, Oliver Flasch, Patrick Koch, and Wolfgang Konen.
SPOT: A Toolbox for Interactive and Automatic Tuning in the
R Environment.
In Proceedings 20. Workshop Computational Intelligence, pages
264–273, Karlsruhe, 2010. KIT Scientific Publishing.
[ bib ]

[140]

Richard S. Barr, Bruce L. Golden, James P. Kelly, Mauricio G. C. Resende, and
Jr. William R. Stewart.
Designing and Reporting on Computational Experiments with
Heuristic Methods.
Journal of Heuristics, 1(1):9–32, 1995.
[ bib 
DOI ]

[141]

Cynthia Barnhart, Ellis L. Johnson, George L. Nemhauser, Martin W. P.
Savelsbergh, and Pamela H. Vance.
Branchandprice: Column generation for solving huge integer
programs.
Operations Research, 46(3):316–329, 1998.
[ bib ]

[142]

Erin Bartholomew and Jan H. Kwakkel.
On considering robustness in the search phase of Robust Decision
Making: A comparison of ManyObjective Robust Decision Making, multiscenario
ManyObjective Robust Decision Making, and Many Objective Robust
Optimization.
Environmental Modelling & Software, 127:104699, 2020.
[ bib 
DOI ]

[143]

Thomas BartzBeielstein, C. Lasarczyk, and Mike Preuss.
Sequential Parameter Optimization.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pages 773–780, Piscataway, NJ, September 2005. IEEE Press.
[ bib ]

[144]

Thomas BartzBeielstein, C. Lasarczyk, and Mike Preuss.
The Sequential Parameter Optimization Toolbox.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 337–360. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Keywords: SPOT

[145]

Thomas BartzBeielstein and Sandor Markon.
Tuning search algorithms for realworld applications: A
regression tree based approach.
In Proceedings of the 2004 Congress on Evolutionary Computation
(CEC 2004), pages 1111–1118, Piscataway, NJ, September 2004. IEEE Press.
[ bib ]

[146]

Elias Bareinboim and Judea Pearl.
Transportability of causal effects: Completeness results.
In J. Hoffmann and B. Selman, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pages 698,704. AAAI Press, 2012.
[ bib ]

[147]

Elias Bareinboim and Judea Pearl.
Causal inference and the datafusion problem.
Proceedings of the National Academy of Sciences,
113(27):7345–7352, 2016.
[ bib 
DOI ]

[148]

Thomas BartzBeielstein and Mike Preuss.
Considerations of budget allocation for sequential parameter
optimization (SPO).
In L. Paquete, M. Chiarandini, and D. Basso, editors, Empirical
Methods for the Analysis of Algorithms, Workshop EMAA 2006, Proceedings,
pages 35–40, Reykjavik, Iceland, 2006.
[ bib ]

[149]

Thomas BartzBeielstein and Mike Preuss.
Experimental Analysis of Optimization Algorithms: Tuning and
Beyond.
In Y. Borenstein and A. Moraglio, editors, Theory and Principled
Methods for the Design of Metaheuristics, Natural Computing Series, pages
205–245. Springer, Berlin/Heidelberg, 2014.
[ bib 
DOI ]

[150]

Benjamín Barán and Matilde Schaerer.
A multiobjective ant colony system for vehicle routing problem
with time windows.
In Proceedings of the Twentyfirst IASTED International
Conference on Applied Informatics, pages 97–102, Insbruck, Austria, 2003.
[ bib ]

[151]

Thomas BartzBeielstein and Martin Zaefferer.
Modelbased methods for continuous and discrete global
optimization.
Applied Soft Computing, 55:154–167, June 2017.
[ bib 
DOI ]

[152]

Atanu Basu and L. Neil Frazer.
Rapid Determination of the Critical Temperature in Simulated
Annealing Inversion.
Science, 249(4975):1409–1412, 1990.
[ bib ]

[153]

Matthieu Basseur, Adrien Goëffon, Arnaud Liefooghe, and Sébastien
Verel.
On Setbased Local Search for Multiobjective Combinatorial
Optimization.
In C. Blum and E. Alba, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2013, pages 471–478. ACM Press,
New York, NY, 2013.
[ bib 
DOI ]

[154]

Roberto Battiti, M. Brunato, and Franco Mascia.
Reactive Search and Intelligent Optimization, volume 45 of
Operations Research/Computer Science Interfaces.
Springer, New York, NY, 2008.
[ bib 
DOI ]

[155]

Roberto Battiti and Paolo Campigotto.
Reactive search optimization: Learning while optimizing. An
experiment in interactive multiobjective optimization.
In M. Caserta and S. Voß, editors, Proceedings of MIC 2009,
the 8th Metaheuristics International Conference, Hamburg, Germany, 2010.
University of Hamburg.
[ bib ]

[156]

Roberto Battiti and Andrea Passerini.
BrainComputer Evolutionary Multiobjective Optimization: A
Genetic Algorithm Adapting to the Decision Maker.
IEEE Transactions on Evolutionary Computation, 14(5):671–687,
2010.
[ bib 
DOI ]
Errata: DTLZ6 and DTLZ7 in the paper are actually DTLZ7 and
DTLZ8 in [551]
Keywords: BCEMOA

[157]

Roberto Battiti and M. Protasi.
Reactive Search, A HistoryBased Heuristic for MAXSAT.
ACM Journal of Experimental Algorithmics, 2, 1997.
[ bib ]

[158]

Michele Battistutta, Andrea Schaerf, and Tommaso Urli.
Featurebased tuning of singlestage simulated annealing for
examination timetabling.
In E. Özcan, E. K. Burke, and B. McCollum, editors, PATAT
2014: Proceedings of the 10th International Conference of the Practice and
Theory of Automated Timetabling, pages 53–61. PATAT, 2014.
[ bib ]
Keywords: Frace

[159]

Michele Battistutta, Andrea Schaerf, and Tommaso Urli.
Featurebased Tuning of Singlestage Simulated Annealing for
Examination Timetabling.
Annals of Operations Research, 252(2):239–254, 2017.
[ bib ]

[160]

Roberto Battiti and Giampietro Tecchiolli.
Simulated annealing and Tabu search in the long run: A
comparison on QAP tasks.
Computer and Mathematics with Applications, 28(6):1–8, 1994.
[ bib 
DOI ]

[161]

Roberto Battiti and Giampietro Tecchiolli.
The Reactive Tabu Search.
ORSA Journal on Computing, 6(2):126–140, 1994.
[ bib ]

[162]

Roberto Battiti and Giampietro Tecchiolli.
The continuous reactive tabu search: blending combinatorial
optimization and stochastic search for global optimization.
Annals of Operations Research, 63(2):151–188, 1996.
[ bib ]

[163]

E. B. Baum.
Iterated Descent: A Better Algorithm for Local Search in
Combinatorial Optimization Problems.
Manuscript, 1986.
[ bib ]

[164]

E. B. Baum.
Towards Practical “Neural” Computation for Combinatorial
Optimization Problems.
In Neural Networks for Computing, AIP Conference Proceedings,
pages 53–64, 1986.
[ bib ]

[165]

J. Bautista and J. Pereira.
Ant algorithms for a time and space constrained assembly line
balancing problem.
European Journal of Operational Research, 177(3):2016–2032,
2007.
[ bib 
DOI ]

[166]

William J. Baumol.
Management models and industrial applications of linear
programming.
Naval Research Logistics Quarterly, 9(1):63–64, 1962.
[ bib 
DOI ]

[167]

John Baxter.
Local Optima Avoidance in Depot Location.
Journal of the Operational Research Society, 32(9):815–819,
1981.
[ bib ]

[168]

A. Baykasoglu, T. Dereli, and I. Sabuncu.
A multiple objective ant colony optimization approach to
assembly line balancing problems.
In 35th International Conference on Computers and Industrial
Engineering (CIE35), pages 263–268, Istanbul, Turkey, 2005.
[ bib ]

[169]

John E. Beasley and P. C. Chu.
A Genetic Algorithm for the Set Covering Problem.
European Journal of Operational Research, 94(2):392–404, 1996.
[ bib ]

[170]

John E. Beasley and P. C. Chu.
A Genetic Algorithm for the Multidimensional Knapsack Problem.
Journal of Heuristics, 4(1):63–86, 1998.
[ bib ]

[171]

Brian Beachkofski and Ramana Grandhi.
Improved Distributed Hypercube Sampling.
In Proceedings of the 43rd AIAA/ASME/ASCE/AHS/ASC Structures,
Structural Dynamics, and Materials Conference. AIAA paper 20021274,
American Institute of Aeronautics and Astronautics, 2002.
[ bib ]

[172]

Jennifer Bealt, Duncan Shaw, Chris M. Smith, and Manuel
LópezIbáñez.
Peer Reviews for Making Cities Resilient: A Systematic
Literature Review.
International Journal of Emergency Management, 15(4):334–359,
2019.
[ bib 
DOI ]
Peer reviews are a unique governance tool that use expertise
from one city or country to assess and strengthen the
capabilities of another. Peer review tools are gaining
momentum in disaster management and remain an important but
understudied topic in risk governance. Methodologies to
conduct a peer review are still in their infancy. To enhance
these, a systematic literature review (SLR) of academic and
nonacademic literature was conducted on city resilience peer
reviews. Thirtythree attributes of resilience are
identified, which provides useful insights into how research
and practice can inform risk governance, and utilise peer
reviews, to drive meaningful change. Moreover, it situates
the challenges associated with resilience building tools
within risk governance to support the development of
interdisciplinary perspectives for integrated city resilience
frameworks. Results of this research have been used to
develop a peer review methodology and an international
standard on conducting peer reviews for disaster risk
reduction.
Keywords: city resilience, city peer review, disaster risk governance

[173]

John E. Beasley.
ORLibrary: distributing test problems by electronic mail.
Journal of the Operational Research Society, pages 1069–1072,
1990.
Currently available from
http://people.brunel.ac.uk/~mastjjb/jeb/info.html.
[ bib ]

[174]

John E. Beasley.
Heuristic algorithms for the unconstrained binary quadratic
programming problem.
Technical report, The Management School, Imperial College, London,
England, 1998.
[ bib ]

[175]

S. Becker, J. Gottlieb, and Thomas Stützle.
Applications of Racing Algorithms: An Industrial Perspective.
In E.G. Talbi, P. Liardet, P. Collet, E. Lutton, and M. Schoenauer,
editors, Artificial Evolution, volume 3871 of Lecture Notes in
Computer Science, pages 271–283, Heidelberg, 2005. Springer.
[ bib ]

[176]

David D. Bedworth and James E. Bailey.
Integrated Production Control Systems: Management, Analysis,
Design, volume 2.
John Wiley & Sons, New York, NY, 1982.
[ bib ]

[177]

Andreas Beham, Michael Affenzeller, and Stefan Wagner.
Instancebased algorithm selection on quadratic assignment
problem landscapes.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2017, pages
1471–1478, New York, NY, 2017. ACM Press.
[ bib ]

[178]

J. Behnamian and S.M.T. Fatemi Ghomi.
Hybrid Flowshop Scheduling with Machine and Resourcedependent
Processing Times.
Applied Mathematical Modelling, 35(3):1107–1123, 2011.
[ bib ]

[179]

Richard Bellman.
The theory of dynamic programming.
Bulletin of the American Mathematical Society, 60:503–515,
1954.
[ bib ]

[180]

Valerie Belton, Jürgen Branke, Petri Eskelinen, Salvatore Greco, Julián
Molina, Francisco Ruiz, and Roman Slowiński.
Interactive Multiobjective Optimization from a Learning
Perspective.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science, pages 405–433.
Springer, Heidelberg, 2008.
[ bib 
DOI ]

[181]

Ruggero Bellio, Sara Ceschia, Luca Di Gaspero, Andrea Schaerf, and Tommaso
Urli.
Featurebased tuning of simulated annealing applied to the
curriculumbased course timetabling problem.
Computers & Operations Research, 65:83–92, 2016.
[ bib ]

[182]

Nacim Belkhir, Johann Dréo, Pierre Savéant, and Marc Schoenauer.
Feature Based Algorithm Configuration: A Case Study with
Differential Evolution.
In J. Handl, E. Hart, P. R. Lewis, M. LópezIbáñez,
G. Ochoa, and B. Paechter, editors, Parallel Problem Solving from Nature
– PPSN XIV, volume 9921 of Lecture Notes in Computer Science, pages
156–166, Heidelberg, 2016. Springer.
[ bib 
DOI ]

[183]

Nacim Belkhir, Johann Dréo, Pierre Savéant, and Marc Schoenauer.
Per Instance Algorithm Configuration of CMAES with Limited
Budget.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pages 681–688. ACM Press,
New York, NY, 2017.
[ bib ]

[184]

J. L. Bentley.
Experiments on Traveling Salesman Heuristics.
In D. S. Johnson, editor, Proceedings of the First Annual
ACMSIAM Symposium on Discrete Algorithms, pages 91–99. Society for
Industrial and Applied Mathematics, Philadelphia, PA, USA, 1990.
[ bib ]

[185]

J. L. Bentley.
Fast Algorithms for Geometric Traveling Salesman Problems.
ORSA Journal on Computing, 4(4):387–411, 1992.
[ bib ]

[186]

Nawal Benabbou, Cassandre Leroy, and Thibaut Lust.
An Interactive RegretBased Genetic Algorithm for Solving
MultiObjective Combinatorial Optimization Problems.
In Proceedings of the AAAI Conference on Artificial
Intelligence, pages 2335–2342. AAAI Press, 2020.
[ bib 
DOI ]
Keywords: interactive, multiobjective, decisionmakers

[187]

Una Benlic and JinKao Hao.
Breakout Local Search for the Quadratic Assignment Problem.
Applied Mathematics and Computation, 219(9):4800–4815, 2013.
[ bib ]

[188]

Calem J. Bendell, Shalon Liu, Tristan AumentadoArmstrong, Bogdan Istrate,
Paul T. Cernek, Samuel Khan, Sergiu Picioreanu, Michael Zhao, and Robert A.
Murgita.
Transient proteinprotein interface prediction: datasets,
features, algorithms, and the RADT predictor.
BMC Bioinformatics, 15:82, 2014.
[ bib ]

[189]

Alexander Javier Benavides and Marcus Ritt.
Iterated Local Search Heuristics for Minimizing Total Completion
Time in Permutation and Nonpermutation Flow Shops.
In R. I. Brafman, C. Domshlak, P. Haslum, and S. Zilberstein,
editors, Proceedings of the TwentyFifth International Conference on
Automated Planning and Scheduling, ICAPS 2015, pages 34–41. AAAI Press,
Menlo Park, CA, 2015.
[ bib ]
Keywords: irace

[190]

Alexander Javier Benavides and Marcus Ritt.
Two Simple and Effective Heuristics for Minimizing the Makespan
in Nonpermutation Flow Shops.
Computers & Operations Research, 66:160–169, 2016.
[ bib 
DOI ]

[191]

Stefano Benedettini, Andrea Roli, and Christian Blum.
A Randomized Iterated Greedy Algorithm for the Founder Sequence
Reconstruction Problem.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pages 37–51. Springer, Heidelberg, 2010.
[ bib 
DOI ]

[192]

Stefano Benedettini, Andrea Roli, and Luca Di Gaspero.
Twolevel ACO for Haplotype Inference under Pure Parsimony.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 6th International Conference, ANTS 2008, volume 5217 of
Lecture Notes in Computer Science, pages 179–190. Springer, Heidelberg,
2008.
[ bib ]

[193]

J. F. Benders.
Partitioning Procedures for Solving Mixedvariables Programming
Problems.
Numerische Mathematik, 4(3):238–252, 1962.
[ bib ]

[194]

D. Bertsekas.
Dynamic Programming and Optimal Control.
Athena Scientific, Belmont, MA, 1995.
[ bib ]

[195]

D. Bertsekas.
Network Optimization: Continuous and Discrete Models.
Athena Scientific, Belmont, MA, 1998.
[ bib ]

[196]

James S. Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl.
Algorithms for HyperParameter Optimization.
In J. ShaweTaylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and
K. Q. Weinberger, editors, Advances in Neural Information Processing
Systems (NIPS 24), pages 2546–2554. Curran Associates, Red Hook, NY, 2011.
[ bib 
http ]

[197]

James S. Bergstra and Yoshua Bengio.
Random Search for HyperParameter Optimization.
Journal of Machine Learning Research, 13:281–305, 2012.
[ bib 
eprint ]
Grid search and manual search are the most widely
used strategies for hyperparameter
optimization. This paper shows empirically and
theoretically that randomly chosen trials are more
efficient for hyperparameter optimization than
trials on a grid. Empirical evidence comes from a
comparison with a large previous study that used
grid search and manual search to configure neural
networks and deep belief networks. Compared with
neural networks configured by a pure grid search, we
find that random search over the same domain is able
to find models that are as good or better within a
small fraction of the computation time. Granting
random search the same computational budget, random
search finds better models by effectively searching
a larger, less promising configuration
space. Compared with deep belief networks configured
by a thoughtful combination of manual search and
grid search, purely random search over the same
32dimensional configuration space found
statistically equal performance on four of seven
data sets, and superior performance on one of
seven. A Gaussian process analysis of the function
from hyperparameters to validation set performance
reveals that for most data sets only a few of the
hyperparameters really matter, but that different
hyperparameters are important on different data
sets. This phenomenon makes grid search a poor
choice for configuring algorithms for new data
sets. Our analysis casts some light on why recent
"High Throughput" methods achieve surprising
success: they appear to search through a large number
of hyperparameters because most hyperparameters do
not matter much. We anticipate that growing interest
in large hierarchical models will place an
increasing burden on techniques for hyperparameter
optimization; this work shows that random search is
a natural baseline against which to judge progress
in the development of adaptive (sequential)
hyperparameter optimization algorithms.

[198]

Hughes Bersini, Marco Dorigo, S. Langerman, G. Seront, and L. M. Gambardella.
Results of the First International Contest on Evolutionary
Optimisation.
In T. Bäck, T. Fukuda, and Z. Michalewicz, editors,
Proceedings of the 1996 IEEE International Conference on Evolutionary
Computation (ICEC'96), pages 611–615, Piscataway, NJ, 1996. IEEE Press.
[ bib ]

[199]

Loïc Berger, Johannes Emmerling, and Massimo Tavoni.
Managing catastrophic climate risks under model uncertainty
aversion.
Management Science, 63(3):749–765, 2017.
[ bib ]

[200]

Livio Bertacco, Matteo Fischetti, and Andrea Lodi.
A feasibility pump heuristic for general mixedinteger
problems.
Discrete Optimization, 4(1):63–76, 2007.
[ bib ]

[201]

Dimitris Bertsimas and Nathan Kallus.
From predictive to prescriptive analytics.
Management Science, 66(3):1025–1044, 2020.
[ bib ]

[202]

Felix Berkenkamp, Andreas Krause, and Angela P. Schoellig.
Bayesian Optimization with Safety Constraints: Safe and
Automatic Parameter Tuning in Robotics.
Arxiv preprint arXiv:1602.04450, 2016.
[ bib 
http ]

[203]

Felix Berkenkamp, Angela P. Schoellig, and Andreas Krause.
Safe controller optimization for quadrotors with Gaussian
processes.
In 2016 IEEE International Conference on Robotics and Automation
(ICRA), pages 491–496. IEEE, 2016.
[ bib 
DOI ]

[204]

Dimitri P. Bertsekas, John N. Tsitsiklis, and Cynara Wu.
Rollout Algorithms for Combinatorial Optimization.
Journal of Heuristics, 3(3):245–262, 1997.
[ bib ]

[205]

Judith O. Berkey and Pearl Y. Wang.
Twodimensional finite binpacking algorithms.
Journal of the Operational Research Society, 38(5):423–429,
1987.
[ bib 
DOI ]

[206]

James S. Bergstra, Daniel Yasmin, and David Cox.
Making a science of model search: Hyperparameter optimization in
hundreds of dimensions for vision architectures.
In S. Dasgupta and D. McAllester, editors, Proceedings of the
30th International Conference on Machine Learning, ICML 2013, volume 28,
pages 115–123, 2013.
[ bib 
http ]

[207]

Matthijs L. den Besten, Thomas Stützle, and Marco Dorigo.
Ant Colony Optimization for the Total Weighted Tardiness
Problem.
In M. Schoenauer et al., editors, Parallel Problem Solving from
Nature – PPSN VI, volume 1917 of Lecture Notes in Computer Science,
pages 611–620, Heidelberg, 2000. Springer.
[ bib ]

[208]

Matthijs L. den Besten, Thomas Stützle, and Marco Dorigo.
Design of Iterated Local Search Algorithms: An Example
Application to the Single Machine Total Weighted Tardiness Problem.
In E. J. W. Boers et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2001, volume 2037 of Lecture
Notes in Computer Science, pages 441–452. Springer, Heidelberg, 2001.
[ bib ]

[209]

Nicola Beume, Carlos M. Fonseca, Manuel LópezIbáñez, Luís
Paquete, and Jan Vahrenhold.
On the complexity of computing the hypervolume indicator.
IEEE Transactions on Evolutionary Computation,
13(5):1075–1082, 2009.
[ bib 
DOI ]
The goal of multiobjective optimization is to find
a set of best compromise solutions for typically
conflicting objectives. Due to the complex nature of
most reallife problems, only an approximation to
such an optimal set can be obtained within
reasonable (computing) time. To compare such
approximations, and thereby the performance of
multiobjective optimizers providing them, unary
quality measures are usually applied. Among these,
the hypervolume indicator (or
Smetric) is of particular relevance due to
its favorable properties. Moreover, this indicator
has been successfully integrated into stochastic
optimizers, such as evolutionary algorithms, where
it serves as a guidance criterion for finding good
approximations to the Pareto front. Recent results
show that computing the hypervolume indicator can be
seen as solving a specialized version of Klee's
Measure Problem. In general, Klee's Measure Problem
can be solved with O(n logn +
n^{d/2}logn) comparisons for an input instance of
size n in d dimensions; as of this writing, it
is unknown whether a lower bound higher than
Ω(n logn) can be proven. In this article,
we derive a lower bound of Ω(nlogn) for the
complexity of computing the hypervolume indicator in
any number of dimensions d>1 by reducing the
socalled UniformGap problem to it. For
the three dimensional case, we also present a
matching upper bound of O(nlogn)
comparisons that is obtained by extending an
algorithm for finding the maxima of a point set.

[210]

Nicola Beume, Boris Naujoks, and Michael T. M. Emmerich.
SMSEMOA: Multiobjective selection based on dominated
hypervolume.
European Journal of Operational Research, 181(3):1653–1669,
2007.
[ bib 
DOI ]

[211]

Nicola Beume and Günther Rudolph.
Faster SMetric Calculation by Considering Dominated
Hypervolume as Klee's Measure Problem.
In B. Kovalerchuk, editor, Proceedings of the Second IASTED
Conference on Computational Intelligence, pages 231–236. ACTA Press,
Anaheim, 2006.
[ bib ]

[212]

HansGeorg Beyer and HansPaul Schwefel.
Evolution Strategies: A Comprehensive Introduction.
Natural Computing, 1:3–52, 2002.
[ bib ]

[213]

HansGeorg Beyer, HansPaul Schwefel, and Ingo Wegener.
How to analyse evolutionary algorithms.
Theoretical Computer Science, 287(1):101–130, 2002.
[ bib ]

[214]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Generation of MultiObjective ACO Algorithms for the
Biobjective Knapsack.
In M. Dorigo et al., editors, Swarm Intelligence, 8th
International Conference, ANTS 2012, volume 7461 of Lecture Notes in
Computer Science, pages 37–48. Springer, Heidelberg, 2012.
[ bib 
DOI 
supplementary material ]

[215]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Generation of MOACO Algorithms for the Biobjective
Bidimensional Knapsack Problem: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012008/, 2012.
[ bib ]

[216]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An Analysis of Local Search for the Biobjective Bidimensional
Knapsack: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012016/, 2013.
[ bib ]

[217]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Deconstructing MultiObjective Evolutionary Algorithms: An
Iterative Analysis on the Permutation Flowshop: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2013010/, 2013.
[ bib ]

[218]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An Analysis of Local Search for the Biobjective Bidimensional
Knapsack Problem.
In M. Middendorf and C. Blum, editors, Proceedings of EvoCOP
2013 – 13th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7832 of Lecture Notes in Computer Science, pages
85–96. Springer, Heidelberg, 2013.
[ bib 
DOI ]

[219]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic ComponentWise Design of MultiObjective
Evolutionary Algorithms.
Technical Report TR/IRIDIA/2014012, IRIDIA, Université Libre de
Bruxelles, Belgium, August 2014.
[ bib ]

[220]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Deconstructing MultiObjective Evolutionary Algorithms: An
Iterative Analysis on the Permutation Flowshop.
In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, and J. L.
Walteros, editors, Learning and Intelligent Optimization, 8th
International Conference, LION 8, volume 8426 of Lecture Notes in
Computer Science, pages 57–172. Springer, Heidelberg, 2014.
[ bib 
DOI 
supplementary material ]

[221]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Design of Evolutionary Algorithms for MultiObjective
Combinatorial Optimization.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, PPSN 2014, volume 8672 of Lecture Notes in Computer
Science, pages 508–517. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[222]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Design of Evolutionary Algorithms for MultiObjective
Combinatorial Optimization.
http://iridia.ulb.ac.be/supp/IridiaSupp2014007/, 2014.
[ bib ]

[223]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic ComponentWise Design of MultiObjective Evolutionary
Algorithms.
http://iridia.ulb.ac.be/supp/IridiaSupp2014010/, 2015.
[ bib ]

[224]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
To DE or Not to DE? Multiobjective Differential Evolution
Revisited from a ComponentWise Perspective: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2015001/, 2015.
[ bib ]

[225]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
To DE or Not to DE? Multiobjective Differential Evolution
Revisited from a ComponentWise Perspective.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pages 48–63.
Springer, Heidelberg, 2015.
[ bib 
DOI ]

[226]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Comparing DecompositionBased and Automatically
ComponentWise Designed MultiObjective Evolutionary Algorithms.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pages 396–410,
Heidelberg, 2015. Springer.
[ bib 
DOI ]

[227]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic ComponentWise Design of MultiObjective Evolutionary
Algorithms.
IEEE Transactions on Evolutionary Computation, 20(3):403–417,
2016.
[ bib 
DOI 
supplementary material ]

[228]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
http://iridia.ulb.ac.be/supp/IridiaSupp2015007/, 2017.
[ bib ]

[229]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
Technical Report TR/IRIDIA/2017005, IRIDIA, Université Libre de
Bruxelles, Belgium, February 2017.
[ bib ]

[230]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
Evolutionary Computation, 26(4):621–656, 2018.
[ bib 
DOI 
supplementary material ]
Research on multiobjective evolutionary algorithms (MOEAs)
has produced over the past decades a large number of
algorithms and a rich literature on performance assessment
tools to evaluate and compare them. Yet, newly proposed MOEAs
are typically compared against very few, often a decade older
MOEAs. One reason for this apparent contradiction is the lack
of a common baseline for comparison, with each subsequent
study often devising its own experimental scenario, slightly
different from other studies. As a result, the state of the
art in MOEAs is a disputed topic. This article reports a
systematic, comprehensive evaluation of a large number of
MOEAs that covers a wide range of experimental scenarios. A
novelty of this study is the separation between the
higherlevel algorithmic components related to
multiobjective optimization (MO), which characterize each
particular MOEA, and the underlying parameterssuch as
evolutionary operators, population size, etc.whose
configuration may be tuned for each scenario. Instead of
relying on a common or "default" parameter configuration that
may be lowperforming for particular MOEAs or scenarios and
unintentionally biased, we tune the parameters of each MOEA
for each scenario using automatic algorithm configuration
methods. Our results confirm some of the assumed knowledge in
the field, while at the same time they provide new insights
on the relative performance of MOEAs for manyobjective
problems. For example, under certain conditions,
indicatorbased MOEAs are more competitive for such problems
than previously assumed. We also analyze problemspecific
features affecting performance, the agreement between
performance metrics, and the improvement of tuned
configurations over the default configurations used in the
literature. Finally, the data produced is made publicly
available to motivate further analysis and a baseline for
future comparisons.

[231]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An Empirical Assessment of the Properties of Inverted
Generational Distance Indicators on Multi and Manyobjective Optimization.
In H. Trautmann, G. Rudolph, K. Klamroth, O. Schütze, M. M.
Wiecek, Y. Jin, and C. Grimme, editors, Evolutionary Multicriterion
Optimization, EMO 2017, Lecture Notes in Computer Science, pages 31–45.
Springer International Publishing, Cham, Switzerland, 2017.
[ bib 
DOI ]

[232]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An empirical assessment of the properties of inverted
generational distance indicators on multi and manyobjective optimization:
Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2016006/, 2016.
[ bib ]

[233]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatically Designing StateoftheArt Multi and
ManyObjective Evolutionary Algorithms.
Evolutionary Computation, 28(2):195–226, 2020.
[ bib 
DOI 
supplementary material ]
A recent comparison of wellestablished multiobjective
evolutionary algorithms (MOEAs) has helped better identify
the current stateoftheart by considering (i) parameter
tuning through automatic configuration, (ii) a wide range of
different setups, and (iii) various performance
metrics. Here, we automatically devise MOEAs with verified
stateoftheart performance for multi and manyobjective
continuous optimization. Our work is based on two main
considerations. The first is that highperforming algorithms
can be obtained from a configurable algorithmic framework in
an automated way. The second is that multiple performance
metrics may be required to guide this automatic design
process. In the first part of this work, we extend our
previously proposed algorithmic framework, increasing the
number of MOEAs, underlying evolutionary algorithms, and
search paradigms that it comprises. These components can be
combined following a general MOEA template, and an automatic
configuration method is used to instantiate highperforming
MOEA designs that optimize a given performance metric and
present stateoftheart performance. In the second part, we
propose a multiobjective formulation for the automatic MOEA
design, which proves critical for the context of
manyobjective optimization due to the disagreement of
established performance metrics. Our proposed formulation
leads to an automatically designed MOEA that presents
stateoftheart performance according to a set of metrics,
rather than a single one.

[234]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatically Designing StateoftheArt Multi and
ManyObjective Evolutionary Algorithms: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2016004/, 2019.
[ bib ]

[235]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Archiver Effects on the Performance of Stateoftheart Multi
and Manyobjective Evolutionary Algorithms: Supplementary material.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019. ACM Press, New York, NY, 2019.
[ bib 
DOI 
supplementary material ]

[236]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Archiver Effects on the Performance of Stateoftheart Multi
and Manyobjective Evolutionary Algorithms: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2019004/, 2019.
[ bib ]

[237]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Configuration of Multiobjective Optimizers and
Multiobjective Configuration.
In T. BartzBeielstein, B. Filipič, P. Korošec, and E.G.
Talbi, editors, HighPerformance SimulationBased Optimization, pages
69–92. Springer International Publishing, Cham, Switzerland, 2020.
[ bib 
DOI ]
Heuristic optimizers are an important tool in academia and industry, and their performanceoptimizing configuration requires a significant amount of expertise. As the proper configuration of algorithms is a crucial aspect in the engineering of heuristic algorithms, a significant research effort has been dedicated over the last years towards moving this step to the computer and, thus, make it automatic. These research efforts go way beyond tuning only numerical parameters of already fully defined algorithms, but exploit automatic configuration as a means for automatic algorithm design. In this chapter, we review two main aspects where the research on automatic configuration and multiobjective optimization intersect. The first is the automatic configuration of multiobjective optimizers, where we discuss means and specific approaches. In addition, we detail a case study that shows how these approaches can be used to design new, highperforming multiobjective evolutionary algorithms. The second aspect is the research on multiobjective configuration, that is, the possibility of using multiple performance metrics for the evaluation of algorithm configurations. We highlight some few examples in this direction.

[238]

Leonardo C. T. Bezerra.
A componentwise approach to multiobjective evolutionary
algorithms: from flexible frameworks to automatic design.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2016.
[ bib ]
Supervised by Thomas Stützle and Manuel LópezIbáñez

[239]

Leonora Bianchi, Mauro Birattari, M. Manfrin, M. Mastrolilli, Luís Paquete,
O. RossiDoria, and Tommaso Schiavinotto.
Hybrid Metaheuristics for the Vehicle Routing Problem with
Stochastic Demands.
Journal of Mathematical Modelling and Algorithms, 5(1):91–110,
2006.
[ bib ]

[240]

Leonora Bianchi, Marco Dorigo, L. M. Gambardella, and Walter J. Gutjahr.
A survey on metaheuristics for stochastic combinatorial
optimization.
Natural Computing, 8(2):239–287, 2009.
[ bib ]

[241]

Leonora Bianchi, L. M. Gambardella, and Marco Dorigo.
An Ant Colony Optimization Approach to the Probabilistic
Traveling Salesman Problem.
In J.J. Merelo et al., editors, Parallel Problem Solving from
Nature – PPSN VII, volume 2439 of Lecture Notes in Computer Science,
pages 883–892, Heidelberg, 2002. Springer.
[ bib ]

[242]

Armin Biere.
Yet another Local Search Solver and Lingeling and Friends
Entering the SAT Competition 2014.
In A. Belov, D. Diepold, M. Heule, and M. Järvisalo, editors,
Proceedings of SAT Competition 2014: Solver and Benchmark Descriptions,
volume B20142 of Science Series of Publications B, pages 39–40.
University of Helsinki, 2014.
[ bib ]

[243]

André Biedenkapp, H. Furkan Bozkurt, Theresa Eimer, Frank Hutter, and
Marius Thomas Lindauer.
Dynamic Algorithm Configuration: Foundation of a New
MetaAlgorithmic Framework.
In G. D. Giacomo, A. Catala, B. Dilkina, M. Milano, S. Barro,
A. BugarÃn, and J. Lang, editors, ECAI 2020, volume 325 of
Frontiers in Artificial Intelligence and Applications, pages 427–434. IOS
Press, 2020.
[ bib 
DOI ]

[244]

André Biedenkapp, Marius Thomas Lindauer, Katharina Eggensperger, Frank
Hutter, Chris Fawcett, and Holger H. Hoos.
Efficient Parameter Importance Analysis via Ablation with
Surrogates.
In S. P. Singh and S. Markovitch, editors, Proceedings of the
AAAI Conference on Artificial Intelligence. AAAI Press, February 2017.
[ bib 
http ]

[245]

André Biedenkapp, Joshua Marben, Marius Thomas Lindauer, and Frank Hutter.
Cave: Configuration assessment, visualization and evaluation.
In R. Battiti, M. Brunato, I. Kotsireas, and P. M. Pardalos, editors,
Learning and Intelligent Optimization, 12th International Conference,
LION 12, volume 11353 of Lecture Notes in Computer Science, pages
115–130, Cham, Switzerland, 2018. Springer.
[ bib ]

[246]

George Bilchev and Ian C. Parmee.
The Ant Colony Metaphor for Searching Continuous Design Spaces.
In T. C. Fogarty, editor, Evolutionary Computing, AISB
Workshop, volume 993 of Lecture Notes in Computer Science, pages
25–39. Springer, Berlin, Germany, 1995.
[ bib 
DOI ]

[247]

M. Binois, D. Ginsbourger, and O. Roustant.
Quantifying uncertainty on Pareto fronts with Gaussian
process conditional simulations.
European Journal of Operational Research, 243(2):386–394,
2015.
[ bib 
DOI ]
Multiobjective optimization algorithms aim at finding
Paretooptimal solutions. Recovering Pareto fronts or Pareto
sets from a limited number of function evaluations are
challenging problems. A popular approach in the case of
expensivetoevaluate functions is to appeal to
metamodels. Kriging has been shown efficient as a base for
sequential multiobjective optimization, notably through
infill sampling criteria balancing exploitation and
exploration such as the Expected Hypervolume
Improvement. Here we consider Kriging metamodels not only for
selecting new points, but as a tool for estimating the whole
Pareto front and quantifying how much uncertainty remains on
it at any stage of Krigingbased multiobjective optimization
algorithms. Our approach relies on the Gaussian process
interpretation of Kriging, and bases upon conditional
simulations. Using concepts from random set theory, we
propose to adapt the Vorob'ev expectation and deviation to
capture the variability of the set of nondominated
points. Numerical experiments illustrate the potential of the
proposed workflow, and it is shown on examples how Gaussian
process simulations and the estimated Vorob'ev deviation can
be used to monitor the ability of Krigingbased
multiobjective optimization algorithms to accurately learn
the Pareto front.
Keywords: Attainment function, Expected Hypervolume Improvement,
Kriging, Multiobjective optimization, Vorob'ev expectation

[248]

Mauro Birattari, Prasanna Balaprakash, and Marco Dorigo.
The ACO/FRACE algorithm for combinatorial optimization under
uncertainty.
In K. F. Doerner, M. Gendreau, P. Greistorfer, W. J. Gutjahr, R. F.
Hartl, and M. Reimann, editors, Metaheuristics – Progress in Complex
Systems Optimization, volume 39 of Operations Research/Computer Science
Interfaces Series, pages 189–203. Springer, New York, NY, 2006.
[ bib ]

[249]

Mauro Birattari, Prasanna Balaprakash, Thomas Stützle, and Marco Dorigo.
Estimation Based Local Search for Stochastic Combinatorial
Optimization.
INFORMS Journal on Computing, 20(4):644–658, 2008.
[ bib ]

[250]

Mauro Birattari, Marco Chiarandini, Marco Saerens, and Thomas Stützle.
Learning Graphical Models for Algorithm Configuration.
In T. Berthold, A. M. Gleixner, S. Heinz, and T. Koch, editors,
Integration of AI and OR Techniques in Contraint Programming for
Combinatorial Optimization Problems, Lecture Notes in Computer Science.
Springer, Heidelberg, 2011.
[ bib ]

[251]

Mauro Birattari, Gianni A. Di Caro, and Marco Dorigo.
Toward the formal foundation of Ant Programming.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 188–201. Springer, Heidelberg, 2002.
[ bib ]

[252]

Steven Bird, Ewan Klein, and Edward Loper.
Natural language processing with Python: analyzing text with
the natural language toolkit.
O'Reilly Media, Inc., 2009.
[ bib ]

[253]

Mauro Birattari, Paola Pellegrini, and Marco Dorigo.
On the invariance of ant colony optimization.
IEEE Transactions on Evolutionary Computation, 11(6):732–742,
2007.
[ bib 
DOI ]

[254]

Mauro Birattari, Thomas Stützle, Luís Paquete, and Klaus Varrentrapp.
A Racing Algorithm for Configuring Metaheuristics.
In W. B. Langdon et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2002, pages 11–18. Morgan
Kaufmann Publishers, San Francisco, CA, 2002.
[ bib 
eprint ]
Keywords: Frace

[255]

Mauro Birattari, Zhi Yuan, Prasanna Balaprakash, and Thomas Stützle.
FRace and Iterated FRace: An Overview.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 311–336. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Keywords: Frace, iterated Frace, irace, tuning

[256]

Mauro Birattari, Zhi Yuan, Prasanna Balaprakash, and Thomas Stützle.
Parameter Adaptation in Ant Colony Optimization.
In M. Caserta and S. Voß, editors, Proceedings of MIC 2009,
the 8th Metaheuristics International Conference, Hamburg, Germany, 2010.
University of Hamburg.
[ bib ]

[257]

Mauro Birattari, M. Zlochin, and Marco Dorigo.
Towards a theory of practice in metaheuristics design: A machine
learning perspective.
Theoretical Informatics and Applications, 40(2):353–369, 2006.
[ bib ]

[258]

Mauro Birattari.
Tuning Metaheuristics: A Machine Learning Perspective, volume
197 of Studies in Computational Intelligence.
Springer, Berlin/Heidelberg, 2009.
[ bib 
DOI ]

[259]

Mauro Birattari.
The Problem of Tuning Metaheuristics as Seen from a Machine
Learning Perspective.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2004.
[ bib ]
Supervised by Marco Dorigo

[260]

Francesco Biscani, Dario Izzo, and Chit Hong Yam.
A Global Optimisation Toolbox for Massively Parallel Engineering
Optimisation.
In Astrodynamics Tools and Techniques (ICATT 2010), 4th
International Conference on, 2010.
[ bib 
http ]
Keywords: PaGMO

[261]

Francesco Biscani, Dario Izzo, and Chit Hong Yam.
A Global Optimisation Toolbox for Massively Parallel Engineering
Optimisation.
Arxiv preprint arXiv:1004.3824, 2010.
[ bib 
http ]
A software platform for global optimisation, called PaGMO,
has been developed within the Advanced Concepts Team (ACT) at
the European Space Agency, and was recently released as an
opensource project. PaGMO is built to tackle
highdimensional global optimisation problems, and it has
been successfully used to find solutions to reallife
engineering problems among which the preliminary design of
interplanetary spacecraft trajectories  both chemical
(including multiple flybys and deepspace maneuvers) and
lowthrust (limited, at the moment, to single phase
trajectories), the inverse design of nanostructured
radiators and the design of nonreactive controllers for
planetary rovers. Featuring an arsenal of global and local
optimisation algorithms (including genetic algorithms,
differential evolution, simulated annealing, particle swarm
optimisation, compass search, improved harmony search, and
various interfaces to libraries for local optimisation such
as SNOPT, IPOPT, GSL and NLopt), PaGMO is at its core a C++
library which employs an objectoriented architecture
providing a clean and easilyextensible optimisation
framework. Adoption of multithreaded programming ensures the
efficient exploitation of modern multicore architectures and
allows for a straightforward implementation of the island
model paradigm, in which multiple populations of candidate
solutions asynchronously exchange information in order to
speedup and improve the optimisation process. In addition to
the C++ interface, PaGMO's capabilities are exposed to the
highlevel language Python, so that it is possible to easily
use PaGMO in an interactive session and take advantage of the
numerous scientific Python libraries available.
Keywords: PaGMO

[262]

Bernd Bischl, Pascal Kerschke, Lars Kotthoff, Marius Thomas Lindauer, Yuri
Malitsky, Alexandre Fréchette, Holger H. Hoos, Frank Hutter, Kevin
LeytonBrown, Kevin Tierney, and Joaquin Vanschoren.
ASlib: A Benchmark Library for Algorithm Selection.
Artificial Intelligence, 237:41–58, 2016.
[ bib ]

[263]

Bernd Bischl, Michel Lang, Lars Kotthoff, Julia Schiffner, Jakob Richter, Erich
Studerus, Giuseppe Casalicchio, and Zachary M. Jones.
mlr: Machine Learning in R.
Journal of Machine Learning Research, 17(170):1–5, 2016.
[ bib 
eprint ]

[264]

Bernd Bischl, Olaf Mersmann, Heike Trautmann, and Mike Preuss.
Algorithm Selection Based on Exploratory Landscape Analysis and
Costsensitive Learning.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pages 313–320. ACM
Press, New York, NY, 2012.
[ bib ]
Keywords: continuous optimization, landscape analysis, algorithm selection

[265]

Christopher M. Bishop.
Pattern recognition and machine learning.
Springer, 2006.
[ bib ]

[266]

Erdem Biyik, Jonathan Margoliash, Shahrouz Ryan Alimo, and Dorsa
Sadigh.
Efficient and Safe Exploration in Deterministic Markov
Decision Processes with Unknown Transition Models.
In 2019 American Control Conference (ACC), pages 1792–1799.
IEEE, 2019.
[ bib 
DOI ]

[267]

Xavier Blasco, Juan M. Herrero, Javier Sanchis, and Manuel Martínez.
A new graphical visualization of ndimensional Pareto front
for decisionmaking in multiobjective optimization.
Information Sciences, 178(20):3908–3924, 2008.
[ bib ]

[268]

Craig Blackmore, Oliver Ray, and Kerstin Eder.
Automatically Tuning the GCC Compiler to Optimize the
Performance of Applications Running on Embedded Systems.
Arxiv preprint arXiv:1703.08228, 2017.
[ bib 
http ]

[269]

María J. Blesa and Christian Blum.
Ant Colony Optimization for the Maximum EdgeDisjoint Paths
Problem.
In G. R. Raidl et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2004, volume 3005 of Lecture
Notes in Computer Science, pages 160–169. Springer, Heidelberg, 2004.
[ bib ]

[270]

María J. Blesa and Christian Blum.
Finding edgedisjoint paths in networks by means of artificial
ant colonies.
Journal of Mathematical Modelling and Algorithms,
6(3):361–391, 2007.
[ bib ]

[271]

Aymeric Blot, Holger H. Hoos, Laetitia Jourdan, MarieEléonore
KessaciMarmion, and Heike Trautmann.
MOParamILS: A Multiobjective Automatic Algorithm
Configuration Framework.
In P. Festa, M. Sellmann, and J. Vanschoren, editors, Learning
and Intelligent Optimization, 10th International Conference, LION 10, volume
10079 of Lecture Notes in Computer Science, pages 32–47. Springer,
Cham, Switzerland, 2016.
[ bib ]

[272]

Aymeric Blot, Laetitia Jourdan, and MarieEléonore KessaciMarmion.
Automatic design of multiobjective local search algorithms:
case study on a biobjective permutation flowshop scheduling problem.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pages 227–234. ACM Press,
New York, NY, 2017.
[ bib 
DOI ]

[273]

Aymeric Blot, Manuel LópezIbáñez, MarieEléonore
KessaciMarmion, and Laetitia Jourdan.
New Initialisation Techniques for MultiObjective Local Search:
Application to the Biobjective Permutation Flowshop.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature  PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pages 323–334.
Springer, Cham, Switzerland, 2018.
[ bib 
DOI ]

[274]

Aymeric Blot, Alexis Pernet, Laetitia Jourdan, MarieEléonore
KessaciMarmion, and Holger H. Hoos.
Automatically Configuring Multiobjective Local Search Using
Multiobjective Optimisation.
In H. Trautmann, G. Rudolph, K. Klamroth, O. Schütze, M. M.
Wiecek, Y. Jin, and C. Grimme, editors, Evolutionary Multicriterion
Optimization, EMO 2017, Lecture Notes in Computer Science, pages 61–76.
Springer International Publishing, Cham, Switzerland, 2017.
[ bib ]

[275]

Christian Blum.
BeamACO—Hybridizing Ant Colony Optimization with Beam
Search: An Application to Open Shop Scheduling.
Computers & Operations Research, 32(6):1565–1591, 2005.
[ bib ]

[276]

Christian Blum.
BeamACO for simple assembly line balancing.
INFORMS Journal on Computing, 20(4):618–627, 2008.
[ bib 
DOI ]

[277]

Christian Blum, J. Bautista, and J. Pereira.
BeamACO applied to assembly line balancing.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 5th International Workshop, ANTS 2006, volume 4150 of
Lecture Notes in Computer Science, pages 96–107. Springer, Heidelberg,
2006.
[ bib 
DOI ]

[278]

Christian Blum, María J. Blesa, and Manuel LópezIbáñez.
Beam Search for the Longest Common Subsequence Problem.
Technical Report LSI0829, Department LSI, Universitat
Politècnica de Catalunya, 2008.
Published in Computers & Operations
Research [279].
[ bib ]

[279]

Christian Blum, María J. Blesa, and Manuel LópezIbáñez.
Beam search for the longest common subsequence problem.
Computers & Operations Research, 36(12):3178–3186, 2009.
[ bib 
DOI ]
The longest common subsequence problem is a classical string
problem that concerns finding the common part of a set of
strings. It has several important applications, for example,
pattern recognition or computational biology. Most research
efforts up to now have focused on solving this problem
optimally. In comparison, only few works exist dealing with
heuristic approaches. In this work we present a deterministic
beam search algorithm. The results show that our algorithm
outperforms the current stateoftheart approaches not only
in solution quality but often also in computation time.

[280]

Christian Blum, Borja Calvo, and María J. Blesa.
FrogCOL and FrogMIS: New Decentralized Algorithms for
Finding Large Independent Sets in Graphs.
Swarm Intelligence, 9(23):205–227, 2015.
[ bib 
DOI ]
Keywords: irace

[281]

Christian Blum, Carlos Cotta, Antonio J. Fernández, and J. E. Gallardo.
A probabilistic beam search algorithm for the shortest common
supersequence problem.
In C. Cotta et al., editors, Proceedings of EvoCOP 2007 –
Seventh European Conference on Evolutionary Computation in Combinatorial
Optimisation, volume 4446 of Lecture Notes in Computer Science, pages
36–47. Springer, Berlin, Germany, 2007.
[ bib ]

[282]

Christian Blum and Marco Dorigo.
The hypercube framework for ant colony optimization.
IEEE Transactions on Systems, Man, and Cybernetics – Part B,
34(2):1161–1172, 2004.
[ bib ]

[283]

Christian Blum and Marco Dorigo.
Search Bias in Ant Colony Optimization: On the Role of
CompetitionBalanced Systems.
IEEE Transactions on Evolutionary Computation, 9(2):159–174,
2005.
[ bib ]

[284]

Christian Blum and Manuel LópezIbáñez.
Ant Colony Optimization.
In The Industrial Electronics Handbook: Intelligent Systems.
CRC Press, 2nd edition, 2011.
[ bib 
http ]

[285]

Christian Blum and M. Mastrolilli.
Using Branch & Bound Concepts in ConstructionBased
Metaheuristics: Exploiting the Dual Problem Knowledge.
In T. BartzBeielstein, M. J. Blesa, C. Blum, B. Naujoks, A. Roli,
G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics, volume 4771
of Lecture Notes in Computer Science, pages 123–139. Springer,
Heidelberg, 2007.
[ bib ]

[286]

C. Blum and D. Merkle, editors.
Swarm Intelligence–Introduction and Applications.
Natural Computing Series. Springer Verlag, Berlin, Germany, 2008.
[ bib ]

[287]

Christian Blum and Gabriela Ochoa.
A comparative analysis of two matheuristics by means of merged
local optima networks.
European Journal of Operational Research, 290(1):36–56, 2021.
[ bib ]

[288]

Christian Blum, Pedro Pinacho, Manuel LópezIbáñez, and José A.
Lozano.
Construct, Merge, Solve & Adapt: A New General Algorithm for
Combinatorial Optimization.
Computers & Operations Research, 68:75–88, 2016.
[ bib 
DOI ]
Keywords: irace

[289]

Christian Blum, Jakob Puchinger, Günther R. Raidl, and Andrea Roli.
Hybrid Metaheuristics in Combinatorial Optimization: A Survey.
Applied Soft Computing, 11(6):4135–4151, 2011.
[ bib ]

[290]

Christian Blum and Günther R. Raidl.
Hybrid Metaheuristics—Powerful Tools for Optimization.
Artificial Intelligence: Foundations, Theory, and Algorithms.
Springer, Berlin, Germany, 2016.
[ bib ]

[291]

Christian Blum and Andrea Roli.
Metaheuristics in Combinatorial Optimization: Overview and
Conceptual Comparison.
ACM Computing Surveys, 35(3):268–308, 2003.
[ bib ]

[292]

Christian Blum and Andrea Roli.
Hybrid metaheuristics: an introduction.
In C. Blum, M. J. Blesa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics: An emergent approach for optimization, volume 114 of
Studies in Computational Intelligence, pages 1–30. Springer, Berlin,
Germany, 2008.
[ bib ]

[293]

Christian Blum and M. Sampels.
An Ant Colony Optimization Algorithm for Shop Scheduling
Problems.
Journal of Mathematical Modelling and Algorithms,
3(3):285–308, 2004.
[ bib 
DOI ]

[294]

Christian Blum and M. Yábar Vallès.
Multilevel ant colony optimization for DNA sequencing by
hybridization.
In F. Almeida et al., editors, Hybrid Metaheuristics, volume
4030 of Lecture Notes in Computer Science, pages 94–109. Springer,
Heidelberg, 2006.
[ bib 
DOI ]

[295]

Christian Blum, M. Yábar Vallès, and María J. Blesa.
An ant colony optimization algorithm for DNA sequencing by
hybridization.
Computers & Operations Research, 35(11):3620–3635, 2008.
[ bib ]

[296]

Andrea F. Bocchese, Chris Fawcett, Mauro Vallati, Alfonso E. Gerevini, and
Holger H. Hoos.
Performance robustness of AI planners in the 2014
International Planning Competition.
AI Communications, 31(6):445–463, December 2018.
[ bib 
DOI ]
Solver competitions have been used in many areas of AI to
assess the current state of the art and guide future research
and development. AI planning is no exception, and the
International Planning Competition (IPC) has been frequently
run for nearly two decades. Due to the organisational and
computational burden involved in running these competitions,
solvers are generally compared using a single homogeneous
hardware and software environment for all competitors. To
what extent does the specific choice of hardware and software
environment have an effect on solver performance, and is that
effect distributed equally across the competing solvers? In
this work, we use the competing planners and benchmark
instance sets from the 2014 IPC to investigate these two
questions. We recreate the 2014 IPC Optimal and Agile tracks
on two distinct hardware environments and eight distinct
software environments. We show that solver performance varies
significantly based on the hardware and software environment,
and that this variation is not equal for all
planners. Furthermore, the observed variation is sufficient
to change the competition rankings, including the topranked
planners for some tracks.

[297]

Kenneth D. Boese, Andrew B. Kahng, and Sudhakar Muddu.
A New Adaptive MultiStart Technique for Combinatorial Global
Optimization.
Operations Research Letters, 16(2):101–113, 1994.
[ bib ]
Keywords: bigvalley hypothesis, TSP, landscape analysis

[298]

K. D. Boese.
Models for Iterative Global Optimization.
PhD thesis, University of California, Computer Science Department,
Los Angeles, CA, 1996.
[ bib ]

[299]

Marko Bohanec.
Decision making: a computerscience and informationtechnology
viewpoint.
Interdisciplinary Description of Complex Systems, 7(2):22–37,
2009.
[ bib ]

[300]

Ihor O. Bohachevsky, Mark E. Johnson, and Myron L. Stein.
Generalized Simulated Annealing for Function Optimization.
Technometrics, 28(3):209–217, 1986.
[ bib ]

[301]

Béla Bollobás.
Random Graphs.
Cambridge University Press, New York, NY, 2nd edition, 2001.
[ bib ]

[302]

Grady Booch, James E. Rumbaugh, and Ivar Jacobson.
The Unified Modeling Language User Guide.
AddisonWesley, 2nd edition, 2005.
[ bib ]

[303]

P. C. Borges and Michael Pilegaard Hansen.
A basis for future successes in multiobjective combinatorial
optimization.
Technical Report IMMREP19988, Institute of Mathematical Modelling,
Technical University of Denmark, Lyngby, Denmark, 1998.
[ bib ]

[304]

P. C. Borges.
CHESS  Changing Horizon Efficient Set Search: A simple
principle for multiobjective optimization.
Journal of Heuristics, 6(3):405–418, 2000.
[ bib ]

[305]

Allan Borodin and Ran ElYaniv.
Online computation and competitive analysis.
Cambridge University Press, New York, NY, 1998.
[ bib ]

[306]

Endre Boros, Peter L. Hammer, and Gabriel Tavares.
Local search heuristics for Quadratic Unconstrained Binary
Optimization (QUBO).
Journal of Heuristics, 13(2):99–132, 2007.
[ bib ]

[307]

Michael Borenstein, Larry V. Hedges, Julian P. T. Higgins, and Hannah R.
Rothstein.
Introduction to MetaAnalysis.
Wiley, 2009.
[ bib ]

[308]

JeanCharles de Borda.
Mémoire sur les Élections au Scrutin.
Histoire de l'Académie Royal des Sciences, 1781.
[ bib ]
Keywords: ranking

[309]

Bernhard E. Boser, Isabelle Guyon, and Vladimir Vapnik.
A Training Algorithm for Optimal Margin Classifiers.
In D. Haussler, editor, COLT'92, pages 144–152. ACM Press,
1992.
[ bib 
DOI ]
Proposed SVM

[310]

Hozefa M. Botee and Eric Bonabeau.
Evolving Ant Colony Optimization.
Advances in Complex Systems, 1:149–159, 1998.
[ bib ]

[311]

Marco Botte and Anita Schöbel.
Dominance for multiobjective robust optimization concepts.
European Journal of Operational Research, 273(2):430–440,
2019.
[ bib ]

[312]

Salim Bouamama, Christian Blum, and Abdellah Boukerram.
A Populationbased Iterated Greedy Algorithm for the Minimum
Weight Vertex Cover Problem.
Applied Soft Computing, 12(6):1632–1639, 2012.
[ bib ]

[313]

Géraldine Bous, Philippe Fortemps, François Glineur, and Marc Pirlot.
ACUTA: A novel method for eliciting additive value functions
on the basis of holistic preference statements.
European Journal of Operational Research, 206(2):435–444,
2010.
[ bib ]

[314]

K. Bouleimen and H. Lecocq.
A new efficient simulated annealing algorithm for the
resourceconstrained project scheduling problem and its multiple mode
version.
European Journal of Operational Research, 149(2):268–281,
2003.
[ bib 
DOI ]
This paper describes new simulated annealing (SA)
algorithms for the resourceconstrained project
scheduling problem (RCPSP) and its multiple mode
version (MRCPSP). The objective function
considered is minimisation of the makespan. The
conventional SA search scheme is replaced by a new
design that takes into account the specificity of
the solution space of project scheduling
problems. For RCPSP, the search was based on an
alternated activity and time incrementing process,
and all parameters were set after preliminary
statistical experiments done on test instances. For
MRCPSP, we introduced an original approach using
two embedded search loops alternating activity and
mode neighbourhood exploration. The performance
evaluation done on the benchmark instances available
in the literature proved the efficiency of both
adaptations that are currently among the most
competitive algorithms for these problems.
Keywords: multimode resourceconstrained project scheduling,
project scheduling, simulated annealing

[315]

Paul F. Boulos, Chun Hou Orr, Werner de Schaetzen, J. G. Chatila, Michael
Moore, Paul Hsiung, and Devan Thomas.
Optimal pump operation of water distribution systems using
genetic algorithms.
In AWWA Distribution System Symp., Denver, USA, 2001. American
Water Works Association.
[ bib ]

[316]

V. Bowman and Jr. Joseph.
On the Relationship of the Tchebycheff Norm and the Efficient
Frontier of MultipleCriteria Objectives.
In H. Thiriez and S. Zionts, editors, Multiple Criteria Decision
Making, volume 130 of Lecture Notes in Economics and Mathematical
Systems, pages 76–86. Springer, Berlin/Heidelberg, 1976.
[ bib 
DOI ]

[317]

George E. P. Box and Norman R. Draper.
Response surfaces, mixtures, and ridge analyses.
John Wiley & Sons, 2007.
[ bib ]

[318]

G. E. P. Box, W. G. Hunter, and J. S. Hunter.
Statistics for experimenters: an introduction to design, data
analysis, and model building.
John Wiley & Sons, New York, NY, 1978.
[ bib ]

[319]

A. Brandt.
Multilevel Computations: Review and Recent Developments.
In S. F. McCormick, editor, Multigrid Methods: Theory,
Applications, and Supercomputing, Proceedings of the 3rd Copper Mountain
Conference on Multigrid Methods, volume 110 of Lecture Notes in Pure
and Applied Mathematics, pages 35–62. Marcel Dekker, New York, NY, 1988.
[ bib ]

[320]

L. Bradstreet, L. Barone, L. While, S. Huband, and P. Hingston.
Use of the WFG Toolkit and PISA for Comparison of MOEAs.
In IEEE Symposium on Computational Intelligence in
Multicriteria DecisionMaking, IEEE MCDM, pages 382–389, 2007.
[ bib ]

[321]

Jürgen Branke, Salvatore Corrente, Salvatore Greco, Milosz Kadzinski,
Manuel LópezIbáñez, Vincent Mousseau, Mauro Munerato, and Roman
Slowiński.
BehaviorRealistic Artificial DecisionMakers to Test
PreferenceBased Multiobjective Optimization Method (Working Group
“Machine DecisionMaking”).
In S. Greco, K. Klamroth, J. D. Knowles, and G. Rudolph, editors,
Understanding Complexity in Multiobjective Optimization (Dagstuhl
Seminar 15031), volume 5(1) of Dagstuhl Reports, pages 110–116.
Schloss Dagstuhl–LeibnizZentrum für Informatik, Germany, 2015.
[ bib 
DOI ]
Keywords: multiple criteria decision making, evolutionary
multiobjective optimization

[322]

Yesnier Bravo, Javier Ferrer, Gabriel J. Luque, and Enrique Alba.
Smart Mobility by Optimizing the Traffic Lights: A New Tool for
Traffic Control Centers.
In E. Alba, F. Chicano, and G. J. Luque, editors, Smart Cities
(SmartCT 2016), Lecture Notes in Computer Science, pages 147–156.
Springer, Cham, Switzerland, 2016.
[ bib 
DOI ]
Urban traffic planning is a fertile area of Smart Cities to
improve efficiency, environmental care, and safety, since the
traffic jams and congestion are one of the biggest sources of
pollution and noise. Traffic lights play an important role in
solving these problems since they control the flow of the
vehicular network at the city. However, the increasing number
of vehicles makes necessary to go from a local control at one
single intersection to a holistic approach considering a
large urban area, only possible using advanced computational
resources and techniques. Here we propose HITUL, a system
that supports the decisions of the traffic control managers
in a large urban area. HITUL takes the real traffic
conditions and compute optimal traffic lights plans using
bioinspired techniques and microsimulations. We compare our
system against plans provided by experts. Our solutions not
only enable continuous traffic flows but reduce the
pollution. A case study of Málaga city allows us to
validate the approach and show its benefits for other cities
as well.
Keywords: Multiobjective optimization, Smart mobility, Traffic lights
planning

[323]

Jürgen Branke, Salvatore Greco, Roman Slowiński, and P Zielniewicz.
Interactive evolutionary multiobjective optimization driven by
robust ordinal regression.
Bulletin of the Polish Academy of Sciences: Technical Sciences,
58(3):347–358, 2010.
[ bib 
DOI ]

[324]

S. C. Brailsford, Walter J. Gutjahr, M. S. Rauner, and W. Zeppelzauer.
Combined Discreteevent Simulation and Ant Colony Optimisation
Approach for Selecting Optimal Screening Policies for Diabetic Retinopathy.
Computational Management Science, 4(1):59–83, 2006.
[ bib ]

[325]

Jürgen Branke, T. Kaussler, and H. Schmeck.
Guidance in evolutionary multiobjective optimization.
Advances in Engineering Software, 32:499–507, 2001.
[ bib ]

[326]

JeanPierre Brans and Bertrand Mareschal.
PROMETHEEGAIA. Une méthode d'aide à la décision
en présence de critères multiples.
Editions Ellipses, Paris, FR, 2002.
[ bib ]

[327]

JeanPierre Brans and Bertrand Mareschal.
PROMETHEE Methods.
In J. R. Figueira, S. Greco, and M. Ehrgott, editors, Multiple
Criteria Decision Analysis, State of the Art Surveys, chapter 5, pages
163–195. Springer, 2005.
[ bib ]

[328]

Jürgen Branke, S. Nguyen, C. W. Pickardt, and M. Zhang.
Automated Design of Production Scheduling Heuristics: A Review.
IEEE Transactions on Evolutionary Computation, 20(1):110–124,
2016.
[ bib ]

[329]

Jürgen Branke, C. Schmidt, and H. Schmeck.
Efficient fitness estimation in noisy environments.
In E. D. Goodman, editor, Proceedings of the 3rd Annual
Conference on Genetic and Evolutionary Computation, GECCO 2001, pages
243–250. Morgan Kaufmann Publishers, San Francisco, CA, 2001.
[ bib ]

[330]

Roland Braune and G. Zäpfel.
Shifting Bottleneck Scheduling for Total Weighted Tardiness
Minimization—A Computational Evaluation of Subproblem and Reoptimization
Heuristics.
Computers & Operations Research, 66:130–140, 2016.
[ bib ]

[331]

Jürgen Branke, Salvatore Corrente, Salvatore Greco, Roman Slowiński,
and P. Zielniewicz.
Using Choquet integral as preference model in interactive
evolutionary multiobjective optimization.
Technical report, WBS, University of Warwick, 2014.
[ bib ]

[332]

Jürgen Branke, Salvatore Corrente, Salvatore Greco, Roman Slowiński,
and P. Zielniewicz.
Using Choquet integral as preference model in interactive
evolutionary multiobjective optimization.
European Journal of Operational Research, 250(3):884–901,
2016.
[ bib 
DOI ]

[333]

Jürgen Branke and Jawad Elomari.
Simultaneous tuning of metaheuristic parameters for various
computing budgets.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 263–264.
ACM Press, New York, NY, 2011.
[ bib 
DOI ]
Keywords: metaoptimization, offline parameter optimization

[334]

Jürgen Branke and Jawad Elomari.
Racing with a Fixed Budget and a SelfAdaptive Significance
Level.
In P. M. Pardalos and G. Nicosia, editors, Learning and
Intelligent Optimization, 7th International Conference, LION 7, volume 7997
of Lecture Notes in Computer Science. Springer, Heidelberg, 2013.
[ bib ]

[335]

Jürgen Branke, S. S. Farid, and N. Shah.
Industry 4.0: a vision for personalized medicine supply chains?
Cell and Gene Therapy Insights, 2(2):263–270, 2016.
[ bib 
DOI ]

[336]

Jürgen Branke, Salvatore Greco, Roman Slowiński, and Piotr
Zielniewicz.
Learning Value Functions in Interactive Evolutionary
Multiobjective Optimization.
IEEE Transactions on Evolutionary Computation, 19(1):88–102,
2015.
[ bib ]

[337]

Yaochu Jin and Jürgen Branke.
Evolutionary Optimization in Uncertain Environments—A Survey.
IEEE Transactions on Evolutionary Computation, 9(5):303–317,
2005.
[ bib ]

[338]

Mátyás Brendel and Marc Schoenauer.
LearnandOptimize: A Parameter Tuning Framework for
Evolutionary AI Planning.
In J.K. Hao, P. Legrand, P. Collet, N. Monmarché, E. Lutton, and
M. Schoenauer, editors, Artificial Evolution: 10th International
Conference, Evolution Artificielle, EA, 2011, volume 7401 of Lecture
Notes in Computer Science, pages 145–155. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[339]

Mátyás Brendel and Marc Schoenauer.
Instancebased Parameter Tuning for Evolutionary AI Planning.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference Companion, GECCO Companion
2011, pages 591–598, New York, NY, 2011. ACM Press.
[ bib 
DOI ]

[340]

Leo Breiman.
Random Forests.
Machine Learning, 45(1):5–32, 2001.
[ bib 
DOI ]

[341]

Karl Bringmann and Tobias Friedrich.
Approximating the Least Hypervolume Contributor: NPHard in
General, But Fast in Practice.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pages 6–20.
Springer, Heidelberg, 2009.
[ bib ]

[342]

Karl Bringmann and Tobias Friedrich.
The Maximum Hypervolume Set Yields Nearoptimal Approximation.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pages 511–518. ACM
Press, New York, NY, 2010.
[ bib ]

[343]

Karl Bringmann and Tobias Friedrich.
Convergence of HypervolumeBased Archiving Algorithms I:
Effectiveness.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 745–752.
ACM Press, New York, NY, 2011.
[ bib 
DOI 
eprint ]

[344]

Karl Bringmann and Tobias Friedrich.
Convergence of HypervolumeBased Archiving Algorithms II:
Competitiveness.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pages 457–464. ACM
Press, New York, NY, 2012.
[ bib 
DOI 
eprint ]

[345]

Karl Bringmann and Tobias Friedrich.
Don't be greedy when calculating hypervolume contributions.
In I. I. Garibay, T. Jansen, R. P. Wiegand, and A. S. Wu, editors,
Proceedings of the Tenth ACM SIGEVO Workshop on Foundations of Genetic
Algorithms (FOGA), pages 103–112. ACM, 2009.
[ bib ]

[346]

Karl Bringmann, Tobias Friedrich, Frank Neumann, and Markus Wagner.
Approximationguided Evolutionary Multiobjective Optimization.
In T. Walsh, editor, Proceedings of the TwentySecond
International Joint Conference on Artificial Intelligence (IJCAI11), pages
1198–1203. IJCAI/AAAI Press, Menlo Park, CA, 2011.
[ bib ]

[347]

Dimo Brockhoff.
A Bug in the Multiobjective Optimizer IBEA: Salutary Lessons
for Code Release and a Performance ReAssessment.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pages 187–201.
Springer, Heidelberg, 2015.
[ bib 
DOI ]

[348]

Dimo Brockhoff, Johannes Bader, Lothar Thiele, and Eckart Zitzler.
Directed Multiobjective Optimization Based on the Weighted
Hypervolume Indicator.
Journal of MultiCriteria Decision Analysis, 20(56):291–317,
2013.
[ bib 
DOI ]
Recently, there has been a large interest in setbased
evolutionary algorithms for multi objective
optimization. They are based on the definition of indicators
that characterize the quality of the current population while
being compliant with the concept of Paretooptimality. It has
been shown that the hypervolume indicator, which measures the
dominated volume in the objective space, enables the design
of efficient search algorithms and, at the same time, opens
up opportunities to express user preferences in the search by
means of weight functions. The present paper contains the
necessary theoretical foundations and corresponding
algorithms to (i) select appropriate weight functions, to
(ii) transform user preferences into weight functions and to
(iii) efficiently evaluate the weighted hypervolume indicator
through Monte Carlo sampling. The algorithm WHypE, which
implements the previous concepts, is introduced, and the
effectiveness of the search, directed towards the user's
preferred solutions, is shown using an extensive set of
experiments including the necessary statistical performance
assessment.
Keywords: hypervolume, preferencebased search, multi objective
optimization, evolutionary algorithm

[349]

Dimo Brockhoff, Roberto Calandra, Manuel LópezIbáñez, Frank
Neumann, and Selvakumar Ulaganathan.
Metamodeling for (interactive) multiobjective optimization
(WG5).
In K. Klamroth, J. D. Knowles, G. Rudolph, and M. M. Wiecek, editors,
Personalized Multiobjective Optimization: An Analytics Perspective
(Dagstuhl Seminar 18031), volume 8(1) of Dagstuhl Reports, pages
85–94. Schloss Dagstuhl–LeibnizZentrum für Informatik, Germany, 2018.
[ bib 
DOI ]
Keywords: multiple criteria decision making, evolutionary
multiobjective optimization

[350]

Eric Brochu, Vlad Cora, and Nando de Freitas.
A Tutorial on Bayesian Optimization of Expensive Cost
Functions, with Application to Active User Modeling and Hierarchical
Reinforcement Learning.
Arxiv preprint arXiv:1012.2599, December 2010.
[ bib 
http ]

[351]

Dimo Brockhoff, Manuel LópezIbáñez, Boris Naujoks, and Günther
Rudolph.
Runtime Analysis of Simple Interactive Evolutionary Biobjective
Optimization Algorithms.
In C. A. Coello Coello et al., editors, Parallel Problem
Solving from Nature, PPSN XII, volume 7491 of Lecture Notes in Computer
Science, pages 123–132. Springer, Heidelberg, 2012.
[ bib 
DOI ]
Development and deployment of interactive evolutionary
multiobjective optimization algorithms (EMOAs) have recently
gained broad interest. In this study, first steps towards a
theory of interactive EMOAs are made by deriving bounds on
the expected number of function evaluations and queries to a
decision maker. We analyze randomized local search and the
(1+1)EA on the biobjective problems LOTZ and COCZ under the
scenario that the decision maker interacts with these
algorithms by providing a subjective preference whenever
solutions are incomparable. It is assumed that this decision
is based on the decision maker's internal utility
function. We show that the performance of the interactive
EMOAs may dramatically worsen if the utility function is
nonlinear instead of linear.

[352]

Dimo Brockhoff, Dhish Kumar Saxena, Kalyanmoy Deb, and Eckart Zitzler.
On Handling a Large Number of Objectives A Posteriori and During
Optimization.
In J. D. Knowles, D. Corne, K. Deb, and D. R. Chair, editors,
Multiobjective Problem Solving from Nature, Natural Computing Series, pages
377–403. Springer, Berlin/Heidelberg, 2008.
[ bib 
DOI ]
Dimensionality reduction methods are used routinely in
statistics, pattern recognition, data mining, and machine
learning to cope with highdimensional spaces. Also in the
case of highdimensional multiobjective optimization
problems, a reduction of the objective space can be
beneficial both for search and decision making. New questions
arise in this context, e.g., how to select a subset of
objectives while preserving most of the problem structure. In
this chapter, two different approaches to the task of
objective reduction are developed, one based on assessing
explicit conflicts, the other based on principal component
analysis (PCA). Although both methods use different
principles and preserve different properties of the
underlying optimization problems, they can be effectively
utilized either in an a posteriori scenario or during
search. Here, we demonstrate the usability of the
conflictbased approach in a decisionmaking scenario after
the search and show how the principalcomponentbased
approach can be integrated into an evolutionary
multicriterion optimization (EMO) procedure.

[353]

Dimo Brockhoff and Eckart Zitzler.
Dimensionality Reduction in Multiobjective Optimization: The
Minimum Objective Subset Problem.
In K.H. Waldmann and U. M. Stocker, editors, Operations
Research Proceedings 2006, pages 423–429. Springer, Berlin/
Heidelberg, 2007.
[ bib 
DOI ]
The number of objectives in a multiobjective optimization
problem strongly influences both the performance of
generating methods and the decision making process in
general. On the one hand, with more objectives, more
incomparable solutions can arise, the number of which affects
the generating method's performance. On the other hand, the
more objectives are involved the more complex is the choice
of an appropriate solution for a (human) decision maker. In
this context, the question arises whether all objectives are
actually necessary and whether some of the objectives may be
omitted; this question in turn is closely linked to the
fundamental issue of conflicting and nonconflicting
optimization criteria. Besides a general definition of
conflicts between objective sets, we here introduce the
NPhard problem of computing a minimum subset of objectives
without losing information (MOSS). Furthermore, we present
for MOSS both an approximation algorithm with optimum
approximation ratio and an exact algorithm which works well
for small input instances. We conclude with experimental
results for a random problem and the multiobjective
0/1knapsack problem

[354]

Peter Brucker, Johann Hurink, and Frank Werner.
Improving Local Search Heuristics for some Scheduling Problems
— Part I.
Discrete Applied Mathematics, 65(1–3):97–122, 1996.
[ bib ]

[355]

Peter Brucker, Johann Hurink, and Frank Werner.
Improving Local Search Heuristics for some Scheduling Problems
— Part II.
Discrete Applied Mathematics, 72(1–2):47–69, 1997.
[ bib ]

[356]

M. J. Brusco, L. W. Jacobs, and G. M. Thompson.
A Morphing Procedure to Supplement a Simulated Annealing
Heuristic for Cost and Coveragecorrelated Set Covering Problems.
Annals of Operations Research, 86:611–627, 1999.
[ bib ]

[357]

Artur Brum and Marcus Ritt.
Automatic Design of Heuristics for Minimizing the Makespan in
Permutation Flow Shops.
In Proceedings of the 2018 Congress on Evolutionary Computation
(CEC 2018), pages 1–8, Piscataway, NJ, 2018. IEEE Press.
[ bib 
DOI ]

[358]

Artur Brum and Marcus Ritt.
Automatic Algorithm Configuration for the Permutation Flow Shop
Scheduling Problem Minimizing Total Completion Time.
In A. Liefooghe and M. LópezIbáñez, editors,
Proceedings of EvoCOP 2018 – 18th European Conference on Evolutionary
Computation in Combinatorial Optimization, volume 10782 of Lecture
Notes in Computer Science, pages 85–100, Heidelberg, 2018. Springer.
[ bib 
DOI ]

[359]

John T. Buchanan.
An experimental evaluation of interactive MCDM methods and the
decision making process.
Journal of the Operational Research Society, 45(9):1050–1059,
1994.
[ bib ]

[360]

A. L. Buchsbaum and M. T. Goodrich.
ThreeDimensional Layers of Maxima.
Algorithmica, 39:275–289, 2004.
[ bib ]

[361]

T. N. Bui and J. R. Rizzo, Jr.
Finding Maximum Cliques with Distributed Ants.
In K. Deb et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2004, Part I, volume 3102 of
Lecture Notes in Computer Science, pages 24–35. Springer, Heidelberg, 2004.
[ bib ]

[362]

B. Bullnheimer, Richard F. Hartl, and Christine Strauss.
An Improved Ant System Algorithm for the Vehicle Routing
Problem.
Annals of Operations Research, 89:319–328, 1999.
[ bib ]

[363]

B. Bullnheimer, Richard F. Hartl, and Christine Strauss.
A new rankbased version of the Ant System: A
computational study.
Central European Journal for Operations Research and Economics,
7(1):25–38, 1999.
[ bib ]

[364]

Edmund K. Burke and Yuri Bykov.
The Late Acceptance HillClimbing Heuristic.
Technical Report CSM192, University of Stirling, 2012.
[ bib ]

[365]

Edmund K. Burke and Yuri Bykov.
The Late Acceptance HillClimbing Heuristic.
European Journal of Operational Research, 258(1):70–78, 2017.
[ bib ]

[366]

Rainer E. Burkard and Ulrich Fincke.
The asymptotic probabilistic behaviour of quadratic sum
assignment problems.
Zeitschrift für Operations Research, 27(1):73–81, 1983.
[ bib ]

[367]

Luciana Buriol, Paulo M. França, and Pablo Moscato.
A New Memetic Algorithm for the Asymmetric Traveling Salesman
Problem.
Journal of Heuristics, 10(5):483–506, 2004.
[ bib ]

[368]

Edmund K. Burke, Michel Gendreau, Matthew R. Hyde, Graham Kendall, Gabriela
Ochoa, Ender Özcan, and Rong Qu.
Hyperheuristics: A Survey of the State of the Art.
Journal of the Operational Research Society, 64(12):1695–1724,
2013.
[ bib ]

[369]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, and John R. Woodward.
Automatic Heuristic Generation with Genetic Programming:
Evolving a Jackofalltrades or a Master of One.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pages 1559–1565, New York,
NY, 2007. ACM Press.
[ bib 
DOI ]

[370]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, and John R. Woodward.
A Genetic Programming HyperHeuristic Approach for Evolving 2D
Strip Packing Heuristics.
IEEE Transactions on Evolutionary Computation, 14(6):942–958,
2010.
[ bib 
DOI ]

[371]

Edmund K. Burke, Matthew R. Hyde, and Graham Kendall.
Grammatical Evolution of Local Search Heuristics.
IEEE Transactions on Evolutionary Computation, 16(7):406–417,
2012.
[ bib 
DOI ]

[372]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, Gabriela Ochoa, Ender
Özcan, and John R. Woodward.
A Classification of HyperHeuristic Approaches: Revisited.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 272 of International Series in Operations
Research & Management Science, chapter 14, pages 453–477. Springer, 2019.
[ bib 
DOI ]

[373]

Rainer E. Burkard, Stefan E. Karisch, and Franz Rendl.
QAPLIB–a Quadratic Assignment Problem Library.
Journal of Global Optimization, 10(4):391–403, 1997.
[ bib ]

[374]

Rainer E. Burkard and Franz Rendl.
A Thermodynamically Motivated Simulation Procedure for
Combinatorial Optimization Problems.
European Journal of Operational Research, 17(2):169–174, 1984.
[ bib 
DOI ]
Keywords: 2exchange delta evaluation for QAP

[375]

Rainer E. Burkard, Eranda Çela, Panos M. Pardalos, and L. S. Pitsoulis.
The quadratic assignment problem.
In P. M. Pardalos and D.Z. Du, editors, Handbook of
Combinatorial Optimization, volume 2, pages 241–338. Kluwer Academic
Publishers, 1998.
[ bib ]

[376]

Erika Buson, Roberto Roberti, and Paolo Toth.
A ReducedCost Iterated Local Search Heuristic for the
FixedCharge Transportation Problem.
Operations Research, 62(5):1095–1106, 2014.
[ bib ]

[377]

Maxim Buzdalov.
Towards better estimation of statistical significance when
comparing evolutionary algorithms.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference Companion, GECCO Companion 2019, pages 1782–1788. ACM Press, New
York, NY, 2019.
[ bib 
DOI ]

[378]

Nicola Beume, Carlos M. Fonseca, Manuel LópezIbáñez, Luís
Paquete, and Jan Vahrenhold.
On the Complexity of Computing the Hypervolume Indicator.
Technical Report CI235/07, University of Dortmund, December 2007.
Published in IEEE Transactions on Evolutionary
Computation [209].
[ bib ]

[379]

COnfiguration and SElection of ALgorithms.
http://www.coseal.net, 2017.
[ bib ]

[380]

IBM.
ILOG CPLEX Optimizer.
http://www.ibm.com/software/integration/optimization/cplexoptimizer/,
2017.
[ bib ]

[381]

R. Caballero, Mariano Luque, J. Molina, and F. Ruiz.
PROMOIN: An Interactive System for Multiobjective
Programming.
Information Technologies and Decision Making, 1:635–656, 2002.
[ bib ]
Keywords: preferences, multi interactive methods framework

[382]

Leslie Pérez Cáceres and Thomas Stützle.
Exploring Variable Neighborhood Search for Automatic Algorithm
Configuration.
Electronic Notes in Discrete Mathematics, 58:167–174, 2017.
[ bib 
DOI ]

[383]

Sebastien Cahon, Nordine Melab, and ElGhazali Talbi.
ParadisEO: A Framework for the Reusable Design of Parallel and
Distributed Metaheuristics.
Journal of Heuristics, 10(3):357–380, 2004.
[ bib 
DOI ]

[384]

Zhaoquan Cai, Han Huang, Yong Qin, and Xianheng Ma.
Ant Colony Optimization Based on Adaptive Volatility Rate of
Pheromone Trail.
International Journal of Communications, Network and System
Sciences, 2(8):792–796, 2009.
[ bib ]

[385]

Roberto Wolfler Calvo.
A New Heuristic for the Traveling Salesman Problem with Time
Windows.
Transportation Science, 34(1):113–124, 2000.
[ bib 
DOI ]

[386]

Felipe Campelo, Lucas S. Batista, and Claus Aranha.
The MOEADr Package: A ComponentBased Framework for
Multiobjective Evolutionary Algorithms Based on Decomposition.
Journal of Statistical Software, 92, 2020.
[ bib 
DOI ]

[387]

Christian Leonardo CamachoVillalón, Marco Dorigo, and Thomas Stützle.
Why the Intelligent Water Drops Cannot Be Considered as a Novel
Algorithm.
In M. Dorigo, M. Birattari, A. L. Christensen, A. Reina, and
V. Trianni, editors, Swarm Intelligence, 11th International Conference,
ANTS 2018, volume 11172 of Lecture Notes in Computer Science, pages
302–314, Heidelberg, 2018. Springer.
[ bib ]

[388]

Christian Leonardo CamachoVillalón, Marco Dorigo, and Thomas Stützle.
The intelligent water drops algorithm: why it cannot be
considered a novel algorithm.
Swarm Intelligence, 13:173–192, 2019.
[ bib ]

[389]

Ann Melissa Campbell and Philip C. Jones.
Prepositioning supplies in preparation for disasters.
European Journal of Operational Research, 209(2):156–165,
2011.
[ bib ]

[390]

Paolo Campigotto and Andrea Passerini.
Adapting to a realistic decision maker: experiments towards a
reactive multiobjective optimizer.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pages 338–341. Springer, Heidelberg,
2010.
[ bib 
DOI ]

[391]

E Cambria, B Schuller, Y Xia, and C Havasi.
New avenues in opinion mining and sentiment analysis.
IEEE Intelligent Systems, 28(2):15–21, 2013.
[ bib 
DOI ]

[392]

Felipe Campelo, Áthila R. Trindade, and Manuel LópezIbáñez.
Pseudoreplication in Racing Methods for Tuning Metaheuristics.
In preparation, 2017.
[ bib ]

[393]

Felipe Campelo and Elizabeth F. Wanner.
Sample size calculations for the experimental comparison of
multiple algorithms on multiple problem instances.
Journal of Heuristics, 2020.
[ bib 
DOI ]

[394]

E. CantúPaz.
Efficient and Accurate Parallel Genetic Algorithms.
Kluwer Academic Publishers, Boston, MA, 2000.
[ bib ]

[395]

Z. Cao, S. Jiang, J. Zhang, and H. Guo.
A unified framework for vehicle rerouting and traffic light
control to reduce traffic congestion.
IEEE Transactions on Intelligent Transportation Systems,
18(7):1958–1973, 2017.
[ bib ]

[396]

Gilles Caporossi.
Variable Neighborhood Search for Extremal Vertices : The
AutoGraphiXIII System.
Computers & Operations Research, 78:431–438, 2017.
[ bib ]

[397]

J. Carlier.
The Onemachine Sequencing Problem.
European Journal of Operational Research, 11(1):42–47, 1982.
[ bib ]

[398]

William B. Carlton and J. Wesley Barnes.
Solving the travelingsalesman problem with time windows using
tabu search.
IIE Transactions, 28:617–629, 1996.
[ bib ]

[399]

P. Cardoso, M. Jesus, and A. Marquez.
MONACO: multiobjective network optimisation based on an
ACO.
In Proc. X Encuentros de Geometría Computacional, Seville,
Spain, 2003.
[ bib ]

[400]

Alex Guimarães Cardoso de Sá, Walter José G. S. Pinto, Luiz
Otávio Vilas Boas Oliveira, and Gisele L. Pappa.
RECIPE: A GrammarBased Framework for Automatically Evolving
Classification Pipelines.
In J. McDermott, M. Castelli, L. Sekanina, E. Haasdijk, and
P. GarcíaSánchez, editors, Proceedings of the 20th European
Conference on Genetic Programming, EuroGP 2017, volume 10196 of Lecture
Notes in Computer Science, pages 246–261. Springer, Heidelberg, 2017.
[ bib 
DOI ]

[401]

Ioannis Caragiannis, Ariel D. Procaccia, and Nisarg Shah.
When Do Noisy Votes Reveal the Truth?
In M. J. Kearns, R. P. McAfee, and É. Tardos, editors,
Proceedings of the Fourteenth ACM Conference on Electronic Commerce, pages
143–160, New York, NY, 2013. ACM Press.
[ bib 
DOI ]
A wellstudied approach to the design of voting rules views
them as maximum likelihood estimators; given votes that are
seen as noisy estimates of a true ranking of the
alternatives, the rule must reconstruct the most likely true
ranking. We argue that this is too stringent a requirement,
and instead ask: How many votes does a voting rule need to
reconstruct the true ranking? We define the family of
pairwisemajority consistent rules, and show that for all
rules in this family the number of samples required from the
Mallows noise model is logarithmic in the number of
alternatives, and that no rule can do asymptotically better
(while some rules like plurality do much worse). Taking a
more normative point of view, we consider voting rules that
surely return the true ranking as the number of samples tends
to infinity (we call this property accuracy in the limit);
this allows us to move to a higher level of abstraction. We
study families of noise models that are parametrized by
distance functions, and find voting rules that are accurate
in the limit for all noise models in such general
families. We characterize the distance functions that induce
noise models for which pairwisemajority consistent rules are
accurate in the limit, and provide a similar result for
another novel family of positiondominance consistent
rules. These characterizations capture three wellknown
distance functions.
Keywords: computer social choice, mallows model, sample complexity

[402]

Yves Caseau and François Laburthe.
Heuristics for large constrained vehicle routing problems.
Journal of Heuristics, 5(3):281–303, 1999.
[ bib ]

[403]

Yves Caseau, Glenn Silverstein, and François Laburthe.
Learning Hybrid Algorithms for Vehicle Routing Problems.
Theory and Practice of Logic Programming, 1(6):779–806, 2001.
[ bib ]

[404]

Diego Cattaruzza, Nabil Absi, Dominique Feillet, and Daniele Vigo.
An Iterated Local Search for the Multicommodity Multitrip
Vehicle Routing Problem with Time Windows.
Computers & Operations Research, 51:257–267, 2014.
[ bib ]

[405]

Aakil M. Caunhye, Xiaofeng Nie, and Shaligram Pokharel.
Optimization models in emergency logistics: A literature
review.
SocioEconomic Planning Sciences, 46(1):4–13, 2012.
[ bib ]

[406]

Josu Ceberio, Ekhine Irurozki, Alexander Mendiburu, and José A. Lozano.
A distancebased ranking model estimation of distribution
algorithm for the flowshop scheduling problem.
IEEE Transactions on Evolutionary Computation, 18(2):286–300,
2014.
[ bib 
DOI ]
The aim of this paper is twofold. First, we introduce a
novel general estimation of distribution algorithm to deal
with permutationbased optimization problems. The algorithm
is based on the use of a probabilistic model for permutations
called the generalized Mallows model. In order to prove the
potential of the proposed algorithm, our second aim is to
solve the permutation flowshop scheduling problem. A hybrid
approach consisting of the new estimation of distribution
algorithm and a variable neighborhood search is
proposed. Conducted experiments demonstrate that the proposed
algorithm is able to outperform the stateoftheart
approaches. Moreover, from the 220 benchmark instances
tested, the proposed hybrid approach obtains new best known
results in 152 cases. An indepth study of the results
suggests that the successful performance of the introduced
approach is due to the ability of the generalized Mallows
estimation of distribution algorithm to discover promising
regions in the search space.
Keywords: Estimation of distribution algorithms,Generalized Mallows
model,Permutation flowshop scheduling
problem,Permutationsbased optimization problems

[407]

Josu Ceberio, Alexander Mendiburu, and José A. Lozano.
Kernels of Mallows Models for Solving Permutationbased
Problems.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pages 505–512. ACM Press, New York, NY, 2015.
[ bib ]

[408]

Eranda Çela.
The Quadratic Assignment Problem: Theory and Algorithms.
Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998.
[ bib ]

[409]

Vladimír Černý.
A Thermodynamical Approach to the Traveling Salesman Problem: An
Efficient Simulation Algorithm.
Journal of Optimization Theory and Applications, 45(1):41–51,
1985.
[ bib ]

[410]

Sara Ceschia, Luca Di Gaspero, and Andrea Schaerf.
Design, Engineering, and Experimental Analysis of a Simulated
Annealing Approach to the PostEnrolment Course Timetabling Problem.
Computers & Operations Research, 39(7):1615–1624, 2012.
[ bib ]

[411]

Amadeo Cesta, Angelo Oddi, and Stephen F. Smith.
Iterative Flattening: A Scalable Method for Solving
MultiCapacity Scheduling Problems.
In H. A. Kautz and B. W. Porter, editors, Proceedings of AAAI
2000 – Seventeenth National Conference on Artificial Intelligence, pages
742–747. AAAI Press/MIT Press, Menlo Park, CA, 2000.
[ bib ]

[412]

Sara Ceschia and Andrea Schaerf.
Modeling and solving the dynamic patient admission scheduling
problem under uncertainty.
Artificial Intelligence in Medicine, 56(3):199–205, 2012.
[ bib 
DOI ]
Keywords: Frace

[413]

Sara Ceschia, Andrea Schaerf, and Thomas Stützle.
Local Search Techniques for a Routingpacking Problem.
Computers and Industrial Engineering, 66(4):1138–1149, 2013.
[ bib ]

[414]

Shelvin Chand and Markus Wagner.
Evolutionary manyobjective optimization: A quickstart guide.
Surveys in Operations Research and Management Science,
20(2):35–42, 2015.
[ bib 
DOI ]

[415]

S. T. H. Chang.
Optimizing the Real Time Operation of a Pumping Station at a
Water Filtration Plant using Genetic Algorithms.
Honors thesis, Department of Civil and Environmental Engineering, The
University of Adelaide, 1999.
[ bib ]

[416]

Donald V. Chase and Lindell E. Ormsbee.
Optimal pump operation of water distribution systems with
multiple storage tanks.
In Proceedings of American Water Works Association Computer
Specialty Conference, pages 205–214, Denver, USA, 1989. AWWA.
[ bib ]

[417]

Donald V. Chase and Lindell E. Ormsbee.
An alternate formulation of time as a decision variable to
facilitate realtime operation of water supply systems.
In Proceedings of the 18th Annual Conference of Water Resources
Planning and Management, pages 923–927, New York, NY, 1991. ASCE.
[ bib ]

[418]

Donald V. Chase and Lindell E. Ormsbee.
Computergenerated pumping schedules for satisfying operation
objectives.
J. Am. Water Works Assoc., 85(7):54–61, 1993.
[ bib ]

[419]

Shamik Chaudhuri and Kalyanmoy Deb.
An interactive evolutionary multiobjective optimization and
decision making procedure.
Applied Soft Computing, 10(2):496–511, 2010.
[ bib ]

[420]

Hsinchun Chen, Roger H. L. Chiang, and Veda C. Storey.
Business Intelligence and Analytics: From Big Data to Big
Impact.
MIS Quarterly, 36(4):1165–1188, 2012.
[ bib ]

[421]

Fei Chen, Yang Gao, Zhaoqian Chen, and Shifu Chen.
SCGA: Controlling genetic algorithms with Sarsa(0).
In Computational Intelligence for Modelling, Control and
Automation, 2005 and International Conference on Intelligent Agents, Web
Technologies and Internet Commerce, International Conference on, volume 1,
pages 1177–1183. IEEE, 2005.
[ bib 
DOI ]

[422]

Clément Chevalier, David Ginsbourger, Julien Bect, and Ilya Molchanov.
Estimating and Quantifying Uncertainties on Level Sets Using the
Vorob'ev Expectation and Deviation with Gaussian Process Models.
In D. Ucinski, A. C. Atkinson, and M. Patan, editors, mODa
10–Advances in ModelOriented Design and Analysis, pages 35–43. Springer
International Publishing, Heidelberg, 2013.
[ bib 
DOI ]
Several methods based on Kriging have recently been proposed
for calculating a probability of failure involving
costlytoevaluate functions. A closely related problem is to
estimate the set of inputs leading to a response exceeding a
given threshold. Now, estimating such a level set—and not
solely its volume—and quantifying uncertainties on it are
not straightforward. Here we use notions from random set
theory to obtain an estimate of the level set, together with
a quantification of estimation uncertainty. We give explicit
formulae in the Gaussian process setup and provide a
consistency result. We then illustrate how spacefilling
versus adaptive design strategies may sequentially reduce
level set estimation uncertainty.

[423]

Yuning Chen, JinKao Hao, and Fred Glover.
A hybrid metaheuristic approach for the capacitated arc routing
problem.
European Journal of Operational Research, 553(1):25–39, 2016.
[ bib 
DOI ]
Keywords: irace

[424]

RueyMaw Chen and FuRen Hsieh.
An exchange local search heuristic based scheme for permutation
flow shop problems.
Applied Mathematics & Information Sciences, 8(1):209–215,
2014.
[ bib ]

[425]

Peter C. Cheeseman, Bob Kanefsky, and William M. Taylor.
Where the Really Hard Problems Are.
In J. Mylopoulos and R. Reiter, editors, Proceedings of the
Twelfth International Joint Conference on Artificial Intelligence
(IJCAI91), pages 331–340. Morgan Kaufmann Publishers, 1995.
[ bib ]

[426]

F. Y. Cheng and X. S. Li.
Generalized center method for multiobjective engineering
optimization.
Engineering Optimization, 31(5):641–661, 1999.
[ bib 
DOI ]

[427]

Rachid Chelouah and Patrick Siarry.
Tabu search applied to global optimization.
European Journal of Operational Research, 123(2):256–270,
2000.
[ bib ]

[428]

L. Chen, X. H. Xu, and Y. X. Chen.
An adaptive ant colony clustering algorithm.
In I. Cloete, K.P. Wong, and M. Berthold, editors, Proceedings
of the International Conference on Machine Learning and Cybernetics, pages
1387–1392. IEEE Press, 2004.
[ bib ]

[429]

ChinBin Cheng and ChunPin Mao.
A modified ant colony system for solving the travelling salesman
problem with time windows.
Mathematical and Computer Modelling, 46:1225–1235, 2007.
[ bib 
DOI ]

[430]

Marco Chiarandini, Mauro Birattari, Krzysztof Socha, and O. RossiDoria.
An Effective Hybrid Algorithm for University Course
Timetabling.
Journal of Scheduling, 9(5):403–432, October 2006.
[ bib 
DOI ]
Keywords: 2003 international timetabling competition, Frace

[431]

Manuel Chica, Oscar Cordón, Sergio Damas, and Joaquín Bautista.
A New Diversity Induction Mechanism for a Multiobjective Ant
Colony Algorithm to Solve a Realworld time and Space Assembly Line Balancing
Problem.
Memetic Computing, 3(1):15–24, 2011.
[ bib ]

[432]

Marco Chiarandini and Yuri Goegebeur.
Mixed Models for the Analysis of Optimization Algorithms.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 225–264. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Preliminary version available as Tech. Rep. MF200907001 at the The Danish Mathematical Society

[433]

D. S. Chivilikhin, V. I. Ulyantsev, and A. A. Shalyto.
Modified ant colony algorithm for constructing finite state
machines from execution scenarios and temporal formulas.
Automation and Remote Control, 77(3):473–484, 2016.
[ bib 
DOI ]
Keywords: irace

[434]

Marco Chiarandini.
Stochastic Local Search Methods for Highly Constrained
Combinatorial Optimisation Problems.
PhD thesis, FB Informatik, TU Darmstadt, Germany, 2005.
[ bib ]

[435]

Francisco Chicano, Darrell Whitley, and Enrique Alba.
A Methodology to Find the Elementary Landscape Decomposition of
Combinatorial Optimization Problems.
Evolutionary Computation, 19(4):597–637, 2011.
[ bib ]

[436]

Francisco Chicano, Gabriel J. Luque, and Enrique Alba.
Autocorrelation Measures for the Quadratic Assignment Problem.
Applied Mathematics Letters, 25:698–705, 2012.
[ bib 
DOI ]

[437]

TsungChe Chiang.
nsga3cpp: A C++ implementation of NSGAIII.
http://web.ntnu.edu.tw/~tcchiang/publications/nsga3cpp/nsga3cpp.htm,
2014.
[ bib ]

[438]

Nicos Christofides, A. Mingozzi, and Paolo Toth.
Statespace relaxation procedures for the computation of bounds
to routing problems.
Networks, 11(2):145–164, 1981.
[ bib 
DOI ]

[439]

Matthias Christen, Olaf Schenk, and Helmar Burkhart.
PATUS: A Code Generation and Autotuning Framework for Parallel
Iterative Stencil Computations on Modern Microarchitectures.
In F. Mueller, editor, Proceedings of the 2011 IEEE
International Parallel & Distributed Processing Symposium, IPDPS '11, pages
676–687. IEEE Computer Society, 2011.
[ bib 
DOI ]

[440]

Jan Christiaens and Greet Vanden Berghe.
Slack Induction by String Removals for Vehicle Routing
Problems.
Technical Report 7052018, Department of Computing Science, KU
Leuven, Gent, Belgium, 2018.
[ bib ]

[441]

Nicos Christofides.
Worstcase analysis of a new heuristic for the travelling
salesman problem.
Technical Report 388, Graduate School of Industrial Administration,
CarnegieMellon University, Pittsburgh, PA, 1976.
[ bib ]

[442]

Tinkle Chugh and Manuel LópezIbáñez.
Maximising Hypervolume and Minimising εIndicators
using Bayesian Optimisation over Sets.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference Companion, GECCO Companion
2021, pages 1326–1334. ACM Press, New York, NY, 2021.
[ bib 
DOI 
supplementary material ]
Keywords: multiobjective, surrogate models, epsilon, hypervolume

[443]

S. Chusanapiputt, D. Nualhong, S. Jantarang, and S. Phoomvuthisarn.
Selective selfadaptive approach to ant system for solving unit
commitment problem.
In M. Cattolico et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2006, pages 1729–1736. ACM
Press, New York, NY, 2006.
[ bib ]

[444]

Tinkle Chugh, Karthik Sindhya, Jussi Hakanen, and Kaisa Miettinen.
A survey on handling computationally expensive multiobjective
optimization problems with evolutionary algorithms.
Soft Computing, 23(9):3137–3166, 2019.
[ bib 
DOI ]
Evolutionary algorithms are widely used for solving
multiobjective optimization problems but are often criticized
because of a large number of function evaluations
needed. Approximations, especially function approximations,
also referred to as surrogates or metamodels are commonly
used in the literature to reduce the computation time. This
paper presents a survey of 45 different recent algorithms
proposed in the literature between 2008 and 2016 to handle
computationally expensive multiobjective optimization
problems. Several algorithms are discussed based on what kind
of an approximation such as problem, function or fitness
approximation they use. Most emphasis is given to function
approximationbased algorithms. We also compare these
algorithms based on different criteria such as metamodeling
technique and evolutionary algorithm used, type and
dimensions of the problem solved, handling constraints,
training time and the type of evolution control. Furthermore,
we identify and discuss some promising elements and major
issues among algorithms in the literature related to using an
approximation and numerical settings used. In addition, we
discuss selecting an algorithm to solve a given
computationally expensive multiobjective optimization problem
based on the dimensions in both objective and decision spaces
and the computation budget available.

[445]

Tinkle Chugh.
Handling expensive multiobjective optimization problems with
evolutionary algorithms.
PhD thesis, University of JyvÃ¤skylÃ¤, 2017.
[ bib ]

[446]

Tinkle Chugh.
Scalarizing Functions in Bayesian Multiobjective Optimization.
In Proceedings of the 2020 Congress on Evolutionary Computation
(CEC 2020), pages 1–8, Piscataway, NJ, 2020. IEEE Press.
[ bib 
DOI ]

[447]

Christian Cintrano, Javier Ferrer, Manuel LópezIbáñez, and Enrique
Alba.
Hybridization of Racing Methods with Evolutionary Operators for
Simulation Optimization of Traffic Lights Programs.
In C. Zarges and S. Verel, editors, Proceedings of EvoCOP 2021
– 21th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 12692 of Lecture Notes in Computer Science, pages
17–33. Springer, Cham, Switzerland, 2021.
[ bib 
DOI ]
In many realworld optimization problems, like the traffic
light scheduling problem tackled here, the evaluation of
candidate solu tions requires the simulation of a process
under various scenarios. Thus, good solutions should not only
achieve good objective function values, but they must be
robust (low variance) across all different scenarios.
Previous work has revealed the effectiveness of IRACE for
this task. However, the operators used by IRACE to generate
new solutions were designed for configuring algorithmic
parameters, that have various data types (categorical,
numerical, etc.). Meanwhile, evolutionary algorithms have
powerful operators for numerical optimization, which could
help to sample new solutions from the best ones found in the
search. Therefore, in this work, we propose a hybridization
of the elitist iterated racing mechanism of IRACE with
evolutionary operators from differential evo lution and
genetic algorithms. We consider a realistic case study
derived from the traffic network of Malaga (Spain) with 275
traffic lights that should be scheduled optimally. After a
meticulous study, we discovered that the hybrid algorithm
comprising IRACE plus differential evolution offers
statistically better results than conventional algorithms and
also improves travel times and reduces pollution.
Keywords: Hybrid algorithms, Evolutionary algorithms, Simulation
optimization, Uncertainty, Traffic light planning

[448]

Jill Cirasella, David S. Johnson, Lyle A. McGeoch, and Weixiong Zhang.
The Asymmetric Traveling Salesman Problem: Algorithms, Instance
Generators, and Tests.
In A. L. Buchsbaum and J. Snoeyink, editors, Algorithm
Engineering and Experimentation, Third International Workshop, ALENEX 2001,
Washington, DC, USA, January 56, 2001, Revised Papers, volume 2153 of
Lecture Notes in Computer Science, pages 32–59, Berlin, Germany, 2001.
Springer.
[ bib 
DOI ]

[449]

Jon Claerbout and Martin Karrenbach.
Electronic documents give reproducible research a new meaning.
In SEG Technical Program Expanded Abstracts 1992, pages
601–604. Society of Exploration Geophysicists, 1992.
[ bib 
DOI ]
Proposed a reproducibility taxonomy, defined reproducibility
and taxonomy

[450]

R. M. Clark, L. A. Rossman, and L. J. Wymer.
Modeling distribution system water quality: regulatory
implications.
Journal of Water Resources Planning and Management, ASCE,
121(6):423–428, 1995.
[ bib ]

[451]

M. Clerc and J. Kennedy.
Standard PSO 2011.
Particle Swarm Central, 2011.
[ bib 
http ]

[452]

Andy Cockburn, Pierre Dragicevic, Lonni Besançon, and Carl Gutwin.
Threats of a Replication Crisis in Empirical Computer Science.
Communications of the ACM, 63(8):70–79, July 2020.
[ bib 
DOI ]
Research replication only works if there is confidence built
into the results.

[453]

B. Codenotti, G. Manzini, L. Margara, and G. Resta.
Perturbation: An Efficient Technique for the Solution of Very
Large Instances of the Euclidean TSP.
INFORMS Journal on Computing, 8(2):125–133, 1996.
[ bib ]

[454]

Carlos A. Coello Coello.
Handling preferences in evolutionary multiobjective
optimization: A survey.
In Proceedings of the 2000 Congress on Evolutionary Computation
(CEC'00), pages 30–37. IEEE Press, Piscataway, NJ, July 2000.
[ bib ]

[455]

Carlos A. Coello Coello.
Theoretical and numerical constrainthandling techniques used
with evolutionary algorithms: a survey of the state of the art.
Computer Methods in Applied Mechanics and Engineering,
191(1112):1245–1287, 2002.
[ bib 
DOI ]

[456]

Carlos A. Coello Coello.
Multiobjective Evolutionary Algorithms in RealWorld
Applications: Some Recent Results and Current Challenges.
In Advances in Evolutionary and Deterministic Methods for
Design, Optimization and Control in Engineering and Sciences, pages 3–18.
Springer, 2015.
[ bib 
DOI ]

[457]

Carlos A. Coello Coello, Gary B. Lamont, and David A. Van Veldhuizen.
Evolutionary Algorithms for Solving MultiObjective Problems.
Springer, New York, NY, 2007.
[ bib ]

[458]

Carlos A. Coello Coello and Margarita ReyesSierra.
A Study of the Parallelization of a Coevolutionary
Multiobjective Evolutionary Algorithm.
In R. Monroy, G. ArroyoFigueroa, L. E. Sucar, and H. Sossa, editors,
Proceedings of MICAI, volume 2972 of Lecture Notes in Artificial
Intelligence, pages 688–697. Springer, Heidelberg, 2004.
[ bib ]
Introduces Inverted Generational Distance (IGD)
Keywords: IGD

[459]

Carlos A. Coello Coello.
Handling Preferences in Evolutionary Multiobjective
Optimization: A Survey.
In Proceedings of the 2000 Congress on Evolutionary Computation
(CEC'00), pages 30–37. IEEE Press, Piscataway, NJ, July 2000.
[ bib ]

[460]

Carlos A. Coello Coello.
Special Issue on Evolutionary Multiobjective
Optimization.
IEEE Transactions on Evolutionary Computation, 7(2), 2003.
[ bib ]

[461]

Carlos A. Coello Coello.
Evolutionary multiobjective optimization: a historical view of
the field.
IEEE Computational Intelligence Magazine, 1(1):28–36, 2006.
[ bib ]

[462]

Carlos A. Coello Coello.
Recent Results and Open Problems in Evolutionary Multiobjective
Optimization.
In C. MartínVide, R. Neruda, and M. A. VegaRodríguez,
editors, Theory and Practice of Natural Computing  6th International
Conference, TPNC 2017, volume 10687 of Lecture Notes in Computer
Science, pages 3–21. Springer International Publishing, Cham, Switzerland,
2017.
[ bib ]

[463]

Harry Cohn and Mark J. Fielding.
Simulated Annealing: Searching for an Optimal Temperature.
SIAM Journal on Optimization, 9(3):779–802, 1999.
[ bib ]

[464]

Paul R. Cohen.
Empirical Methods for Artificial Intelligence.
MIT Press, Cambridge, MA, 1995.
[ bib ]

[465]

G. Cohen.
Optimal Control of Water Supply Networks.
In S. G. Tzafestas, editor, Optimization and Control of Dynamic
Operational Research Models, volume 4, chapter 8, pages 251–276.
NorthHolland Publishing Company, Amsterdam, 1982.
[ bib ]

[466]

Andrew F. Colombo and Bryan W. Karney.
Impacts of Leaks on Energy Consumption in Pumped Systems with
Storage.
Journal of Water Resources Planning and Management, ASCE,
131(2):146–155, March 2005.
[ bib ]

[467]

Alberto Colorni, Marco Dorigo, and Vittorio Maniezzo.
Distributed Optimization by Ant Colonies.
In F. J. Varela and P. Bourgine, editors, Proceedings of the
First European Conference on Artificial Life, pages 134–142. MIT Press,
Cambridge, MA, 1992.
[ bib ]

[468]

Alberto Colorni, Marco Dorigo, Vittorio Maniezzo, and M. Trubian.
Ant System for Jobshop Scheduling.
JORBEL — Belgian Journal of Operations Research, Statistics
and Computer Science, 34(1):39–53, 1994.
[ bib ]

[469]

Sonia Colas, Nicolas Monmarché, Pierre Gaucher, and Mohamed Slimane.
Artificial Ants for the Optimization of Virtual Keyboard
Arrangement for Disabled People.
In N. Monmarché, E.G. Talbi, P. Collet, M. Schoenauer, and
E. Lutton, editors, Artificial Evolution, volume 4926 of Lecture
Notes in Computer Science, pages 87–99. Springer, Heidelberg, 2008.
[ bib 
DOI ]

[470]

Barry McCollum, Andrea Schaerf, Ben Paechter, Paul McMullan, Rhyd M. R. Lewis,
Andrew J. Parkes, Luca Di Gaspero, Rong Qu, and Edmund K. Burke.
Setting the Research Agenda in Automated Timetabling: The Second
International Timetabling Competition.
INFORMS, 22(1):120–130, February 2010.
[ bib 
DOI ]

[471]

Richard K. Congram, Chris N. Potts, and Steve van de Velde.
An Iterated Dynasearch Algorithm for the SingleMachine Total
Weighted Tardiness Scheduling Problem.
INFORMS Journal on Computing, 14(1):52–67, 2002.
[ bib ]

[472]

Andrew R. Conn, Katya Scheinberg, and Luis N. Vicente.
Introduction to DerivativeFree Optimization.
MPS–SIAM Series on Optimization. Society for Industrial and Applied
Mathematics, Philadelphia, PA, USA, 2009.
[ bib ]

[473]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Concorde TSP Solver.
http://www.math.uwaterloo.ca/tsp/concorde.html, 2014.
Version visited last on 15 April 2014.
[ bib ]

[474]

David T. Connolly.
An Improved Annealing Scheme for the QAP.
European Journal of Operational Research, 46(1):93–100, 1990.
[ bib ]

[475]

W. J. Conover.
Practical Nonparametric Statistics.
John Wiley & Sons, New York, NY, 3rd edition, 1999.
[ bib ]

[476]

Richard J. Cook and Vern T. Farewell.
Multiplicity Considerations in the Design and Analysis of
Clinical Trials.
Journal of the Royal Statistical Society: Series A,
159:93–110, 1996.
[ bib ]
multiplicity; multiple endpoints; multiple treatments;
pvalue adjustment; type I error; argues that if results are
intended to be interpreted marginally, there may be no need
for controlling experimentwise error rate

[477]

Stephen A. Cook.
The Complexity of Theoremproving Procedures.
In Proceedings of the Third Annual ACM Symposium on Theory of
Computing, STOC '71, pages 151–158. ACM, 1971.
[ bib 
DOI ]

[478]

William J. Cook.
In Pursuit of the Traveling Salesman.
Princeton University Press, Princeton, NJ, 2012.
[ bib ]

[479]

William J. Cook.
Computing in Combinatorial Optimization.
In B. Steffen and G. Woeginger, editors, Computing and Software
Science: State of the Art and Perspectives, volume 10000 of Lecture
Notes in Computer Science, pages 27–47. Springer, Cham, Switzerland, 2019.
[ bib 
DOI ]

[480]

Don Coppersmith, Lisa K. Fleischer, and Atri Rurda.
Ordering by Weighted Number of Wins Gives a Good Ranking for
Weighted Tournaments.
ACM Transactions on Algorithms, 6(3):1–13, July 2010.
[ bib 
DOI ]
Keywords: Approximation algorithms,Borda's method,feedback arc set
problem,rank aggregation,tournaments

[481]

Oscar Cordón and Sergio Damas.
Image Registration with Iterated Local Search.
Journal of Heuristics, 12(1–2):73–94, 2006.
[ bib ]

[482]

Jeroen Corstjens, Nguyen Dang, Benoît Depaire, An Caris, and Patrick De
Causmaecker.
A combined approach for analysing heuristic algorithms.
Journal of Heuristics, 25(4):591–628, 2019.
[ bib 
DOI ]

[483]

Jeroen Corstjens, Benoît Depaire, An Caris, and Kenneth Sörensen.
A multilevel evaluation method for heuristics with an
application to the VRPTW.
International Transactions in Operational Research,
27(1):168–196, 2020.
[ bib 
DOI ]

[484]

David Corne, Nick R. Jerram, Joshua D. Knowles, and Martin J. Oates.
PESAII: RegionBased Selection in Evolutionary Multiobjective
Optimization.
In E. D. Goodman, editor, Proceedings of the 3rd Annual
Conference on Genetic and Evolutionary Computation, GECCO 2001, pages
283–290. Morgan Kaufmann Publishers, San Francisco, CA, 2001.
[ bib 
DOI ]
Keywords: PESAII

[485]

David Corne and Joshua D. Knowles.
Some Multiobjective Optimizers are Better than Others.
In Proceedings of the 2003 Congress on Evolutionary Computation
(CEC'03), pages 2506–2512. IEEE Press, Piscataway, NJ, December 2003.
[ bib ]

[486]

David Corne and Joshua D. Knowles.
No free lunch and free leftovers theorems for multiobjective
optimisation problems.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 327–341. Springer,
Heidelberg, 2003.
[ bib 
DOI ]

[487]

David Corne, Joshua D. Knowles, and M. J. Oates.
The Pareto EnvelopeBased Selection Algorithm for
Multiobjective Optimization.
In M. Schoenauer et al., editors, Parallel Problem Solving from
Nature – PPSN VI, volume 1917 of Lecture Notes in Computer Science,
pages 839–848. Springer, Heidelberg, 2000.
[ bib ]

[488]

P. Corry and E. Kozan.
Ant Colony Optimisation for Machine Layout Problems.
Computational Optimization and Applications, 28(3):287–310,
2004.
[ bib ]

[489]

JeanFrançois Cordeau, Gilbert Laporte, and A. Mercier.
A unified tabu search heuristic for vehicle routing problems
with time windows.
Journal of the Operational Research Society, 52(8):928–936,
2001.
[ bib ]

[490]

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein.
Introduction to algorithms.
MIT Press, Cambridge, MA, 2009.
[ bib ]

[491]

JeanFrançois Cordeau and Mirko Maischberger.
A Parallel Iterated Tabu Search Heuristic for Vehicle Routing
Problems.
Computers & Operations Research, 39(9):2033–2050, 2012.
[ bib ]

[492]

David Corne and Alan Reynolds.
Evaluating optimization algorithms: bounds on the performance of
optimizers on unseen problems.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 707–710,
New York, NY, 2011. ACM Press.
[ bib 
DOI 
supplementary material ]

[493]

Oscar Cordón, I. Fernández de Viana, Francisco Herrera, and L. Moreno.
A New ACO Model Integrating Evolutionary Computation Concepts:
The BestWorst Ant System.
In M. Dorigo et al., editors, Abstract proceedings of ANTS 2000
– From Ant Colonies to Artificial Ants: Second International Workshop on Ant
Algorithms, pages 22–29. IRIDIA, Université Libre de Bruxelles,
Belgium, September, 7–9 2000.
[ bib ]

[494]

Wagner Emanoel Costa, Marco C. Goldbarg, and Elizabeth F. G. Goldbarg.
Hybridizing VNS and pathrelinking on a particle swarm
framework to minimize total flowtime.
Expert Systems with Applications, 39(18):13118–13126, 2012.
[ bib ]

[495]

D. Costa and A. Hertz.
Ants can color graphs.
Journal of the Operational Research Society, 48:295–305, 1997.
[ bib ]

[496]

S. P. Coy, B. L. Golden, G. C. Runger, and E. A. Wasil.
Using Experimental Design to Find Effective Parameter Settings
for Heuristics.
Journal of Heuristics, 7(1):77–97, 2001.
[ bib ]

[497]

I. Barry Crabtree.
Resource Scheduling: Comparing Simulated Annealing with
Constraint Programming.
BT Technology Journal, 13(1):121–127, 1995.
[ bib ]

[498]

M. J. Crawley.
The R Book.
Wiley, 2nd edition, 2012.
[ bib ]

[499]

Douglas Edward Critchlow, Michael A. Fligner, and Joseph S. Verducci.
Probability Models on Rankings.
Journal of Mathematical Psychology, 35:294–318, 1991.
[ bib ]

[500]

G. A. Croes.
A Method for Solving Traveling Salesman Problems.
Operations Research, 6:791–812, 1958.
[ bib ]

[501]

W. B. Crowston, F. Glover, G. L. Thompson, and J. D. Trawick.
Probabilistic and Parametric Learning Combinations of Local Job
Shop Scheduling Rules.
ONR Research Memorandum No. 117, GSIA, CarnegieMellon University,
Pittsburgh, PA, USA, 1963.
[ bib ]

[502]

Carlos Cruz, Juan Ramón González, and David A. Pelta.
Optimization in Dynamic Environments: A Survey on Problems,
Methods and Measures.
Soft Computing, 15(7):1427–1448, 2011.
[ bib ]

[503]

Fábio Cruz, Anand Subramanian, Bruno P. Bruck, and Manuel Iori.
A Heuristic Algorithm for a Single Vehicle Static Bike Sharing
Rebalancing Problem.
Computers & Operations Research, 79:19–33, 2017.
[ bib ]

[504]

J. C. Culberson.
On the Futility of Blind Search: An Algorithmic View of “No
Free Lunch”.
Evolutionary Computation, 6(2):109–127, 1998.
[ bib 
DOI ]
Keywords: NFL

[505]

J. C. Culberson.
Iterated Greedy Graph Coloring and the Difficulty Landscape.
Technical Report 9207, Department of Computing Science, The
University of Alberta, Edmonton, Alberta, Canada, 1992.
[ bib ]

[506]

J. C. Culberson, A. Beacham, and D. Papp.
Hiding our Colors.
In Proceedings of the CP'95 Workshop on Studying and Solving
Really Hard Problems, pages 31–42, Cassis, France, September 1995.
[ bib ]

[507]

J. C. Culberson and F. Luo.
Exploring the kcolorable Landscape with Iterated Greedy.
In D. S. Johnson and M. A. Trick, editors, Cliques, Coloring,
and Satisfiability: Second DIMACS Implementation Challenge, volume 26 of
DIMACS Series on Discrete Mathematics and Theoretical Computer
Science, pages 245–284. American Mathematical Society, Providence, RI,
1996.
[ bib ]

[508]

Jeff Cumming.
Understanding the New Statistics – Effect Sizes, Confidence
Intervals, and Metaanalysis.
Taylor & Francis, 2012.
[ bib ]

[509]

P. Czyzżak and Andrzej Jaszkiewicz.
Pareto simulated annealing – a metaheuristic technique for
multipleobjective combinatorial optimization.
Journal of MultiCriteria Decision Analysis, 7(1):34–47, 1998.
[ bib ]

[510]

Steven B. Damelin, Fred J. Hickernell, David L. Ragozin, and Xiaoyan Zeng.
On Energy, Discrepancy and Group Invariant Measures on
Measurable Subsets of Euclidean Space.
Journal of Fourier Analysis and Applications, 16(6):813–839,
2010.
[ bib ]
Keywords: Capacity; Cubature; Discrepancy; Distribution; Group
invariant kernel; Group invariant measure; Energy minimizer;
Equilibrium measure; Numerical integration; Positive
definite; Potential field; Riesz kernel; Reproducing Hilbert
space; Signed measure

[511]

M. Damas, M. Salmerón, J. Ortega, G. Olivares, and H. Pomares.
Parallel Dynamic Water Supply Scheduling in a Cluster of
Computers.
Concurrency and Computation: Practice and Experience,
13(15):1281–1302, December 2001.
[ bib ]

[512]

Nguyen Dang Thi Thanh and Patrick De Causmaecker.
Motivations for the Development of a Multiobjective Algorithm
Configurator.
In B. Vitoriano, E. Pinson, and F. Valente, editors, ICORES
2014  Proceedings of the 3rd International Conference on Operations Research
and Enterprise Systems, pages 328–333. SciTePress, 2014.
[ bib ]

[513]

Nguyen Dang and Carola Doerr.
Hyperparameter tuning for the (1 + (λ,
λ)) GA.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pages 889–897. ACM Press, New York, NY, 2019.
[ bib 
DOI ]
Keywords: irace; theory

[514]

Nguyen Dang Thi Thanh, Leslie Pérez Cáceres, Patrick De
Causmaecker, and Thomas Stützle.
Configuring irace Using Surrogate Configuration
Benchmarks.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pages 243–250. ACM Press,
New York, NY, 2017.
[ bib 
DOI ]
Keywords: irace

[515]

Augusto Lopez Dantas and Aurora Trinidad Ramirez Pozo.
A MetaLearning Algorithm Selection Approach for the Quadratic
Assignment Problem.
In Proceedings of the 2018 Congress on Evolutionary Computation
(CEC 2018), pages 1–8, Piscataway, NJ, 2018. IEEE Press.
[ bib ]

[516]

Augusto Dantas and Aurora Pozo.
On the use of fitness landscape features in metalearning based
algorithm selection for the quadratic assignment problem.
Theoretical Computer Science, 805:62–75, 2020.
[ bib 
DOI ]

[517]

Emilie Danna, Edward Rothberg, and Claude Le Pape.
Exploring relaxation induced neighborhoods to improve MIP
solutions.
Mathematical Programming, 102(1):71–90, 2005.
[ bib ]

[518]

George B. Dantzig and Philip Wolfe.
Decomposition Principle for Linear Programs.
Operations Research, 8(1):101–111, 1960.
[ bib ]

[519]

Graeme C. Dandy and Matthew S. Gibbs.
Optimizing System Operations and Water Quality.
In P. Bizier and P. DeBarry, editors, Proceedings of World Water
and Environmental Resources Congress. ASCE, Philadelphia, USA, 2003.
on CDROM.
[ bib 
DOI ]

[520]

Nguyen Dang Thi Thanh.
Data analytics for algorithm design.
PhD thesis, KU Leuven, Belgium, 2018.
[ bib ]
Supervised by Patrick De Causmaecker

[521]

Fabio Daolio, Sébastien Verel, Gabriela Ochoa, and Marco Tomassini.
Local Optima Networks and the Performance of Iterated Local
Search.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pages 369–376. ACM
Press, New York, NY, 2012.
[ bib ]

[522]

Indraneel Das and John E. Dennis.
A closer look at drawbacks of minimizing weighted sums of
objectives for Pareto set generation in multicriteria optimization
problems.
Structural Optimization, 14(1):63–69, 1997.
[ bib 
DOI ]

[523]

Swagatam Das, Sankha Subhra Mullick, and Ponnuthurai N. Suganthan.
Recent advances in differential evolution–An updated survey.
Swarm and Evolutionary Computation, 27:1–30, 2016.
[ bib ]

[524]

Swagatam Das and Ponnuthurai N. Suganthan.
Differential Evolution: A Survey of the Stateoftheart.
IEEE Transactions on Evolutionary Computation, 15(1), February
2011.
[ bib ]

[525]

Sanjeeb Dash.
Exponential Lower Bounds on the Lengths of Some Classes of
BranchandCut Proofs.
Mathematics of Operations Research, 30(3):678–700, 2005.
[ bib ]

[526]

Samuel Daulton, Maximilian Balandat, and Eytan Bakshy.
Differentiable Expected Hypervolume Improvement for Parallel
MultiObjective Bayesian Optimization.
In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin,
editors, Advances in Neural Information Processing Systems (NeurIPS
33), pages 9851–9864, 2020.
[ bib ]

[527]

Jean Daunizeau, Hanneke E. M. den Ouden, Matthias Pessiglione, Stefan J.
Kiebel, Karl J. Friston, and Klaas E. Stephan.
Observing the observer (II): deciding when to decide.
PLoS One, 5(12):e15555, 2010.
[ bib 
DOI ]

[528]

Jean Daunizeau, Hanneke E. M. den Ouden, Matthias Pessiglione, Klaas E.
Stephan, Stefan J. Kiebel, and Karl J. Friston.
Observing the observer (I): metaBayesian models of learning
and decisionmaking.
PLoS One, 5(12):e15554, 2010.
[ bib 
DOI ]

[529]

Werner de Schaetzen, Dragan A. Savic, and Godfrey A. Walters.
A genetic algorithm approach to pump scheduling in water
supply.
In V. Babovic and L. C. Larsen, editors, Hydroinformatics '98,
pages 897–899, Rotterdam, Balkema, 1998.
[ bib ]

[530]

Thomas Dean and Mark S. Boddy.
An Analysis of TimeDependent Planning.
In H. E. Shrobe, T. M. Mitchell, and R. G. Smith, editors,
Proceedings of the 7th National Conference on Artificial Intelligence,
AAAI88, pages 49–54. AAAI Press/MIT Press, Menlo Park, CA, 1988.
[ bib 
http ]
Keywords: anytime, performance profiles

[531]

Angela Dean and Daniel Voss.
Design and Analysis of Experiments.
Springer, London, UK, 1999.
[ bib 
DOI ]

[532]

Kalyanmoy Deb.
An efficient constraint handling method for genetic algorithms.
Computer Methods in Applied Mechanics and Engineering,
186(2/4):311–338, 2000.
[ bib 
DOI ]

[533]

Kalyanmoy Deb, A. Pratap, S. Agarwal, and T. Meyarivan.
A fast and elitist multiobjective genetic algorithm:
NSGAII.
IEEE Transactions on Evolutionary Computation, 6(2):182–197,
2002.
[ bib 
DOI ]

[534]

Kalyanmoy Deb.
Multiobjective genetic algorithms: problem difficulties and
construction of test problems.
Evolutionary Computation, 7(3):205–230, 1999.
[ bib ]
Naive definition of PLOset

[535]

Kalyanmoy Deb.
Introduction to evolutionary multiobjective optimization.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science, pages 59–96.
Springer, Heidelberg, 2008.
[ bib 
DOI ]
In its current state, evolutionary multiobjective
optimization (EMO) is an established field of research and
application with more than 150 PhD theses, more than ten
dedicated texts and edited books, commercial softwares and
numerous freely downloadable codes, a biannual conference
series running successfully since 2001, special sessions and
workshops held at all major evolutionary computing
conferences, and fulltime researchers from universities and
industries from all around the globe. In this chapter, we
provide a brief introduction to EMO principles, illustrate
some EMO algorithms with simulated results, and outline the
current research and application potential of EMO. For
solving multiobjective optimization problems, EMO procedures
attempt to find a set of welldistributed Paretooptimal
points, so that an idea of the extent and shape of the
Paretooptimal front can be obtained. Although this task was
the early motivation of EMO research, EMO principles are now
being found to be useful in various other problem solving
tasks, enabling one to treat problems naturally as they
are. One of the major current research thrusts is to combine
EMO procedures with other multiple criterion decision making
(MCDM) tools so as to develop hybrid and interactive
multiobjective optimization algorithms for finding a set of
tradeoff optimal solutions and then choose a preferred
solution for implementation. This chapter provides the
background of EMO principles and their potential to launch
such collaborative studies with MCDM researchers in the
coming years.

[536]

Kalyanmoy Deb.
Multiobjective optimization.
In E. K. Burke and G. Kendall, editors, Search Methodologies,
pages 403–449. Springer, Boston, MA, 2005.
[ bib 
DOI ]

[537]

Kalyanmoy Deb.
MultiObjective Optimization Using Evolutionary Algorithms.
Wiley, Chichester, UK, 2001.
[ bib ]

[538]

Kalyanmoy Deb and S. Agrawal.
A NichedPenalty Approach for Constraint Handling in Genetic
Algorithms.
In A. Dobnikar, N. C. Steele, D. W. Pearson, and R. F. Albrecht,
editors, Artificial Neural Nets and Genetic Algorithms (ICANNGA99),
pages 235–243. Springer Verlag, 1999.
[ bib 
DOI ]
Keywords: polynomial mutation

[539]

Kalyanmoy Deb and Ram Bhushan Agrawal.
Simulated binary crossover for continuous search spaces.
Complex Systems, 9(2):115–148, 1995.
[ bib 
eprint ]
Keywords: SBX

[540]

Kalyanmoy Deb, S. Agarwal, A. Pratap, and T. Meyarivan.
A fast elitist nondominated sorting genetic algorithm for
multiobjective optimization: NSGAII.
In M. Schoenauer et al., editors, Parallel Problem Solving from
Nature – PPSN VI, volume 1917 of Lecture Notes in Computer Science,
pages 849–858. Springer, Heidelberg, 2000.
[ bib ]

[541]

Kalyanmoy Deb and Debayan Deb.
Analysing mutation schemes for realparameter genetic
algorithms.
International Journal of Artificial Intelligence and Soft
Computing, 4(1):1–28, 2014.
[ bib ]

[542]

Kalyanmoy Deb, S. Gupta, D. Daum, Jürgen Branke, A. Mall, and
D. Padmanabhan.
Reliabilitybased optimization using evolutionary algorithms.
IEEE Transactions on Evolutionary Computation,
13(5):1054–1074, October 2009.
[ bib 
DOI ]

[543]

Kalyanmoy Deb and Sachin Jain.
MultiSpeed Gearbox Design Using MultiObjective Evolutionary
Algorithms.
Technical Report 2002001, KanGAL, February 2002.
[ bib ]

[544]

Kalyanmoy Deb and Sachin Jain.
An Evolutionary ManyObjective Optimization Algorithm Using
ReferencePointBased Nondominated Sorting Approach, Part I: Solving
Problems With Box Constraints.
IEEE Transactions on Evolutionary Computation, 18(4):577–601,
2014.
[ bib ]
Proposed NSGAIII

[545]

Kalyanmoy Deb and Murat Köksalan.
Guest Editorial: Special Issue on Preferencebased
Multiobjective Evolutionary Algorithms.
IEEE Transactions on Evolutionary Computation, 14(5):669–670,
October 2010.
[ bib 
DOI ]

[546]

Kalyanmoy Deb and Christie Myburgh.
Breaking the billionvariable barrier in realworld optimization
using a customized evolutionary algorithm.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2016, pages 653–660. ACM Press, New York, NY, 2016.
[ bib ]

[547]

Kalyanmoy Deb and Ankur Sinha.
Solving Bilevel MultiObjective Optimization Problems Using
Evolutionary Algorithms.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pages
110–124. Springer, Heidelberg, 2009.
[ bib ]

[548]

Kalyanmoy Deb and J. Sundar.
Reference point based multiobjective optimization using
evolutionary algorithms.
In M. Cattolico et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2006, pages 635–642. ACM Press,
New York, NY, 2006.
[ bib 
DOI ]

[549]

Kalyanmoy Deb, Rahul Tewari, Mayur Dixit, and Joydeep Dutta.
Finding tradeoff solutions close to KKT points using
evolutionary multiobjective optimization.
In Proceedings of the 2007 Congress on Evolutionary Computation
(CEC 2007), pages 2109–2116. IEEE Press, Piscataway, NJ, 2007.
[ bib ]

[550]

Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler.
Scalable Test Problems for Evolutionary MultiObjective
Optimization.
Technical Report 112, Computer Engineering and Networks Laboratory
(TIK), Swiss Federal Institute of Technology (ETH), Zürich, Switzerland,
2001.
Do not cite this TR! It is incorrect and it is superseeded
by [551].
[ bib ]
Keywords: DTLZ benchmark

[551]

Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler.
Scalable Test Problems for Evolutionary Multiobjective
Optimization.
In A. Abraham, L. Jain, and R. Goldberg, editors, Evolutionary
Multiobjective Optimization, Advanced Information and Knowledge Processing,
pages 105–145. Springer, London, UK, January 2005.
[ bib ]
Keywords: DTLZ benchmark

[552]

Kalyanmoy Deb, Ling Zhu, and Sandeep Kulkarni.
Handling Multiple Scenarios in Evolutionary MultiObjective
Numerical Optimization.
IEEE Transactions on Evolutionary Computation, 22(6):920–933,
2018.
[ bib 
DOI ]
Solutions to most practical numerical optimization problems
must be evaluated for their performance over a number of
different loading or operating conditions, which we refer
here as scenarios. Therefore, a meaningful and resilient
optimal solution must be such that it remains feasible under
all scenarios and performs close to an individual optimal
solution corresponding to each scenario. Despite its
practical importance, multiscenario consideration has
received a lukewarm attention, particularly in the context of
multiobjective optimization. The usual practice is to
optimize for the worstcase scenario. In this paper, we
review existing methodologies in this direction and set our
goal to suggest a new and potential populationbased method
for handling multiple scenarios by defining scenariowise
domination principle and scenariowise diversitypreserving
operators. To evaluate, the proposed method is applied to a
number of numerical test problems and engineering design
problems with a detail explanation of the obtained results
and compared with an existing method. This first systematic
evolutionary based multiscenario, multiobjective,
optimization study on numerical problems indicates that
multiple scenarios can be handled in an integrated manner
using an EMO framework to find a wellbalanced compromise set
of solutions to multiple scenarios and maintain a tradeoff
among multiple objectives. In comparison to an existing
serial multiple optimization approach, the proposed approach
finds a set of compromised tradeoff solutions
simultaneously. An achievement of multiobjective tradeoff
and multiscenario tradeoff is algorithmically challenging,
but due to its practical appeal, further research and
application must be spent.
Keywords: scenariobased

[553]

Annelies De Corte and Kenneth Sörensen.
Optimisation of gravityfed water distribution network design: A
critical review.
European Journal of Operational Research, 228(1):1–10, 2013.
[ bib 
DOI ]

[554]

Annelies De Corte and Kenneth Sörensen.
An Iterated Local Search Algorithm for Water Distribution
Network Design Optimization.
Networks, 67(3):187–198, 2016.
[ bib ]

[555]

Annelies De Corte and Kenneth Sörensen.
An Iterated Local Search Algorithm for multiperiod water
distribution network design optimization.
Water, 8(8):359, 2016.
[ bib 
DOI ]

[556]

William A. Dees, Jr. and Patrick G. Karger.
Automated Ripup and Reroute Techniques.
In DAC'82, Proceedings of the 19th Design Automation Workshop,
pages 432–439. IEEE Press, 1982.
[ bib ]

[557]

V. Dekhtyarenko.
Verification of weight coefficients in multicriteria
optimization problems.
ComputerAided Design, 13(6):339–344, 1981.
[ bib ]

[558]

X. Delorme, Xavier Gandibleux, and F. Degoutin.
Evolutionary, constructive and hybrid procedures for the
biobjective set packing problem.
European Journal of Operational Research, 204(2):206–217,
2010.
[ bib ]
This paper cannot be found on internet!! Does it exist?

[559]

Federico Della Croce, Thierry Garaix, and Andrea Grosso.
Iterated Local Search and Very Large Neighborhoods for the
Parallelmachines Total Tardiness Problem.
Computers & Operations Research, 39(6):1213–1217, 2012.
[ bib ]

[560]

Maxence Delorme, Manuel Iori, and Silvano Martello.
Bin packing and cutting stock problems: Mathematical models and
exact algorithms.
European Journal of Operational Research, 255(1):1–20, 2016.
[ bib 
DOI ]

[561]

Mauro Dell'Amico, Manuel Iori, Silvano Martello, and Michele Monaci.
Heuristic and Exact Algorithms for the Identical Parallel
Machine Scheduling Problem.
INFORMS Journal on Computing, 20(3):333–344, 2016.
[ bib ]

[562]

Maxence Delorme, Manuel Iori, and Silvano Martello.
BPPLIB: a library for bin packing and cutting stock problems.
Optimization Letters, 12(2):235–250, 2018.
[ bib 
DOI ]

[563]

Mauro Dell'Amico, Manuel Iori, Stefano Novellani, and Thomas Stützle.
A destroy and repair algorithm for the Bike sharing Rebalancing
Problem.
Computers & Operations Research, 71:146–162, 2016.
[ bib 
DOI ]
Keywords: irace

[564]

Robert F. Dell and Mark H. Karwan.
An interactive MCDM weight space reduction method utilizing a
Tchebycheff utility function.
Naval Research Logistics, 37(2):263–277, 1990.
[ bib ]

[565]

Mauro Dell'Amico and Marco Trubian.
Applying Tabu Search to the Job Shop Scheduling Problem.
Annals of Operations Research, 41:231–252, 1993.
[ bib ]

[566]

Stephan Dempe, Gabriele Eichfelder, and Jörg Fliege.
On the effects of combining objectives in multiobjective
optimization.
Mathematical Methods of Operations Research, 82(1):1–18, 2015.
[ bib ]

[567]

JeanLouis Deneubourg, S. Aron, S. Goss, and J.M. Pasteels.
The SelfOrganizing Exploratory Pattern of the Argentine Ant.
Journal of Insect Behavior, 3(2):159–168, 1990.
[ bib 
DOI ]

[568]

Matthijs L. den Besten.
Simple Metaheuristics for Scheduling.
PhD thesis, FB Informatik, TU Darmstadt, Germany, 2004.
[ bib 
http ]

[569]

Roman Denysiuk, Lino Costa, and Isabel Espírito Santo.
Manyobjective optimization using differential evolution with
variablewise mutation restriction.
In C. Blum and E. Alba, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2013, pages 591–598. ACM Press,
New York, NY, 2013.
[ bib ]

[570]

Jia Deng, Wei Dong, Richard Socher, LiJia Li, Kai Li, and Li FeiFei.
Imagenet: A largescale hierarchical image database.
In Computer Vision and Pattern Recognition, 2009. CVPR 2009.
IEEE Conference on, pages 248–255. IEEE, 2009.
[ bib ]

[571]

Joaquín Derrac, Salvador García, Daniel Molina, and Francisco Herrera.
A practical tutorial on the use of nonparametric statistical
tests as a methodology for comparing evolutionary and swarm intelligence
algorithms.
Swarm and Evolutionary Computation, 1(1):3–18, 2011.
[ bib ]

[572]

Ulrich Derigs and Ulrich Vogel.
Experience with a Framework for Developing Heuristics for
Solving Rich Vehicle Routing Problems.
Journal of Heuristics, 20(1):75–106, 2014.
[ bib ]

[573]

Marcelo De Souza and Marcus Ritt.
An Automatically Designed Recombination Heuristic for the
TestAssignment Problem.
In Proceedings of the 2018 Congress on Evolutionary Computation
(CEC 2018), pages 1–8, Piscataway, NJ, 2018. IEEE Press.
[ bib 
DOI ]

[574]

Marcelo De Souza and Marcus Ritt.
Automatic GrammarBased Design of Heuristic Algorithms for
Unconstrained Binary Quadratic Programming.
In A. Liefooghe and M. LópezIbáñez, editors,
Proceedings of EvoCOP 2018 – 18th European Conference on Evolutionary
Computation in Combinatorial Optimization, volume 10782 of Lecture
Notes in Computer Science, pages 67–84. Springer, Heidelberg, 2018.
[ bib 
DOI ]

[575]

Marcelo De Souza and Marcus Ritt.
Hybrid Heuristic for Unconstrained Binary Quadratic Programming
– Source Code of HHBQP.
https://github.com/souzamarcelo/hhbqp, 2018.
[ bib ]

[576]

Marcelo De Souza, Marcus Ritt, Manuel LópezIbáñez, and Leslie
Pérez Cáceres.
ACVIZ: A Tool for the Visual Analysis of the
Configuration of Algorithms with irace – Source Code.
https://github.com/souzamarcelo/acviz, 2020.
[ bib ]

[577]

Marcelo De Souza, Marcus Ritt, Manuel LópezIbáñez, and Leslie
Pérez Cáceres.
ACVIZ: Algorithm Configuration
Visualizations for irace: Supplementary material.
http://doi.org/10.5281/zenodo.4714582, September 2020.
[ bib ]

[578]

Marcelo De Souza, Marcus Ritt, Manuel LópezIbáñez, and Leslie
Pérez Cáceres.
ACVIZ: A Tool for the Visual Analysis of the
Configuration of Algorithms with irace.
Operations Research Perspectives, 8:100186, 2021.
[ bib 
DOI 
supplementary material ]
This paper introduces acviz, a tool that helps to analyze the
automatic configuration of algorithms with irace. It provides
a visual representation of the configuration process,
allowing users to extract useful information, e.g. how the
configurations evolve over time. When test data is available,
acviz also shows the performance of each configuration on the
test instances. Using this visualization, users can analyze
and compare the quality of the resulting configurations and
observe the performance differences on training and test
instances.

[579]

Paolo Detti, Francesco Papalini, and Garazi Zabalo Manrique de Lara.
A multidepot dialaride problem with heterogeneous vehicles
and compatibility constraints in healthcare.
Omega, 70:1–14, 2017.
[ bib 
DOI ]

[580]

Sven De Vries and Rakesh V. Vohra.
Combinatorial Auctions: A Survey.
INFORMS Journal on Computing, 15(3):284–309, 2003.
[ bib ]

[581]

Sophie Dewez.
On the toll setting problem.
PhD thesis, Faculté de Sciences, Université Libre de
Bruxelles, 2014.
[ bib ]
Supervised by Dr. Martine Labbé

[582]

Juan Esteban Diaz, Julia Handl, and DongLing Xu.
Evolutionary robust optimization in production planning:
interactions between number of objectives, sample size and choice of
robustness measure.
Computers & Operations Research, 79:266–278, 2017.
[ bib 
DOI ]
Keywords: Evolutionary multiobjective optimization, Production
planning, Robust optimization, Simulationbased optimization,
Uncertainty modelling

[583]

Juan Esteban Diaz, Julia Handl, and DongLing Xu.
Integrating metaheuristics, simulation and exact techniques for
production planning of a failureprone manufacturing system.
European Journal of Operational Research, 266(3):976–989,
2018.
[ bib 
DOI ]
Keywords: Genetic algorithms, Combinatorial optimization, Production
planning, Simulationbased optimization, Uncertainty
modelling

[584]

Juan Esteban Diaz and Manuel LópezIbáñez.
Incorporating DecisionMaker's Preferences into the Automatic
Configuration of BiObjective Optimisation Algorithms.
European Journal of Operational Research, 289(3):1209–1222,
2021.
[ bib 
DOI 
supplementary material ]
Automatic configuration (AC) methods are increasingly used to
tune and design optimisation algorithms for problems with
multiple objectives. Most AC methods use unary quality
indicators, which assign a single scalar value to an
approximation to the Pareto front, to compare the performance
of different optimisers. These quality indicators, however,
imply preferences beyond Paretooptimality that may differ
from those of the decision maker (DM). Although it is
possible to incorporate DM's preferences into quality
indicators, e.g., by means of the weighted hypervolume
indicator (HV^{w}), expressing preferences in terms of weight
function is not always intuitive nor an easy task for a DM,
in particular, when comparing the stochastic outcomes of
several algorithm configurations. A more visual approach to
compare such outcomes is the visualisation of their empirical
attainment functions (EAFs) differences. This paper proposes
using such visualisations as a way of eliciting information
about regions of the objective space that are preferred by
the DM. We present a method to convert the information about
EAF differences into a HV^{w} that will assign higher quality
values to approximation fronts that result in EAF differences
preferred by the DM. We show that the resulting HV^{w} may be
used by an AC method to guide the configuration of
multiobjective optimisers according to the preferences of
the DM. We evaluate the proposed approach on a wellknown
benchmark problem. Finally, we apply our approach to
reconfiguring, according to different DM's preferences, a
multiobjective optimiser tackling a realworld production
planning problem arising in the manufacturing industry.

[585]

L. C. Dias, Vincent Mousseau, José Rui Figueira, and J. N. Clímaco.
An aggregation/disaggregation approach to obtain robust
conclusions with ELECTRE TRI.
European Journal of Operational Research, 138(2):332–348,
April 2002.
[ bib ]

[586]

Gianni A. Di Caro and Marco Dorigo.
AntNet: Distributed Stigmergetic Control for Communications
Networks.
Journal of Artificial Intelligence Research, 9:317–365, 1998.
[ bib ]

[587]

Gianni A. Di Caro, F. Ducatelle, and L. M. Gambardella.
AntHocNet: An adaptive natureinspired algorithm for routing
in mobile ad hoc networks.
European Transactions on Telecommunications, 16(5):443–455,
2005.
[ bib ]

[588]

Diego Díaz, Pablo Valledor, Paula Areces, Jorge Rodil, and Montserrat
Suárez.
An ACO Algorithm to Solve an Extended Cutting Stock Problem
for Scrap Minimization in a Bar Mill.
In M. Dorigo et al., editors, Swarm Intelligence, 9th
International Conference, ANTS 2014, volume 8667 of Lecture Notes in
Computer Science, pages 13–24. Springer, Heidelberg, 2014.
[ bib ]

[589]

Luca Di Gaspero, Marco Chiarandini, and Andrea Schaerf.
A Study on the ShortTerm Prohibition Mechanisms in Tabu
Search.
In G. Brewka, S. Coradeschi, A. Perini, and P. Traverso, editors,
Proceedings of the 17th European Conference on Artificial Intelligence,
ECAI 2006, Riva del Garda, Italy, August29  September 1, 2006, pages
83–87. IOS Press, 2006.
[ bib ]

[590]

Luca Di Gaspero, Andrea Rendl, and Tommaso Urli.
ConstraintBased Approaches for Balancing Bike Sharing Systems.
In C. Schulte, editor, Principles and Practice of Constraint
Programming, volume 8124 of Lecture Notes in Computer Science, pages
758–773. Springer, Heidelberg, 2013.
[ bib 
DOI ]
Keywords: Frace

[591]

Luca Di Gaspero, Andrea Rendl, and Tommaso Urli.
A Hybrid ACO+CP for Balancing Bicycle Sharing Systems.
In M. J. Blesa, C. Blum, P. Festa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics, volume 7919 of Lecture Notes in Computer
Science, pages 198–212. Springer, Heidelberg, 2013.
[ bib 
DOI ]
Keywords: Frace

[592]

Luca Di Gaspero and Andrea Schaerf.
EasyLocal++: An objectoriented framework for flexible
design of local search algorithms.
Software — Practice & Experience, 33(8):733–765, July 2003.
[ bib 
eprint ]
Keywords: software engineering, local search, easylocal

[593]

J.Y. Ding, S. Song, J. N. D. Gupta, R. Zhang, R. Chiong, and C. Wu.
An Improved Iterated Greedy Algorithm with a Tabubased
Reconstruction Strategy for the Nowait Flowshop Scheduling Problem.
Applied Soft Computing, 30:604–613, 2015.
[ bib ]

[594]

Benjamin Doerr, Carola Doerr, and Franziska Ebel.
From blackbox complexity to designing new genetic algorithms.
Theoretical Computer Science, 567:87–104, 2015.
[ bib 
DOI ]

[595]

Benjamin Doerr, Carola Doerr, and Jing Yang.
Optimal parameter choices via precise blackbox analysis.
Theoretical Computer Science, 801:1–34, 2020.
[ bib 
DOI ]

[596]

Karl F. Doerner, Guenther Fuellerer, Manfred Gronalt, Richard F. Hartl, and
Manuel Iori.
Metaheuristics for the Vehicle Routing Problem with Loading
Constraints.
Networks, 49(4):294–307, 2006.
[ bib ]

[597]

Benjamin Doerr, Christian Gießen, Carsten Witt, and Jing Yang.
The (1+λ) evolutionary algorithm with selfadjusting
mutation rate.
Algorithmica, 81(2):593–631, 2019.
[ bib ]

[598]

Karl F. Doerner, Walter J. Gutjahr, Richard F. Hartl, Christine Strauss, and
Christian Stummer.
NatureInspired Metaheuristics in Multiobjective Activity
Crashing.
Omega, 36(6):1019–1037, 2008.
[ bib ]

[599]

Karl F. Doerner, Walter J. Gutjahr, Richard F. Hartl, Christine Strauss, and
Christian Stummer.
Pareto Ant Colony Optimization: A Metaheuristic Approach to
Multiobjective Portfolio Selection.
Annals of Operations Research, 131:79–99, 2004.
[ bib ]

[600]

Karl F. Doerner, Walter J. Gutjahr, Richard F. Hartl, Christine Strauss, and
Christian Stummer.
Pareto ant colony optimization with ILP preprocessing in
multiobjective project portfolio selection.
European Journal of Operational Research, 171:830–841, 2006.
[ bib ]

[601]

Karl F. Doerner, Richard F. Hartl, and Marc Reimann.
Are COMPETants more competent for problem solving? The case
of a multiple objective transportation problem.
Central European Journal for Operations Research and Economics,
11(2):115–141, 2003.
[ bib ]

[602]

Benjamin Doerr, Daniel Johannsen, and Carola Winzen.
Multiplicative drift analysis.
Algorithmica, 64(4):673–697, 2012.
[ bib ]

[603]

Benjamin Doerr, Timo Kötzing, Johannes Lengler, and Carola Winzen.
Blackbox complexities of combinatorial problems.
Theoretical Computer Science, 471:84–106, 2013.
[ bib ]

[604]

Karl F. Doerner, D. Merkle, and Thomas Stützle.
Special issue on Ant Colony Optimization.
Swarm Intelligence, 3(1), 2009.
[ bib ]

[605]

Benjamin Doerr, Frank Neumann, Dirk Sudholt, and Carsten Witt.
Runtime analysis of the 1ANT ant colony optimizer.
Theoretical Computer Science, 412(1):1629–1644, 2011.
[ bib ]

[606]

Dogan Aydin.
Composite artificial bee colony algorithms: From componentbased
analysis to highperforming algorithms.
Applied Soft Computing, 32:266–285, 2015.
[ bib 
DOI ]
Keywords: irace

[607]

JeanPaul Doignon, Aleksandar Pekeč, and Michel Regenwetter.
The repeated insertion model for rankings: Missing link between
two subset choice models.
Psychometrika, 69(1):33–54, March 2004.
[ bib 
DOI ]
Several probabilistic models for subset choice have been
proposed in the literature, for example, to explain approval
voting data. We show that Marley et al.'s latent scale model
is subsumed by Falmagne and Regenwetter's sizeindependent
model, in the sense that every choice probability
distribution generated by the former can also be explained by
the latter. Our proof relies on the construction of a
probabilistic ranking model which we label the “repeated
insertion model”. This model is a special case of Marden's
orthogonal contrast model class and, in turn, includes the
classical Mallows φmodel as a special case. We
explore its basic properties as well as its relationship to
Fligner and Verducci's multistage ranking model.

[608]

Elizabeth D. Dolan and Jorge J. Moré.
Benchmarking optimization software with performance profiles.
Mathematical Programming, 91(2):201–213, 2002.
[ bib ]
Keywords: performance profiles; convergence

[609]

Pedro Domingos and Geoff Hulten.
Mining highspeed data streams.
In R. Ramakrishnan, S. J. Stolfo, R. J. Bayardo, and I. Parsa,
editors, The 6th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, KDD 2000, pages 71–80. ACM Press, New York,
NY, 2000.
[ bib ]
http://dl.acm.org/citation.cfm?id=347090

[610]

Xingye Dong, Ping, Houkuan Huang, and Maciek Nowak.
A Multirestart Iterated Local Search Algorithm for the
Permutation Flow Shop Problem Minimizing Total Flow Time.
Computers & Operations Research, 40(2):627–632, 2013.
[ bib ]

[611]

X. Dong, H. Huang, and P. Chen.
An Iterated Local Search Algorithm for the Permutation Flowshop
Problem with Total Flowtime Criterion.
Computers & Operations Research, 36(5):1664–1669, 2009.
[ bib ]

[612]

A. V. Donati, Roberto Montemanni, N. Casagrande, A. E. Rizzoli, and L. M.
Gambardella.
Time dependent vehicle routing problem with a multi ant colony
system.
European Journal of Operational Research, 185(3):1174–1191,
2008.
[ bib ]

[613]

Marco Dorigo.
Ant Colony Optimization.
Scholarpedia, 2(3):1461, 2007.
[ bib 
DOI ]

[614]

Marco Dorigo.
Swarm intelligence: A few things you need to know if you want to
publish in this journal.
Swarm Intelligence, November 2016.
[ bib 
http ]

[615]

Marco Dorigo, Mauro Birattari, Xiaodong Li, Manuel LópezIbáñez,
Kazuhiro Ohkura, Carlo Pinciroli, and Thomas Stützle.
ANTS 2016 Special Issue: Editorial.
Swarm Intelligence, November 2017.
[ bib 
DOI ]

[616]

Marco Dorigo, Mauro Birattari, and Thomas Stützle.
Ant Colony Optimization: Artificial Ants as a Computational
Intelligence Technique.
IEEE Computational Intelligence Magazine, 1(4):28–39, 2006.
[ bib ]

[617]

Marco Dorigo and Christian Blum.
Ant colony optimization theory: A survey.
Theoretical Computer Science, 344(23):243–278, 2005.
[ bib ]

[618]

Marco Dorigo and Gianni A. Di Caro.
The Ant Colony Optimization MetaHeuristic.
In D. Corne, M. Dorigo, and F. Glover, editors, New Ideas in
Optimization, pages 11–32. McGraw Hill, London, UK, 1999.
[ bib ]

[619]

Marco Dorigo, Gianni A. Di Caro, and L. M. Gambardella.
Ant Algorithms for Discrete Optimization.
Artificial Life, 5(2):137–172, 1999.
[ bib ]

[620]

Marco Dorigo and L. M. Gambardella.
Ant Colony System.
Technical Report IRIDIA/9605, IRIDIA, Université Libre de
Bruxelles, Belgium, 1996.
[ bib ]

[621]

Marco Dorigo and L. M. Gambardella.
Ant Colonies for the Traveling Salesman Problem.
BioSystems, 43(2):73–81, 1997.
[ bib 
DOI ]

[622]

Marco Dorigo and L. M. Gambardella.
Ant Colony System: A Cooperative Learning Approach to the
Traveling Salesman Problem.
IEEE Transactions on Evolutionary Computation, 1(1):53–66,
1997.
[ bib ]
Keywords: Ant Colony System

[623]

Marco Dorigo, L. M. Gambardella, Martin Middendorf, and Thomas Stützle.
Guest Editorial: Special Section on Ant Colony Optimization.
IEEE Transactions on Evolutionary Computation, 6(4):317–320,
2002.
[ bib 
DOI ]
Keywords: ant colony optimization, swarm intelligence

[624]

Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni.
The Ant System: An autocatalytic optimizing process.
Technical Report 91016 Revised, Dipartimento di Elettronica,
Politecnico di Milano, Italy, 1991.
[ bib ]

[625]

Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni.
Ant System: Optimization by a Colony of Cooperating Agents.
IEEE Transactions on Systems, Man, and Cybernetics – Part B,
26(1):29–41, 1996.
[ bib ]

[626]

Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni.
Positive Feedback as a Search Strategy.
Technical Report 91016, Dipartimento di Elettronica, Politecnico di
Milano, Italy, 1991.
[ bib ]

[627]

Marco Dorigo, Marco A. Montes de Oca, Sabrina Oliveira, and Thomas
Stützle.
Ant Colony Optimization.
In J. J. Cochran, editor, Wiley Encyclopedia of Operations
Research and Management Science, volume 1, pages 114–125. John Wiley &
Sons, 2011.
[ bib 
DOI ]

[628]

Marco Dorigo and Thomas Stützle.
The Ant Colony Optimization Metaheuristic: Algorithms,
Applications and Advances.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 251–285. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib ]

[629]

Marco Dorigo and Thomas Stützle.
Ant Colony Optimization.
MIT Press, Cambridge, MA, 2004.
[ bib ]

[630]

Marco Dorigo, Thomas Stützle, and Gianni A. Di Caro.
Special Issue on “Ant Algorithms”.
Future Generation Computer Systems, 16(8), 2000.
[ bib ]
Keywords: swarm intelligence, ant colony optimization

[631]

Marco Dorigo.
Optimization, Learning and Natural Algorithms.
PhD thesis, Dipartimento di Elettronica, Politecnico di Milano,
Italy, 1992.
In Italian.
[ bib ]

[632]

Michael Doumpos and Constantin Zopounidis.
Preference disaggregation and statistical learning for
multicriteria decision support: A review.
European Journal of Operational Research, 209(3):203–214,
2011.
[ bib ]

[633]

Erik Dovgan, Tea Tušar, and Bogdan Filipič.
Parameter tuning in an evolutionary algorithm for commodity
transportation optimization.
Evolutionary Computation, pages 1–8, 2010.
[ bib ]

[634]

Johann Dréo.
Using performance fronts for parameter setting of stochastic
metaheuristics.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2009, pages
2197–2200. ACM Press, New York, NY, 2009.
[ bib 
DOI ]

[635]

Johann Dréo, Carola Doerr, and Yann Semet.
Coupling the design of benchmark with algorithm in
landscapeaware solver design.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference Companion, GECCO Companion 2019, pages 1419–1420. ACM Press, New
York, NY, 2019.
[ bib 
DOI ]

[636]

Johann Dréo and P. Siarry.
A New Ant Colony Algorithm Using the Heterarchical Concept Aimed
at Optimization of Multiminima Continuous Functions.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 216–221. Springer, Heidelberg, 2002.
[ bib ]

[637]

Johann Dréo and P. Siarry.
Continuous interacting ant colony algorithm based on dense
heterarchy.
Future Generation Computer Systems, 20(5):841–856, 2004.
[ bib ]

[638]

Stefan Droste, Thomas Jansen, and Ingo Wegener.
A new framework for the valuation of algorithms for
blackboxoptimization.
In K. A. De Jong, R. Poli, and J. E. Rowe, editors, Proceedings
of the Seventh Workshop on Foundations of Genetic Algorithms (FOGA), pages
253–270. Morgan Kaufmann Publishers, 2002.
[ bib ]

[639]

Stefan Droste, Thomas Jansen, and Ingo Wegener.
Upper and lower bounds for randomized search heuristics in
blackbox optimization.
Theory of Computing Systems, 39(4):525–544, 2006.
[ bib ]

[640]

Chris Drummond.
Replicability is not Reproducibility: Nor is it Good Science.
In Proceedings of the Evaluation Methods for Machine Learning
Workshop at the 26th ICML, Montreal, Canada, 2009.
[ bib 
http ]

[641]

Mădălina M. Drugan and Dirk Thierens.
PathGuided Mutation for Stochastic Pareto Local Search
Algorithms.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of Lecture
Notes in Computer Science, pages 485–495. Springer, Heidelberg, 2010.
[ bib ]

[642]

Mădălina M. Drugan and Dirk Thierens.
Stochastic Pareto local search: Pareto neighbourhood
exploration and perturbation strategies.
Journal of Heuristics, 18(5):727–766, 2012.
[ bib ]

[643]

J. Du and Joseph Y.T. Leung.
Minimizing Total Tardiness on One Machine is NPHard.
Mathematics of Operations Research, 15(3):483–495, 1990.
[ bib ]

[644]

Jérémie DuboisLacoste.
Weight Setting Strategies for TwoPhase Local Search: A Study on
Biobjective Permutation Flowshop Scheduling.
Technical Report TR/IRIDIA/2009024, IRIDIA, Université Libre de
Bruxelles, Belgium, 2009.
[ bib ]

[645]

Jérémie DuboisLacoste, Holger H. Hoos, and Thomas Stützle.
On the Empirical Scaling Behaviour of Stateoftheart Local
Search Algorithms for the Euclidean TSP.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pages 377–384, New York, NY, 2015. ACM Press.
[ bib 
DOI ]

[646]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Effective Hybrid Stochastic Local Search Algorithms for
Biobjective Permutation Flowshop Scheduling.
In M. J. Blesa, C. Blum, L. Di Gaspero, A. Roli, M. Sampels, and
A. Schaerf, editors, Hybrid Metaheuristics, volume 5818 of Lecture
Notes in Computer Science, pages 100–114. Springer, Heidelberg, 2009.
[ bib 
DOI ]

[647]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary material: Improving the Anytime Behavior of
TwoPhase Local Search.
http://iridia.ulb.ac.be/supp/IridiaSupp2010012, 2010.
[ bib ]

[648]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary material: A Hybrid TP+PLS Algorithm for
Biobjective Flowshop Scheduling Problems.
http://iridia.ulb.ac.be/supp/IridiaSupp2010001, 2010.
[ bib ]

[649]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Adaptive “Anytime” TwoPhase Local Search.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pages 52–67. Springer, Heidelberg, 2010.
[ bib 
DOI ]

[650]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary material: Automatic Configuration of
Stateoftheart Multiobjective Optimizers Using the TPLS+PLS Framework.
http://iridia.ulb.ac.be/supp/IridiaSupp2011005, 2011.
[ bib ]

[651]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Improving the Anytime Behavior of TwoPhase Local Search.
Annals of Mathematics and Artificial Intelligence,
61(2):125–154, 2011.
[ bib 
DOI ]

[652]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
A Hybrid TP+PLS Algorithm for Biobjective FlowShop
Scheduling Problems.
Computers & Operations Research, 38(8):1219–1236, 2011.
[ bib 
DOI 
supplementary material ]

[653]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Configuration of Stateoftheart MultiObjective
Optimizers Using the TP+PLS Framework.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages
2019–2026. ACM Press, New York, NY, 2011.
[ bib 
DOI ]

[654]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary Material: Pareto Local Search Variants for
Anytime BiObjective Optimization.
http://iridia.ulb.ac.be/supp/IridiaSupp2012004, 2012.
[ bib ]

[655]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Pareto Local Search Algorithms for Anytime Biobjective
Optimization.
In J.K. Hao and M. Middendorf, editors, Proceedings of EvoCOP
2012 – 12th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7245 of Lecture Notes in Computer Science, pages
206–217. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[656]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Combining Two Search Paradigms for Multiobjective Optimization:
TwoPhase and Pareto Local Search.
In E.G. Talbi, editor, Hybrid Metaheuristics, volume 434 of
Studies in Computational Intelligence, pages 97–117. Springer Verlag,
2013.
[ bib 
DOI 
http ]

[657]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Anytime Pareto Local Search.
European Journal of Operational Research, 243(2):369–385,
2015.
[ bib 
DOI 
supplementary material ]

[658]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary material: Anytime Pareto Local Search.
http://iridia.ulb.ac.be/supp/IridiaSupp2013003, 2013.
[ bib ]

[659]

Jérémie DuboisLacoste, Federico Pagnozzi, and Thomas Stützle.
Supplementary material: An iterated greedy algorithm with
optimization of partial solutions for the permutation flowshop problem.
http://iridia.ulb.ac.be/supp/IridiaSupp2013006, 2017.
[ bib ]

[660]

Jérémie DuboisLacoste, Federico Pagnozzi, and Thomas Stützle.
An Iterated Greedy Algorithm with Optimization of Partial
Solutions for the Permutation Flowshop Problem.
Computers & Operations Research, 81:160–166, 2017.
[ bib 
DOI 
supplementary material ]

[661]

Jérémie DuboisLacoste and Thomas Stützle.
Tuning of a Stigmergybased Traffic Light Controller as a
Dynamic Optimization Problem.
In Proceedings of the 2017 Congress on Evolutionary Computation
(CEC 2017), pages 1–8. IEEE Press, Piscataway, NJ, 2017.
[ bib ]

[662]

Jérémie DuboisLacoste.
A study of Pareto and TwoPhase Local Search Algorithms for
Biobjective Permutation Flowshop Scheduling.
Master's thesis, IRIDIA, Université Libre de Bruxelles, Belgium,
2009.
[ bib ]

[663]

Jérémie DuboisLacoste.
Effective Stochastic Local Search Algorithms For BiObjective
Permutation Flowshop Scheduling.
Rapport d'avancement de recherches présenté pour la formation
doctorale en sciences de l'ingénieur, IRIDIA, Université Libre de
Bruxelles, Belgium, 2010.
[ bib ]

[664]

Jérémie DuboisLacoste.
Anytime Local Search for MultiObjective Combinatorial
Optimization: Design, Analysis and Automatic Configuration.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2014.
[ bib ]
Supervised by Thomas Stützle and Manuel LópezIbáñez

[665]

Fabian Duddeck.
Multidisciplinary optimization of car bodies.
Structural and Multidisciplinary Optimization, 35(4):375–389,
2008.
[ bib 
DOI ]
Evolutionary optimization of car bodies at General Motors

[666]

Gunter Dueck and T. Scheuer.
Threshold Accepting: A General Purpose Optimization Algorithm
Appearing Superior to Simulated Annealing.
Journal of Computational Physics, 90(1):161–175, 1990.
[ bib ]

[667]

Gunter Dueck.
New Optimization Heuristics: the Great Deluge Algorithm and the
RecordToRecord Travel.
Journal of Computational Physics, 104(1):86–92, 1993.
[ bib ]

[668]

Gunter Dueck, Martin Maehler, Johannes Schneider, Gerhard Schrimpf, and Hermann
StammWilbrandt.
Optimization with Ruin Recreate.
US Patent 6,418,398 B1, July 2002.
Filed on October 1, 1999 and granted on July 9, 2002; Assignee is IBM
Corporation, Armonk, NY (US).
[ bib ]

[669]

Rikky R. P. R. Duivenvoorden, Felix Berkenkamp, Nicolas Carion, Andreas Krause,
and Angela P. Schoellig.
Constrained Bayesian Optimization with Particle Swarms for
Safe Adaptive Controller Tuning.
IFACPapersOnLine, 50(1):11800–11807, 2017.
[ bib 
DOI ]
Tuning controller parameters is a recurring and
timeconsuming problem in control. This is especially true in
the field of adaptive control, where good performance is
typically only achieved after significant tuning. Recently,
it has been shown that constrained Bayesian optimization is a
promising approach to automate the tuning process without
risking system failures during the optimization
process. However, this approach is computationally too
expensive for tuning more than a couple of parameters. In
this paper, we provide a heuristic in order to efficiently
perform constrained Bayesian optimization in highdimensional
parameter spaces by using an adaptive discretization based on
particle swarms. We apply the method to the tuning problem of
an L1 adaptive controller on a quadrotor vehicle and show
that we can reliably and automatically tune parameters in
experiments.
20th IFAC World Congress
Keywords: Adaptive Control, Constrained Bayesian Optimization, Safety,
Gaussian Process, Particle Swarm Optimization, Policy Search,
Reinforcement learning

[670]

Cees Duin and Stefan Voß.
The Pilot Method: A Strategy for Heuristic Repetition with
Application to the Steiner Problem in Graphs.
Networks, 34(3):181–191, 1999.
[ bib ]

[671]

Irina Dumitrescu and Thomas Stützle.
Combinations of Local Search and Exact Algorithms.
In G. R. Raidl and J. Gottlieb, editors, Proceedings of EvoCOP
2003 – 3rd European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 2611 of Lecture Notes in Computer Science, pages
211–223. Springer, Heidelberg, 2003.
[ bib 
DOI ]

[672]

Irina Dumitrescu and Thomas Stützle.
Usage of Exact Algorithms to Enhance Stochastic Local Search
Algorithms.
In V. Maniezzo, T. Stützle, and S. Voß, editors,
Matheuristics—Hybridizing Metaheuristics and Mathematical Programming,
volume 10 of Annals of Information Systems, pages 103–134. Springer,
New York, NY, 2009.
[ bib ]

[673]

Y. Dumas, J. Desrosiers, E. Gelinas, and M. M. Solomon.
An Optimal Algorithm for the Traveling Salesman Problem with
Time Windows.
Operations Research, 43(2):367–371, 1995.
[ bib ]

[674]

Olive Jean Dunn.
Multiple Comparisons Using Rank Sums.
Technometrics, 6(3):241–252, 1964.
[ bib ]

[675]

Olive Jean Dunn.
Multiple Comparisons Among Means.
Journal of the American Statistical Association,
56(293):52–64, 1961.
[ bib ]

[676]

Juan J. Durillo, Antonio J. Nebro, Francisco Luna, and Enrique Alba.
On the Effect of the SteadyState Selection Scheme in
MultiObjective Genetic Algorithms.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pages
183–197. Springer, Heidelberg, 2009.
[ bib ]

[677]

Cynthia Dwork, Ravi Kumar, Moni Naor, and D. Sivakumar.
Rank aggregation methods for the Web.
In V. Y. Shen, N. Saito, M. R. Lyu, and M. E. Zurko, editors,
Proceedings of the Tenth International World Wide Web Conference, WWW 10,
pages 613–622. ACM Press, New York, NY, 2001.
[ bib 
DOI ]
Keywords: Kemeny ranking,multiword queries,rank aggregation,ranking
functions,spam

[678]

L. A. Rossman.
EPANET 2 Users Manual.
U.S. Environmental Protection Agency, Cincinnati, USA, 2000.
[ bib ]

[679]

L. A. Rossman.
EPANET User's Guide.
Risk Reduction Engineering Laboratory, Office of Research and
Development, U.S. Environmental Protection Agency, Cincinnati, USA, 1994.
[ bib ]

[680]

L. A. Rossman.
The EPANET Programmer's Toolkit for Analysis of Water
Distribution Systems.
In Proceedings of the Annual Water Resources Planning and
Management Conference, Reston, USA, 1999. ASCE.
[ bib ]

[681]

Russell C. Eberhart and J. Kennedy.
A New Optimizer Using Particle Swarm Theory.
In Proceedings of the Sixth International Symposium on Micro
Machine and Human Science, pages 39–43, 1995.
[ bib ]

[682]

Katharina Eggensperger, Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Efficient Benchmarking of Hyperparameter Optimizers via
Surrogates.
In B. Bonet and S. Koenig, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pages 1114–1120. AAAI Press, 2015.
[ bib ]

[683]

Richard W. Eglese.
Simulated Annealing: a Tool for Operational Research.
European Journal of Operational Research, 46(3):271–281, 1990.
[ bib ]

[684]

Werner Ehm.
Reproducibility from the perspective of metaanalysis.
In H. Atmanspacher and S. Maasen, editors, Reproducibility –
Principles, problems, practices and prospects, pages 141–168. Wiley, 2016.
[ bib ]

[685]

Matthias Ehrgott.
A discussion of scalarization techniques for multiple objective
integer programming.
Annals of Operations Research, 147(1):343–360, 2006.
[ bib ]

[686]

Matthias Ehrgott and Xavier Gandibleux.
Approximative Solution Methods for Combinatorial Multicriteria
Optimization.
TOP, 12(1):1–88, 2004.
[ bib ]

[687]

Matthias Ehrgott and Xavier Gandibleux.
Hybrid Metaheuristics for Multiobjective Combinatorial
Optimization.
In C. Blum, M. J. Blesa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics: An emergent approach for optimization, volume 114 of
Studies in Computational Intelligence, pages 221–259. Springer,
Berlin, Germany, 2008.
[ bib 
DOI ]
Many realworld optimization problems can be
modelled as combinatorial optimization
problems. Often, these problems are characterized by
their large size and the presence of multiple,
conflicting objectives. Despite progress in solving
multiobjective combinatorial optimization problems
exactly, the large size often means that heuristics
are required for their solution in acceptable time.
Since the middle of the nineties the trend is
towards heuristics that “pick and choose” elements
from several of the established metaheuristic
schemes. Such hybrid approximation techniques may
even combine exact and heuristic approaches. In this
chapter we give an overview over approximation
methods in multiobjective combinatorial
optimization. We briefly summarize “classical”
metaheuristics and focus on recent approaches, where
metaheuristics are hybridized and/or combined with
exact methods.

[688]

Matthias Ehrgott.
Multicriteria Optimization, volume 491 of Lecture Notes in
Economics and Mathematical Systems.
Springer, Berlin, Germany, 2000.
[ bib ]

[689]

Matthias Ehrgott.
Multicriteria Optimization.
Springer, Berlin, Germany, 2nd edition, 2005.
[ bib 
DOI ]

[690]

Agoston E. Eiben, Robert Hinterding, and Zbigniew Michalewicz.
Parameter Control in Evolutionary Algorithms.
IEEE Transactions on Evolutionary Computation, 3(2):124–141,
1999.
[ bib ]

[691]

Agoston E. Eiben, Mark Horvath, Wojtek Kowalczyk, and Martijn C. Schut.
Reinforcement learning for online control of evolutionary
algorithms.
In International Workshop on Engineering SelfOrganising
Applications, pages 151–160. Springer, 2006.
[ bib ]

[692]

Agoston E. Eiben and M. Jelasity.
A critical note on experimental research methodology in EC.
In Proceedings of the 2002 Congress on Evolutionary Computation
(CEC'02), pages 582–587, Piscataway, NJ, 2002. IEEE Press.
[ bib 
DOI ]
Discusses reproducibility, generalizability and separation
between training (for tuning and experimentation) and testing
instances (for comparisons).

[693]

Agoston E. Eiben, Zbigniew Michalewicz, Marc Schoenauer, and James E. Smith.
Parameter Control in Evolutionary Algorithms.
In F. Lobo, C. F. Lima, and Z. Michalewicz, editors, Parameter
Setting in Evolutionary Algorithms, pages 19–46. Springer, Berlin, Germany,
2007.
[ bib ]

[694]

Agoston E. Eiben and Günther Rudolph.
Theory of evolutionary algorithms: A bird's eye view.
Theoretical Computer Science, 229(12):3–9, 1999.
[ bib ]

[695]

Agoston E. Eiben and James E. Smith.
Introduction to Evolutionary Computing.
Springer, 2003.
[ bib ]

[696]

Agoston E. Eiben and James E. Smith.
Introduction to Evolutionary Computing.
Natural Computing Series. Springer, 2nd edition, 2007.
[ bib ]

[697]

Agoston E. Eiben and Selmar K. Smit.
Parameter Tuning for Configuring and Analyzing Evolutionary
Algorithms.
Swarm and Evolutionary Computation, 1(1):19–31, 2011.
[ bib 
DOI ]

[698]

Sibel Eker and Jan H. Kwakkel.
Including robustness considerations in the search phase of
ManyObjective Robust Decision Making.
Environmental Modelling & Software, 105:201–216, 2018.
[ bib ]
Keywords: scenariobased

[699]

Mohammed ElAbd.
Oppositionbased Artificial Bee Colony Algorithm.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 109–116.
ACM Press, New York, NY, 2011.
[ bib ]

[700]

Jeffrey L Elman.
Distributed representations, simple recurrent networks, and
grammatical structure.
Machine Learning, 7(23):195–225, 1991.
[ bib ]

[701]

V. A. Emelichev and V. A. Perepelitsa.
Complexity of Vector Optimization Problems on Graphs.
Optimization, 22(6):906–918, 1991.
[ bib 
DOI ]

[702]

V. A. Emelichev and V. A. Perepelitsa.
On the Cardinality of the Set of Alternatives in Discrete
Manycriterion Problems.
Discrete Mathematics and Applications, 2(5):461–471, 1992.
[ bib ]

[703]

Michael T. M. Emmerich, André H. Deutz, and J. W. Klinkenberg.
Hypervolumebased expected improvement: Monotonicity properties
and exact computation.
In Proceedings of the 2011 Congress on Evolutionary Computation
(CEC 2011), pages 2147–2154, Piscataway, NJ, 2011. IEEE Press.
[ bib 
DOI ]
Proposed Expected Hypervolume Improvement (EHVI)

[704]

Michael T. M. Emmerich and Carlos M. Fonseca.
Computing Hypervolume Contributions in Low Dimensions:
Asymptotically Optimal Algorithm and Complexity Results.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pages 121–135. Springer, Heidelberg, 2011.
[ bib 
DOI ]
Given a finite set Y ⊂R^{d} of n mutually
nondominated vectors in d ≥2 dimensions, the
hypervolume contribution of a point y ∈Y is the
difference between the hypervolume indicator of Y
and the hypervolume indicator of Y ∖ {y}. In
multiobjective metaheuristics, hypervolume
contributions are computed in several selection and
boundedsize archiving procedures. This paper
presents new results on the (time) complexity of
computing all hypervolume contributions. It is
proved that for d = 2,3 the problem has time
complexity Θ(n logn), and, for d > 3,
the time complexity is bounded below by Ω(n
logn). Moreover, complexity bounds are derived for
computing a single hypervolume contribution. A
dimension sweep algorithm with time complexity
O (n logn) and space
complexity O(n) is
proposed for computing all hypervolume contributions
in three dimensions. It improves the complexity of
the best known algorithm for d = 3 by a factor of
√(n) . Theoretical results
are complemented by performance tests on randomly
generated testproblems.

[705]

Michael T. M. Emmerich, K. C. Giannakoglou, and Boris Naujoks.
Single and multiobjective evolutionary optimization assisted by
Gaussian random field metamodels.
IEEE Transactions on Evolutionary Computation, 10(4):421–439,
2006.
[ bib 
DOI ]

[706]

Alexander Engau and Margaret M. Wiecek.
2D decisionmaking for multicriteria design optimization.
Structural and Multidisciplinary Optimization, 34:301–315,
2007.
[ bib 
DOI ]

[707]

Alexander Engau and Margaret M. Wiecek.
Interactive coordination of objective decompositions in
multiobjective programming.
Management Science, 54(7):1350–1363, 2008.
[ bib ]

[708]

Stefan Eppe, Yves De Smet, and Thomas Stützle.
A biobjective optimization model to eliciting decision maker's
preferences for the PROMETHEE II method.
In R. I. Brafman, F. Roberts, and A. Tsoukiàs, editors,
Algorithmic Decision Theory, Third International Conference, ADT 2011,
volume 6992 of Lecture Notes in Artificial Intelligence, pages 56–66.
Springer, Heidelberg, 2011.
[ bib ]

[709]

Stefan Eppe, Manuel LópezIbáñez, Thomas Stützle, and Yves De
Smet.
An Experimental Study of Preference Model Integration into
MultiObjective Optimization Heuristics.
In Proceedings of the 2011 Congress on Evolutionary Computation
(CEC 2011), pages 2751–2758. IEEE Press, Piscataway, NJ, 2011.
[ bib 
DOI ]

[710]

David Eriksson, Michael Pearce, Jacob Gardner, Ryan D. Turner, and Matthias
Poloczek.
Scalable Global Optimization via Local Bayesian Optimization.
In H. M. Wallach, H. Larochelle, A. Beygelzimer,
F. d'AlchéBuc, E. B. Fox, and R. Garnett, editors, Advances in
Neural Information Processing Systems (NeurIPS 32), pages 5496–5507, 2019.
[ bib 
eprint ]

[711]

Emre Ertin, Anthony N. Dean, Mathew L. Moore, and Kevin L. Priddy.
Dynamic Optimization for Optimal Control of Water Distribution
Systems.
In K. L. Priddy, P. E. Keller, and P. J. Angeline, editors,
Applications and Science of Computational Intelligence IV, Proceedings of
SPIE, volume 4390, pages 142–149, March 2001.
[ bib ]

[712]

V. Esat and M. Hall.
Water resources system optimization using genetic algorithms.
In A. Verwey, A. Minns, V. Babovic, and C. Maksimović, editors,
Hydroinformatics'94, pages 225–231, Balkema, Rotterdam, The
Netherlands, 1994.
[ bib ]

[713]

Larry J. Eshelman and J. David Schaffer.
RealCoded Genetic Algorithms and IntervalSchemata.
In D. Whitley, editor, Foundations of Genetic Algorithms
(FOGA), pages 187–202. Morgan Kaufmann Publishers, 1993.
[ bib ]

[714]

Larry J. Eshelman, A. Caruana, and J. David Schaffer.
Biases in the Crossover Landscape.
In J. D. Schaffer, editor, Proc. of the Third Int. Conf. on
Genetic Algorithms, pages 86–91. Morgan Kaufmann Publishers, San Mateo, CA,
1989.
[ bib ]

[715]

Imen Essafi, Yazid Mati, and Stéphane DauzèrePèrés.
A Genetic Local Search Algorithm for Minimizing Total Weighted
Tardiness in the Jobshop Scheduling Problem.
Computers & Operations Research, 35(8):2599–2616, 2008.
[ bib ]

[716]

C. J. Eyckelhof and M. Snoek.
Ant Systems for a Dynamic TSP: Ants Caught in a Traffic
Jam.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 88–99. Springer, Heidelberg, 2002.
[ bib ]

[717]

Stefan Falkner, Marius Thomas Lindauer, and Frank Hutter.
SpySMAC: Automated configuration and performance analysis of
SAT solvers.
In M. Heule and S. Weaver, editors, Theory and Applications of
Satisfiability Testing – SAT 2015, volume 9340 of Lecture Notes in
Computer Science, pages 215–222. Springer, Cham, Switzerland, 2015.
[ bib 
DOI ]

[718]

Wei Fan and Albert Bifet.
Mining big data: current status, and forecast to the future.
ACM sIGKDD Explorations Newsletter, 14(2):1–5, 2013.
[ bib ]

[719]

Daniele Fanelli.
Negative results are disappearing from most disciplines and
countries.
Scientometrics, 90(3):891–904, 2012.
[ bib 
DOI ]
Concerns that the growing competition for funding and
citations might distort science are frequently discussed, but
have not been verified directly. Of the hypothesized
problems, perhaps the most worrying is a worsening of
positiveoutcome bias. A system that disfavours negative
results not only distorts the scientific literature directly,
but might also discourage highrisk projects and pressure
scientists to fabricate and falsify their data. This study
analysed over 4,600 papers published in all disciplines
between 1990 and 2007, measuring the frequency of papers
that, having declared to have “tested” a hypothesis,
reported a positive support for it. The overall frequency of
positive supports has grown by over 22% between 1990 and
2007, with significant differences between disciplines and
countries. The increase was stronger in the social and some
biomedical disciplines. The United States had published, over
the years, significantly fewer positive results than Asian
countries (and particularly Japan) but more than European
countries (and in particular the United
Kingdom). Methodological artefacts cannot explain away these
patterns, which support the hypotheses that research is
becoming less pioneering and/or that the objectivity with
which results are produced and published is decreasing.

[720]

M. Farina and P. Amato.
On the Optimal Solution Definition for Manycriteria
Optimization Problems.
In Proceedings of the NAFIPSFLINT International
Conference'2002, pages 233–238, Piscataway, New Jersey, June 2002. IEEE
Service Center.
[ bib ]

[721]

H. Faria, Jr, S. Binato, Mauricio G. C. Resende, and D. J. Falcão.
Power transmission network design by a greedy randomized
adaptive path relinking approach.
IEEE Transactions on Power Systems, 20(1):43–49, 2005.
[ bib ]

[722]

R. Farmani, Godfrey A. Walters, and Dragan A. Savic.
Evolutionary multiobjective optimization of the design and
operation of water distribution network: total cost vs. reliability vs. water
quality.
Journal of Hydroinformatics, 8(3):165–179, 2006.
[ bib ]

[723]

D. Favaretto, E. Moretti, and Paola Pellegrini.
Ant colony system approach for variants of the traveling
salesman problem with time windows.
Journal of Information and Optimization Sciences, 27(1):35–54,
2006.
[ bib ]

[724]

D. Favaretto, E. Moretti, and Paola Pellegrini.
Ant Colony System for a VRP with Multiple Time Windows and
Multiple Visits.
Journal of Interdisciplinary Mathematics, 10(2):263–284, 2007.
[ bib ]

[725]

D. Favaretto, E. Moretti, and Paola Pellegrini.
On the explorative behavior of MaxMin Ant System.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2009, volume 5752 of Lecture Notes
in Computer Science, pages 115–119. Springer, Heidelberg, 2009.
[ bib ]

[726]

Chris Fawcett, Malte Helmert, Holger H. Hoos, Erez Karpas, Gabriele Röger,
and Jendrik Seipp.
FDAutotune: DomainSpecific Configuration using
FastDownward.
In E. Karpas, S. Jiménez Celorrio, and S. Kambhampati, editors,
Proceedings of ICAPSPAL11, 2011.
[ bib ]

[727]

Chris Fawcett and Holger H. Hoos.
Analysing Differences between Algorithm Configurations through
Ablation.
In Proceedings of MIC 2013, the 10th Metaheuristics
International Conference, pages 123–132, 2013.
[ bib 
eprint ]

[728]

Chris Fawcett and Holger H. Hoos.
Analysing Differences Between Algorithm Configurations through
Ablation.
Journal of Heuristics, 22(4):431–458, 2016.
[ bib ]

[729]

T. A. Feo and Mauricio G. C. Resende.
A Probabilistic Heuristic for a Computationally Difficult Set
Covering Problem.
Operations Research Letters, 8(2):67–71, 1989.
[ bib ]

[730]

T. A. Feo and Mauricio G. C. Resende.
Greedy Randomized Adaptive Search Procedures.
Journal of Global Optimization, 6(2):109–113, 1995.
[ bib ]

[731]

T. A. Feo, Mauricio G. C. Resende, and S. H. Smith.
A Greedy Randomized Adaptive Search Procedure for Maximum
Independent Set.
Operations Research, 42:860–878, October 1994.
[ bib ]

[732]

Silvino Fernández, Segundo Álvarez, Diego Díaz, Miguel Iglesias,
and Borja Ena.
Scheduling a Galvanizing Line by Ant Colony Optimization.
In M. Dorigo et al., editors, Swarm Intelligence, 9th
International Conference, ANTS 2014, volume 8667 of Lecture Notes in
Computer Science, pages 146–157. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[733]

Silvino Fernández, Segundo Álvarez, Eneko Malatsetxebarria, Pablo
Valledor, and Diego Díaz.
Performance Comparison of Ant Colony Algorithms for the
Scheduling of Steel Production Lines.
In J. L. Jiménez Laredo, S. Silva, and A. I.
EsparciaAlcázar, editors, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2015. ACM
Press, New York, NY, 2015.
[ bib 
DOI ]
Keywords: irace

[734]

José C. Ferreira, Carlos M. Fonseca, and António GasparCunha.
Methodology to select solutions from the Paretooptimal set: a
comparative study.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pages 789–796. ACM Press,
New York, NY, 2007.
[ bib ]

[735]

Victor FernandezViagas and Jose M. Framiñán.
On Insertion Tiebreaking Rules in Heuristics for the
Permutation Flowshop Scheduling Problem.
Computers & Operations Research, 45:60–67, 2014.
[ bib ]

[736]

Victor FernandezViagas and Jose M. Framiñán.
A Beamsearchbased Constructive Heuristic for the PFSP to
Minimise Total Flowtime.
Computers & Operations Research, 81:167–177, 2017.
[ bib ]

[737]

Victor FernandezViagas and Jose M. Framiñán.
Iteratedgreedybased algorithms with beam search initialization
for the permutation flowshop to minimise total tardiness.
Expert Systems with Applications, 94:58–69, 2018.
[ bib ]

[738]

Javier Ferrer, José GarcíaNieto, Enrique Alba, and Francisco Chicano.
Intelligent Testing of Traffic Light Programs: Validation in
Smart Mobility Scenarios.
Mathematical Problems in Engineering, 2016:1–19, 2016.
[ bib 
DOI ]

[739]

Alberto Ferrer, Daniel Guimarans, Helena Ramalhinho Lourenço, and
Angel A. Juan.
A BRILS Metaheuristic for Nonsmooth Flowshop Problems with
Failurerisk Costs.
Expert Systems with Applications, 44:177–186, 2016.
[ bib ]

[740]

Javier Ferrer, Manuel LópezIbáñez, and Enrique Alba.
Reliable SimulationOptimization of Traffic Lights in a
RealWorld City.
Applied Soft Computing, 78:697–711, 2019.
[ bib 
DOI 
supplementary material ]

[741]

Eduardo Fernandez, Jorge Navarro, and Sergio Bernal.
Multicriteria Sorting Using a Valued Indifference Relation Under
a Preference Disaggregation Paradigm.
European Journal of Operational Research, 198(2):602–609,
2009.
[ bib ]

[742]

Victor FernandezViagas, Rubén Ruiz, and Jose M. Framiñán.
A New Vision of Approximate Methods for the Permutation Flowshop
to Minimise Makespan: Stateoftheart and Computational Evaluation.
European Journal of Operational Research, 257(3):707–721,
2017.
[ bib ]

[743]

R. Ferreira da Silva and S. Urrutia.
A general VNS heuristic for the traveling salesman problem
with time windows.
Discrete Optimization, 7(4):203–211, 2010.
[ bib ]
Keywords: TSPTW

[744]

Silvino Fernández, Pablo Valledor, Diego Díaz, Eneko Malatsetxebarria,
and Miguel Iglesias.
Criticality of Response Time in the usage of Metaheuristics in
Industry.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference Companion,
GECCO Companion 2016, pages 937–940. ACM Press, New York, NY, 2016.
[ bib ]

[745]

Victor FernandezViagas, Jorge M. S. Valente, and Jose M. Framiñán.
Iteratedgreedybased algorithms with Beam Search Initialization
for the Permutation Flowshop to Minimise Total Tardiness.
Expert Systems with Applications, 94:58–69, 2018.
[ bib ]

[746]

F. J. Ferri, P. Pudil, M. Hatef, and J. Kittler.
Comparative study of techniques for largescale feature
selection.
In E. S. Gelsema and L. S. Kanal, editors, Pattern Recognition
in Practice IV, volume 16 of Machine Intelligence and Pattern
Recognition, pages 403–413. NorthHolland, 1994.
[ bib 
DOI ]
The combinatorial search problem arising in feature selection
in high dimensional spaces is considered. Recently developed
techniques based on the classical sequential methods and the
(l, r) search called Floating search algorithms are compared
against the Genetic approach to feature subset search. Both
approaches have been designed with the view to give a good
compromise between efficiency and effectiveness for large
problems. The purpose of this paper is to investigate the
applicability of these techniques to high dimensional
problems of feature selection. The aim is to establish
whether the properties inferred for these techniques from
medium scale experiments involving up to a few tens of
dimensions extend to dimensionalities of one order of
magnitude higher. Further, relative merits of these
techniques visavis such high dimensional problems are
explored and the possibility of exploiting the best aspects
of these methods to create a composite feature selection
procedure with superior properties is considered.
Describes sequential forward / backward selection

[747]

Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel
Blum, and Frank Hutter.
Efficient and robust automated machine learning.
In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett,
editors, Advances in Neural Information Processing Systems (NIPS 28),
pages 2962–2970, 2015.
[ bib 
http ]

[748]

Álvaro Fialho, Luis Da Costa, Marc Schoenauer, and Michèle Sebag.
Analyzing Banditbased Adaptive Operator Selection Mechanisms.
Annals of Mathematics and Artificial Intelligence,
60(1–2):25–64, 2010.
[ bib ]

[749]

Álvaro Fialho, Raymond Ros, Marc Schoenauer, and Michèle Sebag.
Comparisonbased adaptive strategy selection with bandits in
differential evolution.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of Lecture
Notes in Computer Science, pages 194–203. Springer, Heidelberg, 2010.
[ bib ]

[750]

Álvaro Fialho, Marc Schoenauer, and Michèle Sebag.
FitnessAUC bandit adaptive strategy selection vs. the
probability matching one within differential evolution: an empirical
comparison on the BBOB2010 noiseless testbed.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference Companion, GECCO Companion 2010,
pages 1535–1542. ACM Press, New York, NY, 2010.
[ bib ]

[751]

Álvaro Fialho, Marc Schoenauer, and Michèle Sebag.
Toward comparisonbased adaptive operator selection.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pages 767–774. ACM
Press, New York, NY, 2010.
[ bib ]
Proposed FAUC and FSR

[752]

Álvaro Fialho.
Adaptive operator selection for optimization.
PhD thesis, Université Paris SudParis XI, 2010.
[ bib ]

[753]

Mark J. Fielding.
Simulated Annealing with an Optimal Fixed Temperature.
SIAM Journal on Optimization, 11(2):289–307, 2000.
[ bib ]

[754]

Jonathan E. Fieldsend and Richard M. Everson.
Visualising highdimensional Pareto relationships in
twodimensional scatterplots.
In R. C. Purshouse, P. J. Fleming, C. M. Fonseca, S. Greco, and
J. Shaw, editors, Evolutionary Multicriterion Optimization, EMO 2013,
volume 7811 of Lecture Notes in Computer Science, pages 558–572.
Springer, Heidelberg, 2013.
[ bib 
DOI ]

[755]

Jonathan E. Fieldsend, Richard M. Everson, and Sameer Singh.
Using unconstrained elite archives for multiobjective
optimization.
IEEE Transactions on Evolutionary Computation, 7(3):305–323,
2003.
[ bib ]

[756]

José Rui Figueira, Carlos M. Fonseca, Pascal Halffmann, Kathrin Klamroth,
Luís Paquete, Stefan Ruzika, Britta Schulze, Michael Stiglmayr, and David
Willems.
Easy to say they are Hard, but Hard to see they are EasyTowards
a Categorization of Tractable Multiobjective Combinatorial Optimization
Problems.
Journal of MultiCriteria Decision Analysis, 24(12):82–98,
2017.
[ bib 
DOI ]

[757]

Andreas Fink and Stefan Voß.
HotFrame: A Heuristic Optimization Framework.
In S. Voß and D. L. Woodruff, editors, Optimization Software
Class Libraries, pages 81–154. Kluwer Academic Publishers, Boston, MA,
2002.
[ bib ]

[758]

Benjamin Fisset, Clarisse Dhaenens, and Laetitia Jourdan.
MOMine_{}clust: A Framework for MultiObjective
Clustering.
In C. Dhaenens, L. Jourdan, and M.E. Marmion, editors, Learning
and Intelligent Optimization, 9th International Conference, LION 9, volume
8994 of Lecture Notes in Computer Science, pages 293–305. Springer,
Heidelberg, 2015.
[ bib ]
Keywords: irace

[759]

Matteo Fischetti, Fred Glover, and Andrea Lodi.
The feasibility pump.
Mathematical Programming, 104(1):91–104, 2005.
[ bib ]

[760]

Matteo Fischetti and Andrea Lodi.
Local Branching.
Mathematical Programming Series B, 98:23–47, 2003.
[ bib ]

[761]

Matteo Fischetti and Michele Monaci.
Proximity search for 01 mixedinteger convex programming.
Journal of Heuristics, 20(6):709–731, 2014.
[ bib ]

[762]

Matteo Fischetti and Michele Monaci.
Exploiting Erraticism in Search.
Operations Research, 62(1):114–122, 2014.
[ bib 
DOI ]
High sensitivity to initial conditions is generally viewed
as a drawback of tree search methods because it leads to
erratic behavior to be mitigated somehow. In this paper we
investigate the opposite viewpoint and consider this behavior
as an opportunity to exploit. Our working hypothesis is that
erraticism is in fact just a consequence of the exponential
nature of tree search that acts as a chaotic amplifier, so it
is largely unavoidable. We propose a betandrun approach to
actually turn erraticism to one's advantage. The idea is to
make a number of short sample runs with randomized initial
conditions, to bet on the "most promising" run selected
according to certain simple criteria, and to bring it to
completion. Computational results on a large testbed of mixed
integer linear programs from the literature are presented,
showing the potential of this approach even when embedded in
a proofofconcept implementation.
http://mat.tepper.cmu.edu/blog/?p=1695

[763]

Matteo Fischetti, Michele Monaci, and Domenico Salvagnin.
Three Ideas for the Quadratic Assignment Problem.
Operations Research, 60(4):954–964, 2012.
[ bib ]

[764]

Matteo Fischetti and Domenico Salvagnin.
Feasibility pump 2.0.
Mathematical Programming Computation, 1(2–3):201–222, 2009.
[ bib ]

[765]

Charles Fleurent and Fred Glover.
Improved constructive multistart strategies for the quadratic
assignment problem using adaptive memory.
INFORMS Journal on Computing, 11(2):198–204, 1999.
[ bib ]

[766]

Peter J. Fleming, Robin C. Purshouse, and Robert J. Lygoe.
Manyobjective optimization: An engineering design perspective.
In C. A. Coello Coello, A. H. Aguirre, and E. Zitzler, editors,
Evolutionary Multicriterion Optimization, EMO 2005, volume 3410 of
Lecture Notes in Computer Science, pages 14–32. Springer, Heidelberg,
2005.
[ bib ]

[767]

M. Fleischer.
The Measure of Pareto Optima. Applications to Multiobjective
Metaheuristics.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 519–533. Springer,
Heidelberg, 2003.
[ bib ]

[768]

R. Fletcher.
Practical methods of optimization.
John Wiley & Sons, New York, NY, 1987.
[ bib ]
BFGS

[769]

Jörg Fliege.
The effects of adding objectives to an optimisation problem on
the solution set.
Operations Research Letters, 35(6):782–790, 2007.
[ bib ]

[770]

Michael A. Fligner and Joseph S. Verducci.
Distance Based Ranking Models.
Journal of the Royal Statistical Society: Series B
(Methodological), 48(3):359–369, 1986.
[ bib 
DOI ]
Keywords: Mallows model, ranking, probabilistic models

[771]

M. M. Flood.
The Travelling Salesman Problem.
Operations Research, 4:61–75, 1956.
[ bib ]

[772]

D. Floreano and L. Keller.
Evolution of Adaptive Behaviour in Robots by Means of
Darwinian Selection.
PLoS Biology, 8(1):e1000292, 2010.
[ bib 
DOI ]

[773]

Dario Floreano and Francesco Mondada.
Automatic creation of an autonomous agent: Genetic evolution of
a neural network driven robot.
In D. Cliff, P. Husbands, J.A. Meyer, and S. Wilson, editors,
Proceedings of the third international conference on Simulation of adaptive
behavior: From Animals to Animats 3, pages 421–430. MIT Press, Cambridge,
MA, 1994.
[ bib ]
LISCONF1994003

[774]

D. Floreano and J. Urzelai.
Evolutionary robots with online selforganization and
behavioral fitness.
Neural Networks, 13(45):431–443, 2000.
[ bib ]

[775]

Filippo Focacci, François Laburthe, and Andrea Lodi.
Local Search and Constraint Programming.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 369–403. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib ]

[776]

Filippo Focacci, Andrea Lodi, and Michela Milano.
A Hybrid Exact Algorithm for the TSPTW.
INFORMS Journal on Computing, 14:403–417, 2002.
[ bib ]

[777]

David B. Fogel, Alvin J. Owens, and Michael J. Walsh.
Artificial Intelligence Through Simulated Evolution.
John Wiley & Sons, 1966.
[ bib ]

[778]

David B. Fogel.
Evolutionary Computation. Toward a New Philosophy of Machine
Intelligence.
IEEE Press, 1995.
[ bib ]

[779]

Carlos M. Fonseca and Peter J. Fleming.
Genetic Algorithms for Multiobjective Optimization: Formulation,
Discussion and Generalization.
In S. Forrest, editor, ICGA, pages 416–423. Morgan Kaufmann
Publishers, 1993.
[ bib 
eprint ]
Proposes MOGA

[780]

Carlos M. Fonseca and Peter J. Fleming.
On the Performance Assessment and Comparison of Stochastic
Multiobjective Optimizers.
In H.M. Voigt et al., editors, Parallel Problem Solving from
Nature – PPSN IV, volume 1141 of Lecture Notes in Computer Science,
pages 584–593. Springer, Heidelberg, 1996.
[ bib ]

[781]

Carlos M. Fonseca and Peter J. Fleming.
Multiobjective Optimization and Multiple Constraint Handling
with Evolutionary Algorithms (II): Application Example.
IEEE Transactions on Systems, Man, and Cybernetics – Part A,
28(1):38–44, January 1998.
[ bib 
DOI ]

[782]

Carlos M. Fonseca and Peter J. Fleming.
Multiobjective Optimization and Multiple Constraint Handling
with Evolutionary Algorithms (I): A Unified Formulation.
IEEE Transactions on Systems, Man, and Cybernetics – Part A,
28(1):26–37, January 1998.
[ bib 
DOI ]

[783]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
The Relationship between the Covered Fraction, Completeness and
Hypervolume Indicators.
In J.K. Hao, P. Legrand, P. Collet, N. Monmarché, E. Lutton, and
M. Schoenauer, editors, Artificial Evolution: 10th International
Conference, Evolution Artificielle, EA, 2011, volume 7401 of Lecture
Notes in Computer Science, pages 25–36. Springer, Heidelberg, 2012.
[ bib ]

[784]

Carlos M. Fonseca, Viviane Grunert da Fonseca, and Luís Paquete.
Exploring the Performance of Stochastic Multiobjective
Optimisers with the SecondOrder Attainment Function.
In C. A. Coello Coello, A. H. Aguirre, and E. Zitzler, editors,
Evolutionary Multicriterion Optimization, EMO 2005, volume 3410 of
Lecture Notes in Computer Science, pages 250–264. Springer,
Heidelberg, 2005.
[ bib 
DOI ]
The attainment function has been proposed as a
measure of the statistical performance of stochastic
multiobjective optimisers which encompasses both the
quality of individual nondominated solutions in
objective space and their spread along the tradeoff
surface. It has also been related to results from
random closedset theory, and cast as a meanlike,
firstorder moment measure of the outcomes of
multiobjective optimisers. In this work, the use of
more informative, secondorder moment measures for
the evaluation and comparison of multiobjective
optimiser performance is explored experimentally,
with emphasis on the interpretability of the
results.

[785]

Carlos M. Fonseca, Andreia P. Guerreiro, Manuel LópezIbáñez, and
Luís Paquete.
On the Computation of the Empirical Attainment Function.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pages 106–120. Springer, Heidelberg, 2011.
[ bib 
DOI ]

[786]

Carlos M. Fonseca, Luís Paquete, and Manuel LópezIbáñez.
An improved dimension sweep algorithm for the
hypervolume indicator.
In Proceedings of the 2006 Congress on Evolutionary Computation
(CEC 2006), pages 1157–1163. IEEE Press, Piscataway, NJ, July 2006.
[ bib 
DOI ]
This paper presents a recursive, dimensionsweep
algorithm for computing the hypervolume indicator of
the quality of a set of n nondominated points in
d>2 dimensions. It improves upon the existing HSO
(Hypervolume by Slicing Objectives) algorithm by
pruning the recursion tree to avoid repeated
dominance checks and the recalculation of partial
hypervolumes. Additionally, it incorporates a recent
result for the threedimensional special case. The
proposed algorithm achieves O(n^{d2} logn) time
and linear space complexity in the worstcase, but
experimental results show that the pruning
techniques used may reduce the time complexity
exponent even further.

[787]

Jorge Ramón Fonseca Cacho and Kazem Taghva.
The State of Reproducible Research in Computer Science.
In S. Latifi, editor, 17th International Conference on
Information TechnologyNew Generations (ITNG 2020), Advances in
Intelligent Systems and Computing, pages 519–524. Springer International
Publishing, 2020.
[ bib 
DOI ]
Reproducible research is the cornerstone of cumulative
science and yet is one of the most serious crisis that we
face today in all fields. This paper aims to describe the
ongoing reproducible research crisis along with
counterarguments of whether it really is a crisis, suggest
solutions to problems limiting reproducible research along
with the tools to implement such solutions by covering the
latest publications involving reproducible research.
Keywords: Docker, Improving transparency, OCR, Open science,
Replicability, Reproducibility

[788]

Alexander I. J. Forrester and Andy J. Keane.
Recent advances in surrogatebased optimization.
Progress in Aerospace Sciences, 45(13):50–79, 2009.
[ bib 
DOI ]
Keywords: Kriging; Gaussian Process; EGO; Design of Experiments

[789]

Manuel Förster, Bettina Bickel, Bernd Hardung, and Gabriella Kókai.
Selfadaptive ant colony optimisation applied to function
allocation in vehicle networks.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pages 1991–1998. ACM
Press, New York, NY, 2007.
[ bib ]

[790]

Michael Foster, Matthew Hughes, George O'Brien, Pietro S. Oliveto, James Pyle,
Dirk Sudholt, and James Williams.
Do sophisticated evolutionary algorithms perform better than
simple ones?
In C. A. Coello Coello, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2020, pages 184–192, New York,
NY, 2020. ACM Press.
[ bib 
DOI ]

[791]

Robert Fourer, David M. Gay, and Brian W. Kernighan.
AMPL: A Modeling Language for Mathematical Programming.
Duxbury, 2nd edition, 2002.
[ bib ]

[792]

Bennett L. Fox.
Uniting probabilistic methods for optimization.
In Proceedings of the 24th conference on Winter simulation,
pages 500–505. ACM, 1992.
[ bib ]

[793]

Bennett L. Fox.
Integrating and accelerating tabu search, simulated annealing,
and genetic algorithms.
Annals of Operations Research, 41(2):47–67, 1993.
[ bib ]

[794]

Bennett L. Fox.
Simulated annealing: folklore, facts, and directions.
In Monte Carlo and QuasiMonte Carlo Methods in Scientific
Computing, pages 17–48. Springer, 1995.
[ bib ]

[795]

Peter I. Frazier.
A Tutorial on Bayesian Optimization.
Arxiv preprint arXiv:1807.02811, 2018.
[ bib 
http ]

[796]

C. B. Fraser.
Subsequences and Supersequences of Strings.
PhD thesis, University of Glasgow, 1995.
[ bib ]

[797]

G. Francesca, M. Brambilla, A. Brutschy, Vito Trianni, and Mauro Birattari.
AutoMoDe: A Novel Approach to the Automatic Design of Control
Software for Robot Swarms.
Swarm Intelligence, 8(2):89–112, 2014.
[ bib 
DOI ]

[798]

Gianpiero Francesca, Manuele Brambilla, Arne Brutschy, Lorenzo Garattoni, Roman
Miletitch, Gaetan Podevijn, Andreagiovanni Reina, Touraj Soleymani, Mattia
Salvaro, Carlo Pinciroli, Franco Mascia, Vito Trianni, and Mauro Birattari.
AutoMoDeChocolate: Automatic Design of Control Software for
Robot Swarms.
Swarm Intelligence, 2015.
[ bib 
DOI ]
Keywords: Swarm robotics; Automatic design; AutoMoDe

[799]

Jose M. Framiñán, Jatinder N.D. Gupta, and Rainer Leisten.
A Review and Classification of Heuristics for Permutation
Flowshop Scheduling with Makespan Objective.
Journal of the Operational Research Society, 55(12):1243–1255,
2004.
[ bib ]

[800]

Alberto Franzin, Raphaël Gyory, JeanCharles Nadé, Guillaume Aubert,
Georges Klenkle, and Hughes Bersini.
Philéas: Anomaly Detection for IoT Monitoring.
In L. Cao, W. Kosters, and J. Lijffijt, editors, Proceedings of
the 32nd Benelux Conference on Artificial Intelligence, BNAIC 2020, Leiden,
The Netherlands, 1920 November 2020, pages 56–70, 2020.
[ bib 
http ]

[801]

Jose M. Framiñán, Rainer Leisten, and Rubén Ruiz.
Manufacturing Scheduling Systems: An Integrated View on Models,
Methods, and Tools.
Springer, New York, NY, 2014.
[ bib ]

[802]

Alberto Franzin, Leslie Pérez Cáceres, and Thomas Stützle.
Effect of Transformations of Numerical Parameters in Automatic
Algorithm Configuration.
Optimization Letters, 12(8):1741–1753, 2018.
[ bib 
DOI ]

[803]

Alberto Franzin, Francesco Sambo, and Barbara Di Camillo.
bnstruct: an R package for Bayesian Network
structure learning in the presence of missing data.
Bioinformatics, 33(8):1250–1252, 2016.
[ bib ]

[804]

Alberto Franzin and Thomas Stützle.
Exploration of Metaheuristics through Automatic Algorithm
Configuration Techniques and Algorithmic Frameworks.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference Companion,
GECCO Companion 2016, pages 1341–1347. ACM Press, New York, NY, 2016.
[ bib ]

[805]

Alberto Franzin and Thomas Stützle.
Comparison of Acceptance Criteria in Randomized Local Searches.
In E. Lutton, P. Legrand, P. Parrend, N. Monmarché, and
M. Schoenauer, editors, EA 2017: Artificial Evolution, volume 10764 of
Lecture Notes in Computer Science, pages 16–29, Heidelberg, 2017.
Springer.
[ bib ]

[806]

Alberto Franzin and Thomas Stützle.
Revisiting Simulated Annealing: a ComponentBased Analysis:
Supplementaty Material.
http://iridia.ulb.ac.be/supp/IridiaSupp2018001, 2018.
[ bib ]

[807]

Alberto Franzin and Thomas Stützle.
Revisiting Simulated Annealing: A ComponentBased Analysis.
Computers & Operations Research, 104:191–206, 2019.
[ bib 
DOI ]

[808]

Alberto Franzin and Thomas Stützle.
Towards transferring algorithm configurations across problems.
In M. Vlastelica, J. Song, A. Ferber, B. Amos, G. Martius,
B. Dilkina, and Y. Yue, editors, Learning Meets Combinatorial Algorithms
Workshop at NeurIPS 2020, LMCA 2020, Vancouver, Canada, December 12, 2020,
pages 1–6, 2020.
[ bib ]

[809]

Alberto Franzin and Thomas Stützle.
A causal framework for understanding optimisation algorithms.
In F. Heintz, M. Milano, and B. O'Sullivan, editors, Trustworthy
AI – Integrating Learning, Optimization and Reasoning. TAILOR 2020, volume
12641 of Lecture Notes in Computer Science, pages 140–145. Springer,
Cham, Switzerland, 2020.
[ bib ]

[810]

Alberto Franzin and Thomas Stützle.
A Landscapebased Analysis of Fixed Temperature and Simulated
Annealing: Supplementaty Material.
http://iridia.ulb.ac.be/supp/IridiaSupp2021002, 2021.
[ bib ]

[811]

Brendan J. Frey and Delbert Dueck.
Clustering by Passing Messages Between Data Points.
Science, 315(5814):972–976, February 2007.
[ bib 
DOI ]
Keywords: clustering; affinity propagation

[812]

Alan R. R. de Freitas, Peter J. Fleming, and Frederico G. Guimarães.
Aggregation trees for visualization and dimension reduction in
manyobjective optimization.
Information Sciences, 298:288–314, 2015.
[ bib ]

[813]

B. Freisleben and P. Merz.
A Genetic Local Search Algorithm for Solving Symmetric and
Asymmetric Traveling Salesman Problems.
In T. Bäck, T. Fukuda, and Z. Michalewicz, editors,
Proceedings of the 1996 IEEE International Conference on Evolutionary
Computation (ICEC'96), pages 616–621, Piscataway, NJ, 1996. IEEE Press.
[ bib ]

[814]

Hela Frikha, Habib Chabchoub, and JeanMarc Martel.
Inferring criteria's relative importance coefficients in
PROMETHEE II.
International Journal of Operational Research, 7(2):257–275,
2010.
[ bib ]

[815]

Tobias Friedrich, Andreas Göbel, Francesco Quinzan, and Markus Wagner.
HeavyTailed Mutation Operators in SingleObjective
Combinatorial Optimization.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature  PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pages 134–145.
Springer, Cham, Switzerland, 2018.
[ bib ]
A core feature of evolutionary algorithms is their mutation
operator. Recently, much attention has been devoted to the
study of mutation operators with dynamic and nonuniform
mutation rates. Following up on this line of work, we propose
a new mutation operator and analyze its performance on the
(1+1) Evolutionary Algorithm (EA). Our analyses show that
this mutation operator competes with preexisting ones, when
used by the (1+1)EA on classes of problems for which
results on the other mutation operators are available. We
present a “jump” function for which the performance of the
(1+1)EA using any static uniform mutation and any restart
strategy can be worse than the performance of the (1+1)EA
using our mutation operator with no restarts. We show that
the (1+1)EA using our mutation operator finds a
(1/3)approximation ratio on any nonnegative submodular
function in polynomial time. This performance matches that of
combinatorial local search algorithms specifically designed
to solve this problem.

[816]

Matteo Frigo and Steven G. Johnson.
The Design and Implementation of FFTW3.
Proceedings of the IEEE, 93(2):216–231, 2005.
Special issue on “Program Generation, Optimization, and Platform
Adaptation”.
[ bib ]

[817]

Tobias Friedrich, Timo Kötzing, Martin S. Krejca, and Andrew M. Sutton.
Robustness of Ant Colony Optimization to Noise.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pages 17–24. ACM Press, New York, NY, 2015.
[ bib 
DOI ]
Keywords: ant colony optimization, noisy fitness, run time analysis,
theory

[818]

Tobias Friedrich, Timo Kötzing, and Markus Wagner.
A Generic BetandRun Strategy for Speeding Up Stochastic Local
Search.
In S. P. Singh and S. Markovitch, editors, Proceedings of the
AAAI Conference on Artificial Intelligence, pages 801–807. AAAI Press,
February 2017.
[ bib ]

[819]

Tobias Friedrich, Francesco Quinzan, and Markus Wagner.
Escaping Large Deceptive Basins of Attraction with Heavytailed
Mutation Operators.
In H. E. Aguirre and K. Takadama, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2018, pages 293–300.
ACM Press, New York, NY, 2018.
[ bib 
DOI ]
Keywords: combinatorial optimization, heavytailed mutation,
singleobjective optimization, experimentsmotivated theory,
irace

[820]

Milton Friedman.
The use of ranks to avoid the assumption of normality implicit
in the analysis of variance.
Journal of the American Statistical Association,
32(200):675–701, 1937.
[ bib ]

[821]

Michael Friendly.
Statistical graphics for multivariate data.
In SAS Conference Proceedings: SAS Users Group International 16
(SUGI 16), 1991.
[ bib ]
February 1720, 1991, New Orleans, Louisiana, 297 papers

[822]

Z Fu, R Eglese, and L Y O Li.
A unified tabu search algorithm for vehicle routing problems
with soft time windows.
Journal of the Operational Research Society, 59(5):663–673,
2008.
[ bib ]

[823]

D. Fudenberg and J. Tirole.
Game Theory.
MIT Press, Cambridge, MA, 1983.
[ bib ]

[824]

Guenther Fuellerer, Karl F. Doerner, Richard F. Hartl, and Manuel Iori.
Metaheuristics for vehicle routing problems with
threedimensional loading constraints.
European Journal of Operational Research, 201(3):751–759,
2009.
[ bib 
DOI ]

[825]

Guenther Fuellerer, Karl F. Doerner, Richard F. Hartl, and Manuel Iori.
Ant colony optimization for the twodimensional loading vehicle
routing problem.
Computers & Operations Research, 36(3):655–673, 2009.
[ bib ]

[826]

Alex S. Fukunaga.
Evolving Local Search Heuristics for SAT Using Genetic
Programming.
In K. Deb et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2004, Part II, volume 3103 of
Lecture Notes in Computer Science, pages 483–494. Springer,
Heidelberg, 2004.
[ bib ]
Satisfiability testing (SAT) is a very active area
of research today, with numerous realworld
applications. We describe CLASS2.0, a genetic
programming system for semiautomatically designing
SAT local search heuristics. An empirical
comparison shows that that the heuristics generated
by our GP system outperform the state of the art
humandesigned local search algorithms, as well as
previously proposed evolutionary approaches, with
respect to both runtime as well as search efficiency
(number of variable flips to solve a problem).

[827]

Alex S. Fukunaga.
Automated Discovery of Local Search Heuristics for
Satisfiability Testing.
Evolutionary Computation, 16(1):31–61, March 2008.
[ bib 
DOI ]
The development of successful metaheuristic
algorithms such as local search for a difficult
problem such as satisfiability testing (SAT) is a
challenging task. We investigate an evolutionary
approach to automating the discovery of new local
search heuristics for SAT. We show that several
wellknown SAT local search algorithms such as
Walksat and Novelty are composite heuristics that
are derived from novel combinations of a set of
building blocks. Based on this observation, we
developed CLASS, a genetic programming system that
uses a simple composition operator to automatically
discover SAT local search heuristics. New
heuristics discovered by CLASS are shown to be
competitive with the best Walksat variants,
including Novelty+. Evolutionary algorithms have
previously been applied to directly evolve a
solution for a particular SAT instance. We show
that the heuristics discovered by CLASS are also
competitive with these previous, direct evolutionary
approaches for SAT. We also analyze the local
search behavior of the learned heuristics using the
depth, mobility, and coverage metrics proposed by
Schuurmans and Southey.

[828]

Nancy E. Furlong, Eugene A. Lovelace, and Kristin L. Lovelace.
Research Methods and Statistics: An Integrated Approach.
Harcourt College Publishers, 2000.
[ bib ]

[829]

Grigori Fursin, Yuriy Kashnikov, Abdul Wahid Memon, Zbigniew Chamski, Olivier
Temam, Mircea Namolaru, Elad YomTov, Bilha Mendelson, Ayal Zaks, Eric
Courtois, Francois Bodin, Phil Barnard, Elton Ashton, Edwin Bonilla, John
Thomson, Christopher K. I. Williams, and Michael O'Boyle.
Milepost GCC: Machine Learning Enabled Selftuning Compiler.
International Journal of Parallel Programming, 39(3):296–327,
2011.
[ bib 
DOI ]

[830]

D. Gaertner and K. Clark.
On Optimal Parameters for Ant Colony Optimization Algorithms.
In H. R. Arabnia and R. Joshua, editors, Proceedings of the 2005
International Conference on Artificial Intelligence, ICAI 2005, pages
83–89. CSREA Press, 2005.
[ bib ]

[831]

Matteo Gagliolo and Catherine Legrand.
Algorithm Survival Analysis.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 161–184. Springer, Berlin, Germany, 2010.
[ bib 
DOI 
eprint ]
Algorithm selection is typically based on models of
algorithm performance,learned during a separate
offline training sequence, which can be
prohibitively expensive. In recent work, we adopted
an online approach, in which models of the runtime
distributions of the available algorithms are
iteratively updated and used to guide the allocation
of computational resources, while solving a sequence
of problem instances. The models are estimated using
survival analysis techniques, which allow us to
reduce computation time, censoring the runtimes of
the slower algorithms. Here, we review the
statistical aspects of our online selection method,
discussing the bias induced in the runtime
distributions (RTD) models by the competition of
different algorithms on the same problem instances.

[832]

Caroline Gagné, W. L. Price, and M. Gravel.
Comparing an ACO algorithm with other heuristics for the
single machine scheduling problem with sequencedependent setup times.
Journal of the Operational Research Society, 53:895–906, 2002.
[ bib ]

[833]

Philippe Galinier and JinKao Hao.
Hybrid evolutionary algorithms for graph coloring.
Journal of Combinatorial Optimization, 3(4):379–397, 1999.
[ bib 
DOI ]

[834]

L. M. Gambardella and Marco Dorigo.
Ant Colony System Hybridized with a New Local Search for the
Sequential Ordering Problem.
INFORMS Journal on Computing, 12(3):237–255, 2000.
[ bib ]

[835]

L. M. Gambardella and Marco Dorigo.
AntQ: A Reinforcement Learning Approach to the Traveling
Salesman Problem.
In A. Prieditis and S. Russell, editors, Proceedings of the
Twelfth International Conference on Machine Learning (ML95), pages
252–260. Morgan Kaufmann Publishers, Palo Alto, CA, 1995.
[ bib ]

[836]

L. M. Gambardella and Marco Dorigo.
Solving Symmetric and Asymmetric TSPs by Ant Colonies.
In T. Bäck, T. Fukuda, and Z. Michalewicz, editors,
Proceedings of the 1996 IEEE International Conference on Evolutionary
Computation (ICEC'96), pages 622–627, Piscataway, NJ, 1996. IEEE Press.
[ bib ]

[837]

L. M. Gambardella, Roberto Montemanni, and Dennis Weyland.
Coupling Ant Colony Systems with Strong Local Searches.
European Journal of Operational Research, 220(3):831–843,
2012.
[ bib 
DOI ]

[838]

L. M. Gambardella, Éric D. Taillard, and G. Agazzi.
MACSVRPTW: A Multiple Ant Colony System for Vehicle Routing
Problems with Time Windows.
In D. Corne, M. Dorigo, and F. Glover, editors, New Ideas in
Optimization, pages 63–76. McGraw Hill, London, UK, 1999.
[ bib ]

[839]

Xavier Gandibleux, X. Delorme, and V. T'Kindt.
An Ant Colony Optimisation Algorithm for the Set Packing
Problem.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 49–60. Springer, Heidelberg, 2004.
[ bib ]

[840]

Xavier Gandibleux, Andrzej Jaszkiewicz, A. Fréville, and Roman
Slowiński.
Special Issue on Multiple Objective Metaheuristics.
Journal of Heuristics, 6(3), 2000.
[ bib ]

[841]

Xavier Gandibleux, N. Mezdaoui, and A. Fréville.
A tabu search procedure to solve multiobjective combinatorial
optimization problem.
In R. Caballero, F. Ruiz, and R. Steuer, editors, Advances in
Multiple Objective and Goal Programming, volume 455 of Lecture Notes in
Economics and Mathematical Systems, pages 291–300, Heidelberg, 1997.
Springer.
[ bib ]

[842]

Xavier Gandibleux, H. Morita, and N. Katoh.
Use of a genetic heritage for solving the assignment problem
with two objectives.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 43–57. Springer,
Heidelberg, 2003.
[ bib ]

[843]

Kaizhou Gao, Yicheng Zhang, Ali Sadollah, and Rong Su.
Optimizing urban traffic light scheduling problem using harmony
search with ensemble of local search.
Applied Soft Computing, 48:359–372, November 2016.
[ bib 
DOI ]
Keywords: harmony search algorithm,traffic light scheduling

[844]

Huiru Gao, Haifeng Nie, and Ke Li.
Visualisation of Pareto Front Approximation: A Short Survey
and Empirical Comparisons.
In Proceedings of the 2019 Congress on Evolutionary Computation
(CEC 2019), pages 1750–1757, Piscataway, NJ, 2019. IEEE Press.
[ bib 
DOI ]

[845]

Huiru Gao, Haifeng Nie, and Ke Li.
Visualisation of Pareto Front Approximation: A Short Survey
and Empirical Comparisons.
Arxiv preprint arXiv:1903.01768, 2019.
[ bib 
http ]

[846]

José GarcíaNieto, Enrique Alba, and Ana Carolina Olivera.
Swarm intelligence for traffic light scheduling: Application to
real urban areas.
Engineering Applications of Artificial Intelligence,
25(2):274–283, March 2012.
[ bib ]
Keywords: Cycle program optimization,Particle swarm
optimization,Realistic traffic instances,SUMO microscopic
simulator of urban mobility,Traffic light scheduling

[847]

Carlos GarcíaMartínez, Oscar Cordón, and Francisco Herrera.
A taxonomy and an empirical analysis of multiple objective ant
colony optimization algorithms for the bicriteria TSP.
European Journal of Operational Research, 180(1):116–148,
2007.
[ bib ]

[848]

Deon Garrett and Dipankar Dasgupta.
Multiobjective landscape analysis and the generalized assignment
problem.
In V. Maniezzo, R. Battiti, and J.P. Watson, editors, Learning
and Intelligent Optimization, Second International Conference, LION 2,
volume 5313 of Lecture Notes in Computer Science, pages 110–124.
Springer, Heidelberg, 2008.
[ bib ]

[849]

Javier García and Fernando Fernández.
A comprehensive survey on safe reinforcement learning.
Journal of Machine Learning Research, 16(1):1437–1480, 2015.
[ bib 
eprint ]

[850]

Salvador García, Alberto Fernández, Julián Luengo, and Francisco
Herrera.
Advanced nonparametric tests for multiple comparisons in the
design of experiments in computational intelligence and data mining:
Experimental analysis of power.
Information Sciences, 180(10):2044–2064, 2010.
[ bib ]

[851]

Carlos GarcíaMartínez, Fred Glover, Francisco J. Rodríguez, Manuel
Lozano, and Rafael Martí.
Strategic Oscillation for the Quadratic Multiple Knapsack
Problem.
Computational Optimization and Applications, 58(1):161–185,
2014.
[ bib ]

[852]

M. R. Garey and David S. Johnson.
Computers and Intractability: A Guide to the Theory of
NPCompleteness.
Freeman & Co, San Francisco, CA, 1979.
[ bib ]

[853]

M. R. Garey, David S. Johnson, and R. Sethi.
The Complexity of Flowshop and Jobshop Scheduling.
Mathematics of Operations Research, 1:117–129, 1976.
[ bib ]

[854]

Salvador García, Daniel Molina, Manuel Lozano, and Francisco Herrera.
A study on the use of nonparametric tests for analyzing the
evolutionary algorithms' behaviour: a case study on the CEC'2005 Special
Session on Real Parameter Optimization.
Journal of Heuristics, 15(617):617–644, 2009.
[ bib 
DOI ]

[855]

José GarcíaNieto, Ana Carolina Olivera, and Enrique Alba.
Optimal Cycle Program of Traffic Lights With Particle Swarm
Optimization.
IEEE Transactions on Evolutionary Computation, 17(6):823–839,
December 2013.
[ bib 
DOI ]

[856]

Carlos GarcíaMartínez, Francisco J. Rodríguez, and Manuel Lozano.
Arbitrary function optimisation with metaheuristics: No free
lunch and realworld problems.
Soft Computing, 16(12):2115–2133, 2012.
[ bib 
DOI ]

[857]

Carlos GarcíaMartínez, Francisco J. Rodríguez, and Manuel Lozano.
Tabuenhanced Iterated Greedy Algorithm: A Case Study in the
Quadratic Multiple Knapsack Problem.
European Journal of Operational Research, 232(3):454–463,
2014.
[ bib ]

[858]

Beatriz A. Garro, Humberto Sossa, and Roberto A. Vazquez.
Evolving ant colony system for optimizing path planning in
mobile robots.
In Electronics, Robotics and Automotive Mechanics Conference,
pages 444–449, Los Alamitos, CA, 2007. IEEE Computer Society.
[ bib 
DOI ]

[859]

Luca Di Gaspero and Andrea Schaerf.
Easysyn++: A tool for automatic synthesis of stochastic local
search algorithms.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2007, volume 4638 of Lecture Notes
in Computer Science, pages 177–181. Springer, Heidelberg, 2007.
[ bib ]

[860]

Gauci Melvin, Tony J. Dodd, and Roderich Groß.
Why `GSA: a gravitational search algorithm' is not genuinely
based on the law of gravity.
Natural Computing, 11(4):719–720, 2012.
[ bib ]

[861]

Martin Gebser, Roland Kaminski, Benjamin Kaufmann, Torsten Schaub,
Marius Thomas Schneider, and Stefan Ziller.
A portfolio solver for answer set programming: Preliminary
report.
In P. Calabar and T. C. Son, editors, Logic Programming and
Nonmonotonic Reasoning, volume 8148 of Lecture Notes in Artificial
Intelligence, pages 352–357. Springer, Heidelberg, 2013.
[ bib ]

[862]

Peter Geibel.
Reinforcement Learning for MDPs with Constraints.
In J. Fürnkranz, T. Scheffer, and M. Spiliopoulou, editors,
Machine Learning: ECML 2006, volume 4212 of Lecture Notes in Computer
Science, pages 646–653, 2006.
[ bib 
DOI ]
In this article, I will consider Markov Decision Processes
with two criteria, each defined as the expected value of an
infinite horizon cumulative return. The second criterion is
either itself subject to an inequality constraint, or there
is maximum allowable probability that the single returns
violate the constraint. I describe and discuss three new
reinforcement learning approaches for solving such control
problems.
Keywords: Safe RL

[863]

Martin Josef Geiger.
Decision Support for Multiobjective Flow Shop Scheduling by the
Pareto Iterated Local Search Methodology.
Computers and Industrial Engineering, 61(3):805–812, 2011.
[ bib ]

[864]

Martin Josef Geiger.
A Multithreaded Local Search Algorithm and Computer
Implementation for the Multimode, Resourceconstrained Multiproject
Scheduling Problem.
European Journal of Operational Research, 256:729–741, 2017.
[ bib ]

[865]

Stuart Geman and Donald Geman.
Stochastic Relaxation, Gibbs Distributions, and the Bayesian
Restoration of Images.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
6(6):721–741, 1984.
[ bib ]

[866]

Ian P. Gent, Stuart A. Grant, Ewen MacIntyre, Patrick Prosser, Paul Shaw,
Barbara M. Smith, and Toby Walsh.
How Not To Do It.
Technical Report 97.27, School of Computer Studies, University of
Leeds, May 1997.
[ bib ]
We give some dos and don'ts for those analysing algorithms
experimentally. We illustrate these with many examples from
our own research on the study of algorithms for NPcomplete
problems such as satisfiability and constraint
satisfaction. Where we have not followed these maxims, we
have suffered as a result.

[867]

Michel Gendreau, Francois Guertin, JeanYves Potvin, and Éric D. Taillard.
Parallel tabu search for realtime vehicle routing and
dispatching.
Transportation Science, 33(4):381–390, 1999.
[ bib ]

[868]

Michel Gendreau, Francois Guertin, JeanYves Potvin, and René Séguin.
Neighborhood search heuristics for a dynamic vehicle dispatching
problem with pickups and deliveries.
Transportation Research Part C: Emerging Technologies,
14(3):157–174, 2006.
[ bib ]

[869]

Ian P. Gent, Holger H. Hoos, P. Prosser, and T. Walsh.
Morphing: Combining Structure and Randomness.
In Proceedings of the Sixteenth National Conference on
Artificial Intelligence, pages 654–660, 1999.
[ bib ]

[870]

Mitsuo Gen and Lin Lin.
Multiobjective evolutionary algorithm for manufacturing
scheduling problems: stateoftheart survey.
Journal of Intelligent Manufacturing, 25(5):849–866, 2014.
[ bib ]

[871]

Robin Genuer, JeanMichel Poggi, and Christine TuleauMalot.
Variable selection using random forests.
Pattern Recognition Letters, 31(14):2225–2236, 2010.
[ bib ]

[872]

Michel Gendreau and JeanYves Potvin.
Tabu Search.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, pages 41–59. Springer, New York, NY, 2nd
edition, 2010.
[ bib ]

[873]

Michel Gendreau, A. Hertz, Gilbert Laporte, and M. Stan.
A Generalized Insertion Heuristic for the Traveling Salesman
Problem with Time Windows.
Operations Research, 46:330–335, 1998.
[ bib ]

[874]

Samuel J. Gershman, Eric J. Horvitz, and Joshua B. Tenenbaum.
Computational rationality: A converging paradigm for
intelligence in brains, minds, and machines.
Science, 349(6245):273–278, 2015.
[ bib 
DOI 
eprint ]
After growing up together, and mostly growing apart in the
second half of the 20th century, the fields of artificial
intelligence (AI), cognitive science, and neuroscience are
reconverging on a shared view of the computational
foundations of intelligence that promotes valuable
crossdisciplinary exchanges on questions, methods, and
results. We chart advances over the past several decades that
address challenges of perception and action under uncertainty
through the lens of computation. Advances include the
development of representations and inferential procedures for
largescale probabilistic inference and machinery for
enabling reflection and decisions about tradeoffs in effort,
precision, and timeliness of computations. These tools are
deployed toward the goal of computational rationality:
identifying decisions with highest expected utility, while
taking into consideration the costs of computation in complex
realworld problems in which most relevant calculations can
only be approximated. We highlight key concepts with examples
that show the potential for interchange between computer
science, cognitive science, and neuroscience.

[875]

Daniel Geschwender, Frank Hutter, Lars Kotthoff, Yuri Malitsky, Holger H. Hoos,
and Kevin LeytonBrown.
Algorithm Configuration in the Cloud: A Feasibility Study.
In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, and J. L.
Walteros, editors, Learning and Intelligent Optimization, 8th
International Conference, LION 8, volume 8426 of Lecture Notes in
Computer Science, pages 41–46. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[876]

Pierre Geurts, Damien Ernst, and Louis Wehenkel.
Extremely randomized trees.
Machine Learning, 63(1):3–42, March 2006.
[ bib 
DOI ]
Proposed ExtraTrees

[877]

Sanjay Ghemawat, Howard Gobioff, and ShunTak Leung.
The Google File System.
SIGOPS Oper. Syst. Rev., 37(5):29–43, December 2003.
[ bib ]

[878]

K. Ghoseiri and B. Nadjari.
An ant colony optimization algorithm for the biobjective
shortest path problem.
Applied Soft Computing, 10(4):1237–1246, 2010.
[ bib ]

[879]

Matthew S. Gibbs, Graeme C. Dandy, Holger R. Maier, and John B. Nixon.
Calibrating genetic algorithms for water distribution system
optimisation.
In 7th Annual Symposium on Water Distribution Systems Analysis.
ASCE, May 2005.
[ bib ]

[880]

Nicolas Girerd, Muriel Rabilloud, Philippe Pibarot, Patrick Mathieu, and Pascal
Roy.
Quantification of Treatment Effect Modification on Both an
Additive and Multiplicative Scale.
PLoS One, 11(4):1–14, 4 2016.
[ bib 
DOI ]

[881]

Fred Glover.
Heuristics for Integer Programming Using Surrogate Constraints.
Decision Sciences, 8:156–166, 1977.
[ bib ]

[882]

Fred Glover.
Future Paths for Integer Programming and Links to Artificial
Intelligence.
Computers & Operations Research, 13(5):533–549, 1986.
[ bib ]

[883]

Fred Glover.
Tabu Search – Part I.
INFORMS Journal on Computing, 1(3):190–206, 1989.
[ bib 
DOI ]

[884]

Fred Glover.
Tabu Search – Part II.
INFORMS Journal on Computing, 2(1):4–32, 1990.
[ bib ]

[885]

Fred Glover.
A Template for Scatter Search and Path Relinking.
In J.K. Hao, E. Lutton, E. M. A. Ronald, M. Schoenauer, and
D. Snyers, editors, Artificial Evolution, volume 1363 of Lecture
Notes in Computer Science, pages 1–51. Springer, Heidelberg, 1998.
[ bib 
DOI ]

[886]

Xavier Glorot and Yoshua Bengio.
Understanding the difficulty of training deep feedforward neural
networks.
In Proceedings of the Thirteenth International Conference on
Artificial Intelligence and Statistics, pages 249–256, 2010.
[ bib ]

[887]

Fred Glover and Gary A. Kochenberger.
Critical Even Tabu Search for Multidimensional Knapsack
Problems.
In I. H. Osman and J. P. Kelly, editors, Metaheuristics: Theory
& Applications, pages 407–427. Kluwer Academic Publishers, Norwell, MA,
1996.
[ bib ]

[888]

Fred Glover, Gary A. Kochenberger, and Bahram Alidaee.
Adaptive Memory Tabu Search for Binary Quadratic Programs.
Management Science, 44(3):336–345, 1998.
[ bib ]

[889]

Fred Glover and Manuel Laguna.
Tabu Search.
Kluwer Academic Publishers, Boston, MA, USA, 1997.
[ bib ]

[890]

Fred Glover, Manuel Laguna, and Rafael Martí.
Scatter Search and Path Relinking: Advances and Applications.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 1–35. Kluwer Academic Publishers, Norwell, MA, 2002.
[ bib ]

[891]

Fred Glover, Zhipeng Lü, and JinKao Hao.
Diversificationdriven tabu search for unconstrained binary
quadratic problems.
4OR: A Quarterly Journal of Operations Research,
8(3):239–253, 2010.
[ bib 
DOI ]

[892]

Marc Goerigk and Anita Schöbel.
Recoverytooptimality: A new twostage approach to robustness
with an application to aperiodic timetabling.
Computers & Operations Research, 52:1–15, 2014.
[ bib ]

[893]

Elizabeth F. Gouvêa Goldbarg, Givanaldo R. Souza, and Marco C. Goldbarg.
Particle Swarm for the Traveling Salesman Problem.
In J. Gottlieb and G. R. Raidl, editors, Proceedings of EvoCOP
2006 – 6th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 3906 of Lecture Notes in Computer Science, pages
99–110. Springer, Heidelberg, 2006.
[ bib ]

[894]

David E. Goldberg.
Probability matching, the magnitude of reinforcement, and
classifier system bidding.
Machine Learning, 5(4):407–425, 1990.
[ bib ]

[895]

David E. Goldberg.
Genetic Algorithms in Search, Optimization and Machine
Learning.
AddisonWesley, Boston, MA, USA, 1989.
[ bib ]

[896]

Fred E. Goldman and Larry W. Mays.
The Application of Simulated Annealing to the Optimal Operation
of Water Systems.
In Proceedings of 26th Annual Water Resources Planning and
Management Conference, Tempe, USA, June 2000. ASCE.
[ bib ]

[897]

Ralph E. Gomory.
An algorithm for integer solutions to linear programs.
In R. Graves and P. Wolfe, editors, Recent Advances in
Mathematical Programming, pages 260–302. McGraw Hill, New York, NY, 1963.
[ bib ]

[898]

Wenyin Gong, Álvaro Fialho, and Zhihua Cai.
Adaptive strategy selection in differential evolution.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pages 409–416. ACM
Press, New York, NY, 2010.
[ bib 
DOI ]

[899]

M. GorgesSchleuter.
Asparagos96 and the Travelling Salesman Problem.
In T. Bäck, Z. Michalewicz, and X. Yao, editors, Proceedings
of the 1997 IEEE International Conference on Evolutionary Computation
(ICEC'97), pages 171–174, Piscataway, NJ, 1997. IEEE Press.
[ bib ]

[900]

Abhijit Gosavi.
Reinforcement Learning: A Tutorial Survey and Recent Advances.
INFORMS Journal on Computing, 21(2):178–192, 2009.
[ bib 
DOI ]

[901]

J. Gottlieb, M. Puchta, and Christine Solnon.
A Study of Greedy, Local Search, and Ant Colony Optimization
Approaches for Car Sequencing Problems.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2003, volume 2611 of Lecture
Notes in Computer Science, pages 246–257, Heidelberg, 2003. Springer.
[ bib ]

[902]

N. I. M. Gould, D. Orban, and P. L. Toint.
CUTEr and SifDec: A constrained and unconstrained testing
environment, revisited.
ACM Transactions on Mathematical Software, 29:373–394, 2003.
[ bib ]

[903]

Alex Grasas, Angel A. Juan, and Helena Ramalhinho Lourenço.
SimILS: A Simulationbased Extension of the Iterated Local
Search Metaheuristic for Stochastic Combinatorial Optimization.
Journal of Simulation, 10(1):69–77, 2016.
[ bib ]

[904]

Alex Graves, Abdelrahman Mohamed, and Geoffrey Hinton.
Speech recognition with deep recurrent neural networks.
In Acoustics, speech and signal processing (icassp), 2013 ieee
international conference on, pages 6645–6649. IEEE, 2013.
[ bib ]

[905]

M. Gravel, W. L. Price, and Caroline Gagné.
Scheduling continuous casting of aluminum using a multiple
objective ant colony optimization metaheuristic.
European Journal of Operational Research, 143(1):218–229,
2002.
[ bib 
DOI ]

[906]

J. J. Grefenstette.
Optimization of Control Parameters for Genetic Algorithms.
IEEE Transactions on Systems, Man, and Cybernetics,
16(1):122–128, 1986.
[ bib ]

[907]

Garrison W. Greenwood, Xiaobo Hu, and Joseph G. D'Ambrosio.
Fitness functions for multiple objective optimization problems:
Combining preferences with Pareto rankings.
In R. K. Belew and M. D. Vose, editors, Foundations of Genetic
Algorithms (FOGA), pages 437–455. Morgan Kaufmann Publishers, 1996.
[ bib ]

[908]

Salvatore Greco, Milosz Kadzinski, Vincent Mousseau, and Roman
Slowiński.
ELECTRE^{}GKMS: Robust ordinal regression for
outranking methods.
European Journal of Operational Research, 214(1):118–135,
2011.
[ bib ]

[909]

Salvatore Greco, Benedetto Matarazzo, and Roman Slowiński.
Interactive evolutionary multiobjective optimization using
dominancebased rough set approach.
In H. Ishibuchi et al., editors, Proceedings of the 2010
Congress on Evolutionary Computation (CEC 2010), pages 1–8. IEEE Press,
Piscataway, NJ, 2010.
[ bib ]

[910]

Salvatore Greco, Vincent Mousseau, and Roman Slowiński.
Robust ordinal regression for value functions handling
interacting criteria.
European Journal of Operational Research, 239(3):711–730,
2014.
[ bib 
DOI ]

[911]

David R. Grimes, Chris T. Bauch, and John P. A. Ioannidis.
Modelling science trustworthiness under publish or perish
pressure.
Royal Society Open Science, 5:171511, 2018.
[ bib ]

[912]

Andrea Grosso, Federico Della Croce, and R. Tadei.
An Enhanced Dynasearch Neighborhood for the SingleMachine Total
Weighted Tardiness Scheduling Problem.
Operations Research Letters, 32(1):68–72, 2004.
[ bib ]

[913]

Andrea Grosso, A. R. M. J. U. Jamali, and Marco Locatelli.
Finding Maximin Latin Hypercube Designs by Iterated Local Search
Heuristics.
European Journal of Operational Research, 197(2):541–547,
2009.
[ bib ]

[914]

Peter Groves, Basel Kayyali, David Knott, and Steve Van Kuiken.
The "big data" revolution in healthcare.
McKinsey Quarterly, 2, 2013.
[ bib ]

[915]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
A link between the multivariate cumulative distribution function
and the hitting function for random closed sets.
Statistics & Probability Letters, 57(2):179–182, 2002.
[ bib 
DOI ]

[916]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
A characterization of the outcomes of stochastic multiobjective
optimizers through a reduction of the hitting function test sets.
Technical report, CSI, Universidade do Algarve, 2004.
[ bib ]
Keywords: highorder EAF

[917]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
The AttainmentFunction Approach to Stochastic Multiobjective
Optimizer Assessment and Comparison.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 103–130. Springer, Berlin, Germany, 2010.
[ bib ]

[918]

Viviane Grunert da Fonseca, Carlos M. Fonseca, and Andreia O. Hall.
Inferential Performance Assessment of Stochastic Optimisers and
the Attainment Function.
In E. Zitzler, K. Deb, L. Thiele, C. A. Coello Coello, and
D. Corne, editors, Evolutionary Multicriterion Optimization, EMO 2001,
volume 1993 of Lecture Notes in Computer Science, pages 213–225.
Springer, Heidelberg, 2001.
[ bib 
DOI ]
Proposed looking at anytime behavior as a multiobjective
problem
Keywords: EAF

[919]

C. Guéret, Nicolas Monmarché, and M. Slimane.
Ants Can Play Music.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 310–317. Springer, Heidelberg,
2004.
[ bib ]

[920]

M. Guntsch and Jürgen Branke.
New Ideas for Applying Ant Colony Optimization to the
Probabilistic TSP.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2003, volume 2611 of Lecture
Notes in Computer Science, pages 165–175, Heidelberg, 2003. Springer.
[ bib ]

[921]

Odd Erik Gundersen, Yolanda Gil, and David W. Aha.
On Reproducible AI: Towards Reproducible Research, Open
Science, and Digital Scholarship in AI Publications.
AI Magazine, 39(3):56–68, September 2018.
[ bib 
DOI ]
The reproducibility guidelines can be found here:
https://folk.idi.ntnu.no/odderik/reproducibility_guidelines.pdf
and a short howto can be found here:
https://folk.idi.ntnu.no/odderik/reproducibility_guidelines_how_to.html

[922]

M. Guntsch and Martin Middendorf.
Pheromone Modification Strategies for Ant Algorithms Applied to
Dynamic TSP.
In E. J. W. Boers et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2001, volume 2037 of Lecture
Notes in Computer Science, pages 213–222. Springer, Heidelberg, 2001.
[ bib ]

[923]

M. Guntsch and Martin Middendorf.
A Population Based Approach for ACO.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2002, volume 2279 of Lecture
Notes in Computer Science, pages 71–80, Heidelberg, 2002. Springer.
[ bib ]

[924]

M. Guntsch and Martin Middendorf.
Solving MultiObjective Permutation Problems with Population
Based ACO.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 464–478. Springer,
Heidelberg, 2003.
[ bib ]

[925]

M. Guntsch and Martin Middendorf.
Applying Population Based ACO to Dynamic Optimization
Problems.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 111–122. Springer, Heidelberg, 2002.
[ bib ]

[926]

Aldy Gunawan, Kien Ming Ng, and Kim Leng Poh.
A Hybridized Lagrangian Relaxation and Simulated Annealing
Method for the Course Timetabling Problem.
Computers & Operations Research, 39(12):3074–3088, 2012.
[ bib ]

[927]

J. N. D. Gupta.
Flowshop schedules with sequence dependent setup times.
Journal of Operations Research Society of Japan, 29:206 – 219,
1986.
[ bib ]

[928]

Gurobi.
Gurobi Optimizer.
http://www.gurobi.com/products/gurobioptimizer, 2017.
[ bib ]

[929]

D. Gusfield.
Algorithms on Strings, Trees, and Sequences.
In Computer Science and Computational Biology. Cambridge
University Press, 1997.
[ bib ]

[930]

Walter J. Gutjahr.
SACO: An AntBased Approach to Combinatorial Optimization
Under Uncertainty.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 238–249. Springer, Heidelberg,
2004.
[ bib ]

[931]

Walter J. Gutjahr.
A Graphbased Ant System and its Convergence.
Future Generation Computer Systems, 16(8):873–888, 2000.
[ bib ]

[932]

Walter J. Gutjahr.
ACO Algorithms with Guaranteed Convergence to the Optimal
Solution.
Information Processing Letters, 82(3):145–153, 2002.
[ bib ]

[933]

Walter J. Gutjahr.
A converging ACO algorithm for stochastic combinatorial
optimization.
In A. Albrecht and K. Steinhöfel, editors, Stochastic
Algorithms: Foundations and Applications, volume 2827 of Lecture Notes
in Computer Science, pages 10–25. Springer Verlag, 2003.
[ bib 
DOI ]

[934]

Walter J. Gutjahr.
On the finitetime dynamics of ant colony optimization.
Methodology and Computing in Applied Probability,
8(1):105–133, 2006.
[ bib ]

[935]

Walter J. Gutjahr.
Mathematical runtime analysis of ACO algorithms: survey on an
emerging issue.
Swarm Intelligence, 1(1):59–79, 2007.
[ bib ]

[936]

Walter J. Gutjahr and Marion S. Rauner.
An ACO algorithm for a dynamic regional nursescheduling
problem in Austria.
Computers & Operations Research, 34(3):642–666, 2007.
[ bib 
DOI ]
To the best of our knowledge, this paper describes the first
ant colony optimization (ACO) approach applied to nurse
scheduling, analyzing a dynamic regional problem which is
currently under discussion at the Vienna hospital
compound. Each day, pool nurses have to be assigned for the
following days to public hospitals while taking into account
a variety of soft and hard constraints regarding working date
and time, working patterns, nurses qualifications, nurses
and hospitals preferences, as well as costs. Extensive
computational experiments based on a four week simulation
period were used to evaluate three different scenarios
varying the number of nurses and hospitals for six different
hospitals demand intensities. The results of our simulations
and optimizations reveal that the proposed ACO algorithm
achieves highly significant improvements compared to a greedy
assignment algorithm.

[937]

Walter J. Gutjahr.
First steps to the runtime complexity analysis of ant colony
optimization.
Computers & Operations Research, 35(9):2711–2727, 2008.
[ bib ]

[938]

Walter J. Gutjahr and G. Sebastiani.
Runtime analysis of ant colony optimization with bestsofar
reinforcement.
Methodology and Computing in Applied Probability,
10(3):409–433, 2008.
[ bib ]

[939]

Gregory Gutin, Anders Yeo, and Alexey Zverovich.
Traveling salesman should not be greedy: domination analysis of
greedytype heuristics for the TSP.
Discrete Applied Mathematics, 117(1–3), 2002.
[ bib ]

[940]

Heikki Haario, Eero Saksman, and Johanna Tamminen.
An adaptive Metropolis algorithm.
Bernoulli, 7(2):223–242, 2001.
[ bib ]

[941]

Evert Haasdijk, Arif Attaul Qayyum, and Agoston E. Eiben.
Racing to improve online, onboard evolutionary robotics.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 187–194.
ACM Press, New York, NY, 2011.
[ bib ]

[942]

S. Häckel, M. Fischer, D. Zechel, and T. Teich.
A multiobjective ant colony approach for Paretooptimization
using dynamic programming.
In C. Ryan, editor, Proceedings of the Genetic and Evolutionary
Computation Conference, GECCO 2008, pages 33–40. ACM Press, New York, NY,
2008.
[ bib ]

[943]

David Hadka and Patrick M. Reed.
Borg: An AutoAdaptive ManyObjective Evolutionary Computing
Framework.
Evolutionary Computation, 21(2):231–259, 2013.
[ bib ]

[944]

David Hadka, Patrick M. Reed, and T. W. Simpson.
Diagnostic assessment of the Borg MOEA for manyobjective
product family design problems.
In Proceedings of the 2012 Congress on Evolutionary Computation
(CEC 2012), pages 1–10, Piscataway, NJ, 2012. IEEE Press.
[ bib ]

[945]

David Hadka and Patrick M. Reed.
Diagnostic Assessment of Search Controls and Failure Modes in
ManyObjective Evolutionary Optimization.
Evolutionary Computation, 20(3):423–452, 2012.
[ bib ]

[946]

Josef Hadar and William R. Russell.
Rules for ordering uncertain prospects.
The American Economic Review, 59(1):25–34, 1969.
[ bib 
eprint ]
Keywords: stochastic dominance

[947]

Apache Software Foundation.
Hadoop, 2008.
[ bib 
http ]

[948]

Thomas M. Walski, Donald V. Chase, Dragan A. Savic, Walter Grayman, Stephen
Beckwith, and Edmundo Koelle.
Advanced Water Distribution Modeling and Management.
Haestad Methods, Inc., Haestad Press, 1st edition, 2003.
[ bib ]

[949]

Y. Haimes, L. Lasdon, and D. Da Wismer.
On a bicriterion formation of the problems of integrated system
identification and system optimization.
IEEE Transactions on Systems, Man, and Cybernetics,
1(3):296–297, 1971.
[ bib 
DOI ]
Keywords: epsilonconstraint method

[950]

Prabhat Hajela and CY Lin.
Genetic search strategies in multicriterion optimal design.
Structural Optimization, 4(2):99–107, 1992.
[ bib ]

[951]

Bruce Hajek and Galen Sasaki.
Simulated annealing–to cool or not.
System & Control Letters, 12(5):443–447, 1989.
[ bib ]

[952]

Bruce Hajek.
Cooling Schedules for Optimal Annealing.
Mathematics of Operations Research, 13(2):311–329, 1988.
[ bib ]

[953]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
On the impact of the cutoff time on the performance of algorithm
configurators.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pages 907–915. ACM Press, New York, NY, 2019.
[ bib 
DOI ]
Keywords: theory, automatic configuration, capping

[954]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
Fast Perturbative Algorithm Configurators.
In T. Bäck, M. Preuss, A. Deutz, H. Wang, C. Doerr, M. T. M.
Emmerich, and H. Trautmann, editors, Parallel Problem Solving from
Nature  PPSN XVI, volume 12269 of Lecture Notes in Computer Science,
pages 19–32. Springer, Cham, Switzerland, 2020.
[ bib 
DOI ]

[955]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
Analysis of the performance of algorithm configurators for
search heuristics with global mutation operators.
In C. A. Coello Coello, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2020, pages 823–831. ACM Press,
New York, NY, 2020.
[ bib 
DOI ]

[956]

Greg Hamerly and Charles Elkan.
Learning the k in kmeans.
In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in
Neural Information Processing Systems (NIPS 16). MIT Press, 2003.
[ bib 
eprint ]

[957]

Horst W. Hamacher and Günter Ruhe.
On spanning tree problems with multiple objectives.
Annals of Operations Research, 52(4):209–230, 1994.
[ bib ]

[958]

Hayfa Hammami and Thomas Stützle.
A Computational Study of Neighborhood Operators for JobShop
Scheduling Problems with Regular Objectives.
In B. Hu and M. LópezIbáñez, editors, Proceedings
of EvoCOP 2017 – 17th European Conference on Evolutionary Computation in
Combinatorial Optimization, volume 10197 of Lecture Notes in Computer
Science, pages 1–17. Springer, Heidelberg, 2017.
[ bib 
DOI ]

[959]

Michael Pilegaard Hansen.
Tabu search for multiobjective optimization: MOTS.
In J. Climaco, editor, Proceedings of the 13th International
Conference on Multiple Criteria Decision Making (MCDM'97), pages 574–586.
Springer Verlag, 1997.
[ bib ]

[960]

Nikolaus Hansen, Anne Auger, S. Finck, and R. Ros.
RealParameter BlackBox Optimization Benchmarking 2009:
Experimental setup.
Technical Report RR6828, INRIA, France, 2009.
[ bib 
supplementary material ]

[961]

Nikolaus Hansen, Anne Auger, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff.
COCO: A platform for comparing continuous optimizers in a
blackbox setting.
Arxiv preprint arXiv:1603.08785, 2016.
Published as [962].
[ bib ]

[962]

Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and
Dimo Brockhoff.
COCO: A platform for comparing continuous optimizers in a
blackbox setting.
Optimization Methods and Software, 36(1):1–31, 2020.
[ bib 
DOI ]

[963]

Nikolaus Hansen, S. Finck, R. Ros, and Anne Auger.
RealParameter BlackBox Optimization Benchmarking 2009:
Noiseless Functions Definitions.
Technical Report RR6829, INRIA, France, 2009.
Updated February 2010.
[ bib ]
http://coco.gforge.inria.fr/bbob2012downloads

[964]

Michael Pilegaard Hansen and Andrzej Jaszkiewicz.
Evaluating the quality of approximations to the nondominated
set.
Technical Report IMMREP19987, Institute of Mathematical Modelling,
Technical University of Denmark, Lyngby, Denmark, 1998.
[ bib ]

[965]

Pierre Hansen and B. Jaumard.
Algorithms for the Maximum Satisfiability Problem.
Computing, 44:279–303, 1990.
[ bib ]

[966]

Julia Handl and Joshua D. Knowles.
Modes of Problem Solving with Multiple Objectives: Implications
for Interpreting the Pareto Set and for Decision Making.
In J. D. Knowles, D. Corne, K. Deb, and D. R. Chair, editors,
Multiobjective Problem Solving from Nature, Natural Computing Series, pages
131–151. Springer, Berlin/Heidelberg, 2008.
[ bib 
DOI ]

[967]

Pierre Hansen and Nenad Mladenović.
Variable neighborhood search: Principles and applications.
European Journal of Operational Research, 130(3):449–467,
2001.
[ bib ]

[968]

Pierre Hansen and Nenad Mladenović.
Variable Neighborhood Search.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 145–184. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib ]

[969]

Pierre Hansen, Nenad Mladenović, Jack Brimberg, and José A. Moreno
Pérez.
Variable Neighborhood Search.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, pages 61–86. Springer, New York, NY, 2nd
edition, 2010.
[ bib ]

[970]

Nikolaus Hansen and A. Ostermeier.
Completely derandomized selfadaptation in evolution
strategies.
Evolutionary Computation, 9(2):159–195, 2001.
[ bib 
DOI ]
Keywords: CMAES

[971]

Nikolaus Hansen, Raymond Ros, Nikolaus Mauny, Marc Schoenauer, and Anne Auger.
Impacts of invariance in search: When CMAES and PSO face
illconditioned and nonseparable problems.
Applied Soft Computing, 11(8):5755–5769, 2011.
[ bib ]

[972]

Thomas Hanne.
On the convergence of multiobjective evolutionary algorithms.
European Journal of Operational Research, 117(3):553–564,
1999.
[ bib ]

[973]

Michael Pilegaard Hansen.
Metaheuristics for multiple objective combinatorial
optimization.
PhD thesis, Institute of Mathematical Modelling, Technical University
of Denmark, March 1998.
[ bib ]

[974]

Nikolaus Hansen.
The CMA evolution strategy: a comparing review.
In Towards a new evolutionary computation, pages 75–102.
Springer, 2006.
[ bib ]

[975]

Nikolaus Hansen.
Benchmarking a BIpopulation CMAES on the BBOB2009
function testbed.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2009, pages
2389–2396. ACM Press, New York, NY, 2009.
[ bib ]
Keywords: bipopcmaes

[976]

Zhifeng Hao, Ruichu Cai, and Han Huang.
An Adaptive Parameter Control Strategy for ACO.
In Proceedings of the International Conference on Machine
Learning and Cybernetics, pages 203–206. IEEE Press, 2006.
[ bib ]

[977]

Zhifeng Hao, Han Huang, Yong Qin, and Ruichu Cai.
An ACO Algorithm with Adaptive Volatility Rate of Pheromone
Trail.
In Y. Shi, G. D. van Albada, J. Dongarra, and P. M. A. Sloot,
editors, Computational Science – ICCS 2007, 7th International
Conference, Proceedings, Part IV, volume 4490 of Lecture Notes in
Computer Science, pages 1167–1170. Springer, Heidelberg, 2007.
[ bib ]

[978]

JinKao Hao and Jêrome Pannier.
Simulated Annealing and Tabu Search for Constraint Solving.
In M. C. Golumbic et al., editors, Fifth International Symposium
on Artificial Intelligence and Mathematics, AIM 1998, Fort Lauderdale,
Florida, USA, January 46, 1998, pages 1–15, 1998.
[ bib ]

[979]

William D. Harvey and Matthew L. Ginsberg.
Limited Discrepancy Search.
In C. S. Mellish, editor, Proceedings of the Fourteenth
International Joint Conference on Artificial Intelligence (IJCAI95), pages
607–615. Morgan Kaufmann Publishers, 1995.
[ bib ]

[980]

Douglas P. Hardin and Edward B. Saff.
Discretizing Manifolds via Minimum Energy Points.
Notices of the American Mathematical Society,
51(10):1186–1194, 2004.
[ bib ]

[981]

J. P. Hart and A. W. Shogan.
Semigreedy heuristics: An empirical study.
Operations Research Letters, 6(3):107–114, 1987.
[ bib ]

[982]

Emma Hart and Kevin Sim.
A HyperHeuristic Ensemble Method for Static JobShop
Scheduling.
Evolutionary Computation, 24(4):609–635, 2016.
[ bib 
DOI ]

[983]

Kazuya Haraguchi.
Iterated Local Search with TrellisNeighborhood for the
Partial Latin Square Extension Problem.
Journal of Heuristics, 22(5):727–757, 2016.
[ bib ]

[984]

Sameer Hasija and Chandrasekharan Rajendran.
Scheduling in flowshops to minimize total tardiness of jobs.
International Journal of Production Research,
42(11):2289–2301, 2004.
[ bib 
DOI ]

[985]

Hideki Hashimoto, Mutsunori Yagiura, and Toshihide Ibaraki.
An Iterated Local Search Algorithm for the Timedependent
Vehicle Routing Problem with Time Windows.
Discrete Optimization, 5(2):434–456, 2008.
[ bib ]

[986]

Hado van Hasselt, Arthur Guez, and David Silver.
Deep Reinforcement Learning with Double QLearning.
In D. Schuurmans and M. P. Wellman, editors, Proceedings of the
AAAI Conference on Artificial Intelligence. AAAI Press, 2016.
[ bib ]

[987]

Simon Haykin.
A comprehensive foundation.
Neural Networks, 2:41, 2004.
[ bib ]

[988]

Öncü Hazir, Yavuz Günalay, and Erdal Erel.
Customer order scheduling problem: a comparative metaheuristics
study.
International Journal of Advanced Manufacturing Technology,
37(5):589–598, May 2008.
[ bib 
DOI ]
The customer order scheduling problem (COSP) is defined as
to determine the sequence of tasks to satisfy the demand of
customers who order several types of products produced on a
single machine. A setup is required whenever a product type
is launched. The objective of the scheduling problem is to
minimize the average customer order flow time. Since the
customer order scheduling problem is known to be strongly
NPhard, we solve it using four major metaheuristics and
compare the performance of these heuristics, namely,
simulated annealing, genetic algorithms, tabu search, and ant
colony optimization. These are selected to represent various
characteristics of metaheuristics: natureinspired
vs. artificially created, populationbased vs. local search,
etc. A set of problems is generated to compare the solution
quality and computational efforts of these heuristics.
Results of the experimentation show that tabu search and ant
colony perform better for large problems whereas simulated
annealing performs best in smallsize problems. Some
conclusions are also drawn on the interactions between
various problem parameters and the performance of the
heuristics.
Keywords: ACO,Customer order scheduling,Genetic
algorithms,Metaheuristics,Simulated annealing,Tabu
search

[989]

Xin He, Kaiyong Zhao, and Xiaowen Chu.
AutoML: A survey of the stateoftheart.
KnowledgeBased Systems, 212:106622, 2021.
[ bib 
DOI ]

[990]

Verena HeidrichMeisner and Christian Igel.
Hoeffding and Bernstein races for selecting policies in
evolutionary direct policy search.
In A. P. Danyluk, L. Bottou, and M. L. Littman, editors,
Proceedings of the 26th International Conference on Machine Learning, ICML
2009, pages 401–408, New York, NY, 2009. ACM Press.
[ bib 
DOI ]
Keywords: automated algorithm configuration, CMAES, racing

[991]

Keld Helsgaun.
Source Code of the LinKernighanHelsgaun Traveling
Salesman Heuristic.
http://webhotel4.ruc.dk/~keld/research/LKH/, 2018.
[ bib ]

[992]

Keld Helsgaun.
Efficient Recombination in the LinKernighanHelsgaun
Traveling Salesman Heuristic.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature  PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pages 95–107.
Springer, Cham, Switzerland, 2018.
[ bib 
DOI ]

[993]

Michael Held and Richard M. Karp.
The TravelingSalesman Problem and Minimum Spanning Trees.
Operations Research, 18(6):1138–1162, 1970.
[ bib ]

[994]

Christoph Helmberg and Franz Rendl.
Solving quadratic (0,1)problems by semidefinite programs and
cutting planes.
Mathematical Programming, 82(3):291–315, 1998.
[ bib ]

[995]

Keld Helsgaun.
An Effective Implementation of the LinKernighan Traveling
Salesman Heuristic.
European Journal of Operational Research, 126:106–130, 2000.
[ bib ]

[996]

Keld Helsgaun.
General kopt Submoves for the LinKernighan TSP
Heuristic.
Mathematical Programming Computation, 1(2–3):119–163, 2009.
[ bib ]

[997]

Pascal van Hentenryck.
The OPL optimization programming language.
MIT Press, Cambridge, MA, 1999.
[ bib ]

[998]

Darrall Henderson, Sheldon H. Jacobson, and Alan W. Johnson.
The Theory and Practice of Simulated Annealing.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 287–319. Springer, Boston, MA, 2003.
[ bib 
DOI ]

[999]

Pascal van Hentenryck and Laurent D. Michel.
Constraintbased Local Search.
MIT Press, Cambridge, MA, 2005.
[ bib ]

[1000]

Pascal van Hentenryck and Laurent D. Michel.
Synthesis of constraintbased local search algorithms from
highlevel models.
In R. C. Holte and A. Howe, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pages 273–278. AAAI Press/
MIT Press, Menlo Park, CA, 2007.
[ bib ]

[1001]

Michael A. Heroux.
Editorial: ACM TOMS Replicated Computational Results
Initiative.
ACM Transactions on Mathematical Software, 41(3):1–5, June
2015.
[ bib 
DOI ]

[1002]

H. Hernández and Christian Blum.
Ant colony optimization for multicasting in static wireless
adhoc networks.
Swarm Intelligence, 3(2):125–148, 2009.
[ bib ]

[1003]

Francisco Herrera, Manuel Lozano, and Daniel Molina.
Test suite for the special issue of Soft Computing on
scalability of evolutionary algorithms and other metaheuristics for large
scale continuous optimization problems.
http://sci2s.ugr.es/eamhco/, 2010.
[ bib ]
Keywords: SOCO benchmark

[1004]

Francisco Herrera, Manuel Lozano, and A. M. Sánchez.
A taxonomy for the crossover operator for realcoded genetic
algorithms: An experimental study.
International Journal of Intelligent Systems, 18(3):309–338,
2003.
[ bib 
DOI ]

[1005]

Francisco Herrera, Manuel Lozano, and J. L. Verdegay.
Tackling RealCoded Genetic Algorithms: Operators and Tools for
Behavioural Analysis.
Artificial Intelligence Review, 12:265–319, 1998.
[ bib ]
Keywords: genetic algorithms, real coding, continuous search
spaces, mutation, recombination

[1006]

Jano I. van Hemert.
Evolving Combinatorial Problem Instances That Are Difficult to
Solve.
Evolutionary Computation, 14(4):433–462, 2006.
[ bib 
DOI ]
This paper demonstrates how evolutionary computation can be
used to acquire difficult to solve combinatorial problem
instances. As a result of this technique, the corresponding
algorithms used to solve these instances are
stresstested. The technique is applied in three important
domains of combinatorial optimisation, binary constraint
satisfaction, Boolean satisfiability, and the travelling
salesman problem. The problem instances acquired through this
technique are more difficult than the ones found in popular
benchmarks. In this paper, these evolved instances are
analysed with the aim to explain their difficulty in terms of
structural properties, thereby exposing the weaknesses of
corresponding algorithms.

[1007]

Robert Heumüller, Sebastian Nielebock, Jacob Krüger, and Frank
Ortmeier.
Publish or perish, but do not forget your software artifacts.
Empirical Software Engineering, 25(6):4585–4616, 2020.
[ bib 
DOI ]

[1008]

Daniel P Heyman and Matthew J Sobel.
Stochastic models in operations research: stochastic
optimization, volume 2.
Courier Corporation, 2003.
[ bib ]

[1009]

Christian Hicks.
A Genetic Algorithm tool for optimising cellular or functional
layouts in the capital goods industry.
International Journal of Production Economics, 104(2):598–614,
2006.
[ bib 
DOI ]

[1010]

Geoffrey E. Hinton and Ruslan R. Salakhutdinov.
Reducing the dimensionality of data with neural networks.
Science, 313(5786):504–507, 2006.
[ bib ]

[1011]

Wassily Hoeffding.
Probability inequalities for sums of bounded random variables.
Journal of the American Statistical Association,
58(301):13–30, 1963.
[ bib ]

[1012]

J. Holland.
Adaptation in Natural and Artificial Systems.
University of Michigan Press, 1975.
[ bib ]

[1013]

Myle Hollander and Douglas A. Wolfe.
Nonparametric statistical inference.
John Wiley & Sons, New York, NY, 1973.
Second edition (1999).
[ bib ]

[1014]

I. Hong, A. B. Kahng, and B. R. Moon.
Improved largestep Markov chain variants for the symmetric
TSP.
Journal of Heuristics, 3(1):63–81, 1997.
[ bib ]

[1015]

John N. Hooker.
Needed: An Empirical Science of Algorithms.
Operations Research, 42(2):201–212, 1994.
[ bib ]

[1016]

John N. Hooker.
Testing Heuristics: We Have It All Wrong.
Journal of Heuristics, 1(1):33–42, 1996.
[ bib 
DOI ]

[1017]

Giles Hooker.
Generalized functional ANOVA diagnostics for highdimensional
functions of dependent variables.
Journal of Computational and Graphical Statistics,
16(3):709–732, 2012.
[ bib 
DOI ]

[1018]

Holger H. Hoos, Marius Thomas Lindauer, and Torsten Schaub.
Claspfolio 2: Advances in Algorithm Selection for Answer Set
Programming.
Theory and Practice of Logic Programming, 14(45):560–585,
2014.
[ bib ]

[1019]

Holger H. Hoos and Thomas Stützle.
Stochastic Local Search: Foundations and Applications.
Elsevier, Amsterdam, The Netherlands, 2004.
[ bib ]

[1020]

Holger H. Hoos and Thomas Stützle.
Stochastic Local Search—Foundations and Applications.
Morgan Kaufmann Publishers, San Francisco, CA, 2005.
[ bib ]

[1021]

Holger H. Hoos and Thomas Stützle.
Evaluating Las Vegas Algorithms — Pitfalls and Remedies.
In G. F. Cooper and S. Moral, editors, Proceedings of the
Fourteenth Conference on Uncertainty in Artificial Intelligence, pages
238–245. Morgan Kaufmann Publishers, San Francisco, CA, 1998.
[ bib ]

[1022]

Holger H. Hoos and Thomas Stützle.
On the Empirical Scaling of Runtime for Finding Optimal
Solutions to the Traveling Salesman Problem.
European Journal of Operational Research, 238(1):87–94, 2014.
[ bib ]

[1023]

Holger H. Hoos and Thomas Stützle.
On the Empirical Time Complexity of Finding Optimal Solutions
vs. Proving Optimality for Euclidean TSP Instances.
Optimization Letters, 9(6):1247–1254, 2015.
[ bib ]

[1024]

Holger H. Hoos.
Programming by Optimisation: Towards a new Paradigm for
Developing HighPerformance Software.
In MIC 2011, the 9th Metaheuristics International Conference,
2011.
Plenary talk.
[ bib 
http ]

[1025]

Holger H. Hoos.
Automated Algorithm Configuration and Parameter Tuning.
In Y. Hamadi, E. Monfroy, and F. Saubion, editors, Autonomous
Search, pages 37–71. Springer, Berlin, Germany, 2012.
[ bib 
DOI ]

[1026]

Holger H. Hoos.
Programming by optimization.
Communications of the ACM, 55(2):70–80, February 2012.
[ bib 
DOI ]

[1027]

Christian Horoba and Frank Neumann.
Benefits and drawbacks for the use of epsilondominance in
evolutionary multiobjective optimization.
In C. Ryan, editor, Proceedings of the Genetic and Evolutionary
Computation Conference, GECCO 2008, pages 641–648, New York, NY, 2008. ACM
Press.
[ bib ]

[1028]

J. Horn, N. Nafpliotis, and David E. Goldberg.
A niched Pareto genetic algorithm for multiobjective
optimization.
In Proceedings of the 1994 World Congress on Computational
Intelligence (WCCI 1994), pages 82–87, Piscataway, NJ, June 1994. IEEE
Press.
[ bib 
DOI ]

[1029]

Kenneth Hoste and Lieven Eeckhout.
Cole: Compiler Optimization Level Exploration.
In M. L. Soffa and E. Duesterwald, editors, Proceedings of the
6th Annual IEEE/ACM International Symposium on Code Generation and
Optimization, CGO '08, pages 165–174, New York, NY, 2008. ACM Press.
[ bib 
DOI ]

[1030]

Stela Pudar Hozo, Benjamin Djulbegovic, and Iztok Hozo.
Estimating the mean and variance from the median, range, and the
size of a sample.
BMC Medical Research Methodology, 5(1):13, 2005.
[ bib ]

[1031]

T. C. Hu, A. B. Kahng, and C.W. A. Tsao.
Old Bachelor Acceptance: A New Class of NonMonotone Threshold
Accepting Methods.
ORSA Journal on Computing, 7(4):417–425, 1995.
[ bib ]

[1032]

Wenbin Hu, Huan Wang, Zhenyu Qiu, Cong Nie, and Liping Yan.
A quantum particle swarm optimization driven urban traffic light
scheduling model.
Neural Computing & Applications, 2018.
[ bib 
DOI ]
Keywords: BML,Optimization,Simulation,Traffic congestion,Updating
rules

[1033]

Wenbin Hu, Liping Yan, Huan Wang, Bo Du, and Dacheng Tao.
Realtime traffic jams prediction inspired by Biham,
Middleton and Levine (BML) model.
Information Sciences, 2017.
[ bib ]
Keywords: BML model,Prediction,Realtime,Traffic jam,Urban traffic
network

[1034]

Deng Huang, Theodore T. Allen, William I. Notz, and Ning Zeng.
Global Optimization of Stochastic BlackBox Systems via
Sequential Kriging MetaModels.
Journal of Global Optimization, 34(3):441–466, 2006.
[ bib 
DOI ]

[1035]

Han Huang, Xiaowei Yang, Zhifeng Hao, and Ruichu Cai.
A Novel ACO Algorithm with Adaptive Parameter.
In D.S. Huang, K. Li, and G. W. Irwin, editors, International
Conference on Computational Science (3), volume 4115 of Lecture Notes
in Computer Science, pages 12–21. Springer, Heidelberg, 2006.
[ bib ]

[1036]

K. Huang, C. Yang, and K. Tseng.
Fast algorithms for finding the common subsequences of multiple
sequences.
In Proceedings of the International Computer Symposium, pages
1006–1011. IEEE Press, 2004.
[ bib ]

[1037]

S. Huband, P. Hingston, L. Barone, and L. While.
A review of multiobjective test problems and a scalable test
problem toolkit.
IEEE Transactions on Evolutionary Computation, 10(5):477–506,
2006.
[ bib 
DOI ]

[1038]

B. Huberman, R. Lukose, and T. Hogg.
An Economic Approach to Hard Computational Problems.
Science, 275:51–54, 1997.
[ bib ]

[1039]

D. L. HuertaMuñoz, R. Z. RíosMercado, and Rubén Ruiz.
An Iterated Greedy Heuristic for a Market Segmentation Problem
with Multiple Attributes.
European Journal of Operational Research, 261(1):75–87, 2017.
[ bib ]

[1040]

Evan J. Hughes.
Multiple single objective Pareto sampling.
In Proceedings of the 2003 Congress on Evolutionary Computation
(CEC'03), pages 2678–2684, Piscataway, NJ, December 2003. IEEE Press.
[ bib ]

[1041]

Evan J. Hughes.
MSOPSII: A generalpurpose manyobjective optimiser.
In Proceedings of the 2007 Congress on Evolutionary Computation
(CEC 2007), pages 3944–3951, Piscataway, NJ, 2007. IEEE Press.
[ bib ]

[1042]

Evan J. Hughes.
Manyobjective directed evolutionary line search.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 761–768.
ACM Press, New York, NY, 2011.
[ bib ]

[1043]

Jérémie Humeau, Arnaud Liefooghe, ElGhazali Talbi, and Sébastien
Verel.
ParadisEOMO: From Fitness Landscape Analysis to Efficient
Local Search Algorithms.
Journal of Heuristics, 19(6):881–915, June 2013.
[ bib 
DOI ]

[1044]

Ying Hung, V. Roshan Joseph, and Shreyes N. Melkote.
Design and Analysis of Computer Experiments With Branching and
Nested Factors.
Technometrics, 51(4):354–365, 2009.
[ bib 
DOI ]

[1045]

Maura Hunt and Manuel LópezIbáñez.
Modeling a DecisionMaker in Goal Programming by means of
Computational Rationality.
In I. Palomares, editor, International Alan Turing Conference on
Decision Support and Recommender systems, pages 17–20, London, UK,
November, 21–22 2019. Alan Turing Institute.
[ bib ]
This paper extends a simulation of cognitive mechanisms in
the context of multicriteria decisionmaking by using ideas
from computational rationality. Specifically, this paper
improves the simulation of a human decisionmaker (DM) by
considering how resource constraints impact their evaluation
process in an interactive Goal Programming problem. Our
analysis confirms and emphasizes a previous simulation study
by showing key areas that could be effected by cognitive
mechanisms. While the results are promising, the effects
should be validated by future experiments with human DMs.

[1046]

M. Hurtgen and J.C. Maun.
Optimal PMU placement using Iterated Local Search.
International Journal of Electrical Power & Energy Systems,
32(8):857–860, 2010.
[ bib ]

[1047]

S. H. Hurlbert.
Pseudoreplication and the Design of Ecological Field
Experiments.
Ecological Monographs, 54(2):187–211, 1984.
[ bib ]

[1048]

Mohamed Saifullah Hussin and Thomas Stützle.
Hierarchical Iterated Local Search for the Quadratic Assignment
Problem.
In M. J. Blesa, C. Blum, L. Di Gaspero, A. Roli, M. Sampels, and
A. Schaerf, editors, Hybrid Metaheuristics, volume 5818 of Lecture
Notes in Computer Science, pages 115–129. Springer, Heidelberg, 2009.
[ bib 
DOI ]

[1049]

Mohamed Saifullah Hussin and Thomas Stützle.
Tabu Search vs. Simulated Annealing for Solving Large Quadratic
Assignment Instances.
Computers & Operations Research, 43:286–291, 2014.
[ bib ]

[1050]

Frank Hutter, Domagoj Babić, Holger H. Hoos, and Alan J. Hu.
Boosting Verification by Automatic Tuning of Decision
Procedures.
In J. Baumgartner and M. Sheeran, editors, FMCAD'07: Proceedings
of the 7th International Conference Formal Methods in Computer Aided Design,
pages 27–34, Austin, Texas, USA, 2007. IEEE Computer Society, Washington,
DC, USA.
[ bib ]

[1051]

Frank Hutter, Holger H. Hoos, Kevin LeytonBrown, and Kevin P. Murphy.
An experimental investigation of modelbased parameter
optimisation: SPO and beyond.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pages 271–278. ACM Press,
New York, NY, 2009.
[ bib 
DOI ]

[1052]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Automated Configuration of Mixed Integer Programming Solvers.
In A. Lodi, M. Milano, and P. Toth, editors, Integration of AI
and OR Techniques in Constraint Programming for Combinatorial Optimization
Problems, 7th International Conference, CPAIOR 2010, volume 6140 of
Lecture Notes in Computer Science, pages 186–202. Springer, Heidelberg,
2010.
[ bib 
DOI ]
Keywords: mipconfig

[1053]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Tradeoffs in the Empirical Evaluation of Competing Algorithm
Designs.
Annals of Mathematics and Artificial Intelligence,
60(1–2):65–89, 2010.
[ bib ]

[1054]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Sequential ModelBased Optimization for General Algorithm
Configuration.
In C. A. Coello Coello, editor, Learning and Intelligent
Optimization, 5th International Conference, LION 5, volume 6683 of
Lecture Notes in Computer Science, pages 507–523. Springer, Heidelberg,
2011.
[ bib 
DOI ]
Keywords: SMAC,ROAR

[1055]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Parallel Algorithm Configuration.
In Y. Hamadi and M. Schoenauer, editors, Learning and
Intelligent Optimization, 6th International Conference, LION 6, volume 7219
of Lecture Notes in Computer Science, pages 55–70. Springer,
Heidelberg, 2012.
[ bib ]

[1056]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Bayesian Optimization With Censored Response Data.
Arxiv preprint arXiv:1310.1947, 2013.
[ bib 
http ]

[1057]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Identifying Key Algorithm Parameters and Instance Features using
Forward Selection.
In P. M. Pardalos and G. Nicosia, editors, Learning and
Intelligent Optimization, 7th International Conference, LION 7, volume 7997
of Lecture Notes in Computer Science, pages 364–381. Springer,
Heidelberg, 2013.
[ bib 
DOI ]
Keywords: parameter importance

[1058]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
An Efficient Approach for Assessing Hyperparameter Importance.
In E. P. Xing and T. Jebara, editors, Proceedings of the 31st
International Conference on Machine Learning, ICML 2014, volume 32, pages
754–762, 2014.
[ bib 
http ]
Keywords: fANOVA, parameter importance

[1059]

Frank Hutter, Holger H. Hoos, Kevin LeytonBrown, and Kevin Murphy.
TimeBounded Sequential Parameter Optimization.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pages 281–298. Springer, Heidelberg,
2010.
[ bib 
DOI ]

[1060]

Frank Hutter, Holger H. Hoos, Kevin LeytonBrown, and Thomas Stützle.
ParamILS: An Automatic Algorithm
Configuration Framework.
Journal of Artificial Intelligence Research, 36:267–306,
October 2009.
[ bib 
DOI ]

[1061]

Frank Hutter, Holger H. Hoos, and Thomas Stützle.
Automatic Algorithm Configuration Based on Local Search.
In R. C. Holte and A. Howe, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pages 1152–1157. AAAI Press/
MIT Press, Menlo Park, CA, 2007.
[ bib ]

[1062]

Frank Hutter, Marius Thomas Lindauer, Adrian Balint, Sam Bayless, Holger H.
Hoos, and Kevin LeytonBrown.
The Configurable SAT Solver Challenge (CSSC).
Artificial Intelligence, 243:1–25, 2017.
[ bib 
DOI ]

[1063]

Frank Hutter, Manuel LópezIbáñez, Chris Fawcett, Marius Thomas
Lindauer, Holger H. Hoos, Kevin LeytonBrown, and Thomas Stützle.
AClib: A Benchmark Library for Algorithm Configuration.
In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, and J. L.
Walteros, editors, Learning and Intelligent Optimization, 8th
International Conference, LION 8, volume 8426 of Lecture Notes in
Computer Science, pages 36–40. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1064]

Frank Hutter, Lin Xu, Holger H. Hoos, and Kevin LeytonBrown.
Algorithm runtime prediction: Methods & evaluation.
Artificial Intelligence, 206:79–111, 2014.
[ bib ]

[1065]

Frank Hutter.
SAT benchmarks used in automated algorithm configuration.
http://www.cs.ubc.ca/labs/beta/Projects/AAC/SATbenchmarks.html, 2007.
[ bib ]

[1066]

Frank Hutter.
Automated Configuration of Algorithms for Solving Hard
Computational Problems.
PhD thesis, University of British Columbia, Department of Computer
Science, Vancouver, Canada, October 2009.
[ bib ]

[1067]

Zhiyuan Liu and Jian Tang.
IJCAI 2021 Reproducibility Guidelines, 35th International Joint
Conference on Artificial Intelligence.
https://ijcai21.org/wpcontent/uploads/2020/12/20201226IJCAIReproducibility.pdf,
2021.
[ bib ]

[1068]

Jérémie Humeau, Arnaud Liefooghe, ElGhazali Talbi, and Sébastien
Verel.
ParadisEOMO: From Fitness Landscape Analysis to Efficient
Local Search Algorithms.
Rapport de recherche RR7871, INRIA, France, 2012.
[ bib 
eprint ]

[1069]

Mauro Birattari.
The race Package for R: Racing Methods
for the Selection of the Best.
Technical Report TR/IRIDIA/2003037, IRIDIA, Université Libre de
Bruxelles, Belgium, 2003.
[ bib ]

[1070]

Mauro Birattari.
On the Estimation of the Expected Performance of a Metaheuristic
on a Class of Instances. How Many Instances, How Many Runs?
Technical Report TR/IRIDIA/2004001, IRIDIA, Université Libre de
Bruxelles, Belgium, 2004.
[ bib ]

[1071]

Krzysztof Socha and Marco Dorigo.
Ant Colony Optimization for MixedVariable Optimization
Problems.
Technical Report TR/IRIDIA/2007019, IRIDIA, Université Libre de
Bruxelles, Belgium, October 2007.
[ bib ]

[1072]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
Exploratory Analysis of Stochastic Local Search Algorithms in
Biobjective Optimization.
Technical Report TR/IRIDIA/2009015, IRIDIA, Université Libre de
Bruxelles, Belgium, May 2009.
Published as a book chapter [1418].
[ bib ]

[1073]

Manuel LópezIbáñez and Thomas Stützle.
An Analysis of Algorithmic Components for Multiobjective Ant
Colony Optimization: A Case Study on the Biobjective TSP.
Technical Report TR/IRIDIA/2009019, IRIDIA, Université Libre de
Bruxelles, Belgium, June 2009.
Published in the proceedings of Evolution Artificielle,
2009 [1428].
[ bib ]

[1074]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Effective Hybrid Stochastic Local Search Algorithms for
Biobjective Permutation Flowshop Scheduling.
Technical Report TR/IRIDIA/2009020, IRIDIA, Université Libre de
Bruxelles, Belgium, June 2009.
Published in the proceedings of Hybrid Metaheuristics
2009 [646].
[ bib 
http ]

[1075]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Adaptive “Anytime” TwoPhase Local Search.
Technical Report TR/IRIDIA/2009026, IRIDIA, Université Libre de
Bruxelles, Belgium, 2010.
Published in the proceedings of LION 4 [649].
[ bib 
http ]

[1076]

Thomas Stützle, Manuel LópezIbáñez, Paola Pellegrini, Michael
Maur, Marco A. Montes de Oca, Mauro Birattari, and Marco Dorigo.
Parameter Adaptation in Ant Colony Optimization.
Technical Report TR/IRIDIA/2010002, IRIDIA, Université Libre de
Bruxelles, Belgium, January 2010.
Published as a book chapter [2089].
[ bib ]

[1077]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
A Hybrid TP+PLS Algorithm for Biobjective FlowShop
Scheduling Problems.
Technical Report TR/IRIDIA/2010019, IRIDIA, Université Libre de
Bruxelles, Belgium, 2010.
Published in Computers & Operations
Research [652].
[ bib 
http ]

[1078]

M. S. Hussin and Thomas Stützle.
Tabu Search vs. Simulated Annealing for Solving Large Quadratic
Assignment Instances.
Technical Report TR/IRIDIA/2010020, IRIDIA, Université Libre de
Bruxelles, Belgium, 2010.
[ bib ]

[1079]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Improving the Anytime Behavior of TwoPhase Local Search.
Technical Report TR/IRIDIA/2010022, IRIDIA, Université Libre de
Bruxelles, Belgium, 2010.
Published in Annals of Mathematics and Artificial
Intelligence [651].
[ bib 
http ]

[1080]

Manuel LópezIbáñez, Joshua D. Knowles, and Marco Laumanns.
On Sequential Online Archiving of Objective Vectors.
Technical Report TR/IRIDIA/2011001, IRIDIA, Université Libre de
Bruxelles, Belgium, 2011.
This is a revised version of the paper published in EMO
2011 [1410].
[ bib 
http ]

[1081]

Mauro Birattari, Marco Chiarandini, Marco Saerens, and Thomas Stützle.
Learning graphical models for parameter tuning.
Technical Report TR/IRIDIA/2011002, IRIDIA, Université Libre de
Bruxelles, Belgium, 2011.
[ bib 
http ]

[1082]

Manuel LópezIbáñez and Thomas Stützle.
The Automatic Design of MultiObjective Ant Colony Optimization
Algorithms.
Technical Report TR/IRIDIA/2011003, IRIDIA, Université Libre de
Bruxelles, Belgium, 2011.
Published in IEEE Transactions on Evolutionary
Computation [1435].
[ bib 
http ]

[1083]

Tianjun Liao, Daniel Molina, Marco A. Montes de Oca, and Thomas Stützle.
A Note on the Effects of Enforcing Bound Constraints on
Algorithm Comparisons using the IEEE CEC'05 Benchmark Function Suite.
Technical Report TR/IRIDIA/2011010, IRIDIA, Université Libre de
Bruxelles, Belgium, 2011.
Published in Evolutionary Computation [1359].
[ bib 
http ]

[1084]

Tianjun Liao, Daniel Molina, Marco A. Montes de Oca, and Thomas Stützle.
Computational Results for an Automatically Tuned IPOPCMAES
on the CEC'05 Benchmark Set.
Technical Report TR/IRIDIA/2011022, IRIDIA, Université Libre de
Bruxelles, Belgium, 2011.
[ bib ]

[1085]

Manuel LópezIbáñez and Thomas Stützle.
Automatically Improving the Anytime Behaviour of Optimisation
Algorithms.
Technical Report TR/IRIDIA/2012012, IRIDIA, Université Libre de
Bruxelles, Belgium, May 2012.
Published in European Journal of Operational
Research [1436].
[ bib ]

[1086]

Andreea Radulescu, Manuel LópezIbáñez, and Thomas Stützle.
Automatically Improving the Anytime Behaviour of Multiobjective
Evolutionary Algorithms.
Technical Report TR/IRIDIA/2012019, IRIDIA, Université Libre de
Bruxelles, Belgium, 2012.
Published in the proceedings of EMO 2013 [1829].
[ bib ]

[1087]

Tianjun Liao, Thomas Stützle, Marco A. Montes de Oca, and Marco Dorigo.
A Unified Ant Colony Optimization Algorithm for Continuous
Optimization.
Technical Report TR/IRIDIA/2013002, IRIDIA, Université Libre de
Bruxelles, Belgium, 2013.
[ bib ]

[1088]

Franco Mascia, Manuel LópezIbáñez, Jérémie DuboisLacoste,
and Thomas Stützle.
Grammarbased generation of stochastic local search heuristics
through automatic algorithm configuration tools.
Technical Report TR/IRIDIA/2013015, IRIDIA, Université Libre de
Bruxelles, Belgium, 2013.
[ bib ]

[1089]

Manuel LópezIbáñez, Arnaud Liefooghe, and Sébastien Verel.
Local Optimal Sets and Bounded Archiving on Multiobjective
NKLandscapes with Correlated Objectives.
Technical Report TR/IRIDIA/2014009, IRIDIA, Université Libre de
Bruxelles, Belgium, 2014.
[ bib ]

[1090]

Vito Trianni and Manuel LópezIbáñez.
Advantages of MultiObjective Optimisation in Evolutionary
Robotics: Survey and Case Studies.
Technical Report TR/IRIDIA/2014014, IRIDIA, Université Libre de
Bruxelles, Belgium, 2014.
[ bib 
http ]

[1091]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
Technical Report TR/IRIDIA/2017005, IRIDIA, Université Libre de
Bruxelles, Belgium, November 2017.
[ bib ]

[1092]

Alberto Franzin, Leslie Pérez Cáceres, and Thomas Stützle.
Effect of Transformations of Numerical Parameters in Automatic
Algorithm Configuration.
Technical Report TR/IRIDIA/2017006, IRIDIA, Université Libre de
Bruxelles, Belgium, March 2017.
[ bib 
http ]

[1093]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Configuration of Multiobjective Optimizers and
Multiobjective Configuration.
Technical Report TR/IRIDIA/2017011, IRIDIA, Université Libre de
Bruxelles, Belgium, November 2017.
Published as [237].
[ bib 
http ]

[1094]

Manuel LópezIbáñez, MarieEléonore Kessaci, and Thomas
Stützle.
Automatic Design of Hybrid Metaheuristics from Algorithmic
Components.
Technical Report TR/IRIDIA/2017012, IRIDIA, Université Libre de
Bruxelles, Belgium, December 2017.
[ bib 
http ]

[1095]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatically Designing StateoftheArt Multi and
ManyObjective Evolutionary Algorithms.
Technical Report TR/IRIDIA/2018001, IRIDIA, Université Libre de
Bruxelles, Belgium, January 2018.
Published as [233].
[ bib 
http ]

[1096]

Alberto Franzin and Thomas Stützle.
Revisiting Simulated Annealing: a ComponentBased Analysis.
Technical Report TR/IRIDIA/2018010, IRIDIA, Université Libre de
Bruxelles, Belgium, 2018.
[ bib 
http ]

[1097]

Christian Leonardo CamachoVillalón, Thomas Stützle, and Marco Dorigo.
PSOX: A ComponentBased Framework for the Automatic Design of
Particle Swarm Optimization Algorithms.
Technical Report TR/IRIDIA/2021002, IRIDIA, Université Libre de
Bruxelles, Belgium, 2021.
[ bib 
http ]

[1098]

Alberto Franzin and Thomas Stützle.
A Landscapebased Analysis of Fixed Temperature and Simulated
Annealing.
Technical Report TR/IRIDIA/2021005, IRIDIA, Université Libre de
Bruxelles, Belgium, 2021.
[ bib 
http ]

[1099]

Christian Leonardo CamachoVillalón, Thomas Stützle, and Marco Dorigo.
Cuckoo Search ≡(μ+ λ)Evolution Strategy – A
Rigorous Analysis of an Algorithm That Has Been Misleading the Research
Community for More Than 10 Years and Nobody Seems to Have Noticed.
Technical Report TR/IRIDIA/2021006, IRIDIA, Université Libre de
Bruxelles, Belgium, 2021.
[ bib 
http ]

[1100]

Claudio Iacopino and Phil Palmer.
The Dynamics of Ant Colony Optimization Algorithms Applied to
Binary Chains.
Swarm Intelligence, 6(4):343–377, 2012.
[ bib ]

[1101]

Claudio Iacopino, Phil Palmer, N. Policella, A. Donati, and A. Brewer.
How Ants Can Manage Your Satellites.
Acta Futura, 9:59–72, 2014.
[ bib 
DOI ]
Keywords: ACO, Space

[1102]

Toshihide Ibaraki, Shinji Imahori, Koji Nonobe, Kensuke Sobue, Takeaki Uno, and
Mutsunori Yagiura.
An Iterated Local Search Algorithm for the Vehicle Routing
Problem with Convex Time Penalty Functions.
Discrete Applied Mathematics, 156(11):2050–2069, 2008.
[ bib ]

[1103]

Toshihide Ibaraki.
A Personal Perspective on Problem Solving by General Purpose
Solvers.
International Transactions in Operational Research,
17(3):303–315, 2010.
[ bib ]

[1104]

Jonas Ide and Anita Schöbel.
Robustness for uncertain multiobjective optimization: a survey
and analysis of different concepts.
OR Spectrum, 38(1):235–271, 2016.
[ bib 
DOI ]
In this paper, we discuss various concepts of robustness for
uncertain multiobjective optimization problems. We extend
the concepts of flimsily, highly, and lightly robust
efficiency and we collect different versions of minmax robust
efficiency and concepts based on set order relations from the
literature. Altogether, we compare and analyze ten different
concepts and point out their relations to each
other. Furthermore, we present reduction results for the
class of objectivewise uncertain multiobjective
optimization problems.

[1105]

Christian Igel.
Multiobjective Model Selection for Support Vector Machines.
In C. A. Coello Coello, A. H. Aguirre, and E. Zitzler, editors,
Evolutionary Multicriterion Optimization, EMO 2005, volume 3410 of
Lecture Notes in Computer Science, pages 534–546. Springer,
Heidelberg, 2005.
[ bib 
DOI ]
Early work on multiobjective hyperparameter optimization
(AutoML)

[1106]

Christian Igel, Nikolaus Hansen, and S. Roth.
Covariance Matrix Adaptation for Multiobjective Optimization.
Evolutionary Computation, 15(1):1–28, 2007.
[ bib ]

[1107]

Christian Igel, V. HeidrichMeisner, and T. Glasmachers.
Shark.
Journal of Machine Learning Research, 9:993–996, June 2008.
[ bib 
http ]

[1108]

Kokolo Ikeda, Hajime Kita, and Shigenobu Kobayashi.
Failure of Paretobased MOEAs: Does nondominated really
mean near to optimal?
In Proceedings of the 2001 Congress on Evolutionary Computation
(CEC'01), pages 957–962. IEEE Press, Piscataway, NJ, 2001.
[ bib ]
Keywords: dominance resistance

[1109]

Nesa Ilich and Slobodan P. Simonovic.
Evolutionary Algorithm for minimization of pumping cost.
Journal of Computing in Civil Engineering, ASCE,
12(4):232–240, October 1998.
[ bib ]
This paper deals with minimizing the total cost of
pumping in a liquid pipeline. Previous experience
with the most common solution procedures in pipeline
optimization is discussed along with their strengths
and weaknesses. The proposed method is an
evolutionary algorithm with two distinct features:
(1) The search is restricted to feasible region
only; and (2) it utilizes a floating point decision
variable rather than integer or binary as is the
case with most other similar approaches. A numerical
example is presented as a basis for verification of
the proposed method and its comparison with the
existing solver that utilizes the nonlinear
Newtonian search. The proposed method provides
promising improvements in terms of optimality when
compared to the widespread gradient search methods
because it does not involve evaluation of the
gradient of the objective function. It also provides
potential to improve the performance of previous
evolutionary programs because it restricts the
search to the feasible region, thus eliminating
large overhead associated with generation and
inspection of solutions that are
infeasible. Comparison of the two solutions revealed
improvement of the solution in favor of the proposed
algorithm, which ranged up to 6% depending on the
initial values of the decision variables in the
Newtonian search. The proposed method was not
sensitive to the starting value of the decision
variables.

[1110]

Janine Illian, Antti Penttinen, Helga Stoyan, and Dietrich Stoyan.
Statistical Analysis and Modelling of Spatial Point Patterns.
Wiley, 2008.
[ bib ]

[1111]

Takashi Imamichi, Mutsunori Yagiura, and Hiroshi Nagamochi.
An Iterated Local Search Algorithm Based on Nonlinear
Programming for the Irregular Strip Packing Problem.
Discrete Optimization, 6(4):345–361, 2009.
[ bib ]

[1112]

Alfred Inselberg.
The Plane with Parallel Coordinates.
The Visual Computer, 1(2):69–91, 1985.
[ bib ]

[1113]

John P. A. Ioannidis.
Why Most Published Research Findings Are False.
PLoS Medicine, 2(8):e124, 2005.
[ bib 
DOI ]

[1114]

S. Iredi, D. Merkle, and Martin Middendorf.
BiCriterion Optimization with Multi Colony Ant Algorithms.
In E. Zitzler, K. Deb, L. Thiele, C. A. Coello Coello, and
D. Corne, editors, Evolutionary Multicriterion Optimization, EMO 2001,
volume 1993 of Lecture Notes in Computer Science, pages 359–372.
Springer, Heidelberg, 2001.
[ bib ]
Keywords: BicriterionAnt

[1115]

Manuel LópezIbáñez and Thomas Stützle.
Automatically Improving the Anytime Behaviour of Optimisation
Algorithms: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012011/, 2012.
[ bib ]

[1116]

Stefan Irnich.
A Unified Modeling and Solution Framework for Vehicle Routing
and Local SearchBased Metaheuristics.
INFORMS Journal on Computing, 20(2):270–287, 2008.
[ bib ]

[1117]

Ekhine Irurozki, Borja Calvo, and José A. Lozano.
Sampling and Learning Mallows and Generalized Mallows Models
Under the Cayley Distance.
Methodology and Computing in Applied Probability, 20(1):1–35,
June 2016.
[ bib 
DOI ]

[1118]

Ekhine Irurozki, Borja Calvo, and José A. Lozano.
PerMallows: An R Package for Mallows
and Generalized Mallows Models.
Journal of Statistical Software, 71, 2019.
[ bib 
DOI ]
In this paper we present the R package PerMallows, which is a
complete toolbox to work with permutations, distances and
some of the most popular probability models for permutations:
Mallows and the Generalized Mallows models. The Mallows model
is an exponential location model, considered as analogous to
the Gaussian distribution. It is based on the definition of a
distance between permutations. The Generalized Mallows model
is its bestknown extension. The package includes functions
for making inference, sampling and learning such
distributions. The distances considered in PerMallows are
Kendall's τ, Cayley, Hamming and Ulam.
Keywords: Cayley,Generalized Mallows,Hamming,Kendall's
τ,Learning,Mallows,Permutation,R,Ranking,Sampling,Ulam

[1119]

Ekhine Irurozki, Jesus Lobo, Aritz Perez, and Javier Del Ser.
Rank aggregation for nonstationary data streams.
Arxiv preprint arXiv:1910.08795 [stat.ML], 2020.
[ bib 
http ]
Keywords: uborda

[1120]

Ekhine Irurozki and Manuel LópezIbáñez.
Unbalanced Mallows Models for Optimizing Expensive BlackBox
Permutation Problems.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2021. ACM Press, New
York, NY, 2021.
[ bib 
DOI 
supplementary material ]
Expensive blackbox combinatorial optimization problems arise
in practice when the objective function is evaluated by means
of a simulator or a realworld experiment. Since each fitness
evaluation is expensive in terms of time or resources, only a
limited number of evaluations is possible, typically several
orders of magnitude smaller than in nonexpensive
problems. In this scenario, classical optimization methods
such as mixedinteger programming and local search are not
useful. In the continuous case, Bayesian optimization, in
particular using Gaussian processes, has proven very
effective under these conditions. Much less research is
available in the combinatorial case. In this paper, we
propose and analyze UMM, an estimationofdistribution (EDA)
algorithm based on a Mallows probabilistic model and
unbalanced rank aggregation (uBorda). Experimental results on
blackbox versions of LOP and PFSP show that UMM is able to
match, and sometimes surpass, the solutions obtained by CEGO,
a Bayesian optimization algorithm for combinatorial
optimization. Moreover, the computational complexity of UMM
increases linearly with both the number of function
evaluations and the permutation size.

[1121]

Hisao Ishibuchi and T. Murata.
A multiobjective genetic local search algorithm and its
application to flowshop scheduling.
IEEE Transactions on Systems, Man, and Cybernetics – Part C,
28(3):392–403, 1998.
[ bib ]

[1122]

Hisao Ishibuchi, N. Akedo, and Y. Nojima.
Behavior of Multiobjective Evolutionary Algorithms on
ManyObjective Knapsack Problems.
IEEE Transactions on Evolutionary Computation, 19(2):264–283,
2015.
[ bib 
DOI ]

[1123]

Hisao Ishibuchi, Hiroyuki Masuda, and Yusuke Nojima.
A Study on Performance Evaluation Ability of a Modified Inverted
Generational Distance Indicator.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pages 695–702. ACM Press, New York, NY, 2015.
[ bib ]

[1124]

Hisao Ishibuchi, Hiroyuki Masuda, Yuki Tanigaki, and Yusuke Nojima.
Modified Distance Calculation in Generational Distance and
Inverted Generational Distance.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pages 110–125.
Springer, Heidelberg, 2015.
[ bib ]
Proposed IGD+
Keywords: Performance metrics, multiobjective, IGD, IGD+

[1125]

Hisao Ishibuchi, Shinta Misaki, and Hideo Tanaka.
Modified simulated annealing algorithms for the flow shop
sequencing problem.
European Journal of Operational Research, 81(2):388–398, 1995.
[ bib ]

[1126]

Hisao Ishibuchi, N. Tsukamoto, and Y. Nojima.
Evolutionary manyobjective optimization: A short review.
In Proceedings of the 2008 Congress on Evolutionary Computation
(CEC 2008), pages 2419–2426, Piscataway, NJ, 2008. IEEE Press.
[ bib 
DOI ]

[1127]

Srikanth K. Iyer and Barkha Saxena.
Improved genetic algorithm for the permutation flowshop
scheduling problem.
Computers & Operations Research, 31(4):593–606, 2004.
[ bib 
DOI ]

[1128]

Christopher H. Jackson.
MultiState Models for Panel Data: The msm Package
for R.
Journal of Statistical Software, 38(8):1–29, 2011.
[ bib 
http ]

[1129]

Richard H. F. Jackson, Paul T. Boggs, Stephen G. Nash, and Susan Powell.
Guidelines for Reporting Results of Computational Experiments.
Report of the Ad Hoc Committee.
Mathematical Programming, 49(3):413–425, 1991.
[ bib ]

[1130]

Larry W. Jacobs and Michael J. Brusco.
A Local Search Heuristic for Large SetCovering Problems.
Naval Research Logistics, 42(7):1129–1140, 1995.
[ bib ]

[1131]

Sophie Jacquin, Laetitia Jourdan, and ElGhazali Talbi.
Dynamic Programming Based Metaheuristic for Energy Planning
Problems.
In A. I. EsparciaAlcázar and A. M. Mora, editors,
Applications of Evolutionary Computation, volume 8602 of Lecture Notes
in Computer Science, pages 165–176. Springer, Heidelberg, 2014.
[ bib 
DOI ]
Keywords: irace

[1132]

Warren G. Jackson, Ender Özcan, and Robert I. John.
Move acceptance in local search metaheuristics for crossdomain
search.
Expert Systems with Applications, 109:131–151, 2018.
[ bib ]

[1133]

Daniel M Jaeggi, Geoffrey T Parks, Timoleon Kipouros, and P John Clarkson.
The development of a multiobjective Tabu Search algorithm for
continuous optimisation problems.
European Journal of Operational Research, 185(3):1192–1212,
2008.
[ bib ]

[1134]

Antonio López Jaimes, Carlos A. Coello Coello, and Debrup Chakraborty.
Objective reduction using a feature selection technique.
In C. Ryan, editor, Proceedings of the Genetic and Evolutionary
Computation Conference, GECCO 2008, pages 673–680. ACM Press, New York, NY,
2008.
[ bib ]

[1135]

Satish Jajodia, Ioannis Minis, George Harhalakis, and JeanMarie Proth.
CLASS: computerized layout solutions using simulated
annealing.
International Journal of Production Research, 30(1):95–108,
1992.
[ bib ]

[1136]

Kevin G. Jamieson and Ameet Talwalkar.
Nonstochastic Best Arm Identification and Hyperparameter
Optimization.
In A. Gretton and C. C. Robert, editors, Proceedings of the 19th
International Conference on Artificial Intelligence and Statistics, AISTATS
2016, Cadiz, Spain, May 911, 2016, volume 51 of JMLR Workshop and
Conference Proceedings, pages 240–248. JMLR.org, 2016.
[ bib 
http ]

[1137]

Andrzej Jaszkiewicz.
Genetic local search for multiobjective combinatorial
optimization.
European Journal of Operational Research, 137(1):50–71, 2002.
[ bib ]

[1138]

Andrzej Jaszkiewicz.
ManyObjective Pareto Local Search.
European Journal of Operational Research, 271(3):1001–1013,
2018.
[ bib 
DOI ]

[1139]

Andrzej Jaszkiewicz and Jürgen Branke.
Interactive Multiobjective Evolutionary Algorithms.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science, pages 179–193.
Springer, Heidelberg, 2008.
[ bib 
DOI ]

[1140]

Andrzej Jaszkiewicz, Hisao Ishibuchi, and Qingfu Zhang.
Multiobjective memetic algorithms.
In F. Neri, C. Cotta, and P. Moscato, editors, Handbook of
Memetic Algorithms, volume 379 of Studies in Computational
Intelligence, pages 201–217. Springer, 2011.
[ bib ]

[1141]

Andrzej Jaszkiewicz.
On the performance of multipleobjective genetic local search on
the 0/1 knapsack problem – A comparative experiment.
IEEE Transactions on Evolutionary Computation, 6(4):402–412,
2002.
[ bib ]

[1142]

Frank Hutter and Steve Ramage.
Manual for SMAC.
University of British Columbia, 2015.
SMAC version 2.10.03.
[ bib 
http ]

[1143]

M. T. Jensen.
Reducing the runtime complexity of multiobjective EAs: The
NSGAII and other algorithms.
IEEE Transactions on Evolutionary Computation, 7(5):503–515,
2003.
[ bib ]

[1144]

Mark Jerrum and Alistair Sinclair.
The Markov chain Monte Carlo method: an approach to
approximate counting and integration.
In D. S. Hochbaum, editor, Approximation Algorithms For
NPhard Problems, pages 482–520. PWS Publishing Co., 1996.
[ bib ]

[1145]

Mark Jerrum and Gregory Sorkin.
The Metropolis algorithm for graph bisection.
Discrete Applied Mathematics, 82(1):155–175, 1998.
[ bib ]

[1146]

Mark Jerrum.
Large cliques elude the Metropolis process.
Random Structures & Algorithms, 3(4):347–359, 1992.
[ bib ]

[1147]

Alexandre D. Jesus, Arnaud Liefooghe, Bilel Derbel, and Luís Paquete.
Algorithm Selection of Anytime Algorithms.
In C. A. Coello Coello, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2020, pages 850—858. ACM Press,
New York, NY, 2020.
[ bib 
DOI ]

[1148]

S. Jiang, Y. S. Ong, J. Zhang, and L. Feng.
Consistencies and Contradictions of Performance Metrics in
Multiobjective Optimization.
IEEE Transactions on Cybernetics, 44(12):2391–2404, 2014.
[ bib ]

[1149]

Yaochu Jin.
A Comprehensive Survey of Fitness Approximation in Evolutionary
Computation.
Soft Computing, 9(1):3–12, 2005.
[ bib ]

[1150]

Yaochu Jin, Handing Wang, Tinkle Chugh, Dan Guo, and Kaisa Miettinen.
DataDriven Evolutionary Optimization: An Overview and Case
Studies.
IEEE Transactions on Evolutionary Computation, 23(3):442–458,
June 2019.
[ bib 
DOI ]

[1151]

Journal of Heuristics. Policies on Heuristic Search Research.
http://www.springer.com/journal/10732, 2015.
Version visited last on June 10, 2015.
[ bib ]

[1152]

David S. Johnson.
Optimal Two and Threestage Production Scheduling with Setup
Times Included.
Naval Research Logistics Quarterly, 1:61–68, 1954.
[ bib ]

[1153]

David S. Johnson, Cecilia R. Aragon, Lyle A. McGeoch, and Catherine Schevon.
Optimization by Simulated Annealing: An Experimental Evaluation:
Part I, Graph Partitioning.
Operations Research, 37(6):865–892, 1989.
[ bib ]

[1154]

David S. Johnson, Cecilia R. Aragon, Lyle A. McGeoch, and Catherine Schevon.
Optimization by Simulated Annealing: An Experimental Evaluation:
Part II, Graph Coloring and Number Partitioning.
Operations Research, 39(3):378–406, 1991.
[ bib ]

[1155]

David S. Johnson, G. Gutin, Lyle A. McGeoch, A. Yeo, W. Zhang, and
A. Zverovitch.
Experimental Analysis of Heuristics for the ATSP.
In G. Gutin and A. Punnen, editors, The Traveling Salesman
Problem and its Variations, pages 445–487. Kluwer Academic Publishers,
Dordrecht, The Netherlands, 2002.
[ bib ]

[1156]

Alan W. Johnson and Sheldon H. Jacobson.
On the Convergence of Generalized Hill Climbing Algorithms.
Discrete Applied Mathematics, 119(1):37–57, 2002.
[ bib ]

[1157]

David S. Johnson and Lyle A. McGeoch.
Experimental Analysis of Heuristics for the STSP.
In G. Gutin and A. Punnen, editors, The Traveling Salesman
Problem and its Variations, pages 369–443. Kluwer Academic Publishers,
Dordrecht, The Netherlands, 2002.
[ bib ]

[1158]

David S. Johnson and Lyle A. McGeoch.
The Traveling Salesman Problem: A Case Study in Local
Optimization.
In E. H. L. Aarts and J. K. Lenstra, editors, Local Search in
Combinatorial Optimization, pages 215–310. John Wiley & Sons, Chichester,
UK, 1997.
[ bib ]

[1159]

Mark E. Johnson, Leslie M. Moore, and Donald Ylvisaker.
Minimax and maximin distance designs.
Journal of Statistical Planning and Inference, 26(2):131–148,
1990.
[ bib ]
Keywords: Bayesian design

[1160]

David S. Johnson, Christos H. Papadimitriou, and M. Yannakakis.
How Easy is Local Search?
Journal of Computer System Science, 37(1):79–100, 1988.
[ bib ]

[1161]

David S. Johnson.
Local Optimization and the Traveling Salesman Problem.
In M. Paterson, editor, Automata, Languages and Programming,
17th International Colloquium, volume 443 of Lecture Notes in Computer
Science, pages 446–461, Heidelberg, 1990. Springer.
[ bib ]

[1162]

David S. Johnson, Lyle A. McGeoch, C. Rego, and Fred Glover.
8th DIMACS Implementation Challenge: The Traveling Salesman
Problem.
http://dimacs.rutgers.edu/archive/Challenges/TSP, 2001.
[ bib ]
Keywords: TSP Challenge, RUE, RCE, generators

[1163]

David S. Johnson.
A Theoretician's Guide to the Experimental Analysis of
Algorithms.
In M. H. Goldwasser, D. S. Johnson, and C. C. McGeoch, editors,
Data Structures, Near Neighbor Searches, and Methodology: Fifth and Sixth
DIMACS Implementation Challenges, pages 215–250. American Mathematical
Society, Providence, RI, 2002.
[ bib ]

[1164]

Kenneth A. De Jong.
Evolutionary computation: a unified approach.
MIT Press, Cambridge, MA, 2006.
[ bib ]

[1165]

Neil C. Jones and Pavel A. Pevzner.
An introduction to bioinformatics algorithms.
MIT Press, Cambridge, MA, 2004.
[ bib ]

[1166]

Donald R. Jones, Matthias Schonlau, and William J. Welch.
Efficient Global Optimization of Expensive BlackBox Functions.
Journal of Global Optimization, 13(4):455–492, 1998.
[ bib ]
Proposed EGO algorithm
Keywords: EGO

[1167]

Kenneth A. De Jong and William M. Spears.
A formal analysis of the role of multipoint crossover in
genetic algorithms.
Annals of Mathematics and Artificial Intelligence, 5(1):1–26,
1992.
[ bib ]

[1168]

D. E. Joslin and D. P. Clements.
Squeaky Wheel Optimization.
Journal of Artificial Intelligence Research, 10:353–373, 1999.
[ bib ]

[1169]

P. W. Jowitt and G. Germanopoulos.
Optimal pump scheduling in water supply networks.
Journal of Water Resources Planning and Management, ASCE,
118(4):406–422, 1992.
[ bib ]
The electricity cost of pumping accounts for a large
part of the total operating cost for watersupply
networks. This study presents a method based on
linear programming for determining an optimal
(minimum cost) schedule of pumping on a 24hr
basis. Both unit and maximum demand electricity
charges are considered. Account is taken of the
relative efficiencies of the available pumps, the
structure of the electricity tariff, the
consumerdemand profile, and the hydraulic
characteristics and operational constraints of the
network. The use of extendedperiod simulation of
the network operation in determining the parameters
of the linearized network equations and constraints
and in studying the optimized network operation is
described. An application of the method to an
existing network in the United Kingdom is presented,
showing that considerable savings are possible. The
method was found to be robust and with low
computationtime requirements, and is therefore
suitable for realtime implementation.

[1170]

Angel A. Juan, Javier Faulin, Scott E. Grasman, Markus Rabe, and Gonçalo
Figueira.
A review of simheuristics: Extending metaheuristics to deal with
stochastic combinatorial optimization problems.
Operations Research Perspectives, 2:62–72, 2015.
[ bib 
DOI ]
Keywords: Metaheuristics; Simulation; Combinatorial optimization;
Stochastic problems

[1171]

Angel A. Juan, Helena R. Lourenço, Manuel Mateo, Rachel Luo, and Quim
Castellà.
Using Iterated Local Search for Solving the Flowshop Problem:
Parallelization, Parametrization, and Randomization Issues.
International Transactions in Operational Research,
21(1):103–126, 2014.
[ bib ]

[1172]

H. Juillé and J. B. Pollack.
A SamplingBased Heuristic for Tree Search Applied to Grammar
Induction.
In J. Mostow and C. Rich, editors, Proceedings of AAAI 1998 –
Fifteenth National Conference on Artificial Intelligence, pages 776–783.
AAAI Press/MIT Press, Menlo Park, CA, 1998.
[ bib ]

[1173]

Bryant A. Julstrom.
What Have You Done for Me Lately? Adapting Operator
Probabilities in a SteadyState Genetic Algorithm.
In L. J. Eshelman, editor, ICGA, pages 81–87. Morgan Kaufmann
Publishers, San Francisco, CA, 1995.
[ bib ]

[1174]

M. Jünger, Gerhard Reinelt, and S. Thienel.
Provably Good Solutions for the Traveling Salesman Problem.
Zeitschrift für Operations Research, 40(2):183–217, 1994.
[ bib ]

[1175]

Elena A. Kabova, Jason C. Cole, Oliver Korb, Manuel LópezIbáñez,
Adrian C. Williams, and Kenneth Shankland.
Improved performance of crystal structure solution from powder
diffraction data through parameter tuning of a simulated annealing
algorithm.
Journal of Applied Crystallography, 50(5):1411–1420, October
2017.
[ bib 
DOI ]
Significant gains in the performance of the simulated
annealing algorithm in the DASH software package have
been realized by using the irace automatic
configuration tool to optimize the values of three key
simulated annealing parameters. Specifically, the success
rate in finding the global minimum in intensity χ^{2}
space is improved by up to an order of magnitude. The general
applicability of these revised simulated annealing parameters
is demonstrated using the crystal structure determinations of
over 100 powder diffraction datasets.
Keywords: crystal structure determination, powder diffraction,
simulated annealing, parameter tuning, irace

[1176]

Serdar Kadioglu, Yuri Malitsky, Meinolf Sellmann, and Kevin Tierney.
ISAC: InstanceSpecific Algorithm Configuration.
In H. Coelho, R. Studer, and M. Wooldridge, editors, Proceedings
of the 19th European Conference on Artificial Intelligence, pages 751–756.
IOS Press, 2010.
[ bib ]

[1177]

Daniel Kahneman and Amos Tversky.
Prospect theory: An analysis of decision under risk.
Econometrica, 47(2):263–291, 1979.
[ bib 
DOI ]

[1178]

Daniel Kahneman.
Maps of bounded rationality: Psychology for behavioral
economics.
American economic review, 93(5):1449–1475, 2003.
[ bib ]

[1179]

H. Kaji, Kokolo Ikeda, and Hajime Kita.
Avoidance of constraint violation for experimentbased
evolutionary multiobjective optimization.
In Proceedings of the 2009 Congress on Evolutionary Computation
(CEC 2009), pages 2756–2763, Piscataway, NJ, 2009. IEEE Press.
[ bib 
DOI ]
Keywords: evolutionary computation;constraint
violation;experimentbased evolutionary multiobjective
optimization;evolutionary algorithm;riskyconstraint
violation;Constraint optimization;Diesel
engines;Calibration;Evolutionary computation;Electric
breakdown;Optimization methods;Uncertainty;Computational
fluid dynamics;Cost function;Temperature

[1180]

Qinma Kang, Hong He, and Jun Wei.
An Effective Iterated Greedy Algorithm for Reliabilityoriented
Task Allocation in Distributed Computing Systems.
Journal of Parallel and Distributed Computing,
73(8):1106–1115, 2013.
[ bib ]

[1181]

Korhan Karabulut.
A hybrid iterated greedy algorithm for total tardiness
minimization in permutation flowshops.
Computers and Industrial Engineering, 98(Supplement C):300 –
307, 2016.
[ bib ]

[1182]

Dervis Karaboga and Bahriye Akay.
A Survey: Algorithms Simulating Bee Swarm Intelligence.
Artificial Intelligence Review, 31(1–4):61–85, 2009.
[ bib ]

[1183]

Giorgos Karafotias, Agoston E. Eiben, and Mark Hoogendoorn.
Generic parameter control with reinforcement learning.
In C. Igel and D. V. Arnold, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2014, pages 1319–1326. ACM
Press, New York, NY, 2014.
[ bib ]

[1184]

Giorgos Karafotias, Mark Hoogendoorn, and Agoston E. Eiben.
Parameter Control in Evolutionary Algorithms: Trends and
Challenges.
IEEE Transactions on Evolutionary Computation, 19(2):167–187,
2015.
[ bib ]

[1185]

Giorgos Karafotias, Mark Hoogendoorn, and Agoston E. Eiben.
Evaluating reward definitions for parameter control.
In A. M. Mora and G. Squillero, editors, Applications of
Evolutionary Computation, volume 9028 of Lecture Notes in Computer
Science, pages 667–680. Springer, Heidelberg, 2015.
[ bib 
DOI ]

[1186]

İbrahim Karahan and Murat Köksalan.
A territory defining multiobjective evolutionary algorithms and
preference incorporation.
IEEE Transactions on Evolutionary Computation, 14(4):636–664,
2010.
[ bib 
DOI ]
Keywords: TDEA

[1187]

Zohar Karnin, Tomer Koren, and Oren Somekh.
Almost optimal exploration in multiarmed bandits.
In S. Dasgupta and D. McAllester, editors, Proceedings of the
30th International Conference on Machine Learning, ICML 2013, volume 28,
pages 1238–1246, 2013.
[ bib 
http ]

[1188]

Daniel Karapetyan, Andrew J. Parkes, and Thomas Stützle.
Algorithm Configuration: Learning policies for the quick
termination of poor performers.
In R. Battiti, M. Brunato, I. Kotsireas, and P. M. Pardalos, editors,
Learning and Intelligent Optimization, 12th International Conference,
LION 12, volume 11353 of Lecture Notes in Computer Science, pages
220–224. Springer, Cham, Switzerland, 2018.
[ bib 
DOI ]

[1189]

Oleksiy Karpenko, Jianming Shi, and Yang Dai.
Prediction of MHC class II binders using the ant colony
search strategy.
Artificial Intelligence in Medicine, 35(1):147–156, 2005.
[ bib ]

[1190]

Giorgos Karafotias, Selmar K. Smit, and Agoston E. Eiben.
A generic approach to parameter control.
In C. Di Chio et al., editors, Applications of Evolutionary
Computation, volume 7248 of Lecture Notes in Computer Science, pages
366–375. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1191]

Korhan Karabulut and Fatih M. Tasgetiren.
A Variable Iterated Greedy Algorithm for the Traveling Salesman
Problem with Time Windows.
Information Sciences, 279:383–395, 2014.
[ bib ]

[1192]

Narendra Karmarkar.
A new polynomialtime algorithm for linear programming.
In R. A. DeMillo, editor, Proceedings of the sixteenth annual
ACM Symposium on Theory of Computing, pages 302–311. ACM Press, 1984.
[ bib ]

[1193]

Richard M. Karp.
Reducibility among combinatorial problems.
In R. E. Miller and W. Thatcher, James, editors, Proceedings of
a symposium on the Complexity of Computer Computations, held March 2022,
1972, at the IBM Thomas J. Watson Research Center, Yorktown Heights,
New York, USA, The IBM Research Symposia Series, pages 85–103. Springer,
1972.
[ bib ]

[1194]

Joseph R. Kasprzyk, Shanthi Nataraj, Patrick M. Reed, and Robert J. Lempert.
Many objective robust decision making for complex environmental
systems undergoing change.
Environmental Modelling & Software, 42:55–71, 2013.
[ bib ]
Keywords: scenariobased

[1195]

Joseph R. Kasprzyk, Patrick M. Reed, Gregory W. Characklis, and Brian R.
Kirsch.
Manyobjective de Novo water supply portfolio planning under
deep uncertainty.
Environmental Modelling & Software, 34:87–104, 2012.
[ bib ]
Keywords: scenariobased

[1196]

K. Katayama and H. Narihisa.
Iterated Local Search Approach using Genetic Transformation to
the Traveling Salesman Problem.
In W. Banzhaf, J. M. Daida, A. E. Eiben, M. H. Garzon, V. Honavar,
M. J. Jakiela, and R. E. Smith, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 1999, volume 1, pages 321–328.
Morgan Kaufmann Publishers, San Francisco, CA, 1999.
[ bib ]

[1197]

S. A. Kauffman.
The Origins of Order.
Oxford University Press, 1993.
[ bib ]

[1198]

Artem Kaznatcheev, David A. Cohen, and Peter Jeavons.
Representing Fitness Landscapes by Valued Constraints to
Understand the Complexity of Local Search.
Journal of Artificial Intelligence Research, 69:1077–1102,
2020.
[ bib 
DOI ]

[1199]

Michael D. Kazantzis, Angus R. Simpson, David Kwong, and Shyh Min Tan.
A new methodology for optimizing the daily operations of a
pumping plant.
In Proceedings of 2002 Conference on Water Resources Planning,
Roanoke, USA, May 2002. ASCE.
[ bib ]

[1200]

Liangjun Ke, Claudia Archetti, and Zuren Feng.
Ants can solve the team orienteering problem.
Computers and Industrial Engineering, 54(3):648–665, 2008.
[ bib 
DOI ]
The team orienteering problem (TOP) involves
finding a set of paths from the starting point to
the ending point such that the total collected
reward received from visiting a subset of locations
is maximized and the length of each path is
restricted by a prespecified limit. In this paper,
an ant colony optimization (ACO) approach is
proposed for the team orienteering problem. Four
methods, i.e., the sequential,
deterministicconcurrent and randomconcurrent and
simultaneous methods, are proposed to construct
candidate solutions in the framework of ACO. We
compare these methods according to the results
obtained on wellknown problems from the
literature. Finally, we compare the algorithm with
several existing algorithms. The results show that
our algorithm is promising.
Keywords: Ant colony optimization, Ant system, Heuristics,
Team orienteering problem

[1201]

Liangjun Ke, Zuren Feng, Zongben Xu, Ke Shang, and Yonggang Wang.
A multiobjective ACO algorithm for rough feature selection.
In Circuits, Communications and System (PACCS), 2010 Second
PacificAsia Conference on, volume 1, pages 207–210, 2010.
[ bib ]

[1202]

R. L. Keeney.
Analysis of preference dependencies among objectives.
Operations Research, 29:1105–1120, 1981.
[ bib ]

[1203]

Eric Kee, Sarah Airey, and Walling Cyre.
An adaptive genetic algorithm.
In E. D. Goodman, editor, Proceedings of the 3rd Annual
Conference on Genetic and Evolutionary Computation, GECCO 2001, pages
391–397. Morgan Kaufmann Publishers, San Francisco, CA, 2001.
[ bib ]

[1204]

Hans Kellerer, Ulrich Pferschy, and David Pisinger.
Knapsack problems.
Springer, 2004.
[ bib ]

[1205]

Robert E. Keller and Riccardo Poli.
Linear genetic programming of parsimonious metaheuristics.
In Proceedings of the 2007 Congress on Evolutionary Computation
(CEC 2007), pages 4508–4515, Piscataway, NJ, 2007. IEEE Press.
[ bib 
DOI ]

[1206]

Robert E. Keller and Riccardo Poli.
CostBenefit Investigation of a GeneticProgramming
Hyperheuristic.
In E. Lutton, P. Legrand, P. Parrend, N. Monmarché, and
M. Schoenauer, editors, EA 2017: Artificial Evolution, volume 10764 of
Lecture Notes in Computer Science, pages 13–24, Heidelberg, 2017.
Springer.
[ bib ]

[1207]

Graham Kendall, Ruibin Bai, Jacek Blazewicz, Patrick De Causmaecker,
Michel Gendreau, Robert John, Jiawei Li, Barry McCollum, Erwin Pesch, Rong
Qu, Nasser Sabar, Greet Vanden Berghe, and Angelina Yee.
Good Laboratory Practice for Optimization Research.
Journal of the Operational Research Society, 67(4):676–689,
2016.
[ bib 
DOI ]

[1208]

J. Kennedy and Russell C. Eberhart.
Particle Swarm Optimization.
In Proceedings of IEEE International Conference on Neural
Networks, pages 1942–1948, Piscataway, NJ, USA, 1995. IEEE Press.
[ bib ]

[1209]

J. Kennedy and Russell C. Eberhart.
A discrete binary version of the particle swarm algorithm.
In Proceedings of the 1997 IEEE International Conference on
Systems, Man, and Cybernetics, pages 4104–4108, Piscataway, NJ, USA, 1997.
IEEE Press.
[ bib ]

[1210]

J. Kennedy, Russell C. Eberhart, and Yuhui Shi.
Swarm Intelligence.
Morgan Kaufmann Publishers, San Francisco, CA, 2001.
[ bib ]

[1211]

Maurice G. Kendall.
Rank correlation methods.
Griffin, London, 1948.
[ bib ]

[1212]

Pascal Kerschke, Holger H. Hoos, Frank Neumann, and Heike Trautmann.
Automated Algorithm Selection: Survey and Perspectives.
Evolutionary Computation, 27(1):3–45, March 2019.
[ bib 
DOI ]

[1213]

B. W. Kernighan and S. Lin.
An Efficient Heuristic Procedure for Partitioning Graphs.
Bell Systems Technology Journal, 49(2):213–219, 1970.
[ bib ]

[1214]

Pascal Kerschke and Heike Trautmann.
The Rpackage FLACCO for exploratory landscape
analysis with applications to multiobjective optimization problems.
In Proceedings of the 2016 Congress on Evolutionary Computation
(CEC 2016), pages 5262–5269, Piscataway, NJ, 2016. IEEE Press.
[ bib 
DOI ]

[1215]

Pascal Kerschke and Heike Trautmann.
Automated Algorithm Selection on Continuous BlackBox Problems
by Combining Exploratory Landscape Analysis and Machine Learning.
Evolutionary Computation, 27(1):99–127, 2019.
[ bib 
DOI ]
In this article, we build upon previous work on designing
informative and efficient Exploratory Landscape Analysis
features for characterizing problems' landscapes and show
their effectiveness in automatically constructing algorithm
selection models in continuous blackbox optimization
problems. Focusing on algorithm performance results of the
COCO platform of several years, we construct a representative
set of highperforming complementary solvers and present an
algorithm selection model that, compared to the portfolio's
single best solver, on average requires less than half of the
resources for solving a given problem. Therefore, there is a
huge gain in efficiency compared to classical ensemble
methods combined with an increased insight into problem
characteristics and algorithm properties by using informative
features. The model acts on the assumption that the function
set of the BlackBox Optimization Benchmark is representative
enough for practical applications. The model allows for
selecting the best suited optimization algorithm within the
considered set for unseen problems prior to the optimization
itself based on a small sample of function evaluations. Note
that such a sample can even be reused for the initial
population of an evolutionary (optimization) algorithm so
that even the feature costs become negligible.

[1216]

Pascal Kerschke, Hao Wang, Mike Preuss, Christian Grimme, André H. Deutz,
Heike Trautmann, and Michael T. M. Emmerich.
Towards Analyzing Multimodality of Continuous Multiobjective
Landscapes.
In J. Handl, E. Hart, P. R. Lewis, M. LópezIbáñez,
G. Ochoa, and B. Paechter, editors, Parallel Problem Solving from Nature
– PPSN XIV, volume 9921 of Lecture Notes in Computer Science, pages
962–972. Springer, Heidelberg, 2016.
[ bib 
DOI ]

[1217]

Pascal Kerschke, Hao Wang, Mike Preuss, Christian Grimme, André H. Deutz,
Heike Trautmann, and Michael T. M. Emmerich.
Search Dynamics on Multimodal Multiobjective Problems.
Evolutionary Computation, 27(4):577–609, 2019.
[ bib 
DOI ]

[1218]

Keras development team.
Keras.
https://https://keras.io, 2017.
[ bib ]

[1219]

Norbert L. Kerr.
HARKing: Hypothesizing After the Results are Known.
Personality and Social Psychology Review, 2(3):196–217, August
1998.
[ bib 
DOI ]

[1220]

M. Kerrisk.
pthreads  POSIX Threads.
In Linux Programmer's Manual, Section 7.
http://www.linuxmanpages.org/man7/pthreads/, 2005.
(Last accessed May 15 2008).
[ bib ]

[1221]

V. Khare, X. Yao, and Kalyanmoy Deb.
Performance Scaling of Multiobjective Evolutionary Algorithms.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 376–390. Springer,
Heidelberg, 2003.
[ bib ]

[1222]

M. Khichane, P. Albert, and Christine Solnon.
Integration of ACO in a Constraint Programming Language.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 6th International Conference, ANTS 2008, volume 5217 of
Lecture Notes in Computer Science, pages 84–95. Springer, Heidelberg, 2008.
[ bib ]

[1223]

M. Khichane, P. Albert, and Christine Solnon.
An ACOBased Reactive Framework for Ant Colony Optimization:
First Experiments on Constraint Satisfaction Problems.
In T. Stützle, editor, Learning and Intelligent
Optimization, Third International Conference, LION 3, volume 5851 of
Lecture Notes in Computer Science, pages 119–133. Springer, Heidelberg,
2009.
[ bib ]

[1224]

A. R. KhudaBukhsh, Lin Xu, Holger H. Hoos, and Kevin LeytonBrown.
SATenstein: Automatically Building Local Search SAT
Solvers from Components.
Artificial Intelligence, 232:20–42, 2016.
[ bib 
DOI ]

[1225]

A. R. KhudaBukhsh, Lin Xu, Holger H. Hoos, and Kevin LeytonBrown.
SATenstein: Automatically Building Local Search SAT Solvers
from Components.
In C. Boutilier, editor, Proceedings of the TwentyFirst
International Joint Conference on Artificial Intelligence (IJCAI09), pages
517–524. AAAI Press, Menlo Park, CA, 2009.
[ bib 
eprint ]

[1226]

Philip Kilby and Tommaso Urli.
Fleet design optimisation from historical data using constraint
programming and large neighbourhood search.
Constraints, pages 1–20, 2015.
[ bib 
DOI ]
Keywords: Frace

[1227]

YeongDae Kim.
Heuristics for Flowshop Scheduling Problems Minimizing Mean
Tardiness.
Journal of the Operational Research Society, 44(1):19–28,
1993.
[ bib 
DOI ]

[1228]

Youngmin Kim, Richard Allmendinger, and Manuel LópezIbáñez.
Safe Learning and Optimization Techniques: Towards a Survey of
the State of the Art.
Arxiv preprint arXiv:2101.09505 [cs.LG], 2020.
[ bib 
http ]
Safe learning and optimization deals with learning and
optimization problems that avoid, as much as possible, the
evaluation of nonsafe input points, which are solutions,
policies, or strategies that cause an irrecoverable loss
(e.g., breakage of a machine or equipment, or life
threat). Although a comprehensive survey of safe
reinforcement learning algorithms was published in 2015, a
number of new algorithms have been proposed thereafter, and
related works in active learning and in optimization were not
considered. This paper reviews those algorithms from a number
of domains including reinforcement learning, Gaussian process
regression and classification, evolutionary algorithms, and
active learning. We provide the fundamental concepts on which
the reviewed algorithms are based and a characterization of
the individual algorithms. We conclude by explaining how the
algorithms are connected and suggestions for future
research.

[1229]

Youngmin Kim, Richard Allmendinger, and Manuel LópezIbáñez.
Safe Learning and Optimization Techniques: Towards a Survey of
the State of the Art.
In F. Heintz, M. Milano, and B. O'Sullivan, editors, Trustworthy
AI – Integrating Learning, Optimization and Reasoning. TAILOR 2020, volume
12641 of Lecture Notes in Computer Science, pages 123–139. Springer,
Cham, Switzerland, 2020.
[ bib 
DOI ]
Safe learning and optimization deals with learning and
optimization problems that avoid, as much as possible, the
evaluation of nonsafe input points, which are solutions,
policies, or strategies that cause an irrecoverable loss
(e.g., breakage of a machine or equipment, or life
threat). Although a comprehensive survey of safe
reinforcement learning algorithms was published in 2015, a
number of new algorithms have been proposed thereafter, and
related works in active learning and in optimization were not
considered. This paper reviews those algorithms from a number
of domains including reinforcement learning, Gaussian process
regression and classification, evolutionary computing, and
active learning. We provide the fundamental concepts on which
the reviewed algorithms are based and a characterization of
the individual algorithms. We conclude by explaining how the
algorithms are connected and suggestions for future
research.

[1230]

Jungtaek Kim, Michael McCourt, Tackgeun You, Saehoon Kim, and Seungjin Choi.
Bayesian Optimization with Approximate Set Kernels.
Machine Learning, 2021.
[ bib 
DOI ]
We propose a practical Bayesian optimization method over
sets, to minimize a blackbox function that takes a set as a
single input. Because set inputs are permutationinvariant,
traditional Gaussian processbased Bayesian optimization
strategies which assume vector inputs can fall short. To
address this, we develop a Bayesian optimization method with
set kernel that is used to build surrogate
functions. This kernel accumulates similarity over set
elements to enforce permutationinvariance, but this comes at
a greater computational cost. To reduce this burden, we
propose two key components: (i) a more efficient approximate
set kernel which is still positivedefinite and is an
unbiased estimator of the true set kernel with upperbounded
variance in terms of the number of subsamples, (ii) a
constrained acquisition function optimization over sets,
which uses symmetry of the feasible region that defines a set
input. Finally, we present several numerical experiments
which demonstrate that our method outperforms other methods.

[1231]

J.S. Kim, J.H. Park, and D.H. Lee.
Iterated Greedy Algorithms to Minimize the Total Family Flow
Time for Jobshop Scheduling with Job Families and Sequencedependent
Setups.
Engineering Optimization, 49(10):1719–1732, 2017.
[ bib ]

[1232]

Diederik P Kingma and Jimmy Ba.
Adam: A method for stochastic optimization.
Arxiv preprint arXiv:1412.6980 [cs.LG], 2014.
[ bib 
http ]
Published as a conference paper at the 3rd International
Conference for Learning Representations, San Diego, 2015

[1233]

Scott Kirkpatrick and G. Toulouse.
Configuration Space Analysis of Travelling Salesman Problems.
Journal de Physique, 46(8):1277–1292, 1985.
[ bib ]

[1234]

Scott Kirkpatrick.
Optimization by Simulated Annealing: Quantitative Studies.
Journal of Statistical Physics, 34(56):975–986, 1984.
[ bib ]

[1235]

Scott Kirkpatrick, C. D. Gelatt, and M. P. Vecchi.
Optimization by Simulated Annealing.
Science, 220(4598):671–680, 1983.
[ bib 
DOI ]

[1236]

Anton J. Kleywegt, Alexander Shapiro, and Tito HomemdeMello.
The Sample Average Approximation Method for Stochastic Discrete
Optimization.
SIAM Journal on Optimization, 12(2):479–502, 2002.
[ bib ]

[1237]

Joshua D. Knowles.
A summaryattainmentsurface plotting method for visualizing the
performance of stochastic multiobjective optimizers.
In A. Abraham and M. Paprzycki, editors, Proceedings of the 5th
International Conference on Intelligent Systems Design and Applications,
pages 552–557, 2005.
[ bib 
DOI 
eprint 
supplementary material ]

[1238]

Joshua D. Knowles.
ParEGO: A hybrid algorithm with online landscape
approximation for expensive multiobjective optimization problems.
IEEE Transactions on Evolutionary Computation, 10(1):50–66,
2006.
[ bib ]
Keywords: ParEGO, online, metamodel

[1239]

Joshua D. Knowles.
Closedloop evolutionary multiobjective optimization.
IEEE Computational Intelligence Magazine, 4:77–91, 2009.
[ bib 
DOI ]

[1240]

Joshua D. Knowles and David Corne.
Approximating the Nondominated Front Using the Pareto Archived
Evolution Strategy.
Evolutionary Computation, 8(2):149–172, 2000.
[ bib 
DOI ]

[1241]

Joshua D. Knowles and David Corne.
The Pareto Archived Evolution Strategy: A New Baseline
Algorithm for Multiobjective Optimisation.
In Proceedings of the 1999 Congress on Evolutionary Computation
(CEC 1999), pages 98–105. IEEE Press, Piscataway, NJ, 1999.
[ bib ]
first mention of Adaptive Grid Archiving

[1242]

Joshua D. Knowles and David Corne.
On Metrics for Comparing NonDominated Sets.
In Proceedings of the 2002 Congress on Evolutionary Computation
(CEC'02), pages 711–716. IEEE Press, Piscataway, NJ, 2002.
[ bib ]

[1243]

Joshua D. Knowles and David Corne.
Instance Generators and Test Suites for the Multiobjective
Quadratic Assignment Problem.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 295–310, Heidelberg,
2003. Springer.
[ bib ]

[1244]

Joshua D. Knowles and David Corne.
Properties of an Adaptive Archiving Algorithm for Storing
Nondominated Vectors.
IEEE Transactions on Evolutionary Computation, 7(2):100–116,
April 2003.
[ bib ]
Proposed to use Smetric (hypervolume metric) for
environmental selection
Keywords: Smetric, hypervolume

[1245]

Joshua D. Knowles and David Corne.
Bounded Pareto Archiving: Theory and Practice.
In X. Gandibleux, M. Sevaux, K. Sörensen, and V. T'Kindt,
editors, Metaheuristics for Multiobjective Optimisation, volume 535 of
Lecture Notes in Economics and Mathematical Systems, pages 39–64.
Springer, Berlin, Germany, 2004.
[ bib 
DOI ]

[1246]

Joshua D. Knowles and David Corne.
Memetic algorithms for multiobjective optimization: issues,
methods and prospects.
In H. W. E., S. J. E., and K. N., editors, Recent Advances in
Memetic Algorithms, volume 166 of Studies in Fuzziness and Soft
Computing, pages 313–352. Springer, Berlin/Heidelberg, 2005.
[ bib 
DOI ]

[1247]

Joshua D. Knowles, David Corne, and Kalyanmoy Deb.
Introduction: Problem solving, EC and EMO.
In J. D. Knowles, D. Corne, K. Deb, and D. R. Chair, editors,
Multiobjective Problem Solving from Nature, Natural Computing Series, pages
1–28. Springer, Berlin/Heidelberg, 2008.
[ bib 
DOI ]

[1248]

Joshua D. Knowles, David Corne, and Mark Fleischer.
Bounded archiving using the Lebesgue measure.
In Proceedings of the 2003 Congress on Evolutionary Computation
(CEC'03), pages 2490–2497. IEEE Press, Piscataway, NJ, December 2003.
[ bib ]

[1249]

Joshua D. Knowles, David Corne, and Alan P. Reynolds.
Noisy Multiobjective Optimization on a Budget of 250
Evaluations.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pages 36–50.
Springer, Heidelberg, 2009.
[ bib ]

[1250]

Joshua D. Knowles, Lothar Thiele, and Eckart Zitzler.
A tutorial on the performance assessment of stochastic
multiobjective optimizers.
TIKReport 214, Computer Engineering and Networks Laboratory (TIK),
Swiss Federal Institute of Technology (ETH), Zürich, Switzerland,
February 2006.
Revised version.
[ bib 
eprint ]

[1251]

Mirjam J. Knol, Tyler J. VanderWeele, Rolf H. H. Groenwold, Olaf H. Klungel,
Maroeska M. Rovers, and Diederick E. Grobbee.
Estimating measures of interaction on an additive scale for
preventive exposures.
European Journal of Epidemiology, 26(6):433–438, 2011.
[ bib ]

[1252]

Joshua D. Knowles, Richard A. Watson, and David Corne.
Reducing Local Optima in SingleObjective Problems by
Multiobjectivization.
In E. Zitzler, K. Deb, L. Thiele, C. A. Coello Coello, and
D. Corne, editors, Evolutionary Multicriterion Optimization, EMO 2001,
volume 1993 of Lecture Notes in Computer Science, pages 269–283.
Springer, Heidelberg, 2001.
[ bib 
DOI ]
Proposed multiobjectivization

[1253]

Joshua D. Knowles.
LocalSearch and Hybrid Evolutionary Algorithms for Pareto
Optimization.
PhD thesis, University of Reading, UK, 2002.
[ bib ]
(Examiners: Prof. K. Deb and Prof. K. Warwick)

[1254]

Gary A. Kochenberger, Fred Glover, Bahram Alidaee, and Cesar Rego.
A unified modeling and solution framework for combinatorial
optimization problems.
OR Spektrum, 26(2):237–250, 2004.
[ bib ]

[1255]

Gary A. Kochenberger, JinKao Hao, Fred Glover, Mark Lewis, Zhipeng Lü,
Haibo Wang, and Yang Wang.
The unconstrained binary quadratic programming problem: a
survey.
Journal of Combinatorial Optimization, 28(1):58–81, 2014.
[ bib 
DOI ]

[1256]

Murat Köksalan.
Multiobjective Combinatorial Optimization: Some Approaches.
Journal of MultiCriteria Decision Analysis, 15:69–78, 2009.
[ bib 
DOI ]

[1257]

Murat Köksalan and İbrahim Karahan.
An Interactive Territory Defining Evolutionary Algorithm:
iTDEA.
IEEE Transactions on Evolutionary Computation, 14(5):702–722,
October 2010.
[ bib 
DOI ]

[1258]

Rainer Kolisch and Sönke Hartmann.
Experimental investigation of heuristics for
resourceconstrained project scheduling: An update.
European Journal of Operational Research, 174(1):23–37,
October 2006.
[ bib 
DOI ]
This paper considers heuristics for the wellknown
resourceconstrained project scheduling problem
(RCPSP). It provides an update of our survey which
was published in 2000. We summarize and categorize a
large number of heuristics that have recently been
proposed in the literature. Most of these heuristics
are then evaluated in a computational study and
compared on the basis of our standardized
experimental design. Based on the computational
results we discuss features of good heuristics. The
paper closes with some remarks on our test design
and a summary of the recent developments in research
on heuristics for the RCPSP.
Keywords: Computational evaluation, Heuristics, Project
scheduling, Resource constraints

[1259]

A. Kolen and Erwin Pesch.
Genetic Local Search in Combinatorial Optimization.
Discrete Applied Mathematics, 48(3):273–284, 1994.
[ bib ]

[1260]

Joshua B. Kollat and Patrick M. Reed.
A framework for visually interactive decisionmaking and design
using evolutionary multiobjective optimization (VIDEO).
Environmental Modelling & Software, 22(12):1691–1704, 2007.
[ bib ]
Keywords: glyph plot

[1261]

T. C. Koopmans and M. J. Beckmann.
Assignment Problems and the Location of Economic Activities.
Econometrica, 25:53–76, 1957.
[ bib ]

[1262]

Mario Koppen and Kaori Yoshida.
Visualization of Paretosets in evolutionary multiobjective
optimization.
In 7th International Conference on Hybrid Intelligent Systems
(HIS 2007), pages 156–161. IEEE, 2007.
[ bib ]

[1263]

Jsh Kornbluth.
Sequential multicriterion decision making.
Omega, 13(6):569–574, 1985.
[ bib 
DOI ]
In this paper we consider a simple sequential
multicriterion decision making problem in which a
decision maker has to accept or reject a series of
multiattributed outcomes. We show that using very
simple programming techniques, a great deal of the
decision making can be automated. The method might
be applicable to situations in which a dealer is
having to consider sequential offers in a trading
market.
Keywords: machine decision making

[1264]

Flip Korn, B.U. Pagel, and Christos Faloutsos.
On the “dimensionality curse” and the “selfsimilarity
blessing”.
IEEE Transactions on Knowledge and Data Engineering,
13(1):96–111, 2001.
[ bib 
DOI ]
Spatial queries in highdimensional spaces have been studied
extensively. Among them, nearest neighbor queries are
important in many settings, including spatial databases (Find
the k closest cities) and multimedia databases (Find the k
most similar images). Previous analyses have concluded that
nearestneighbor search is hopeless in high dimensions due to
the notorious "curse of dimensionality". We show that this
may be overpessimistic. We show that what determines the
search performance (at least for Rtreelike structures) is
the intrinsic dimensionality of the data set and not the
dimensionality of the address space (referred to as the
embedding dimensionality). The typical (and often implicit)
assumption in many previous studies is that the data is
uniformly distributed, with independence between
attributes. However, real data sets overwhelmingly disobey
these assumptions; rather, they typically are skewed and
exhibit intrinsic ("fractal") dimensionalities that are much
lower than their embedding dimension, e.g. due to subtle
dependencies between attributes. We show how the Hausdorff
and Correlation fractal dimensions of a data set can yield
extremely accurate formulas that can predict the I/O
performance to within one standard deviation on multiple real
and synthetic data sets.

[1265]

P. Korošec, Jurij Šilc, K. Oblak, and F. Kosel.
The differential antstigmergy algorithm: an experimental
evaluation and a realworld application.
In Proceedings of the 2007 Congress on Evolutionary Computation
(CEC 2007), pages 157–164. IEEE Press, Piscataway, NJ, 2007.
[ bib ]

[1266]

P. Korošec, Jurij Šilc, and B. Robič.
MeshPartitioning with the Multiple AntColony Algorithm.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 430–431. Springer, Heidelberg,
2004.
[ bib ]

[1267]

P. Korošec, Jurij Šilc, and B. Robič.
Solving the meshpartitioning problem with an antcolony
algorithm.
Parallel Computing, 30:785–801, 2004.
[ bib ]

[1268]

Oliver Korb, Thomas Stützle, and Thomas E. Exner.
PLANTS: Application of ant colony optimization to
structurebased drug design.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 5th International Workshop, ANTS 2006, volume 4150 of
Lecture Notes in Computer Science, pages 247–258. Springer, Heidelberg,
2006.
[ bib 
DOI ]

[1269]

Oliver Korb, Thomas Stützle, and Thomas E. Exner.
An Ant Colony Optimization Approach to Flexible Protein–Ligand
Docking.
Swarm Intelligence, 1(2):115–134, 2007.
[ bib ]

[1270]

Oliver Korb, Thomas Stützle, and Thomas E. Exner.
Empirical Scoring Functions for Advanced ProteinLigand Docking
with PLANTS.
Journal of Chemical Information and Modeling, 49(2):84–96,
2009.
[ bib ]

[1271]

Oliver Korb, Peter Monecke, Gerhard Hessler, Thomas Stützle, and Thomas E.
Exner.
pharmACOphore: Multiple Flexible Ligand Alignment Based on Ant
Colony Optimization.
Journal of Chemical Information and Modeling, 50(9):1669–1681,
2010.
[ bib ]

[1272]

Lars Kotthoff.
Algorithm Selection for Combinatorial Search Problems: A
Survey.
AI Magazine, 35(3):48–60, 2014.
[ bib ]

[1273]

Timo Kötzing, Frank Neumann, Heiko Röglin, and Carsten Witt.
Theoretical Analysis of Two ACO Approaches for the Traveling
Salesman Problem.
Swarm Intelligence, 6(1):1–21, 2012.
[ bib 
DOI ]
Bioinspired algorithms, such as evolutionary algorithms and
ant colony optimization, are widely used for different
combinatorial optimization problems. These algorithms rely
heavily on the use of randomness and are hard to understand
from a theoretical point of view. This paper contributes to
the theoretical analysis of ant colony optimization and
studies this type of algorithm on one of the most prominent
combinatorial optimization problems, namely the traveling
salesperson problem (TSP). We present a new construction
graph and show that it has a stronger local property than one
commonly used for constructing solutions of the TSP. The
rigorous runtime analysis for two ant colony optimization
algorithms, based on these two construction procedures, shows
that they lead to good approximation in expected polynomial
time on random instances. Furthermore, we point out in which
situations our algorithms get trapped in local optima and
show where the use of the right amount of heuristic
information is provably beneficial.

[1274]

Lars Kotthoff, Chris Thornton, Holger H. Hoos, Frank Hutter, and Kevin
LeytonBrown.
AutoWEKA 2.0: Automatic model selection and hyperparameter
optimization in WEKA.
Journal of Machine Learning Research, 17:1–5, 2016.
[ bib ]

[1275]

P. Kouvelis and G. Yu.
Robust discrete optimization and its applications.
Nonconvex optimization and its applications. Kluwer Academic
Publishers, Dordrecht, The Netherlands, 1997.
[ bib ]

[1276]

O. Kovářík and M. Skrbek.
Ant Colony Optimization with Castes.
In V. KurkovaPohlova and J. Koutnik, editors, ICANN'08:
Proceedings of the 18th International Conference on Artificial Neural
Networks, Part I, volume 5163 of Lecture Notes in Computer Science,
pages 435–442. Springer, Heidelberg, 2008.
[ bib ]

[1277]

Katharina Kowalski, Sigrid Stagl, Reinhard Madlener, and Ines Omann.
Sustainable energy futures: Methodological challenges in
combining scenarios and participatory multicriteria analysis.
European Journal of Operational Research, 197(3):1063–1074,
2009.
[ bib ]

[1278]

Slawomir Koziel, David Echeverría Ciaurri, and Leifur Leifsson.
SurrogateBased Methods.
In S. Koziel and X.S. Yang, editors, Computational
Optimization, Methods and Algorithms, volume 356 of Studies in
Computational Intelligence, pages 33–59. Springer, Berlin/Heidelberg,
2011.
[ bib ]

[1279]

J. Koza.
Genetic Programming: On the Programming of Computers By the
Means of Natural Selection.
MIT Press, Cambridge, MA, 1992.
[ bib ]

[1280]

Oliver Kramer.
Iterated Local Search with Powell's Method: A Memetic
Algorithm for Continuous Global Optimization.
Memetic Computing, 2(1):69–83, 2010.
[ bib 
DOI ]

[1281]

Daniel Krajzewicz, Jakob Erdmann, Michael Behrisch, and Laura Bieker.
Recent development and applications of SUMO  Simulation of
Urban MObility.
International Journal On Advances in Systems and Measurements,
5(34):128–138, 2012.
[ bib ]

[1282]

Oliver Kramer, Bartek Gloger, and Andreas Goebels.
An Experimental Analysis of Evolution Strategies and Particle
Swarm Optimisers Using Design of Experiments.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pages 674–681, New York,
NY, 2007. ACM Press.
[ bib ]

[1283]

Daniel Krajzewicz, Marek Heinrich, Michela Milano, Paolo Bellavista, Thomas
Stützle, Jérôme Härri, Thrasyvoulos Spyropoulos, Robbin
Blokpoel, Stefan Hausberger, and Martin Fellendorf.
COLOMBO: Investigating the Potential of V2X for Traffic
Management Purposes assuming low penetration Rates.
In Proceedings of ITS Europe 2013, Dublin, Ireland, 2013.
[ bib ]

[1284]

Daniel Krajzewicz, Andreas Leich, Robbin Blokpoel, Michela Milano, and Thomas
Stützle.
COLOMBO: Exploiting Vehicular Communications at Low Equipment
Rates for Traffic Management Purposes.
In T. Schulze, B. Müller, and G. Meyer, editors, Advanced
Microsystems for Automotive Applications 2015: Smart Systems for Green and
Automated Driving, pages 117–130. Springer International Publishing, Cham,
Switzerland, 2016.
[ bib ]

[1285]

Jakob Krarup and Peter Mark Pruzan.
Computeraided Layout Design.
In M. L. Balinski and C. Lemarechal, editors, Mathematical
Programming in Use, volume 9 of Mathematical Programming Studies,
pages 75–94. Springer, Berlin/Heidelberg, 1978.
[ bib ]

[1286]

Johannes Krettek, Jan Braun, Frank Hoffmann, and Torsten Bertram.
Interactive Incorporation of User Preferences in Multiobjective
Evolutionary Algorithms.
In J. Mehnen, M. Köppen, A. Saad, and A. Tiwari, editors,
Applications of Soft Computing, volume 58 of Advances in Intelligent
and Soft Computing, pages 379–388. Springer, Berlin/Heidelberg, 2009.
[ bib ]

[1287]

Johannes Krettek, Jan Braun, Frank Hoffmann, and Torsten Bertram.
Preference Modeling and Model Management for Interactive
Multiobjective Evolutionary Optimization.
In E. Hüllermeier, R. Kruse, and F. Hoffmann, editors,
Information Processing and Management of Uncertainty, 13th International
Conference, IPMU2010, volume 6178 of Lecture Notes in Artificial
Intelligence, pages 574–583. Springer, Heidelberg, 2010.
[ bib ]

[1288]

S. Kreipl.
A Large Step Random Walk for Minimizing Total Weighted Tardiness
in a Job Shop.
Journal of Scheduling, 3(3):125–138, 2000.
[ bib ]

[1289]

Stefanie Kritzinger, Fabien Tricoire, Karl F. Doerner, Richard F. Hartl, and
Thomas Stützle.
A Unified Framework for Routing Problems with a Fixed Fleet
Size.
International Journal of Metaheuristics, 6(3):160–209, 2017.
[ bib ]

[1290]

William H. Kruskal and Judith M. Tanur.
Linear Hypotheses, volume 1.
Free Press, 1978.
[ bib ]

[1291]

Joseph B Kruskal.
On the shortest spanning subtree of a graph and the traveling
salesman problem.
Proceedings of the American Mathematical Society, 7(1):48–50,
1956.
[ bib ]

[1292]

J Kuhpfahl and Christian Bierwirth.
A Study on Local Search Neighborhoods for the Job Shop
Scheduling Problem with Total Weighted Tardiness Objective.
Computers & Operations Research, 66:44–57, 2016.
[ bib ]

[1293]

Harold W. Kuhn.
The hungarian method for the assignment problem.
Naval Research Logistics Quarterly, 2(1–2):83–97, 1955.
[ bib ]

[1294]

Max Kuhn.
Building Predictive Models in R Using the
caret Package.
Journal of Statistical Software, 28(5):1–26, 2008.
[ bib ]

[1295]

S. Kukkonen and J. Lampinen.
GDE3: the third evolution step of generalized differential
evolution.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pages 443–450. IEEE Press, Piscataway, NJ, September 2005.
[ bib ]

[1296]

R. Kumar and P. K. Singh.
Pareto Evolutionary Algorithm Hybridized with Local Search for
Biobjective TSP.
Studies in Computational Intelligence, 75:361–398, 2007.
[ bib ]

[1297]

Ravi Kumar and Sergei Vassilvitskii.
Generalized Distances between Rankings.
In M. Rappa, P. Jones, J. Freire, and S. Chakrabarti, editors,
Proceedings of the 19th International Conference on World Wide Web, WWW
2010. ACM Press, New York, NY, 2010.
[ bib ]

[1298]

H. T. Kung, F. Luccio, and F. P. Preparata.
On Finding the Maxima of a Set of Vectors.
Journal of the ACM, 22(4):469–476, 1975.
[ bib ]

[1299]

Frank Kursawe.
A variant of evolution strategies for vector optimization.
In H.P. Schwefel and R. Männer, editors, Parallel Problem
Solving from Nature – PPSN I, pages 193–197, Berlin/Heidelberg,
1991. Springer.
[ bib 
DOI ]

[1300]

I. Kurtulus and E. W. Davis.
MultiProject Scheduling: Categorization of Heuristic Rules
Performance.
Management Science, 28(2):161–172, 1982.
[ bib 
DOI ]
Application of heuristic solution procedures to the
practical problem of project scheduling has
previously been studied by numerous
researchers. However, there is little consensus
about their findings, and the practicing manager is
currently at a loss as to which scheduling rule to
use. Furthermore, since no categorization process
was developed, it is assumed that once a rule is
selected it must be used throughout the whole
project. This research breaks away from this
tradition by providing a categorization process
based on two powerful project summary measures. The
first measure identifies the location of the peak of
total resource requirements and the second measure
identifies the rate of utilization of each resource
type. The performance of the rules are classified
according to values of these two measures, and it is
shown that a rule introduced by this research
performs significantly better on most categories of
projects.
Keywords: project management, research and development

[1301]

H. J. Kushner.
A New Method of Locating the Maximum Point of an Arbitrary
Multipeak Curve in the Presence of Noise.
Journal of Basic Engineering, 86(1):97–106, March 1964.
[ bib 
DOI ]
A versatile and practical method of searching a parameter
space is presented. Theoretical and experimental results
illustrate the usefulness of the method for such problems as
the experimental optimization of the performance of a system
with a very general multipeak performance function when the
only available information is noisedistributed samples of
the function. At present, its usefulness is restricted to
optimization with respect to one system parameter. The
observations are taken sequentially; but, as opposed to the
gradient method, the observation may be located anywhere on
the parameter interval. A sequence of estimates of the
location of the curve maximum is generated. The location of
the next observation may be interpreted as the location of
the most likely competitor (with the current best estimate)
for the location of the curve maximum. A Brownian motion
stochastic process is selected as a model for the unknown
function, and the observations are interpreted with respect
to the model. The model gives the results a simple intuitive
interpretation and allows the use of simple but efficient
sampling procedures. The resulting process possesses some
powerful convergence properties in the presence of noise; it
is nonparametric and, despite its generality, is efficient in
the use of observations. The approach seems quite promising
as a solution to many of the problems of experimental system
optimization.

[1302]

Jan H. Kwakkel.
The Exploratory Modeling Workbench: An open source toolkit for
exploratory modeling, scenario discovery, and (multiobjective) robust
decision making.
Environmental Modelling & Software, 96:239–250, 2017.
[ bib ]

[1303]

Marco Laumanns, Lothar Thiele, Kalyanmoy Deb, and Eckart Zitzler.
Combining Convergence and Diversity in Evolutionary
Multiobjective Optimization.
Evolutionary Computation, 10(3):263–282, 2002.
[ bib ]

[1304]

Antonio LaTorre, Santiago Muelas, and JoséMaría Peña.
A MOSbased dynamic memetic differential evolution algorithm
for continuous optimization: a scalability test.
Soft Computing, 15(11):2187–2199, 2011.
[ bib ]

[1305]

Peter J. M. van Laarhoven, Emile H. L. Aarts, and Jan Karel Lenstra.
Job Shop Scheduling by Simulated Annealing.
Operations Research, 40(1):113–125, 1992.
[ bib ]

[1306]

Martine Labbé, Patrice Marcotte, and Gilles Savard.
A Bilevel Model of Taxation and Its Application to Optimal
Highway Pricing.
Management Science, 44(12):1608–1622, 1998.
[ bib 
DOI ]

[1307]

Martine Labbé and Alessia Violin.
Bilevel programming and price setting problems.
4OR: A Quarterly Journal of Operations Research, 11(1):1–30,
2013.
[ bib 
DOI ]

[1308]

Benjamin Lacroix, Daniel Molina, and Francisco Herrera.
Dynamically updated region based memetic algorithm for the 2013
CEC Special Session and Competition on Real Parameter Single Objective
Optimization.
In Proceedings of the 2013 Congress on Evolutionary Computation
(CEC 2013), pages 1945–1951. IEEE Press, Piscataway, NJ, 2013.
[ bib ]

[1309]

Benjamin Lacroix, Daniel Molina, and Francisco Herrera.
Region based memetic algorithm for realparameter optimisation.
Information Sciences, 262:15–31, 2014.
[ bib 
DOI ]
Keywords: irace

[1310]

Manuel Laguna.
Editor's Note on the MIC 2013 Special Issue of the Journal of
Heuristics (Volume 22, Issue 4, August 2016).
Journal of Heuristics, 22(5):665–666, 2016.
[ bib ]

[1311]

Xiangjing Lai and JinKao Hao.
Iterated Maxima Search for the Maximally Diverse Grouping
Problem.
European Journal of Operational Research, 254(3):780–800,
2016.
[ bib ]

[1312]

A. H. Land and A. G. Doig.
An Automatic Method of Solving Discrete Programming Problems.
Econometrica, 28(3):497–520, 1960.
[ bib ]

[1313]

W. B. Langdon and M. Harman.
Optimising Software with Genetic Programming.
IEEE Transactions on Evolutionary Computation, 19(1):118–135,
2015.
[ bib ]

[1314]

M. Lang, H. Kotthaus, P. Marwedel, C. Weihs, J. Rahnenführer, and Bernd
Bischl.
Automatic Model Selection for HighDimensional Survival
Analysis.
Journal of Statistical Computation and Simulation,
85(1):62–76, 2014.
[ bib 
DOI ]

[1315]

A. Langevin, F. Soumis, and J. Desrosiers.
Classification of travelling salesman problem formulations.
Operations Research Letters, 9(2):127–132, 1990.
[ bib ]

[1316]

A. Langevin, M. Desrochers, J. Desrosiers, Sylvie Gélinas, and F. Soumis.
A TwoCommodity Flow Formulation for the Traveling Salesman and
Makespan Problems with Time Windows.
Networks, 23(7):631–640, 1993.
[ bib ]

[1317]

Kevin E. Lansey and K. Awumah.
Optimal Pump Operations Considering Pump Switches.
Journal of Water Resources Planning and Management, ASCE,
120(1):17–35, January / February 1994.
[ bib ]

[1318]

Gilbert Laporte.
Fifty Years of Vehicle Routing.
Transportation Science, 43(4):408–416, 2009.
[ bib ]

[1319]

R. M. Lark and D. J. Lapworth.
A new statistic to express the uncertainty of kriging
predictions for purposes of survey planning.
In EGU General Assembly Conference Abstracts, May 2014.
[ bib 
http ]

[1320]

Pedro Larrañaga and José A. Lozano.
Estimation of Distribution Algorithms: A New Tool for
Evolutionary Computation.
Kluwer Academic Publishers, Boston, MA, 2002.
[ bib ]

[1321]

Craig Larman.
Applying UML and Patterns: An Introduction to ObjectOriented
Analysis and Design and Iterative Development.
Prentice Hall, Englewood Cliffs, NJ, 3rd edition, 2004.
[ bib ]

[1322]

Marco Laumanns.
Stochastic convergence of random search to fixed size Pareto
set approximations.
Arxiv preprint arXiv:0711.2949, 2007.
[ bib 
http ]

[1323]

Benoît Laurent and JinKao Hao.
Iterated Local Search for the Multiple Depot Vehicle Scheduling
Problem.
Computers and Industrial Engineering, 57(1):277–286, 2009.
[ bib ]

[1324]

Marco Laumanns, Lothar Thiele, and Eckart Zitzler.
Running time analysis of multiobjective evolutionary algorithms
on pseudoboolean functions.
IEEE Transactions on Evolutionary Computation, 8(2):170–182,
2004.
[ bib ]

[1325]

Marco Laumanns, Lothar Thiele, and Eckart Zitzler.
Running time analysis of evolutionary algorithms on a simplified
multiobjective knapsack problem.
Natural Computing, 3(1):37–51, 2004.
[ bib ]

[1326]

Marco Laumanns and Rico Zenklusen.
Stochastic convergence of random search methods to fixed size
Pareto front approximations.
(submitted), November 2010.
[ bib ]
Published as [1327]. Keep this reference for historical reasons.

[1327]

Marco Laumanns and R. Zenklusen.
Stochastic convergence of random search methods to fixed size
Pareto front approximations.
European Journal of Operational Research, 213(2):414–421,
2011.
[ bib 
DOI ]

[1328]

Marco Laumanns, Eckart Zitzler, and Lothar Thiele.
A unified model for multiobjective evolutionary algorithms with
elitism.
In Proceedings of the 2000 Congress on Evolutionary Computation
(CEC'00), pages 46–53, Piscataway, NJ, July 2000. IEEE Press.
[ bib ]

[1329]

E. L. Lawler, J. K. Lenstra, A. H. G. Rinnooy Kan, and D. B. Shmoys.
The Traveling Salesman Problem.
John Wiley & Sons, Chichester, UK, 1985.
[ bib ]

[1330]

E. L. Lawler and D. E. Wood.
BranchandBound Methods: A Survey.
Operations Research, 14(4):699–719, 1966.
[ bib 
DOI ]

[1331]

S. E. Lazic.
The problem of pseudoreplication in neuroscientific studies: is
it affecting your analysis?
BMC Neuroscience, 11(5):397–407, 2004.
[ bib 
DOI ]

[1332]

Yann LeCun, Yoshua Bengio, et al.
Convolutional networks for images, speech, and time series.
The handbook of brain theory and neural networks,
3361(10):1995, 1995.
[ bib ]

[1333]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.
Deep learning.
Nature, 521(7553):436–444, 2015.
[ bib ]

[1334]

Vinícius Leal do Forte, Flávio Marcelo Tavares Montenegro,
José André de Moura Brito, and Nelson Maculan.
Iterated Local Search Algorithms for the Euclidean Steiner
Tree Problem in n Dimensions.
International Transactions in Operational Research,
23(6):1185–1199, 2016.
[ bib ]

[1335]

Guillermo Leguizamón and Enrique Alba.
Ant Colony Based Algorithms for Dynamic Optimization Problems.
In E. Alba, A. Nakib, and P. Siarry, editors, Metaheuristics for
Dynamic Optimization, volume 433 of Studies in Computational
Intelligence, pages 189–210. Springer, Berlin/Heidelberg, 2013.
[ bib 
DOI ]

[1336]

Guillermo Leguizamón and Zbigniew Michalewicz.
A New Version of Ant System for Subset Problems.
In Proceedings of the 1999 Congress on Evolutionary Computation
(CEC 1999), pages 1459–1464. IEEE Press, Piscataway, NJ, 1999.
[ bib ]

[1337]

Per Kristian Lehre and Carsten Witt.
Blackbox search by unbiased variation.
Algorithmica, 64(4):623–642, 2012.
[ bib ]

[1338]

Frank Thomson Leighton.
A Graph Coloring Algorithm for Large Scheduling Problems.
Journal of Research of the National Bureau of Standards,
84(6):489–506, 1979.
[ bib ]

[1339]

Robert J. Lempert, David G. Groves, Steven W. Popper, and Steven C. Bankes.
A general analytic method for generating robust strategies and
narrative scenarios.
Management Science, 52(4):514–528, 2006.
[ bib ]

[1340]

R. J. Lempert, S. Popper, and Steven C. Bankes.
Shaping the Next One Hundred Years: New Methods for
Quantitative, Long Term Policy Analysis.
RAND, 2003.
[ bib ]

[1341]

C. Leon, S. Martin, J. M. Elena, and J. Luque.
EXPLORE: Hybrid expert system for water networks management.
Journal of Water Resources Planning and Management, ASCE,
126(2):65–74, 2000.
[ bib ]

[1342]

L. Lessing, Irina Dumitrescu, and Thomas Stützle.
A Comparison Between ACO Algorithms for the Set Covering
Problem.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 1–12. Springer, Heidelberg, 2004.
[ bib ]

[1343]

Leonid Levin.
Universal'nyie perebornyie zadachi.
Problemy Peredachi Informatsii, 9:265–266, 1973.
[ bib ]

[1344]

Rhyd M. R. Lewis.
A Guide to Graph Colouring: Algorithms and Applications.
Springer, Cham, Switzerland, 2016.
[ bib 
DOI ]
Supplementary material available at [1345]

[1345]

Rhyd M. R. Lewis.
Suite of Graph Colouring Algorithms – Supplementary Material to
the Book “A Guide to Graph Colouring: Algorithms and Applications”.
http://rhydlewis.eu/resources/gCol.zip, 2016.
[ bib ]

[1346]

Kevin LeytonBrown, Eugene Nudelman, and Yoav Shoham.
Learning the Empirical Hardness of Optimization Problems: The
Case of Combinatorial Auctions.
In P. van Hentenryck, editor, Principles and Practice of
Constraint Programming, CP 2002, Lecture Notes in Computer Science, pages
556–572. Springer, Heidelberg, 2002.
[ bib ]

[1347]

Kevin LeytonBrown, M. Pearson, and Y. Shoham.
Towards a Universal Test Suite for Combinatorial Auction
Algorithms.
In A. Jhingran et al., editors, ACM Conference on Electronic
Commerce (EC00), pages 66–76. ACM Press, New York, NY, 2000.
[ bib 
DOI ]
CPLEXregions200 benchmark set,
http://www.cs.ubc.ca/labs/beta/Projects/ParamILS/results.html

[1348]

Jianjun David Li.
A twostep rejection procedure for testing multiple hypotheses.
Journal of Statistical Planning and Inference,
138(6):1521–1527, 2008.
[ bib ]

[1349]

Zhiyi Li, Mohammad Shahidehpour, Shay Bahramirad, and Amin Khodaei.
Optimizing Traffic Signal Settings in Smart Cities.
IEEE Transactions on Smart Grid, 3053(4):1–1, 2016.
[ bib 
DOI ]
Traffic signals play a critical role in smart cities for
mitigating traffic congestions and reducing the emission in
metropolitan areas. This paper proposes a bilevel
optimization framework to settle the optimal traffic signal
setting problem. The upperlevel problem determines the
traffic signal settings to minimize the drivers' average
travel time, while the lowerlevel problem aims for achieving
the network equilibrium using the settings calculated at the
upper level. Genetic algorithm is employed with the
integration of microscopictrafficsimulation based dynamic
traffic assignment (DTA) to decouple the complex bilevel
problem into tractable singlelevel problems which are solved
sequentially. Case studies on a synthetic traffic network and
a realworld traffic subnetwork are conducted to examine the
effectiveness of the proposed model and relevant solution
methods. Additional strategies are provided for the extension
of the proposed model and the acceleration solution process
in largearea traffic network applications.

[1350]

Xiaoping Li, Long Chen, Haiyan Xu, and Jatinder N.D. Gupta.
Trajectory Scheduling Methods for Minimizing Total Tardiness in
a Flowshop.
Operations Research Perspectives, 2:13–23, 2015.
[ bib 
DOI ]

[1351]

Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet
Talwalkar.
Hyperband: A Novel BanditBased Approach to Hyperparameter
Optimization.
Journal of Machine Learning Research, 18(185):1–52, 2018.
[ bib 
http ]
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and earlystopping. We formulate hyperparameter optimization as a pureexploration nonstochastic infinitearmed bandit problem where a predefined resource like iterations, data samples, or features is allocated to randomly sampled configurations. We introduce a novel algorithm, our algorithm , for this framework and analyze its theoretical properties, providing several desirable guarantees. Furthermore, we compare our algorithm with popular Bayesian optimization methods on a suite of hyperparameter optimization problems. We observe that our algorithm can provide over an orderofmagnitude speedup over our competitor set on a variety of deeplearning and kernelbased learning problems.
Keywords: racing

[1352]

Y. Li and W. Li.
Adaptive Ant Colony Optimization Algorithm Based on Information
Entropy: Foundation and Application.
Fundamenta Informaticae, 77(3):229–242, 2007.
[ bib ]

[1353]

Bingdong Li, Jinlong Li, Ke Tang, and Xin Yao.
An Improved Two Archive Algorithm for ManyObjective
Optimization.
In Proceedings of the 2014 Congress on Evolutionary Computation
(CEC 2014), pages 2869–2876, Piscataway, NJ, 2014. IEEE Press.
[ bib ]

[1354]

Bingdong Li, Jinlong Li, Ke Tang, and Xin Yao.
ManyObjective Evolutionary Algorithms: A Survey.
ACM Computing Surveys, 48(1):1–35, 2015.
[ bib ]

[1355]

Z. Li, Y. Wang, J. Yu, Y. Zhang, and X. Li.
A Novel CloudBased Fuzzy SelfAdaptive Ant Colony System.
In ICNC'08: Proceedings of the 2008 Fourth International
Conference on Natural Computation, volume 7, pages 460–465, Washington, DC,
2008. IEEE Computer Society.
[ bib ]

[1356]

Miqing Li, Shengxiang Yang, Xiaohui Liu, and Ruimin Shen.
A Comparative Study on Evolutionary Algorithms for
ManyObjective Optimization.
In R. C. Purshouse, P. J. Fleming, C. M. Fonseca, S. Greco, and
J. Shaw, editors, Evolutionary Multicriterion Optimization, EMO 2013,
volume 7811 of Lecture Notes in Computer Science, pages 261–275.
Springer, Heidelberg, 2013.
[ bib ]

[1357]

Hui Li and Qingfu Zhang.
Multiobjective Optimization Problems with Complicated Pareto
sets, MOEA/D and NSGAII.
IEEE Transactions on Evolutionary Computation, 13(2):284–302,
2009.
[ bib ]

[1358]

Tianjun Liao, Dogan Aydin, and Thomas Stützle.
Artificial Bee Colonies for Continuous Optimization:
Experimental Analysis and Improvements.
Swarm Intelligence, 7(4):327–356, 2013.
[ bib ]

[1359]

Tianjun Liao, Daniel Molina, Marco A. Montes de Oca, and Thomas Stützle.
A Note on the Effects of Enforcing Bound Constraints on
Algorithm Comparisons using the IEEE CEC'05 Benchmark Function Suite.
Evolutionary Computation, 22(2):351–359, 2014.
[ bib ]

[1360]

Tianjun Liao, Daniel Molina, and Thomas Stützle.
Performance Evaluation of Automatically Tuned Continuous
Optimizers on Different Benchmark Sets.
Applied Soft Computing, 27:490–503, 2015.
[ bib ]

[1361]

Tianjun Liao, Marco A. Montes de Oca, Dogan Aydin, Thomas Stützle,
and Marco Dorigo.
An Incremental Ant Colony Algorithm with Local Search for
Continuous Optimization.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 125–132.
ACM Press, New York, NY, 2011.
[ bib ]

[1362]

Tianjun Liao, Marco A. Montes de Oca, and Thomas Stützle.
Computational results for an automatically tuned CMAES with
increasing population size on the CEC'05 benchmark set.
Soft Computing, 17(6):1031–1046, 2013.
[ bib 
DOI ]

[1363]

Tianjun Liao, Marco A. Montes de Oca, and Thomas Stützle.
Tuning Parameters across Mixed Dimensional Instances: A
Performance Scalability Study of SepGCMAES.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference Companion, GECCO Companion
2011, pages 703–706, New York, NY, 2011. ACM Press.
[ bib ]
Workshop on Scaling Behaviours of Landscapes,
Parameters and Algorithms

[1364]

Tianjun Liao, Krzysztof Socha, Marco A. Montes de Oca, Thomas Stützle,
and Marco Dorigo.
Ant Colony Optimization for MixedVariable Optimization
Problems.
IEEE Transactions on Evolutionary Computation, 18(4):503–518,
2014.
[ bib ]
Keywords: ACOR

[1365]

Tianjun Liao and Thomas Stützle.
Benchmark results for a simple hybrid algorithm on the CEC
2013 benchmark set for realparameter optimization.
In Proceedings of the 2013 Congress on Evolutionary Computation
(CEC 2013), pages 1938–1944. IEEE Press, Piscataway, NJ, 2013.
[ bib ]

[1366]

Tianjun Liao, Thomas Stützle, Marco A. Montes de Oca, and Marco Dorigo.
A Unified Ant Colony Optimization Algorithm for Continuous
Optimization.
European Journal of Operational Research, 234(3):597–609,
2014.
[ bib ]

[1367]

C.J. Liao, C.T. Tseng, and P. Luarn.
A Discrete Version of Particle Swarm Optimization for Flowshop
Scheduling Problems.
Computers & Operations Research, 34(10):3099–3111, 2007.
[ bib ]

[1368]

Tianjun Liao.
Populationbased Heuristic Algorithms for Continuous and Mixed
DiscreteContinuous Optimization Problem.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2013.
[ bib ]

[1369]

Arnaud Liefooghe, Fabio Daolio, Bilel Derbel, Sébastien Verel,
Hernán E. Aguirre, and Kiyoshi Tanaka.
LandscapeAware Performance Prediction for Evolutionary
Multiobjective Optimization.
IEEE Transactions on Evolutionary Computation,
24(6):1063–1077, 2020.
[ bib ]

[1370]

Arnaud Liefooghe, Bilel Derbel, Sébastien Verel, Hernán E. Aguirre, and
Kiyoshi Tanaka.
Towards LandscapeAware Automatic Algorithm Configuration:
Preliminary Experiments on Neutral and Rugged Landscapes.
In B. Hu and M. LópezIbáñez, editors, Proceedings
of EvoCOP 2017 – 17th European Conference on Evolutionary Computation in
Combinatorial Optimization, volume 10197 of Lecture Notes in Computer
Science, pages 215–232. Springer, Heidelberg, 2017.
[ bib 
DOI ]

[1371]

Arnaud Liefooghe, Bilel Derbel, Sébastien Verel, Manuel
LópezIbáñez, Hernán E. Aguirre, and Kiyoshi Tanaka.
On Pareto Local Optimal Solutions Networks.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature  PPSN
XV, volume 11102 of Lecture Notes in Computer Science, pages 232–244.
Springer, Cham, Switzerland, 2018.
[ bib 
DOI ]

[1372]

Arnaud Liefooghe, Jérémie Humeau, Salma Mesmoudi, Laetitia Jourdan, and
ElGhazali Talbi.
On dominancebased multiobjective local search: design,
implementation and experimental analysis on scheduling and traveling salesman
problems.
Journal of Heuristics, 18(2):317–352, 2012.
[ bib 
DOI ]
This paper discusses simple local search approaches for
approximating the efficient set of multiobjective
combinatorial optimization problems. We focus on algorithms
defined by a neighborhood structure and a dominance relation
that iteratively improve an archive of nondominated
solutions. Such methods are referred to as dominancebased
multiobjective local search. We first provide a concise
overview of existing algorithms, and we propose a model
trying to unify them through a finegrained
decomposition. The main problemindependent search components
of dominance relation, solution selection, neighborhood
exploration and archiving are largely discussed. Then, a
number of stateoftheart and original strategies are
experimented on solving a permutation flowshop scheduling
problem and a traveling salesman problem, both on a two and
a threeobjective formulation. Experimental results and a
statistical comparison are reported in the paper, and some
directions for future research are highlighted.

[1373]

Arnaud Liefooghe, Laetitia Jourdan, and ElGhazali Talbi.
A Software Framework Based on a Conceptual Unified Model for
Evolutionary Multiobjective Optimization: ParadisEOMOEO.
European Journal of Operational Research, 209(2):104–112,
2011.
[ bib ]

[1374]

Arnaud Liefooghe, Manuel LópezIbáñez, Luís Paquete, and
Sébastien Verel.
Dominance, Epsilon, and Hypervolume Local Optimal Sets in
Multiobjective Optimization, and How to Tell the Difference.
In H. E. Aguirre and K. Takadama, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2018, pages 324–331.
ACM Press, New York, NY, 2018.
[ bib 
DOI ]

[1375]

Arnaud Liefooghe, Salma Mesmoudi, Jérémie Humeau, Laetitia Jourdan, and
ElGhazali Talbi.
A Study on Dominancebased Local Search Approaches for
Multiobjective Combinatorial Optimization.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2009, volume 5752 of Lecture Notes
in Computer Science, pages 120–124. Springer, Heidelberg, 2009.
[ bib ]

[1376]

Arnaud Liefooghe, Luís Paquete, Marco Simoes, and José Rui
Figueira.
Connectedness and Local Search for Bicriteria Knapsack
Problems.
In P. Merz and J.K. Hao, editors, Proceedings of EvoCOP 2011 –
11th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 6622 of Lecture Notes in Computer Science, pages
48–59. Springer, Heidelberg, 2011.
[ bib 
DOI ]

[1377]

Arnaud Liefooghe, Sébastien Verel, Benjamin Lacroix, AlexandruCiprian
Zavoianu, and John McCall.
Landscape features and automated algorithm selection for
multiobjective interpolated continuous optimisation problems.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2021, pages 421–429.
ACM Press, New York, NY, 2021.
[ bib 
DOI ]

[1378]

Bojan Likar and Juš Kocijan.
Predictive control of a gas–liquid separation plant based on a
Gaussian process model.
Computers & Chemical Engineering, 31(3):142–152, 2007.
[ bib 
DOI ]

[1379]

David J. Lilja.
Measuring Computer Performance: A Practitioner's Guide.
Cambridge University Press, 2000.
[ bib 
DOI ]
Measuring Computer Performance sets out the fundamental
techniques used in analyzing and understanding the
performance of computer systems. Throughout the book, the
emphasis is on practical methods of measurement, simulation,
and analytical modeling. The author discusses performance
metrics and provides detailed coverage of the strategies used
in benchmark programmes. He gives intuitive explanations of
the key statistical tools needed to interpret measured
performance data. He also describes the general 'design of
experiments' technique, and shows how the maximum amount of
information can be obtained for the minimum effort. The book
closes with a chapter on the technique of queueing
analysis. Appendices listing common probability distributions
and statistical tables are included, along with a glossary of
important technical terms. This practicallyoriented book
will be of great interest to anyone who wants a detailed, yet
intuitive, understanding of computer systems performance
analysis.

[1380]

Marius Thomas Lindauer, Holger H. Hoos, Frank Hutter, and Torsten Schaub.
AutoFolio: Algorithm Configuration for Algorithm Selection.
In B. Bonet and S. Koenig, editors, Proceedings of the AAAI
Conference on Artificial Intelligence. AAAI Press, 2015.
[ bib ]

[1381]

Marius Thomas Lindauer, Holger H. Hoos, Frank Hutter, and Torsten Schaub.
AutoFolio: An Automatically Configured Algorithm Selector.
Journal of Artificial Intelligence Research, 53:745–778, 2015.
[ bib ]

[1382]

S. Lin and B. W. Kernighan.
An Effective Heuristic Algorithm for the Traveling Salesman
Problem.
Operations Research, 21(2):498–516, 1973.
[ bib ]

[1383]

W. Ling and H. Luo.
An Adaptive Parameter Control Strategy for Ant Colony
Optimization.
In CIS'07: Proceedings of the 2007 International Conference on
Computational Intelligence and Security, pages 142–146, Washington, DC,
2007. IEEE Computer Society.
[ bib ]

[1384]

Marius Thomas Lindauer, Jan N. Van Rijn, and Lars Kotthoff.
The algorithm selection competitions 2015 and 2017.
Artificial Intelligence, 272:86–100, 2019.
[ bib ]

[1385]

Andrei Lissovoi and Carsten Witt.
Runtime Analysis of Ant Colony Optimization on Dynamic Shortest
Path Problems.
Theoretical Computer Science, 561(Part A):73–85, 2015.
[ bib 
DOI ]
A simple ACO algorithm called λMMAS for dynamic
variants of the singledestination shortest paths problem is
studied by rigorous runtime analyses. Building upon previous
results for the special case of 1MMAS, it is studied to what
extent an enlarged colony using λ ants per vertex
helps in tracking an oscillating optimum. It is shown that
easy cases of oscillations can be tracked by a constant
number of ants. However, the paper also identifies more
involved oscillations that with overwhelming probability
cannot be tracked with any polynomialsize colony. Finally,
parameters of dynamic shortestpath problems which make the
optimum difficult to track are discussed. Experiments
illustrate theoretical findings and conjectures.

[1386]

J. D. C. Little, K. G. Murty, D. W. Sweeney, and C. Karel.
An Algorithm for the Traveling Salesman Problem.
Operations Research, 11:972–989, 1963.
[ bib ]

[1387]

Shusen Liu, Dan Maljovec, Bei Wang, PeerTimo Bremer, and Valerio Pascucci.
Visualizing HighDimensional Data: Advances in the Past Decade.
IEEE Transactions on Visualization and Computer Graphics,
23(3), 2017.
[ bib 
DOI ]

[1388]

Jiyin Liu and Colin R. Reeves.
Constructive and Composite Heuristic Solutions to the
P//ΣCi Scheduling Problem.
European Journal of Operational Research, 132(2):439–452,
2001.
[ bib 
DOI ]

[1389]

Innovation 24.
LocalSolver.
http://www.localsolver.com/product.html, 2016.
Last visited, August 15, 2016.
[ bib ]

[1390]

Andrea Lodi, Silvano Martello, and Michele Monaci.
Twodimensional packing problems: A survey.
European Journal of Operational Research, 141(2):241–252,
2002.
[ bib 
DOI ]

[1391]

Andrea Lodi, Silvano Martello, and Daniele Vigo.
Heuristic and metaheuristic approaches for a class of
twodimensional bin packing problems.
INFORMS Journal on Computing, 11(4):345–357, 1999.
[ bib 
DOI ]

[1392]

Andrea Lodi, Silvano Martello, and Daniele Vigo.
TSpack: a unified tabu search code for multidimensional bin
packing problems.
Annals of Operations Research, 131(14):203–213, 2004.
[ bib 
DOI ]

[1393]

Andrea Lodi and Andrea Tramontani.
Performance Variability in MixedInteger Programming.
In H. Topaluglu, editor, Theory Driven by Influential
Applications, pages 1–12. INFORMS, 2013.
[ bib ]

[1394]

Andrea Lodi, Silvano Martello, and Daniele Vigo.
Two and ThreeDimensional Bin Packing – Source Code of
TSpack.
http://or.dei.unibo.it/research_pages/ORcodes/TSpack.html,
2004.
[ bib ]

[1395]

Jason D. Lohn, Gregory S. Hornby, and Derek S. Linden.
Humancompetitive Evolved Antennas.
Artificial Intelligence for Engineering Design, Analysis and
Manufacturing, 22(3):235–247, 2008.
[ bib 
DOI ]
Evolutionary optimization of antennas for NASA

[1396]

PoLing Loh and Sebastian Nowozin.
Faster Hoeffding Racing: Bernstein Races via Jackknife
Estimates.
In S. Jain, R. Munos, F. Stephan, and T. Zeugmann, editors,
Proceedings of Algorithmic Learning Theory, volume 8139 of Lecture
Notes in Computer Science, pages 203–217, Berlin, Germany, 2013. Springer.
[ bib 
DOI ]

[1397]

Manuel LópezIbáñez.
High Performance Ant Colony Optimisation of the Pump Scheduling
Problem.
In P. Alberigo, G. Erbacci, F. Garofalo, and S. Monfardini, editors,
Science and Sumpercomputing in Europe, pages 371–375. CINECA, 2007.
[ bib ]

[1398]

Manuel LópezIbáñez and Christian Blum.
BeamACO Based on Stochastic Sampling: A Case Study on the
TSP with Time Windows.
Technical Report LSI0828, Department LSI, Universitat
Politècnica de Catalunya, 2008.
Extended version published in Computers & Operations
Research [1401].
[ bib ]

[1399]

Manuel LópezIbáñez, Christian Blum, Dhananjay Thiruvady,
Andreas T. Ernst, and Bernd Meyer.
BeamACO based on stochastic sampling for makespan
optimization concerning the TSP with time windows.
In C. Cotta and P. Cowling, editors, Proceedings of EvoCOP 2009
– 9th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 5482 of Lecture Notes in Computer Science, pages
97–108. Springer, Heidelberg, 2009.
[ bib 
DOI ]

[1400]

Manuel LópezIbáñez and Christian Blum.
BeamACO Based on Stochastic Sampling: A Case Study on the
TSP with Time Windows.
In T. Stützle, editor, Learning and Intelligent
Optimization, Third International Conference, LION 3, volume 5851 of
Lecture Notes in Computer Science, pages 59–73. Springer, Heidelberg, 2009.
[ bib 
DOI ]

[1401]

Manuel LópezIbáñez and Christian Blum.
BeamACO for the travelling salesman problem with time
windows.
Computers & Operations Research, 37(9):1570–1583, 2010.
[ bib 
DOI ]
The travelling salesman problem with time windows is
a difficult optimization problem that arises, for
example, in logistics. This paper deals with the
minimization of the travelcost. For solving this
problem, this paper proposes a BeamACO algorithm,
which is a hybrid method combining ant colony
optimization with beam search. In general, BeamACO
algorithms heavily rely on accurate and
computationally inexpensive bounding information for
differentiating between partial solutions. This work
uses stochastic sampling as a useful alternative. An
extensive experimental evaluation on seven benchmark
sets from the literature shows that the proposed
BeamACO algorithm is currently a stateoftheart
technique for the travelling salesman problem with
time windows when travelcost optimization is
concerned.
Keywords: Ant colony optimization, Travelling salesman problem with
time windows, Hybridization

[1402]

Manuel LópezIbáñez, Christian Blum, Jeffrey W. Ohlmann, and
Barrett W. Thomas.
The Travelling Salesman Problem with Time Windows: Adapting
Algorithms from Traveltime to Makespan Optimization.
Applied Soft Computing, 13(9):3806–3815, 2013.
[ bib 
DOI 
eprint ]

[1403]

Manuel LópezIbáñez, Jürgen Branke, and Luís Paquete.
Reproducibility in Evolutionary Computation.
Arxiv preprint arXiv:20102.03380 [cs.AI], 2021.
[ bib 
http ]
Experimental studies are prevalent in Evolutionary
Computation (EC), and concerns about the reproducibility and
replicability of such studies have increased in recent times,
reflecting similar concerns in other scientific fields. In
this article, we suggest a classification of different types
of reproducibility that refines the badge system of the
Association of Computing Machinery (ACM) adopted by TELO. We
discuss, within the context of EC, the different types of
reproducibility as well as the concepts of artifact and
measurement, which are crucial for claiming
reproducibility. We identify cultural and technical obstacles
to reproducibility in the EC field. Finally, we provide
guidelines and suggest tools that may help to overcome some
of these reproducibility obstacles.
Keywords: Evolutionary Computation, Reproducibility, Empirical study,
Benchmarking

[1404]

Manuel LópezIbáñez, Jürgen Branke, and Luís Paquete.
Reproducibility in Evolutionary Computation.
ACM Transactions on Evolutionary Learning and Optimization,
1(4):1–21, 2021.
[ bib 
DOI 
eprint ]
Experimental studies are prevalent in Evolutionary
Computation (EC), and concerns about the reproducibility and
replicability of such studies have increased in recent times,
reflecting similar concerns in other scientific fields. In
this article, we suggest a classification of different types
of reproducibility that refines the badge system of the
Association of Computing Machinery (ACM) adopted by TELO. We
discuss, within the context of EC, the different types of
reproducibility as well as the concepts of artifact and
measurement, which are crucial for claiming
reproducibility. We identify cultural and technical obstacles
to reproducibility in the EC field. Finally, we provide
guidelines and suggest tools that may help to overcome some
of these reproducibility obstacles.
Keywords: Evolutionary Computation, Reproducibility, Empirical study,
Benchmarking

[1405]

Manuel LópezIbáñez, Jérémie DuboisLacoste, Leslie
Pérez Cáceres, Thomas Stützle, and Mauro Birattari.
The irace Package: Iterated Racing for Automatic
Algorithm Configuration.
Operations Research Perspectives, 3:43–58, 2016.
[ bib 
DOI 
supplementary material ]

[1406]

Manuel LópezIbáñez, Jérémie DuboisLacoste, Leslie
Pérez Cáceres, Thomas Stützle, and Mauro Birattari.
The irace Package: Iterated Racing for Automatic
Algorithm Configuration (Supplementary Material).
http://iridia.ulb.ac.be/supp/IridiaSupp2016003, 2016.
[ bib ]

[1407]

Manuel LópezIbáñez, Jérémie DuboisLacoste, Thomas
Stützle, and Mauro Birattari.
The irace package, Iterated Race for Automatic
Algorithm Configuration.
Technical Report TR/IRIDIA/2011004, IRIDIA, Université Libre de
Bruxelles, Belgium, 2011.
Published in Operations Research
Perspectives [1405].
[ bib 
http ]

[1408]

Manuel LópezIbáñez, MarieEléonore Kessaci, and Thomas
Stützle.
Automatic Design of Hybrid Metaheuristics from Algorithmic
Components.
Submitted, 2017.
[ bib ]

[1409]

Manuel LópezIbáñez and Joshua D. Knowles.
Machine Decision Makers as a Laboratory for Interactive EMO.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part II,
volume 9019 of Lecture Notes in Computer Science, pages 295–309.
Springer, Heidelberg, 2015.
[ bib 
DOI ]
A key challenge, perhaps the central challenge, of
multiobjective optimization is how to deal with candidate
solutions that are ultimately evaluated by the hidden or
unknown preferences of a human decision maker (DM) who
understands and cares about the optimization problem.
Alternative ways of addressing this challenge exist but
perhaps the favoured one currently is the interactive
approach (proposed in various forms). Here, an evolutionary
multiobjective optimization algorithm (EMOA) is controlled
by a series of interactions with the DM so that preferences
can be elicited and the direction of search controlled. MCDM
has a key role to play in designing and evaluating these
approaches, particularly in testing them with real DMs, but
so far quantitative assessment of interactive EMOAs has been
limited. In this paper, we propose a conceptual framework
for this problem of quantitative assessment, based on the
definition of machine decision makers (machine DMs), made
somewhat realistic by the incorporation of various
nonidealities. The machine DM proposed here draws from
earlier models of DM biases and inconsistencies in the MCDM
literature. As a practical illustration of our approach, we
use the proposed machine DM to study the performance of an
interactive EMOA, and discuss how this framework could help
in the evaluation and development of better interactive
EMOAs.

[1410]

Manuel LópezIbáñez, Joshua D. Knowles, and Marco Laumanns.
On Sequential Online Archiving of Objective Vectors.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pages 46–60. Springer, Heidelberg, 2011.
[ bib 
DOI ]
In this paper, we examine the problem of maintaining
an approximation of the set of nondominated points
visited during a multiobjective optimization, a
problem commonly known as archiving. Most of the
currently available archiving algorithms are
reviewed, and what is known about their convergence
and approximation properties is summarized. The main
scenario considered is the restricted case where the
archive must be updated online as points are
generated one by one, and at most a fixed number of
points are to be stored in the archive at any one
time. In this scenario, the bettermonotonicity of
an archiving algorithm is proposed as a weaker, but
more practical, property than negative efficiency
preservation. This paper shows that
hypervolumebased archivers and a recently proposed
multilevel grid archiver have this property. On the
other hand, the archiving methods used by SPEA2 and
NSGAII do not, and they may betterdeteriorate with
time. The bettermonotonicity property has meaning
on any input sequence of points. We also classify
archivers according to limit properties,
i.e. convergence and approximation properties of the
archiver in the limit of infinite (input) samples
from a finite space with strictly positive
generation probabilities for all points. This paper
establishes a number of research questions, and
provides the initial framework and analysis for
answering them.
Revised version available at http://iridia.ulb.ac.be/IridiaTrSeries/IridiaTr2011001.pdf

[1411]

Manuel LópezIbáñez, Tianjun Liao, and Thomas Stützle.
On the anytime behavior of IPOPCMAES.
In C. A. Coello Coello et al., editors, Parallel Problem
Solving from Nature, PPSN XII, volume 7491 of Lecture Notes in Computer
Science, pages 357–366. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1412]

Manuel LópezIbáñez, Tianjun Liao, and Thomas Stützle.
On the anytime behavior of IPOPCMAES: Supplementary
material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012010/, 2012.
[ bib ]

[1413]

Manuel LópezIbáñez, Arnaud Liefooghe, and Sébastien Verel.
Local Optimal Sets and Bounded Archiving on Multiobjective
NKLandscapes with Correlated Objectives.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, PPSN 2014, volume 8672 of Lecture Notes in Computer
Science, pages 621–630. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1414]

Manuel LópezIbáñez, Franco Mascia, MarieEléonore Marmion, and
Thomas Stützle.
Automatic Design of a Hybrid Iterated Local Search for the
MultiMode ResourceConstrained MultiProject Scheduling Problem.
In G. Kendall, G. V. Berghe, and B. McCollum, editors,
Multidisciplinary International Conference on Scheduling: Theory and
Applications (MISTA 2013), pages 1–6, Gent, Belgium, 2013.
[ bib ]
https://hal.inria.fr/hal01094681

[1415]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
On the Design of ACO for the Biobjective Quadratic Assignment
Problem.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 214–225. Springer, Heidelberg,
2004.
[ bib 
DOI ]

[1416]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
Hybrid Populationbased Algorithms for the Biobjective
Quadratic Assignment Problem.
Technical Report AIDA–04–11, FG Intellektik, FB Informatik, TU
Darmstadt, December 2004.
Published in Journal of Mathematical Modelling and
Algorithms [1417].
[ bib ]

[1417]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
Hybrid Populationbased Algorithms for the Biobjective
Quadratic Assignment Problem.
Journal of Mathematical Modelling and Algorithms,
5(1):111–137, 2006.
[ bib 
DOI ]
We present variants of an ant colony optimization
(MOACO) algorithm and of an evolutionary algorithm
(SPEA2) for tackling multiobjective combinatorial
optimization problems, hybridized with an iterative
improvement algorithm and the robust tabu search
algorithm. The performance of the resulting hybrid
stochastic local search (SLS) algorithms is
experimentally investigated for the biobjective
quadratic assignment problem (bQAP) and compared
against repeated applications of the underlying
local search algorithms for several
scalarizations. The experiments consider structured
and unstructured bQAP instances with various degrees
of correlation between the flow matrices. We do a
systematic experimental analysis of the algorithms
using outperformance relations and the attainment
functions methodology to asses differences in the
performance of the algorithms. The experimental
results show the usefulness of the hybrid algorithms
if the available computation time is not too limited
and identify SPEA2 hybridized with very short tabu
search runs as the most promising variant.

[1418]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
Exploratory Analysis of Stochastic Local Search Algorithms in
Biobjective Optimization.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 209–222. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
This chapter introduces two Perl programs that
implement graphical tools for exploring the
performance of stochastic local search algorithms
for biobjective optimization problems. These tools
are based on the concept of the empirical attainment
function (EAF), which describes the probabilistic
distribution of the outcomes obtained by a
stochastic algorithm in the objective space. In
particular, we consider the visualization of
attainment surfaces and differences between the
firstorder EAFs of the outcomes of two
algorithms. This visualization allows us to identify
certain algorithmic behaviors in a graphical way.
We explain the use of these visualization tools and
illustrate them with examples arising from
practice.

[1419]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
EAF Graphical Tools.
http://lopezibanez.eu/eaftools, 2010.
These tools are described in the book chapter “Exploratory
analysis of stochastic local search algorithms in biobjective
optimization” [1418].
[ bib ]
Please cite the book chapter, not this.

[1420]

Manuel LópezIbáñez, Leslie Pérez Cáceres, Jérémie
DuboisLacoste, Thomas Stützle, and Mauro Birattari.
The irace package: User Guide.
Technical Report TR/IRIDIA/2016004, IRIDIA, Université Libre de
Bruxelles, Belgium, 2016.
[ bib 
http ]

[1421]

Manuel LópezIbáñez, Leslie Pérez Cáceres, and Thomas
Stützle.
irace: A Tool for the Automatic Configuration of Algorithms.
International Federation of Operational Research Societies
(IFORS) News, 14(2):30–32, June 2020.
[ bib 
http ]

[1422]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Parallel Optimisation Of Pump Schedules With A ThreadSafe
Variant Of EPANET Toolkit.
In J. E. van Zyl, A. A. Ilemobade, and H. E. Jacobs, editors,
Proceedings of the 10th Annual Water Distribution Systems Analysis Conference
(WDSA 2008). ASCE, August 2008.
[ bib 
DOI ]

[1423]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Ant Colony Optimisation for the Optimal Control of Pumps in
Water Distribution Networks.
Journal of Water Resources Planning and Management, ASCE,
134(4):337–346, 2008.
[ bib 
DOI 
eprint ]
Reducing energy consumption of water distribution
networks has never had more significance than today. The greatest
energy savings can be obtained by careful scheduling of operation of
pumps. Schedules can be defined either implicitly, in terms of other
elements of the network such as tank levels, or explicitly by
specifying the time during which each pump is on/off. The
traditional representation of explicit schedules is a string of
binary values with each bit representing pump on/off status during a
particular time interval. In this paper a new explicit
representation is presented. It is based on time controlled
triggers, where the maximum number of pump switches is specified
beforehand. In this representation a pump schedule is divided into a
series of integers with each integer representing the number of
hours for which a pump is active/inactive. This reduces the number
of potential schedules (search space) compared to the binary
representation. Ant colony optimization (ACO) is a stochastic
metaheuristic for combinatorial optimization problems that is
inspired by the foraging behavior of some species of ants. In this
paper, an application of the ACO framework was developed for the
optimal scheduling of pumps. The proposed representation was adapted
to an ant colony Optimization framework and solved for the optimal
pump schedules. Minimization of electrical cost was considered as
the objective, while satisfying system constraints. Instead of using
a penalty function approach for constraint violations, constraint
violations were ordered according to their importance and solutions
were ranked based on this order. The proposed approach was tested on
a small test network and on a large realworld network. Results are
compared with those obtained using a simple genetic algorithm based
on binary representation and a hybrid genetic algorithm that uses
levelbased triggers.

[1424]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Representations and Evolutionary Operators for the Scheduling of
Pump Operations in Water Distribution Networks.
Evolutionary Computation, 19(3):429–467, 2011.
[ bib 
DOI ]
Reducing the energy consumption of water
distribution networks has never had more
significance. The greatest energy savings can be
obtained by carefully scheduling the operations of
pumps. Schedules can be defined either implicitly,
in terms of other elements of the network such as
tank levels, or explicitly by specifying the time
during which each pump is on/off. The traditional
representation of explicit schedules is a string of
binary values with each bit representing pump on/off
status during a particular time interval. In this
paper, we formally define and analyze two new
explicit representations based on timecontrolled
triggers, where the maximum number of pump switches
is established beforehand and the schedule may
contain less switches than the maximum. In these
representations, a pump schedule is divided into a
series of integers with each integer representing
the number of hours for which a pump is
active/inactive. This reduces the number of
potential schedules compared to the binary
representation, and allows the algorithm to operate
on the feasible region of the search space. We
propose evolutionary operators for these two new
representations. The new representations and their
corresponding operations are compared with the two
mostused representations in pump scheduling,
namely, binary representation and levelcontrolled
triggers. A detailed statistical analysis of the
results indicates which parameters have the greatest
effect on the performance of evolutionary
algorithms. The empirical results show that an
evolutionary algorithm using the proposed
representations improves over the results obtained
by a recent stateoftheart Hybrid Genetic
Algorithm for pump scheduling using levelcontrolled
triggers.

[1425]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Solving Optimal Pump Control Problem using MaxMin Ant System.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, volume 1, page 176. ACM
Press, New York, NY, 2007.
[ bib 
DOI ]

[1426]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Multiobjective Optimisation of the Pump Scheduling Problem
using SPEA2.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), volume 1, pages 435–442. IEEE Press, Piscataway, NJ, September
2005.
[ bib 
DOI ]

[1427]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Optimal Pump Scheduling: Representation and Multiple
Objectives.
In D. A. Savic, G. A. Walters, R. King, and S. ThiamKhu, editors,
Proceedings of the Eighth International Conference on Computing and
Control for the Water Industry (CCWI 2005), volume 1, pages 117–122,
University of Exeter, UK, September 2005.
[ bib ]

[1428]

Manuel LópezIbáñez and Thomas Stützle.
An Analysis of Algorithmic Components for Multiobjective Ant
Colony Optimization: A Case Study on the Biobjective TSP.
In P. Collet, N. Monmarché, P. Legrand, M. Schoenauer, and
E. Lutton, editors, Artificial Evolution: 9th International Conference,
Evolution Artificielle, EA, 2009, volume 5975 of Lecture Notes in
Computer Science, pages 134–145. Springer, Heidelberg, 2010.
[ bib 
DOI ]

[1429]

Manuel LópezIbáñez and Thomas Stützle.
Automatic Configuration of MultiObjective ACO Algorithms.
In M. Dorigo et al., editors, Swarm Intelligence, 7th
International Conference, ANTS 2010, volume 6234 of Lecture Notes in
Computer Science, pages 95–106. Springer, Heidelberg, 2010.
[ bib 
DOI ]
In the last few years a significant number of ant
colony optimization (ACO) algorithms have been
proposed for tackling multiobjective optimization
problems. In this paper, we propose a software
framework that allows to instantiate the most
prominent multiobjective ACO (MOACO)
algorithms. More importantly, the flexibility of
this MOACO framework allows the application of
automatic algorithm configuration techniques. The
experimental results presented in this paper show
that such an automatic configuration of MOACO
algorithms is highly desirable, given that our
automatically configured algorithms clearly
outperform the best performing MOACO algorithms that
have been proposed in the literature. As far as we
are aware, this paper is also the first to apply
automatic algorithm configuration techniques to
multiobjective stochastic local search algorithms.

[1430]

Manuel LópezIbáñez and Thomas Stützle.
The impact of design choices of multiobjective ant colony
optimization algorithms on performance: An experimental study on the
biobjective TSP.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pages 71–78. ACM
Press, New York, NY, 2010.
[ bib 
DOI ]
Over the last few years, there have been a number of
proposals of ant colony optimization (ACO)
algorithms for tackling multiobjective combinatorial
optimization problems. These proposals adapt ACO
concepts in various ways, for example, some use
multiple pheromone matrices and multiple heuristic
matrices and others use multiple ant colonies.
In
this article, we carefully examine several of the
most prominent of these proposals. In particular, we
identify commonalities among the approaches by
recasting the original formulation of the algorithms
in different terms. For example, several proposals
described in terms of multiple colonies can be cast
equivalently using a single ant colony, where ants
use different weights for aggregating the pheromone
and/or the heuristic information. We study
algorithmic choices for the various proposals and we
identify previously undetected tradeoffs in their
performance.

[1431]

Manuel LópezIbáñez and Thomas Stützle.
The impact of design choices of multiobjective ant colony
optimization algorithms on performance: An experimental study on the
biobjective TSP.
http://iridia.ulb.ac.be/supp/IridiaSupp2010003/, 2010.
Supplementary material of [1430].
[ bib ]

[1432]

Manuel LópezIbáñez and Thomas Stützle.
The Automatic Design of MultiObjective Ant Colony Optimization
Algorithms: Supplementary material, 2011.
[ bib 
http ]

[1433]

Manuel LópezIbáñez and Thomas Stützle.
An experimental analysis of design choices of multiobjective
ant colony optimization algorithms: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012006/, 2012.
[ bib ]

[1434]

Manuel LópezIbáñez and Thomas Stützle.
An experimental analysis of design choices of multiobjective
ant colony optimization algorithms.
Swarm Intelligence, 6(3):207–232, 2012.
[ bib 
DOI 
supplementary material ]

[1435]

Manuel LópezIbáñez and Thomas Stützle.
The Automatic Design of MultiObjective Ant Colony Optimization
Algorithms.
IEEE Transactions on Evolutionary Computation, 16(6):861–875,
2012.
[ bib 
DOI ]
Multiobjective optimization problems are problems with several,
typically conflicting criteria for evaluating solutions. Without
any a priori preference information, the Pareto optimality
principle establishes a partial order among solutions, and the
output of the algorithm becomes a set of nondominated solutions
rather than a single one. Various ant colony optimization (ACO)
algorithms have been proposed in recent years for solving such
problems. These multiobjective ACO (MOACO) algorithms exhibit
different design choices for dealing with the particularities of
the multiobjective context. This paper proposes a formulation of
algorithmic components that suffices to describe most MOACO
algorithms proposed so far. This formulation also shows that
existing MOACO algorithms often share equivalent design choices
but they are described in different terms. Moreover, this
formulation is synthesized into a flexible algorithmic framework,
from which not only existing MOACO algorithms may be
instantiated, but also combinations of components that were never
studied in the literature. In this sense, this paper goes beyond
proposing a new MOACO algorithm, but it rather introduces a
family of MOACO algorithms. The flexibility of the proposed MOACO
framework facilitates the application of automatic algorithm
configuration techniques. The experimental results presented in
this paper show that the automatically configured MOACO framework
outperforms the MOACO algorithms that inspired the framework
itself. This paper is also among the first to apply automatic
algorithm configuration techniques to multiobjective algorithms.

[1436]

Manuel LópezIbáñez and Thomas Stützle.
Automatically Improving the Anytime Behaviour of Optimisation
Algorithms.
European Journal of Operational Research, 235(3):569–582,
2014.
[ bib 
DOI 
supplementary material ]
Optimisation algorithms with good anytime behaviour try to
return as highquality solutions as possible independently of
the computation time allowed. Designing algorithms with good
anytime behaviour is a difficult task, because performance is
often evaluated subjectively, by plotting the tradeoff curve
between computation time and solution quality. Yet, the
tradeoff curve may be modelled also as a set of mutually
nondominated, biobjective points. Using this model, we
propose to combine an automatic configuration tool and the
hypervolume measure, which assigns a single quality measure
to a nondominated set. This allows us to improve the anytime
behaviour of optimisation algorithms by means of
automatically finding algorithmic configurations that produce
the best nondominated sets. Moreover, the recently proposed
weighted hypervolume measure is used here to incorporate the
decisionmaker's preferences into the automatic tuning
procedure. We report on the improvements reached when
applying the proposed method to two relevant scenarios: (i)
the design of parameter variation strategies for MAXMIN Ant
System and (ii) the tuning of the anytime behaviour of SCIP,
an opensource mixed integer programming solver with more
than 200 parameters.

[1437]

Manuel LópezIbáñez, Thomas Stützle, and Marco Dorigo.
Ant Colony Optimization: A ComponentWise Overview.
In R. Martí, P. M. Pardalos, and M. G. C. Resende, editors,
Handbook of Heuristics, pages 371–407. Springer International Publishing,
2018.
[ bib 
DOI 
supplementary material ]

[1438]

Eunice LÃ³pezCamacho, Hugo TerashimaMarin, Peter Ross, and Gabriela Ochoa.
A unified hyperheuristic framework for solving bin packing
problems.
Expert Systems with Applications, 41(15):6876–6889, 2014.
[ bib 
DOI ]

[1439]

Manuel LópezIbáñez.
Multiobjective Ant Colony Optimization.
Diploma thesis, Intellectics Group, Computer Science Department,
Technische Universität Darmstadt, Germany, 2004.
[ bib ]

[1440]

Manuel LópezIbáñez.
Operational Optimisation of Water Distribution Networks.
PhD thesis, School of Engineering and the Built Environment,
Edinburgh Napier University, UK, 2009.
[ bib 
http ]

[1441]

Ilya Loshchilov, Marc Schoenauer, and Michèle Sebag.
Alternative Restart Strategies for CMAES.
In C. A. Coello Coello et al., editors, Parallel Problem
Solving from Nature, PPSN XII, volume 7491 of Lecture Notes in Computer
Science, pages 296–305. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1442]

A. V. Lotov and Kaisa Miettinen.
Visualizing the Pareto Frontier.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science, pages 213–243.
Springer, Heidelberg, 2008.
[ bib ]

[1443]

Samir Loudni and Patrice Boizumault.
Combining VNS with constraint programming for solving anytime
optimization problems.
European Journal of Operational Research, 191:705–735, 2008.
[ bib 
DOI ]

[1444]

Helena R. Lourenço, Olivier Martin, and Thomas Stützle.
Iterated Local Search.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 321–353. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib 
DOI ]

[1445]

Helena R. Lourenço, Olivier Martin, and Thomas Stützle.
Iterated Local Search: Framework and Applications.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, chapter 9, pages 363–397. Springer, New
York, NY, 2nd edition, 2010.
[ bib 
DOI ]

[1446]

Helena R. Lourenço, Olivier Martin, and Thomas Stützle.
Iterated Local Search: Framework and Applications.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 272 of International Series in Operations
Research & Management Science, chapter 5, pages 129–168. Springer, 2019.
[ bib 
DOI ]

[1447]

Helena R. Lourenço.
JobShop Scheduling: Computational Study of Local Search and
LargeStep Optimization Methods.
European Journal of Operational Research, 83(2):347–364, 1995.
[ bib ]

[1448]

Antonio Lova and Pilar Tormos.
Analysis of Scheduling Schemes and Heuristic Rules Performance
in ResourceConstrained Multiproject Scheduling.
Annals of Operations Research, 102(14):263–286, February
2001.
[ bib 
DOI ]
Frequently, the availability of resources assigned
to a project is limited and not sufficient to
execute all the concurrent activities. In this
situation, decision making about their schedule is
necessary. Many times this schedule supposes an
increase in the project completion
time. Additionally, companies commonly manage
various projects simultaneously, sharing a pool of
renewable resources. Given these resource
constraints, we often can only apply heuristic
methods to solve the scheduling problem. In this
work the effect of the schedule generation schemes 
serial or parallel  and priority rules  MINLFT,
MINSLK, MAXTWK, SASP or FCFS  with two
approaches  multiproject and singleproject  are
analysed. The time criteria considered are the mean
project delay and the multiproject duration
increase. Through an extensive computational study,
results show that with the parallel scheduling
generation scheme and the multiproject approach the
project manager can obtain a good multiproject
schedule with the time criterion selected:
minimising mean project delay or minimising
multiproject duration increase. New heuristics 
based on priority rules with a twophase approach 
that outperform classical ones are proposed to
minimise mean project delay with a multiproject
approach. Finally, the best heuristics analysed are
evaluated together with a representative sample of
commercial project management software.
Keywords: Combinatorics, heuristic based on priority rules,
Multiproject scheduling, Operation
Research/Decision Theory, Project management,
project management software, Resource allocation,
Theory of Computation

[1449]

Antonio Lova, Pilar Tormos, Mariamar Cervantes, and Federico Barber.
An efficient hybrid genetic algorithm for scheduling projects
with resource constraints and multiple execution modes.
International Journal of Production Economics, 117(2):302–316,
2009.
[ bib 
DOI ]
Multimode Resource Constrained Project Scheduling
Problem (MRCPSP) aims at finding the start times
and execution modes for the activities of a project
that optimize a given objective function while
verifying a set of precedence and resource
constraints. In this paper, we focus on this problem
and develop a hybrid Genetic Algorithm (MMHGA) to
solve it. Its main contributions are the mode
assignment procedure, the fitness function and the
use of a very efficient improving method. Its
performance is demonstrated by extensive
computational results obtained on a set of standard
instances and against the best currently available
algorithms.
Keywords: genetic algorithm, multimode resourceconstrained
project scheduling

[1450]

Manuel Lozano, Fred Glover, Carlos GarcíaMartínez, Francisco J.
Rodríguez, and Rafael Martí.
Tabu Search with Strategic Oscillation for the Quadratic Minimum
Spanning Tree.
IIE Transactions, 46(4):414–428, 2014.
[ bib ]

[1451]

Manuel Lozano, Daniel Molina, and Carlos GarcíaMartínez.
Iterated Greedy for the Maximum Diversity Problem.
European Journal of Operational Research, 214(1):31–38, 2011.
[ bib ]

[1452]

Zhipeng Lü, Fred Glover, and JinKao Hao.
A hybrid metaheuristic approach to solving the UBQP problem.
European Journal of Operational Research, 207(3):1254–1262,
2010.
[ bib 
DOI ]

[1453]

M. Lundy and A. Mees.
Convergence of an Annealing Algorithm.
Mathematical Programming, 34(1):111–124, 1986.
[ bib ]

[1454]

Thibaut Lust and Jacques Teghem.
Twophase Pareto local search for the biobjective traveling
salesman problem.
Journal of Heuristics, 16(3):475–510, 2010.
[ bib 
DOI ]
In this work, we present a method, called TwoPhase
Pareto Local Search, to find a good approximation of the
efficient set of the biobjective traveling salesman
problem. In the first phase of the method, an initial
population composed of a good approximation of the extreme
supported efficient solutions is generated. We use as second
phase a Pareto Local Search method applied to each solution
of the initial population. We show that using the combination
of these two techniques: good initial population generation
plus Pareto Local Search gives better results than
stateoftheart algorithms. Two other points are introduced:
the notion of ideal set and a simple way to produce
nearefficient solutions of multiobjective problems, by using
an efficient singleobjective solver with a data perturbation
technique.

[1455]

Thibaut Lust and Jacques Teghem.
The multiobjective traveling salesman problem: A survey and a
new approach.
In C. A. Coello Coello, C. Dhaenens, and L. Jourdan, editors,
Advances in MultiObjective Nature Inspired Computing, volume 272 of
Studies in Computational Intelligence, pages 119–141. Springer, 2010.
[ bib ]

[1456]

Thibaut Lust and Jacques Teghem.
The multiobjective multidimensional knapsack problem: a survey
and a new approach.
Arxiv preprint arXiv:1007.4063, 2010.
[ bib ]
Published as [1457]

[1457]

Thibaut Lust and Jacques Teghem.
The multiobjective multidimensional knapsack problem: a survey
and a new approach.
International Transactions in Operational Research,
19(4):495–520, 2012.
[ bib 
DOI ]

[1458]

Thibaut Lust and Andrzej Jaszkiewicz.
Speedup techniques for solving largescale biobjective TSP.
Computers & Operations Research, 37(3):521–533, 2010.
[ bib 
DOI ]
Keywords: Multiobjective combinatorial optimization, Hybrid
metaheuristics, TSP, Local search, Speedup techniques

[1459]

C. von Lücken, Benjamín Barán, and Carlos Brizuela.
A survey on multiobjective evolutionary algorithms for
manyobjective problems.
Computational Optimization and Applications, 58(3):707–756,
2014.
[ bib ]

[1460]

Robert John Lygoe.
Complexity reduction in highdimensional multiobjective
optimisation.
PhD thesis, University of Sheffield Sheffield, UK, 2010.
[ bib ]

[1461]

Qingfu Zhang.
MOEA/D homepage.
https://dces.essex.ac.uk/staff/zhang/webofmoead.htm, 2007.
[ bib ]

[1462]

Laurens van der Maaten and Geoffrey Hinton.
Visualizing Data using tSNE.
Journal of Machine Learning Research, 9(86):2579–2605, 2008.
[ bib ]

[1463]

Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew
Hausknecht, and Michael Bowling.
Revisiting the Arcade Learning Environment: Evaluation
Protocols and Open Problems for General Agents.
Journal of Artificial Intelligence Research, 61(1):523–562,
January 2018.
[ bib ]
The Arcade Learning Environment (ALE) is an evaluation
platform that poses the challenge of building AI agents with
general competency across dozens of Atari 2600 games. It
supports a variety of different problem settings and it has
been receiving increasing attention from the scientific
community, leading to some highpro_le success stories such
as the much publicized Deep QNetworks (DQN). In this article
we take a big picture look at how the ALE is being used by
the research community. We show how diverse the evaluation
methodologies in the ALE have become with time, and highlight
some key concerns when evaluating agents in the ALE. We use
this discussion to present some methodological best practices
and provide new benchmark results using these best
practices. To further the progress in the field, we introduce
a new version of the ALE that supports multiple game modes
and provides a form of stochasticity we call sticky
actions. We conclude this big picture look by revisiting
challenges posed when the ALE was introduced, summarizing the
stateoftheart in various problems and highlighting
problems that remain open.

[1464]

Gunther Mäckle, Dragan A. Savic, and Godfrey A. Walters.
Application of Genetic Algorithms to Pump Scheduling for Water
Supply.
In Genetic Algorithms in Engineering Systems: Innovations and
Applications, GALESIA'95, volume 414, pages 400–405, Sheffield, UK,
September 1995. IEE Conference Publication.
[ bib 
http ]
A simple Genetic Algorithm has been applied to the
scheduling of multiple pumping units in a water
supply system with the objective of minimising the
overall cost of the pumping operation, taking
advantage of storage capacity in the system and the
availability of off peak electricity tariffs. A
simple example shows that the method is easy to
apply and has produced encouraging preliminary
results

[1465]

Nateri K. Madavan.
Multiobjective optimization using a Pareto differential
evolution approach.
In D. B. Fogel et al., editors, Proceedings of the 2002 World
Congress on Computational Intelligence (WCCI 2002), pages 1145–1150,
Piscataway, NJ, 2002. IEEE Press.
[ bib ]

[1466]

Sam Madden.
From Databases to Big Data.
IEEE Internet Computing, 16(3), 2012.
[ bib ]

[1467]

M. Mahdavi, M. Fesanghary, and E. Damangir.
An improved harmony search algorithm for solving optimization
problems.
Applied Mathematics and Computation, 188(2):1567–1579, 2007.
[ bib 
DOI ]
This paper develops an Improved harmony search (IHS)
algorithm for solving optimization problems. IHS employs a
novel method for generating new solution vectors that
enhances accuracy and convergence rate of harmony search (HS)
algorithm. In this paper the impacts of constant parameters
on harmony search algorithm are discussed and a strategy for
tuning these parameters is presented. The IHS algorithm has
been successfully applied to various benchmarking and
standard engineering optimization problems. Numerical results
reveal that the proposed algorithm can find better solutions
when compared to HS and other heuristic or deterministic
methods and is a powerful search algorithm for various
engineering optimization problems.
Keywords: Global optimization, Heuristics, Harmony search algorithm,
Mathematical programming

[1468]

Guilherme B. Mainieri and Débora P. Ronconi.
New heuristics for total tardiness minimization in a flexible
flowshop.
Optimization Letters, pages 1–20, 2012.
[ bib ]

[1469]

D. R. Broad, Graeme C. Dandy, and Holger R. Maier.
A Metamodeling Approach to Water Distribution System
Optimization.
In 6th Annual Symposium on Water Distribution Systems Analysis.
ASCE, June 2004.
[ bib ]

[1470]

Holger R. Maier, Angus R. Simpson, Aaron C. Zecchin, Wai Kuan Foong, Kuang Yeow
Phang, Hsin Yeow Seah, and Chan Lim Tan.
Ant Colony Optimization for Design of Water Distribution
Systems.
Journal of Water Resources Planning and Management, ASCE,
129(3):200–209, May / June 2003.
[ bib ]

[1471]

Yuri Malitsky and Meinolf Sellmann.
Instancespecific algorithm configuration as a method for
nonmodelbased portfolio generation.
In N. Beldiceanu, N. Jussien, and E. Pinson, editors,
Integration of AI and OR Techniques in Contraint Programming for
Combinatorial Optimization Problems, volume 7298 of Lecture Notes in
Computer Science, pages 244–259. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1472]

R. M. Males, R. M. Clark, P. J. Wehrman, and W. E. Gateset.
Algorithm for mixing problems in water systems.
Journal of Hydraulic Engineering, ASCE, 111(2):206–219,
1985.
[ bib ]

[1473]

Yuri Malitsky, Deepak Mehta, Barry O'Sullivan, and Helmut Simonis.
Tuning parameters of large neighborhood search for the machine
reassignment problem.
In C. Gomes and M. Sellmann, editors, Integration of AI and OR
Techniques in Constraint Programming for Combinatorial Optimization Problems,
CPAIOR 2010, volume 7874 of Lecture Notes in Computer Science, pages
176–192. Springer, Heidelberg, 2013.
[ bib 
DOI ]

[1474]

Vittorio Maniezzo.
Exact and Approximate Nondeterministic TreeSearch Procedures
for the Quadratic Assignment Problem.
INFORMS Journal on Computing, 11(4):358–369, 1999.
[ bib ]

[1475]

Vittorio Maniezzo, M. Boschetti, and M. Jelasity.
An Ant Approach to Membership Overlay Design.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 37–48. Springer, Heidelberg, 2004.
[ bib ]

[1476]

Vittorio Maniezzo and A. Carbonaro.
An ANTS Heuristic for the Frequency Assignment Problem.
Future Generation Computer Systems, 16(8):927–935, 2000.
[ bib ]

[1477]

Vittorio Maniezzo and Alberto Colorni.
The Ant System Applied to the Quadratic Assignment Problem.
IEEE Transactions on Knowledge and Data Engineering,
11(5):769–778, 1999.
[ bib ]

[1478]

Vittorio Maniezzo and M. Milandri.
An AntBased Framework for Very Strongly Constrained Problems.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 222–227. Springer, Heidelberg, 2002.
[ bib ]

[1479]

Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel,
Steven J. Bethard, and David McClosky.
The Stanford CoreNLP Natural Language Processing Toolkit.
In Association for Computational Linguistics (ACL) System
Demonstrations, pages 55–60, 2014.
[ bib ]
http://www.aclweb.org/anthology/P/P14/P145010

[1480]

E. Q. V. Martins.
On a multicritera shortest path problem.
European Journal of Operational Research, 16:236–245, 1984.
[ bib ]

[1481]

R. T. Marler and J. S. Arora.
Survey of multiobjective optimization methods for engineering.
Structural and Multidisciplinary Optimization, 26(6):369–395,
April 2004.
[ bib 
DOI ]

[1482]

F. Martínez, V. Bou, V. Hernández, F. Alvarruiz, and J. M. Alonso.
ANN Architectures for Simulating Water Distribution Networks.
In D. A. Savic, G. A. Walters, R. King, and S. ThiamKhu, editors,
Proceedings of the Eighth International Conference on Computing and
Control for the Water Industry (CCWI 2005), volume 1, pages 251–256,
University of Exeter, UK, September 2005.
[ bib ]

[1483]

D. Martens, M. De Backer, R. Haesen, J. Vanthienen, M. Snoeck, and B. Baesens.
Classification With Ant Colony Optimization.
IEEE Transactions on Evolutionary Computation, 11(5):651–665,
2007.
[ bib ]

[1484]

MarieEléonore Marmion, Clarisse Dhaenens, Laetitia Jourdan, Arnaud
Liefooghe, and Sébastien Verel.
NILS: A NeutralityBased Iterated Local Search and Its
Application to Flowshop Scheduling.
In P. Merz and J.K. Hao, editors, Proceedings of EvoCOP 2011 –
11th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 6622 of Lecture Notes in Computer Science, pages
191–202. Springer, Heidelberg, 2011.
[ bib ]

[1485]

Hugues Marchand, Alexander Martin, Robert Weismantel, and Laurence Wolsey.
Cutting planes in integer and mixed integer programming.
Discrete Applied Mathematics, 123(1–3):397–446, 2002.
[ bib ]

[1486]

MarieEléonore Marmion, Franco Mascia, Manuel LópezIbáñez, and
Thomas Stützle.
Automatic Design of Hybrid Stochastic Local Search Algorithms.
In M. J. Blesa, C. Blum, P. Festa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics, volume 7919 of Lecture Notes in Computer
Science, pages 144–158. Springer, Heidelberg, 2013.
[ bib 
DOI ]

[1487]

Oded Maron and Andrew W. Moore.
Hoeffding races: Accelerating model selection search for
classification and function approximation.
In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Advances
in Neural Information Processing Systems, volume 6, pages 59–66. Morgan
Kaufmann Publishers, San Francisco, CA, 1994.
[ bib ]

[1488]

O. Maron and A. W. Moore.
The Racing Algorithm: Model Selection for Lazy Learners.
Artificial Intelligence Research, 11(1–5):193–225, 1997.
[ bib ]

[1489]

C. E. Mariano and E. Morales.
MOAQ: An AntQ Algorithm for Multiple Objective
Optimization Problems.
In W. Banzhaf, J. M. Daida, A. E. Eiben, M. H. Garzon, V. Honavar,
M. J. Jakiela, and R. E. Smith, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 1999, pages 894–901. Morgan
Kaufmann Publishers, San Francisco, CA, 1999.
[ bib ]

[1490]

Olivier Martin and S. W. Otto.
Partitioning of Unstructured Meshes for Load Balancing.
Concurrency: Practice and Experience, 7(4):303–314, 1995.
[ bib ]

[1491]

Olivier Martin and S. W. Otto.
Combining Simulated Annealing with Local Search Heuristics.
Annals of Operations Research, 63:57–75, 1996.
[ bib ]

[1492]

Olivier Martin, S. W. Otto, and E. W. Felten.
LargeStep Markov Chains for the Traveling Salesman Problem.
Complex Systems, 5(3):299–326, 1991.
[ bib ]

[1493]

Olivier Martin, S. W. Otto, and E. W. Felten.
Largestep Markov Chains for the TSP Incorporating Local
Search Heuristics.
Operations Research Letters, 11(4):219–224, 1992.
[ bib ]

[1494]

Rafael Martí, Gerhard Reinelt, and Abraham Duarte.
A Benchmark Library and a Comparison of Heuristic Methods for
the Linear Ordering Problem.
Computational Optimization and Applications, 51(3):1297–1317,
2012.
[ bib ]

[1495]

Elena Marchiori and Adri G. Steenbeek.
An Iterated Heuristic Algorithm for the Set Covering Problem.
In K. Mehlhorn, editor, Algorithm Engineering, 2nd International
Workshop, WAE'92, pages 155–166. MaxPlanckInstitut für
Informatik, Saarbrücken, Germany, 1998.
[ bib ]

[1496]

Elena Marchiori and Adri G. Steenbeek.
An Evolutionary Algorithm for Large Scale Set Covering Problems
with Application to Airline Crew Scheduling.
In S. Cagnoni et al., editors, RealWorld Applications of
Evolutionary Computing, EvoWorkshops 2000, volume 1803 of Lecture Notes
in Computer Science, pages 367–381, Heidelberg, 2000. Springer.
[ bib ]

[1497]

K. Marriott and P. Stuckey.
Programming With Constraints.
MIT Press, Cambridge, MA, 1998.
[ bib ]

[1498]

Silvano Martello and Paolo Toth.
Lower bounds and reduction procedures for the bin packing
problem.
Discrete Applied Mathematics, 28(1):59–70, 1990.
[ bib 
DOI ]

[1499]

Silvano Martello and Paolo Toth.
Knapsack Problems: Algorithms and Computer Implementations.
John Wiley & Sons, Chichester, UK, 1990.
[ bib ]
Keywords: bin packing

[1500]

Silvano Martello and Daniele Vigo.
Exact solution of the twodimensional finite bin packing
problem.
Management Science, 44(3):388–399, 1998.
[ bib 
DOI ]

[1501]

Oded Maron.
Hoeffding Races: Model selection for MRI classification.
Master's thesis, Massachusetts Institute of Technology, 1994.
[ bib ]

[1502]

Franco Mascia, Mauro Birattari, and Thomas Stützle.
Tuning Algorithms for Tackling Large Instances: An Experimental
Protocol.
In P. M. Pardalos and G. Nicosia, editors, Learning and
Intelligent Optimization, 7th International Conference, LION 7, volume 7997
of Lecture Notes in Computer Science, pages 410–422. Springer,
Heidelberg, 2013.
[ bib 
DOI ]

[1503]

Florence Massen, Yves Deville, and Pascal van Hentenryck.
PheromoneBased Heuristic Column Generation for Vehicle Routing
Problems with Black Box Feasibility.
In N. Beldiceanu, N. Jussien, and E. Pinson, editors,
Integration of AI and OR Techniques in Contraint Programming for
Combinatorial Optimization Problems, volume 7298 of Lecture Notes in
Computer Science, pages 260–274. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1504]

Franco Mascia, Manuel LópezIbáñez, Jérémie DuboisLacoste,
and Thomas Stützle.
Grammarbased generation of stochastic local search heuristics
through automatic algorithm configuration tools: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2013009/, 2013.
[ bib ]

[1505]

Franco Mascia, Manuel LópezIbáñez, Jérémie DuboisLacoste,
MarieEléonore Marmion, and Thomas Stützle.
Algorithm Comparison by Automatically Configurable Stochastic
Local Search Frameworks: A Case Study Using FlowShop Scheduling Problems.
In M. J. Blesa, C. Blum, and S. Voß, editors, Hybrid
Metaheuristics, volume 8457 of Lecture Notes in Computer Science,
pages 30–44. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1506]

Franco Mascia, Manuel LópezIbáñez, Jérémie DuboisLacoste,
and Thomas Stützle.
From Grammars to Parameters: Automatic Iterated Greedy Design
for the Permutation Flowshop Problem with Weighted Tardiness.
In P. M. Pardalos and G. Nicosia, editors, Learning and
Intelligent Optimization, 7th International Conference, LION 7, volume 7997
of Lecture Notes in Computer Science, pages 321–334. Springer,
Heidelberg, 2013.
[ bib 
DOI ]

[1507]

Franco Mascia, Manuel LópezIbáñez, Jérémie DuboisLacoste,
and Thomas Stützle.
GrammarBased Generation of Stochastic Local Search Heuristics
through Automatic Algorithm Configuration Tools.
Computers & Operations Research, 51:190–199, 2014.
[ bib 
DOI ]

[1508]

Florence Massen, Manuel LópezIbáñez, Thomas Stützle, and Yves
Deville.
Experimental Analysis of PheromoneBased Heuristic Column
Generation Using irace.
In M. J. Blesa, C. Blum, P. Festa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics, volume 7919 of Lecture Notes in Computer
Science, pages 92–106. Springer, Heidelberg, 2013.
[ bib 
DOI ]

[1509]

Renzo Massobrio, Sergio Nesmachnow, and Bernabé Dorronsoro.
Virtual Savant: learning for optimization.
In M. Vlastelica, J. Song, A. Ferber, B. Amos, G. Martius,
B. Dilkina, and Y. Yue, editors, Learning Meets Combinatorial Algorithms
Workshop at NeurIPS 2020, LMCA 2020, Vancouver, Canada, December 12, 2020,
pages 1–5, 2020.
[ bib ]

[1510]

Franco Mascia, Paola Pellegrini, Thomas Stützle, and Mauro Birattari.
An Analysis of Parameter Adaptation in Reactive Tabu Search.
International Transactions in Operational Research,
21(1):127–152, 2014.
[ bib ]

[1511]

Renaud Masson, Thibaut Vidal, Julien Michallet, Puca Huachi Vaz Penna,
Vinicius Petrucci, Anand Subramanian, and Hugues Dubedout.
An Iterated Local Search Heuristic for Multicapacity Bin
Packing and Machine Reassignment Problems.
Expert Systems with Applications, 40(13):5266–5275, 2013.
[ bib ]

[1512]

Yazid Mati, Stéphane DauzèrePèrés, and Chams Lahlou.
A General Approach for Optimizing Regular Criteria in the
Jobshop Scheduling Problem.
European Journal of Operational Research, 212(1):33–42, 2011.
[ bib ]

[1513]

Michael Maur, Manuel LópezIbáñez, and Thomas Stützle.
Prescheduled and adaptive parameter variation in
MaxMin Ant System.
In H. Ishibuchi et al., editors, Proceedings of the 2010
Congress on Evolutionary Computation (CEC 2010), pages 3823–3830. IEEE
Press, Piscataway, NJ, 2010.
[ bib 
DOI ]

[1514]

Atanu Mazumdar, Tinkle Chugh, Kaisa Miettinen, and Manuel
LópezIbáñez.
On Dealing with Uncertainties from Kriging Models in Offline
DataDriven Evolutionary Multiobjective Optimization.
In K. Deb, E. D. Goodman, C. A. Coello Coello, K. Klamroth,
K. Miettinen, S. Mostaghim, and P. Reed, editors, Evolutionary
Multicriterion Optimization, EMO 2019, volume 11411 of Lecture Notes
in Computer Science, pages 463–474. Springer International Publishing,
Cham, Switzerland, 2019.
[ bib 
DOI ]

[1515]

Ross M. McConnell, Kurt Mehlhorn, Stefan Näher, and Pascal Schweitzer.
Certifying algorithms.
Computer Science Review, 5(2):119–161, 2011.
[ bib 
DOI ]
A certifying algorithm is an algorithm that produces, with
each output, a certificate or witness (easytoverify proof)
that the particular output has not been compromised by a
bug. A user of a certifying algorithm inputs x, receives the
output y and the certificate w, and then checks, either
manually or by use of a program, that w proves that y is a
correct output for input x. In this way, he/she can be sure
of the correctness of the output without having to trust the
algorithm. We put forward the thesis that certifying
algorithms are much superior to noncertifying algorithms,
and that for complex algorithmic tasks, only certifying
algorithms are satisfactory. Acceptance of this thesis would
lead to a change of how algorithms are taught and how
algorithms are researched. The widespread use of certifying
algorithms would greatly enhance the reliability of
algorithmic software. We survey the state of the art in
certifying algorithms and add to it. In particular, we start
a theory of certifying algorithms and prove that the concept
is universal.
Keywords: Algorithms, Software reliability, Certification

[1516]

G. McCormick and R. S. Powell.
Optimal Pump Scheduling in Water Supply Systems with Maximum
Demand Charges.
Journal of Water Resources Planning and Management, ASCE,
129(5):372–379, 2003.
[ bib 
DOI 
eprint ]
Keywords: water supply; pumps; Markov processes; cost optimal
control

[1517]

G. McCormick and R. S. Powell.
A progressive mixed integerprogramming method for pump
scheduling.
In C. Maksimović, D. Butler, and F. A. Memon, editors,
Advances in Water Supply Management, pages 307–313. CRC Press, 2003.
[ bib ]

[1518]

G. McCormick and R. S. Powell.
Derivation of nearoptimal pump schedules for water distribution
by simulated annealing.
Journal of the Operational Research Society, 55(7):728–736,
July 2004.
[ bib 
DOI ]
The scheduling of pumps for clean water distribution
is a partially discrete nonlinear problem with many
variables. The scheduling method described in this
paper typically produces costs within 1% of a
linear programbased solution, and can incorporate
realistic nonlinear costs that may be hard to
incorporate in linear programming
formulations. These costs include pump switching and
maximum demand charges. A simplified model is
derived from a standard hydraulic simulator. An
initial schedule is produced by a descent
method. Twostage simulated annealing then produces
solutions in a few minutes. Iterative recalibration
ensures that the solution agrees closely with the
results from a full hydraulic simulation.

[1519]

James McDermott.
When and Why Metaheuristics Researchers can Ignore "No Free
Lunch" Theorems.
SN Computer Science, 1(60):1–18, 2020.
[ bib 
DOI ]

[1520]

Catherine C. McGeoch.
Analyzing Algorithms by Simulation: Variance Reduction
Techniques and Simulation Speedups.
ACM Computing Surveys, 24(2):195–212, 1992.
[ bib 
DOI ]
Although experimental studies have been widely applied to the
investigation of algorithm performance, very little attention
has been given to experimental method in this area. This is
unfortunate, since much can be done to improve the quality of
the data obtained; often, much improvement may be needed for
the data to be useful. This paper gives a tutorial discussion
of two aspects of good experimental technique: the use of
variance reduction techniques and simulation speedups in
algorithm studies. In an illustrative study, application of
variance reduction techniques produces a decrease in variance
by a factor 1000 in one case, giving a dramatic improvement
in the precision of experimental results. Furthermore, the
complexity of the simulation program is improved from
Θ(m n/H_{n}) to Θ(m + n logn) (where m is
typically much larger than n), giving a much faster
simulation program and therefore more data per unit of
computation time. The general application of variance
reduction techniques is also discussed for a variety of
algorithm problem domains.
Keywords: experimental analysis of algorithms, movetofront rule,
selforganizing sequential search, statistical analysis of
algorithms, transpose rule, variance reduction techniques

[1521]

Catherine C. McGeoch.
Toward an Experimental Method for Algorithm Simulation.
INFORMS Journal on Computing, 8(1):1–15, 1996.
[ bib 
DOI ]

[1522]

Catherine C. McGeoch.
A Guide to Experimental Algorithmics.
Cambridge University Press, 2012.
[ bib ]

[1523]

Michael D. McKay, Richard J. Beckman, and W. J. Conover.
A Comparison of Three Methods for Selecting Values of Input
Variables in the Analysis of Output from a Computer Code.
Technometrics, 21(2):239–245, 1979.
[ bib 
DOI ]
Two types of sampling plans are examined as alternatives to
simple random sampling in Monte Carlo studies. These plans
are shown to be improvements over simple random sampling with
respect to variance for a class of estimators which includes
the sample mean and the empirical distribution function.

[1524]

Russell McKenna, Valentin Bertsch, Kai Mainzer, and Wolf Fichtner.
Combining local preferences with multicriteria decision
analysis and linear optimization to develop feasible energy concepts in small
communities.
European Journal of Operational Research, 268(3):1092–1110,
2018.
[ bib ]

[1525]

Robert I. Mckay, Nguyen Xuan Hoai, Peter Alexander Whigham, Yin Shan, and
Michael O'Neill.
Grammarbased Genetic Programming: A Survey.
Genetic Programming and Evolvable Machines, 11(34):365–396,
September 2010.
[ bib 
DOI ]

[1526]

Klaus Meer.
Simulated annealing versus Metropolis for a TSP instance.
Information Processing Letters, 104(6):216–219, 2007.
[ bib ]

[1527]

J. Fabian Meier and Uwe Clausen.
A versatile heuristic approach for generalized hub location
problems.
Preprint, Provided upon personal request, 2014.
[ bib ]
Keywords: irace

[1528]

Gábor Melis, Chris Dyer, and Phil Blunsom.
On the State of the Art of Evaluation in Neural Language
Models.
Arxiv preprint arXiv:1807.02811, 2017.
[ bib 
http ]

[1529]

L. Melo, F. Pereira, and E. Costa.
MCANT: a Multicolony Ant Algorithm.
In P. Collet, N. Monmarché, P. Legrand, M. Schoenauer, and
E. Lutton, editors, Artificial Evolution: 9th International Conference,
Evolution Artificielle, EA, 2009, volume 5975 of Lecture Notes in
Computer Science. Springer, Heidelberg, 2010.
[ bib ]

[1530]

Ole J. Mengshoel.
Understanding the role of noise in stochastic local search:
Analysis and experiments.
Artificial Intelligence, 172(8):955–990, 2008.
[ bib ]

[1531]

Adriana MenchacaMendez and Carlos A. Coello Coello.
GDMOEA: A New MultiObjective Evolutionary Algorithm Based on
the Generational Distance Indicator.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pages 156–170.
Springer, Heidelberg, 2015.
[ bib ]

[1532]

Adriana MenchacaMendez and Carlos A. Coello Coello.
GDEMOEA: A New MOEA based on the generational distance
indicator and εdominance.
In Proceedings of the 2015 Congress on Evolutionary Computation
(CEC 2015), pages 947–955, Piscataway, NJ, 2015. IEEE Press.
[ bib ]

[1533]

Hector Mendoza, Aaron Klein, Matthias Feurer, Jost Tobias Springenberg, and
Frank Hutter.
Towards automaticallytuned neural networks.
In Workshop on Automatic Machine Learning, pages 58–65, 2016.
[ bib ]

[1534]

Olaf Mersmann, Bernd Bischl, Heike Trautmann, Mike Preuss, Claus Weihs, and
Günther Rudolph.
Exploratory Landscape Analysis.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pages 829–836.
ACM Press, New York, NY, 2011.
[ bib ]
Keywords: continuous optimization, landscape analysis, instance features

[1535]

JuanJulián Merelo and Carlos Cotta.
Building bridges: the role of subfields in metaheuristics.
SIGEVOlution, 1(4):9–15, 2006.
[ bib 
DOI ]

[1536]

Peter Merz and Bernd Freisleben.
Memetic Algorithms for the Traveling Salesman Problem.
Complex Systems, 13(4):297–345, 2001.
[ bib ]

[1537]

Peter Merz and Bernd Freisleben.
Fitness Landscape Analysis and Memetic Algorithms for the
Quadratic Assignment Problem.
IEEE Transactions on Evolutionary Computation, 4(4):337–352,
2000.
[ bib ]

[1538]

Peter Merz and Jutta Huhse.
An Iterated Local Search Approach for Finding Provably Good
Solutions for Very Large TSP Instances.
In G. Rudolph et al., editors, Parallel Problem Solving from
Nature, PPSN X, volume 5199 of Lecture Notes in Computer Science,
pages 929–939. Springer, Heidelberg, 2008.
[ bib ]

[1539]

Peter Merz and Kengo Katayama.
Memetic algorithms for the unconstrained binary quadratic
programming problem.
BioSystems, 78(1):99–118, 2004.
[ bib 
DOI ]

[1540]

D. Merkle and Martin Middendorf.
Prospects for Dynamic Algorithm Control: Lessons from the Phase
Structure of Ant Scheduling Algorithms.
In R. B. Heckendorn, editor, Proceedings of the 2001 Genetic and
Evolutionary Computation Conference – Workshop Program. Workshop “The Next
Ten Years of Scheduling Research”, pages 121–126. Morgan Kaufmann
Publishers, San Francisco, CA, 2001.
[ bib ]

[1541]

D. Merkle and Martin Middendorf.
Ant Colony Optimization with Global Pheromone Evaluation for
Scheduling a Single Machine.
Applied Intelligence, 18(1):105–111, 2003.
[ bib ]

[1542]

D. Merkle and Martin Middendorf.
Modeling the Dynamics of Ant Colony Optimization.
Evolutionary Computation, 10(3):235–262, 2002.
[ bib ]

[1543]

D. Merkle, Martin Middendorf, and Hartmut Schmeck.
Ant Colony Optimization for ResourceConstrained Project
Scheduling.
In D. Whitley et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2000, pages 893–900. Morgan
Kaufmann Publishers, San Francisco, CA, 2000.
[ bib ]

[1544]

D. Merkle, Martin Middendorf, and Hartmut Schmeck.
Ant Colony Optimization for ResourceConstrained Project
Scheduling.
IEEE Transactions on Evolutionary Computation, 6(4):333–346,
2002.
[ bib ]

[1545]

Olaf Mersmann, Heike Trautmann, Boris Naujoks, and Claus Weihs.
Benchmarking Evolutionary Multiobjective Optimization
Algorithms.
In H. Ishibuchi et al., editors, Proceedings of the 2010
Congress on Evolutionary Computation (CEC 2010), pages 1–8, Piscataway, NJ,
2010. IEEE Press.
[ bib ]
TR: http://hdl.handle.net/2003/26671

[1546]

Peter Merz and Bernd Freisleben.
Greedy and Local Search Heuristics for Unconstrained Binary
Quadratic Programming.
Journal of Heuristics, 8(2):197–213, 2002.
[ bib 
DOI ]

[1547]

Rafael G. Mesquita, Ricardo M. A. Silva, Carlos A. B. Mello, and Péricles
B. C. Miranda.
Parameter tuning for document image binarization using a racing
algorithm.
Expert Systems with Applications, 42(5):2593–2603, 2015.
[ bib 
DOI ]
Keywords: irace

[1548]

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. Teller, and E. Teller.
Equation of State Calculations by Fast Computing Machines.
Journal of Chemical Physics, 21:1087–1092, 1953.
[ bib ]

[1549]

Nicolas Meuleau and Marco Dorigo.
Ant Colony Optimization and Stochastic Gradient Descent.
Artificial Life, 8(2):103–121, 2002.
[ bib ]

[1550]

Bernd Meyer.
Convergence control in ACO.
In Genetic and Evolutionary Computation Conference (GECCO),
Seattle, WA, 2004.
Latebreaking paper available on CD.
[ bib ]

[1551]

Bernd Meyer and Andreas T. Ernst.
Integrating ACO and Constraint Propagation.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 166–177. Springer, Heidelberg,
2004.
[ bib ]

[1552]

Efrén MezuraMontes, M. ReyesSierra, and Carlos A. Coello Coello.
Multiobjective optimization using differential evolution: a
survey of the stateoftheart.
In U. K. Chakraborty, editor, Advances in differential
evolution, pages 173–196. Springer, Heidelberg, 2008.
[ bib 
DOI ]

[1553]

Efrén MezuraMontes, Jesús VelázquezReyes, and Carlos A. Coello
Coello.
A comparative study of differential evolution variants for
global optimization.
In M. Cattolico et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2006, pages 485–492. ACM Press,
New York, NY, 2006.
[ bib 
DOI ]

[1554]

R. M'Hallah.
An iterated local search variable neighborhood descent hybrid
heuristic for the total earliness tardiness permutation flow shop.
International Journal of Production Research,
52(13):3802–3819, 2014.
[ bib ]

[1555]

Zbigniew Michalewicz, Dipankar Dasgupta, Rodolphe G. Le Riche, and Marc
Schoenauer.
Evolutionary algorithms for constrained engineering problems.
Computers and Industrial Engineering, 30(4):851–870, 1996.
[ bib 
DOI ]

[1556]

Zbigniew Michalewicz and David B. Fogel.
How to Solve It: Modern Heuristics.
Springer, 2nd edition, 2004.
[ bib ]

[1557]

Laurent D. Michel and Pascal van Hentenryck.
Iterative Relaxations for Iterative Flattening in Cumulative
Scheduling.
In S. Zilberstein, J. Koehler, and S. Koenig, editors,
Proceedings of the Fourteenth International Conference on Automated Planning
and Scheduling (ICAPS 2004), pages 200–208. AAAI Press/MIT Press,
Menlo Park, CA, 2004.
[ bib ]

[1558]

R. Michel and M. Middendorf.
An Island Model based Ant System with Lookahead for the
Shortest Supersequence Problem.
In A. E. Eiben, T. Bäck, M. Schoenauer, and H.P. Schwefel,
editors, Parallel Problem Solving from Nature – PPSN V, volume 1498 of
Lecture Notes in Computer Science, pages 692–701. Springer,
Heidelberg, 1998.
[ bib ]

[1559]

Julien Michallet, Christian Prins, Farouk Yalaoui, and Grégoire Vitry.
Multistart Iterated Local Search for the Periodic Vehicle
Routing Problem with Time Windows and Time Spread Constraints on Services.
Computers & Operations Research, 41:196–207, 2014.
[ bib ]

[1560]

Zbigniew Michalewicz.
Genetic Algorithms + Data Structures = Evolution Programs.
Springer, Berlin, Germany, 3rd edition, 1996.
[ bib ]

[1561]

Kaisa Miettinen.
Survey of methods to visualize alternatives in multiple criteria
decision making problems.
OR Spectrum, 36(1):3–37, 2014.
[ bib ]

[1562]

Kaisa Miettinen.
Nonlinear Multiobjective Optimization.
Kluwer Academic Publishers, Boston, MA, 1999.
[ bib ]
Nonlinear Multiobjective Optimization provides an extensive,
uptodate, selfcontained and consistent survey and review
of the literature and of the state of the art on nonlinear
(deterministic) multiobjective optimization, its methods, its
theory and its background. This book is intended for both
researchers and students in the areas of (applied)
mathematics, engineering, economics, operations research and
management science; it is meant for both professionals and
practitioners in many different fields of application. The
intention is to provide a consistent summary that may help in
selecting an appropriate method for the problem to be
solved. The extensive bibliography will be of value to
researchers.

[1563]

Kaisa Miettinen, Jyri Mustajoki, and T. J. Stewart.
Interactive multiobjective optimization with NIMBUS for
decision making under uncertainty.
OR Spectrum, 36(1):39–56, 2014.
[ bib ]

[1564]

Kaisa Miettinen, Francisco Ruiz, and Andrzej Wierzbicki.
Introduction to Multiobjective Optimization: Interactive
Approaches.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science. Springer, Heidelberg,
2008.
[ bib 
DOI ]
We give an overview of interactive methods developed
for solving nonlinear multiobjective optimization
problems. In interactive methods, a decision maker
plays an important part and the idea is to support
her/him in the search for the most preferred
solution. In interactive methods, steps of an
iterative solution algorithm are repeated and the
decision maker progressively provides preference
information so that the most preferred solution can
be found. We identify three types of specifying
preference information in interactive methods and
give some examples of methods representing each
type. The types are methods based on tradeoff
information, reference points and classification of
objective functions.

[1565]

R. B. Millar and M. J. Anderson.
Remedies for pseudoreplication.
Fisheries Research, 70(2–3):397–407, 2004.
[ bib 
DOI ]

[1566]

George A. Miller.
The magical number seven, plus or minus two: Some limits on our
capacity for processing information.
Psychological Review, 63(2):81–97, 1956.
[ bib 
DOI ]

[1567]

Steven Minton.
Automatically configuring constraint satisfaction programs: A
case study.
Constraints, 1(1):7–43, 1996.
[ bib 
DOI ]

[1568]

Gerardo Minella, Rubén Ruiz, and M. Ciavotta.
A Review and Evaluation of Multiobjective Algorithms for the
Flowshop Scheduling Problem.
INFORMS Journal on Computing, 20(3):451–471, 2008.
[ bib ]

[1569]

Péricles Miranda, Ricardo M. Silva, and Ricardo B. Prudêncio.
FineTuning of Support Vector Machine Parameters Using Racing
Algorithms.
In European Symposium on Artificial Neural Networks, ESSAN,
pages 325–330, 2014.
[ bib ]
https://www.elen.ucl.ac.be/esann/proceedings/papers.php?ann=2014
Keywords: irace

[1570]

Péricles Miranda, Ricardo M. Silva, and Ricardo B. Prudêncio.
I/SRace: An Iterative Multiobjective Racing Algorithm for
the SVM Parameter Selection Problem.
In European Symposium on Artificial Neural Networks, ESSAN,
pages 573–578, 2015.
[ bib ]
https://www.elen.ucl.ac.be/esann/proceedings/papers.php?ann=2015

[1571]

Alfonsas Misevičius and Dovilė Kuznecovaitė.
Investigating some strategies for construction of initial
populations in genetic algorithms.
Computational Science and Techniques, 5(1):560–573, 2018.
[ bib ]

[1572]

Alfonsas Misevičius.
Genetic Algorithm Hybridized with Ruin and Recreate Procedure:
Application to the Quadratic Assignment Problem.
KnowledgeBased Systems, 16(5–6):261–268, 2003.
[ bib ]

[1573]

Alfonsas Misevičius.
Ruin and Recreate Principle Based Approach for the Quadratic
Assignment Problem.
In E. CantúPaz et al., editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2003, Part I, volume 2723 of
Lecture Notes in Computer Science, pages 598–609, Heidelberg, 2003.
Springer.
[ bib ]

[1574]

Alfonsas Misevičius.
A modified simulated annealing algorithm for the quadratic
assignment problem.
Informatica, 14(4):497–514, 2003.
[ bib ]

[1575]

Debasis Mitra, Fabio Romeo, and Alberto SangiovanniVincentelli.
Convergence and FiniteTime Behavior of Simulated Annealing.
In Decision and Control, 1985 24th IEEE Conference on, pages
761–767. IEEE, 1985.
[ bib ]

[1576]

David G. Mitchell, Bart Selman, and Hector J. Levesque.
Hard and Easy Distributions of SAT Problems.
In W. R. Swartout, editor, Proceedings of the 10th National
Conference on Artificial Intelligence, pages 459–465. AAAI Press/
MIT Press, Menlo Park, CA, 1992.
[ bib ]

[1577]

Alfonsas Misevičius, Dovilė Kuznecovaitė, and Juratė
Platužienė.
Some Further Experiments with Crossover Operators for Genetic
Algorithms.
Informatica, 29(3):499–516, 2018.
[ bib ]

[1578]

Nenad Mladenović and Pierre Hansen.
Variable Neighborhood Search.
Computers & Operations Research, 24(11):1097–1100, 1997.
[ bib ]

[1579]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness,
Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland,
Georg Ostrovski, et al.
Humanlevel control through deep reinforcement learning.
Nature, 518(7540):529, 2015.
[ bib ]

[1580]

Volodymyr Mnih, Csaba Szepesvári, and JeanYves Audibert.
Empirical Bernstein stopping.
In W. W. Cohen, A. McCallum, and S. T. Roweis, editors,
Proceedings of the 25th International Conference on Machine Learning, ICML
2008, pages 672–679. ACM Press, New York, NY, 2008.
[ bib ]

[1581]

Jonas Močkus, Vytautas Tiesis, and Antanas Zilinskas.
The application of bayesian methods for seeking the extremum.
Towards global optimization, pages 117–129, 1978.
[ bib ]
Proposed Bayesian optimization

[1582]

Jonas Močkus.
On Bayesian Methods for Seeking the Extremum.
In G. I. Marchuk, editor, Optimization Techniques IFIP Technical
Conference Novosibirsk, July 1–7, 1974, volume 27 of Lecture Notes in
Computer Science, pages 400–404. Springer, Berlin/Heidelberg, 1975.
[ bib 
DOI ]
Proposed Bayesian optimization

[1583]

Jonas Mockus.
Bayesian Approach to Global Optimization: Theory and
Applications.
Kluwer Academic Publishers, 1989.
[ bib ]

[1584]

Atefeh Moghaddam, Farouk Yalaoui, and Lionel Amodeo.
Lorenz versus Pareto Dominance in a Single Machine
Scheduling Problem with Rejection.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pages 520–534. Springer, Heidelberg, 2011.
[ bib ]

[1585]

Teodor Mihai Moldovan and Pieter Abbeel.
Safe Exploration in Markov Decision Processes.
In J. Langford and J. Pineau, editors, Proceedings of the 29th
International Conference on Machine Learning, ICML 2012, pages 1451–1458.
Omnipress, 2012.
[ bib 
eprint ]

[1586]

Marco A. Montes de Oca, Dogan Aydin, and Thomas Stützle.
An Incremental Particle Swarm for LargeScale Continuous
Optimization Problems: An Example of Tuningintheloop (Re)Design of
Optimization Algorithms.
Soft Computing, 15(11):2233–2255, 2011.
[ bib 
DOI ]

[1587]

JeanNoël Monette, Yves Deville, and Pascal van Hentenryck.
Aeon: Synthesizing Scheduling Algorithms from HighLevel
Models.
In J. W. Chinneck, B. Kristjansson, and M. J. Saltzman, editors,
Operations Research and CyberInfrastructure, volume 47 of Operations
Research/Computer Science Interfaces, pages 43–59. Springer, New York, NY,
2009.
[ bib ]

[1588]

Alysson Mondoro, Dan M. Frangopol, and Liang Liu.
Multicriteria robust optimization framework for bridge
adaptation under climate change.
Structural Safety, 74:14–23, 2018.
[ bib ]

[1589]

Roberto Montemanni, L. M. Gambardella, A. E. Rizzoli, and A. V. Donati.
Ant colony system for a dynamic vehicle routing problem.
Journal of Combinatorial Optimization, 10:327–343, 2005.
[ bib ]

[1590]

Elizabeth Montero, Leslie Pérez Cáceres, MaríaCristina Riff, and
Carlos A. Coello Coello.
Are StateoftheArt FineTuning Algorithms Able to Detect a
Dummy Parameter?
In C. A. Coello Coello et al., editors, Parallel Problem
Solving from Nature, PPSN XII, volume 7491 of Lecture Notes in Computer
Science, pages 306–315. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1591]

James Montgomery, Marcus Randall, and Tim Hendtlass.
Solution bias in ant colony optimisation: Lessons for
selecting pheromone models.
Computers & Operations Research, 35(9):2728–2749, 2008.
[ bib 
DOI ]

[1592]

MaríaCristina Riff and Elizabeth Montero.
A new algorithm for reducing metaheuristic design effort.
In Proceedings of the 2013 Congress on Evolutionary Computation
(CEC 2013), pages 3283–3290. IEEE Press, Piscataway, NJ, 2013.
[ bib 
DOI ]

[1593]

Elizabeth Montero and MaríaCristina Riff.
Towards a Method for Automatic Algorithm Configuration: A Design
Evaluation Using Tuners.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, PPSN 2014, volume 8672 of Lecture Notes in Computer
Science, pages 90–99. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1594]

Elizabeth Montero, MaríaCristina Riff, and Bertrand Neveu.
An Evaluation of Offline Calibration Techniques for
Evolutionary Algorithms.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pages 299–300, New
York, NY, 2010. ACM Press.
[ bib ]

[1595]

Elizabeth Montero, MaríaCristina Riff, and Bertrand Neveu.
A Beginner's Buide to Tuning Methods.
Applied Soft Computing, 17:39–51, 2014.
[ bib 
DOI ]

[1596]

Marco A. Montes de Oca, Thomas Stützle, Mauro Birattari, and Marco
Dorigo.
Frankenstein's PSO: A Composite Particle Swarm Optimization
Algorithm.
IEEE Transactions on Evolutionary Computation,
13(5):1120–1132, 2009.
[ bib 
DOI ]

[1597]

Nicolas Monmarché, G. Venturini, and M. Slimane.
On how pachycondyla apicalis ants suggest a new search
algorithm.
Future Generation Computer Systems, 16(8):937–946, 2000.
[ bib ]

[1598]

Gilberto Montibeller and Hugo Yoshizaki.
A Framework for Locating Logistic Facilities with MultiCriteria
Decision Analysis.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pages 505–519. Springer, Heidelberg, 2011.
[ bib ]

[1599]

Marco A. Montes de Oca.
Incremental Social Learning in Swarm Intelligence Systems.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2011.
[ bib ]
Supervised by Marco Dorigo

[1600]

James Montgomery.
Solution Biases and Pheromone Representation Selection in Ant
Colony Optimisation.
PhD thesis, School of Information Technology, Bond University,
Australia, 2005.
[ bib ]

[1601]

Douglas C. Montgomery.
Design and Analysis of Experiments.
John Wiley & Sons, New York, NY, 8th edition, 2012.
[ bib ]

[1602]

Andrew W. Moore and Mary S. Lee.
Efficient Algorithms for Minimizing Cross Validation Error.
In W. W. Cohen and H. Hirsh, editors, Proceedings of the 11th
International Conference on Machine Learning, ICML 1994, pages 190–198,
San Francisco, CA, 1994. Morgan Kaufmann Publishers.
[ bib ]

[1603]

Peter D. Morgan.
Simulation of an adaptive behavior mechanism in an expert
decisionmaker.
IEEE Transactions on Systems, Man, and Cybernetics,
23(1):65–76, 1993.
[ bib ]

[1604]

J. N. Morse.
Reducing the size of the nondominated set: Pruning by
clustering.
Computers & Operations Research, 7(12):55–66, 1980.
[ bib ]

[1605]

Sara Morin, Caroline Gagné, and Marc Gravel.
Ant colony optimization with a specialized pheromone trail for
the carsequencing problem.
European Journal of Operational Research, 197(3):1185–1191,
2009.
[ bib 
DOI ]
This paper studies the learning process in an ant
colony optimization algorithm designed to solve the
problem of ordering cars on an assembly line
(carsequencing problem). This problem has been
shown to be NPhard and evokes a great deal of
interest among practitioners. Learning in an ant
algorithm is achieved by using an artificial
pheromone trail, which is a central element of this
metaheuristic. Many versions of the algorithm are
found in literature, the main distinction among them
being the management of the pheromone
trail. Nevertheless, few of them seek to perfect
learning by modifying the internal structure of the
trail. In this paper, a new pheromone trail
structure is proposed that is specifically adapted
to the type of constraints in the carsequencing
problem. The quality of the results obtained when
solving three sets of benchmark problems is superior
to that of the best solutions found in literature
and shows the efficiency of the specialized trail.
Keywords: Ant colony optimization,Carsequencing
problem,Pheromone trail,Scheduling

[1606]

A. Moraglio and A. Kattan.
Geometric Generalisation of Surrogate Model Based Optimization
to Combinatorial Spaces.
In P. Merz and J.K. Hao, editors, Proceedings of EvoCOP 2011 –
11th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 6622 of Lecture Notes in Computer Science, pages
142–154. Springer, Heidelberg, 2011.
[ bib ]

[1607]

A. Moraglio, YongHyuk Kim, and Yourim Yoon.
Geometric Surrogatebased Optimisation for Permutationbased
Problems.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference Companion, GECCO Companion
2011, pages 133–134. ACM Press, New York, NY, 2011.
[ bib ]

[1608]

A. M. Mora, JuanJulián Merelo, Juan Luis Jiménez Laredo, C. Millan,
and J. Torrecillas.
CHAC, a MOACO algorithm for computation of bicriteria
military unit path in the battlefield: Presentation and first results.
International Journal of Intelligent Systems, 24(7):818–843,
2009.
[ bib ]

[1609]

Max D. Morris and Toby J. Mitchell.
Exploratory designs for computational experiments.
Journal of Statistical Planning and Inference, 43(3):381–402,
1995.
[ bib 
DOI ]
Keywords: Bayesian prediction

[1610]

Pail Morris.
The Breakout Method for Escaping from Local Minima.
In R. Fikes and W. G. Lehnert, editors, Proceedings of the 11th
National Conference on Artificial Intelligence, pages 40–45. AAAI
Press/MIT Press, Menlo Park, CA, 1993.
[ bib ]

[1611]

Pablo Moscato and José F. Fontanari.
Stochastic Versus Deterministic Update in Simulated Annealing.
Physics Letters A, 146(4):204–208, 1990.
[ bib ]

[1612]

J. D. Moss and C. G. Johnson.
An ant colony algorithm for multiple sequence alignment in
bioinformatics.
In D. W. Pearson, N. C. Steele, and R. F. Albrecht, editors,
Artificial Neural Networks and Genetic Algorithms, pages 182–186. Springer
Verlag, 2003.
[ bib ]

[1613]

Pablo Moscato.
On Evolution, Search, Optimization, Genetic Algorithms and
Martial Arts: Towards Memetic Algorithms.
Caltech Concurrent Computation Program, C3P Report 826, Caltech,
1989.
[ bib ]

[1614]

Pablo Moscato.
Memetic algorithms: a short introduction.
In D. Corne, M. Dorigo, and F. Glover, editors, New Ideas in
Optimization, pages 219–234. McGraw Hill, London, UK, 1999.
[ bib ]

[1615]

J. Mote, D. L. Olson, and M. A. Venkataramanan.
A comparative multiobjective programming study.
Mathematical and Computer Modelling, 10(10):719–729, 1988.
[ bib 
DOI ]
The purpose of this study was to systematically evaluate a
number of multiobjective programming concepts relative to
reflection of utility, assurance of nondominated solutions
and practicality for larger problems using conventional
software. In the problem used, the nonlinear simulated DM
utility function applied resulted in a nonextreme point
solution. Very often, the preferred solution could end up
being an extreme point solution, in which case the techniques
relying upon LP concepts would work as well if not better
than utilizing constrained objective attainments. The point
is that there is no reason to expect linear or near linear
utility.
Keywords: artificial DM, interactive

[1616]

Vincent Mousseau.
Elicitation des préférences pour l'aide multicritère
à la décision.
PhD thesis, Université ParisDauphine, Paris, France, 2003.
[ bib ]

[1617]

Sébastien Mouthuy, Yves Deville, and Pascal van Hentenryck.
Constraintbased Very LargeScale Neighborhood Search.
Constraints, 17(2):87–122, 2012.
[ bib 
DOI ]

[1618]

Lucien Mousin, MarieEléonore Kessaci, and Clarisse Dhaenens.
Exploiting Promising SubSequences of Jobs to solve the NoWait
Flowshop Scheduling Problem.
Arxiv preprint arXiv:1903.09035, 2019.
[ bib 
http ]

[1619]

Vincent Mousseau and Roman Slowiński.
Inferring an ELECTRE TRI model from assignment examples.
Journal of Global Optimization, 12(2):157–174, 1998.
[ bib ]

[1620]

J. Moy.
RFC 1583: Open shortest path first protocol, 1994.
[ bib ]

[1621]

Zongxu Mu, Jérémie DuboisLacoste, Holger H. Hoos, and Thomas
Stützle.
On the Empirical Scaling of Running Time for Finding Optimal
Solutions to the TSP: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2017010/, 2017.
[ bib ]

[1622]

Zongxu Mu, Holger H. Hoos, and Thomas Stützle.
The Impact of Automated Algorithm Configuration on the Scaling
Behaviour of StateoftheArt Inexact TSP Solvers.
In P. Festa, M. Sellmann, and J. Vanschoren, editors, Learning
and Intelligent Optimization, 10th International Conference, LION 10, volume
10079 of Lecture Notes in Computer Science, pages 157–172. Springer,
Cham, Switzerland, 2016.
[ bib 
DOI ]

[1623]

Mudita Sharma, Manuel LópezIbáñez, and Dimitar Kazakov.
Deep Reinforcement Learning Based Parameter Control in
Differential Evolution: Supplementary material.
https://github.com/mudita11/DEDDQN, 2019.
[ bib 
DOI ]

[1624]

Christian L. Müller and Ivos F. Sbalzarini.
Energy Landscapes of Atomic Clusters as Black Box Optimization
Benchmarks.
Evolutionary Computation, 20(4):543–573, 2012.
[ bib 
DOI ]

[1625]

H. Mühlenbein and Jörg Zimmermann.
Size of neighborhood more important than temperature for
stochastic local search.
In Proceedings of the 2000 Congress on Evolutionary Computation
(CEC'00), pages 1017–1024, Piscataway, NJ, July 2000. IEEE Press.
[ bib ]

[1626]

H. Mühlenbein.
Evolution in Time and Space—The Parallel Genetic Algorithm.
In G. Rawlins, editor, Foundations of Genetic Algorithms
(FOGA), pages 316–337. Morgan Kaufmann Publishers, San Mateo, CA, 1991.
[ bib ]

[1627]

H. Mühlenbein and D. SchlierkampVoosen.
Predictive models for the breeder genetic algorithm.
Evolutionary Computation, 1(1):25–49, 1993.
[ bib ]
Keywords: crossover, intermediate, line

[1628]

Moritz Mühlenthaler.
Fairness in Academic Course Timetabling.
Springer, 2015.
[ bib 
DOI ]
Keywords: irace

[1629]

Mario A. Muñoz and Kate SmithMiles.
Generating New SpaceFilling Test Instances for Continuous
BlackBox Optimization.
Evolutionary Computation, 28(3):379–404, September 2020.
[ bib 
DOI ]

[1630]

Mario A. Muñoz, Yuan Sun, Michael Kirley, and Saman K. Halgamuge.
Algorithm selection for blackbox continuous optimization
problems: a survey on methods and challenges.
Information Sciences, 317:224–245, 2015.
[ bib ]

[1631]

L. J. Murphy, Graeme C. Dandy, and Angus R. Simpson.
Optimum Design and Operation of Pumped Water Distribution
Systems.
In 1994 International Conference on Hydraulics and Civil
Engineering, Hidraulic working with the Environment, pages 149–155,
Brisbane, Australia, February 1994. The Institution of Engineers.
[ bib ]

[1632]

Yuichi Nagata and Shigenobu Kobayashi.
Edge Assembly Crossover: A Highpower Genetic Algorithm for the
Traveling Salesman Problem.
In T. Bäck, editor, ICGA, pages 450–457. Morgan Kaufmann
Publishers, San Francisco, CA, 1997.
[ bib ]

[1633]

Yuichi Nagata and Shigenobu Kobayashi.
An analysis of edge assembly crossover for the traveling
salesman problem.
In K. Ito, F. Harashima, and K. Tanie, editors, IEEE SMC'99
Conference Proceedings, 1999 IEEE International Conference on Systems, Man,
and Cybernetics, pages 628–633. IEEE Press, 1999.
[ bib 
DOI ]

[1634]

Yuichi Nagata and Shigenobu Kobayashi.
A Powerful Genetic Algorithm Using Edge Assembly Crossover for
the Traveling Salesman Problem.
INFORMS Journal on Computing, 25(2):346–363, 2013.
[ bib 
DOI ]
This paper presents a genetic algorithm (GA) for solving the
traveling salesman problem (TSP). To construct a powerful GA,
we use edge assembly crossover (EAX) and make substantial
enhancements to it: (i) localization of EAX together with its
efficient implementation and (ii) the use of a local search
procedure in EAX to determine good combinations of building
blocks of parent solutions for generating even better
offspring solutions from very highquality parent
solutions. In addition, we develop (iii) an innovative
selection model for maintaining population diversity at a
negligible computational cost. Experimental results on
wellstudied TSP benchmarks demonstrate that the proposed GA
outperforms stateoftheart heuristic algorithms in finding
very highquality solutions on instances with up to 200,000
cities. In contrast to the stateoftheart TSP heuristics,
which are all based on the LinKernighan (LK) algorithm, our
GA achieves top performance without using an LKbased
algorithm.
Keywords: TSP, EAX, evolutionary algorithms

[1635]

Marcelo S. Nagano, Fernando L. Rossi, and Nádia J. Martarelli.
Highperforming heuristics to minimize flowtime in noidle
permutation flowshop.
Engineering Optimization, 51(2):185–198, 2019.
[ bib ]

[1636]

Yuichi Nagata and David Soler.
A New Genetic Algorithm for the Asymmetric TSP.
Expert Systems with Applications, 39(10):8947–8953, 2012.
[ bib ]

[1637]

R. Nagy, M. Suciu, and D. Dumitrescu.
Exploring Lorenz Dominance.
In Symbolic and Numeric Algorithms for Scientific Computing
(SYNASC), 2012 14th International Symposium on, pages 254–259, 2012.
[ bib ]

[1638]

Vinod Nair and Geoffrey E. Hinton.
Rectified linear units improve restricted boltzmann machines.
In J. Fürnkranz and T. Joachims, editors, Proceedings of the
27th International Conference on Machine Learning, ICML 2010, pages
807–814, New York, NY, 2010. ACM Press.
[ bib ]

[1639]

Samadhi Nallaperuma, Pietro S. Oliveto, Jorge Pérez Heredia, and Dirk
Sudholt.
On the Analysis of TrajectoryBased Search Algorithms: When is
it Beneficial to Reject Improvements?
Algorithmica, 81(2):858–885, 2019.
[ bib ]

[1640]

Samadhi Nallaperuma, Markus Wagner, and Frank Neumann.
Parameter Prediction Based on Features of Evolved Instances for
Ant Colony Optimization and the Traveling Salesperson Problem.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, PPSN 2014, volume 8672 of Lecture Notes in Computer
Science, pages 100–109. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1641]

V. Nannen and Agoston E. Eiben.
A Method for Parameter Calibration and Relevance Estimation in
Evolutionary Algorithms.
In M. Cattolico et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2006, pages 183–190. ACM Press,
New York, NY, 2006.
[ bib 
DOI ]
Keywords: REVAC

[1642]

V. Nannen and Agoston E. Eiben.
Relevance Estimation and Value Calibration of Evolutionary
Algorithm Parameters.
In M. M. Veloso, editor, Proceedings of the Twentieth
International Joint Conference on Artificial Intelligence (IJCAI07), pages
975–980. AAAI Press, Menlo Park, CA, 2007.
[ bib ]
Keywords: REVAC

[1643]

Youssef S. G. Nashed, Pablo Mesejo, Roberto Ugolotti, Jérémie
DuboisLacoste, and Stefano Cagnoni.
A Comparative Study of Three GPUBased Metaheuristics.
In C. A. Coello Coello et al., editors, PPSN 2012, Part II,
volume 7492 of Lecture Notes in Computer Science, pages 398–407.
Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1644]

John Nash and Ravi Varadhan.
Unifying Optimization Algorithms to Aid Software System Users:
optimx for R.
Journal of Statistical Software, 43(9):1–14, 2011.
[ bib ]

[1645]

M. Nawaz, E. Enscore, Jr, and I. Ham.
A Heuristic Algorithm for the mMachine, nJob FlowShop
Sequencing Problem.
Omega, 11(1):91–95, 1983.
[ bib ]
Keywords: NEH heuristic

[1646]

Antonio J. Nebro, Juan J. Durillo, and Matthieu Vergne.
Redesigning the jMetal MultiObjective Optimization
Framework.
In J. L. Jiménez Laredo, S. Silva, and A. I.
EsparciaAlcázar, editors, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2015, pages
1093–1100. ACM Press, New York, NY, 2015.
[ bib ]
Keywords: jmetal, multiobjective metaheu ristics, open source,
optimization framework

[1647]

Antonio J. Nebro, Manuel LópezIbáñez, Cristóbal
BarbaGonzález, and José GarcíaNieto.
Automatic Configuration of NSGAII with jMetal and irace.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference Companion, GECCO Companion 2019, pages 1374–1381. ACM Press, New
York, NY, 2019.
[ bib 
DOI ]

[1648]

G. L. Nemhauser and L. A. Wolsey.
Integer and Combinatorial Optimization.
John Wiley & Sons, New York, NY, 1988.
[ bib ]

[1649]

F. Nerri and Carlos Cotta.
Memetic algorithms and memetic computing optimization: A
literature review.
Swarm and Evolutionary Computation, 2:1–14, 2012.
[ bib 
DOI ]

[1650]

Frank Neumann, Dirk Sudholt, and Carsten Witt.
Analysis of different MMAS ACO algorithms on unimodal
functions and plateaus.
Swarm Intelligence, 3(1):35–68, 2009.
[ bib ]

[1651]

Frank Neumann and Carsten Witt.
Runtime Analysis of a Simple Ant Colony Optimization Algorithm.
Electronic Colloquium on Computational Complexity (ECCC),
13(084), 2006.
[ bib ]

[1652]

Allen Newell and Herbert A. Simon.
Computer Science as Empirical Inquiry: Symbols and Search.
Communications of the ACM, 19(3):113–126, March 1976.
[ bib 
DOI ]
Computer science is the study of the phenomena surrounding
computers. The founders of this society understood this very
well when they called themselves the Association for
Computing Machinery. The machinenot just the hardware, but
the programmed, living machineis the organism we study.
Keywords: cognition, Turing, search, problem solving, symbols,
heuristics, list processing, computer science, artificial
intelligence, science, empirical

[1653]

VietPhuong Nguyen, Christian Prins, and Caroline Prodhon.
A Multistart Iterated Local Search with Tabu List and Path
Relinking for the Twoechelon Locationrouting Problem.
Engineering Applications of Artificial Intelligence,
25(1):56–71, 2012.
[ bib ]

[1654]

AnhTuan Nguyen, Sigrid Reiter, and Philippe Rigo.
A review on simulationbased optimization methods applied to
building performance analysis.
Applied Energy, 113:1043–1058, 2014.
[ bib 
DOI ]

[1655]

Trung Thanh Nguyen, Shengxiang Yang, and Jürgen Branke.
Evolutionary Dynamic Optimization: A Survey of the State of the
Art.
Swarm and Evolutionary Computation, 6:1–24, 2012.
[ bib ]

[1656]

Su Nguyen, Mengjie Zhang, Mark Johnston, and Kay Chen Tan.
Genetic Programming for Evolving DueDate Assignment Models in
Job Shop Environments.
Evolutionary Computation, 22(1):105–138, 2014.
[ bib ]

[1657]

Su Nguyen, Mengjie Zhang, Mark Johnston, and Kay Chen Tan.
Automatic Design of Scheduling Policies for Dynamic
Multiobjective Job Shop Scheduling via Cooperative Coevolution Genetic
Programming.
IEEE Transactions on Evolutionary Computation, 18(2):193–208,
2014.
[ bib ]

[1658]

Peter Nightingale, Özguür Akgün, Ian P. Gent, Christopher Jefferson, Ian
Miguel, and Patrick Spracklen.
Automatically Improving Constraint Models in Savile Row.
Artificial Intelligence, 251:35–61, 2017.
[ bib ]

[1659]

Alexander G. Nikolaev and Sheldon H. Jacobson.
Simulated Annealing.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, pages 1–39. Springer, New York, NY, 2nd
edition, 2010.
[ bib ]

[1660]

Mladen Nikolić, Filip Marić, and Predrag Janičić.
Instancebased selection of policies for SAT solvers.
In International Conference on Theory and Applications of
Satisfiability Testing, pages 326–340. Springer, 2009.
[ bib ]

[1661]

Y. Nishio, A. Oyama, Y. Akimoto, Hernán E. Aguirre, and Kiyoshi Tanaka.
Manyobjective Optimization of Trajectory Design for DESTINY
Mission.
In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, and J. L.
Walteros, editors, Learning and Intelligent Optimization, 8th
International Conference, LION 8, volume 8426 of Lecture Notes in
Computer Science, Heidelberg, 2014. Springer.
[ bib ]

[1662]

Vilas Nitivattananon, Elaine C. Sadowski, and Rafael G. Quimpo.
Optimization of Water Supply System Operation.
Journal of Water Resources Planning and Management, ASCE,
122(5):374–384, September / October 1996.
[ bib ]

[1663]

Mark S. Nixon and Alberto S. Aguado.
Feature extraction & image processing for computer vision.
Academic Press, New York, NY, 2012.
[ bib ]

[1664]

Jorge Nocedal and Stephen J. Wright.
Numerical Optimization.
Springer Series in Operations Research and Financial Engineering.
Springer, 2nd edition, 2006.
[ bib ]

[1665]

Bruno Nogueira, Rian G. S. Pinheiro, and Anand Subramanian.
A Hybrid Iterated Local Search Heuristic for the Maximum Weight
Independent Set Problem.
Optimization Letters, 12(3):567–583, 2018.
[ bib 
DOI ]

[1666]

B. A. Nosek, G. Alter, G. C. Banks, D. Borsboom, S. D. Bowman, S. J. Breckler,
S. Buck, C. D. Chambers, G. Chin, G. Christensen, M. Contestabile, A. Dafoe,
E. Eich, J. Freese, R. Glennerster, D. Goroff, D. P. Green, B. Hesse,
M. Humphreys, J. Ishiyama, D. Karlan, A. Kraut, A. Lupia, P. Mabry, T. Madon,
N. Malhotra, E. MayoWilson, M. McNutt, E. Miguel, E. L. Paluck,
U. Simonsohn, C. Soderberg, B. A. Spellman, J. Turitto, G. VandenBos,
S. Vazire, E. J. Wagenmakers, R. Wilson, and T. Yarkoni.
Promoting an open research culture.
Science, 348(6242):1422–1425, June 2015.
[ bib 
DOI ]

[1667]

Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor.
The Preregistration Revolution.
Proceedings of the National Academy of Sciences,
115(11):2600–2606, March 2018.
[ bib 
DOI ]
Progress in science relies in part on generating hypotheses
with existing observations and testing hypotheses with new
observations. This distinction between postdiction and
prediction is appreciated conceptually but is not respected
in practice. Mistaking generation of postdictions with
testing of predictions reduces the credibility of research
findings. However, ordinary biases in human reasoning, such
as hindsight bias, make it hard to avoid this mistake. An
effective solution is to define the research questions and
analysis plan before observing the research outcomes–a
process called preregistration. Preregistration distinguishes
analyses and outcomes that result from predictions from those
that result from postdictions. A variety of practical
strategies are available to make the best possible use of
preregistration in circumstances that fall short of the ideal
application, such as when the data are preexisting. Services
are now available for preregistration across all disciplines,
facilitating a rapid increase in the practice. Widespread
adoption of preregistration will increase distinctiveness
between hypothesis generation and hypothesis testing and will
improve the credibility of research findings.

[1668]

Yaghout Nourani and Bjarne Andresen.
A Comparison of Simulated Annealing Cooling Strategies.
Journal of Physics A, 31(41):8373–8385, 1998.
[ bib ]

[1669]

Houssem Eddine Nouri, Olfa Belkahla Driss, and Khaled Ghédira.
A Classification Schema for the Job Shop Scheduling Problem with
Transportation Resources: StateoftheArt Review.
In R. Silhavy, R. Senkerik, Z. K. Oplatkova, P. Silhavy, and
Z. Prokopova, editors, Artificial Intelligence Perspectives in
Intelligent Systems, volume 464 of Advances in Intelligent Systems and
Computing, pages 1–11. Springer International Publishing, 2016.
[ bib ]

[1670]

Krzysztof Nowak, Marcus Märtens, and Dario Izzo.
Empirical Performance of the Approximation of the Least
Hypervolume Contributor.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, PPSN 2014, volume 8672 of Lecture Notes in Computer
Science, pages 662–671. Springer, Heidelberg, 2014.
[ bib ]

[1671]

Eugeniusz Nowicki and Czeslaw Smutnicki.
A Fast Taboo Search Algorithm for the Job Shop Problem.
Management Science, 42(6):797–813, 1996.
[ bib ]

[1672]

Eugeniusz Nowicki and Czeslaw Smutnicki.
A fast tabu search algorithm for the permutation flowshop
problem.
European Journal of Operational Research, 91(1):160–175, 1996.
[ bib ]

[1673]

Eoin O'Mahony, Emmanuel Hebrard, Alan Holland, Conor Nugent, and Barry
O'Sullivan.
Using casebased reasoning in an algorithm portfolio for
constraint solving.
In Bridge et al., editors, Irish Conference on Artificial
Intelligence and Cognitive Science, pages 210–216, 2008.
[ bib ]

[1674]

Open Science Collaboration.
Estimating the reproducibility of psychological science.
Science, 349(6251):aac4716, 2015.
[ bib 
DOI ]

[1675]

Shigeru Obayashi and Daisuke Sasaki.
Visualization and data mining of Pareto solutions using
selforganizing map.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 796–809, Heidelberg,
2003. Springer.
[ bib ]

[1676]

Gabriela Ochoa, Matthew Hyde, Tim Curtois, Jose A. VazquezRodriguez, James
Walker, Michel Gendreau, Graham Kendall, Barry McCollum, Andrew J. Parkes,
Sanja Petrovic, and Edmund K. Burke.
Hyflex: A benchmark framework for crossdomain heuristic
search.
In J.K. Hao and M. Middendorf, editors, Proceedings of EvoCOP
2012 – 12th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7245 of Lecture Notes in Computer Science, pages
136–147, Heidelberg, 2012. Springer.
[ bib ]

[1677]

Gabriela Ochoa, Marco Tomassini, Sébastien Verel, and Christian Darabos.
A Study of NK Landscapes' Basins and Local Optima Networks.
In C. Ryan, editor, Proceedings of the Genetic and Evolutionary
Computation Conference, GECCO 2008, pages 555–562, New York, NY, 2008. ACM
Press.
[ bib ]

[1678]

Gabriela Ochoa and Nadarajen Veerapen.
Mapping the global structure of TSP fitness landscapes.
Journal of Heuristics, 24(3):265–294, 2018.
[ bib ]

[1679]

Angelo Oddi, Amadeo Cesta, Nicola Policella, and Stephen F. Smith.
Combining Variants of Iterative Flattening Search.
Engineering Applications of Artificial Intelligence,
21(5):683–690, 2008.
[ bib ]

[1680]

Angelo Oddi, Amadeo Cesta, Nicola Policella, and Stephen F. Smith.
Iterative Flattening Search for Resource Constrained
Scheduling.
Journal of Intelligent Manufacturing, 21(1):17–30, 2010.
[ bib ]

[1681]

Angelo Oddi, Riccardo Rasconi, Amadeo Cesta, and Stephen F. Smith.
Iterative Flattening Search for the Flexible Job Shop Scheduling
Problem.
In T. Walsh, editor, Proceedings of the TwentySecond
International Joint Conference on Artificial Intelligence (IJCAI11), pages
1991–1996. IJCAI/AAAI Press, Menlo Park, CA, 2011.
[ bib ]

[1682]

F. A. Ogbu and David K. Smith.
The Application of the Simulated Annealing Algorithm to the
Solution of the n/m/C Max Flowshop Problem.
Computers & Operations Research, 17(3):243–253, 1990.
[ bib ]

[1683]

Jeffrey W. Ohlmann and Barrett W. Thomas.
A CompressedAnnealing Heuristic for the Traveling Salesman
Problem with Time Windows.
INFORMS Journal on Computing, 19(1):80–90, 2007.
[ bib 
DOI ]

[1684]

Vesa Ojalehto, Dmitry Podkopaev, and Kaisa Miettinen.
Towards Automatic Testing of Reference Point Based Interactive
Methods.
In J. Handl, E. Hart, P. R. Lewis, M. LópezIbáñez,
G. Ochoa, and B. Paechter, editors, Parallel Problem Solving from Nature
– PPSN XIV, volume 9921 of Lecture Notes in Computer Science, pages
483–492. Springer, Heidelberg, 2016.
[ bib 
DOI ]
In this research, we proposed to build an automated framework
for testing interactive multiobjective optimization methods,
without utilizing a value function to represent the DM's
preferences. This was achieved by replacing the human DM with
an artificial DM constructed from two distinct parts: the
steady part and the current context. With the steady part the
artificial DM tries to maintain the search towards its
preferences, while at the same time the current context
allows changing the direction as well as ending the solution
process prematurely, mimicking actions of a human DM.
Keywords: artificial DMs

[1685]

Pietro S. Oliveto, Jun He, and Xin Yao.
Time complexity of evolutionary algorithms for combinatorial
optimization: A decade of results.
International Journal of Automation and Computing,
4(3):281–293, 2007.
[ bib ]

[1686]

Sabrina M. Oliveira, Mohamed Saifullah Hussin, Andrea Roli, Marco Dorigo, and
Thomas Stützle.
Analysis of the Populationbased Ant Colony Optimization
Algorithm for the TSP and the QAP.
In Proceedings of the 2017 Congress on Evolutionary Computation
(CEC 2017), pages 1734–1741. IEEE Press, Piscataway, NJ, 2017.
[ bib ]

[1687]

Pietro S. Oliveto and Carsten Witt.
Improved time complexity analysis of the Simple Genetic
Algorithm.
Theoretical Computer Science, 605:21–41, 2015.
[ bib 
DOI ]

[1688]

David L. Olson.
Review of Empirical Studies in Multiobjective Mathematical
Programming: Subject Reflection of Nonlinear Utility and Learning.
Decision Sciences, 23(1):1–20, 1992.
[ bib 
DOI ]
Multiple objective programming provides a means of
aiding decision makers facing complex decisions where
tradeoffs among conflicting objectives must be
reconciled. Interactive multiobjective programming provides a
means for decision makers to learn what these tradeoffs
involve, while the mathematical program generates solutions
that seek improvement of the implied utility of the decision
maker. A variety of multiobjective programming techniques
have been presented in the multicriteria decisionmaking
literature. This study reviews published studies with human
subjects where some of these techniques were applied. While
all of the techniques have the ability to support decision
makers under conditions of multiple objectives, a number of
features in applying these systems have been tested by these
studies. A general evolution of techniques is traced,
starting with methods relying upon linear combinations of
value, to more recent methods capable of reflecting nonlinear
tradeoffs of value. Support of nonlinear utility and
enhancing decisionmaker learning are considered.
Keywords: Decision Analysis, Human Information Processing, Linear
Programming

[1689]

Randal S. Olson, Nathan Bartley, Ryan J. Urbanowicz, and Jason H. Moore.
Evaluation of a Treebased Pipeline Optimization Tool for
Automating Data Science.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2016, pages 485–492. ACM Press, New York, NY, 2016.
[ bib 
DOI ]

[1690]

Roland Olsson and Arne Løkketangen.
Using automatic programming to generate stateoftheart
algorithms for random 3SAT.
Journal of Heuristics, 19(5):819–844, 2013.
[ bib 
DOI ]

[1691]

Randal S. Olson, Ryan J. Urbanowicz, Peter C. Andrews, Nicole A. Lavender,
La Creis Kidd, and Jason H. Moore.
Automating Biomedical Data Science Through TreeBased Pipeline
Optimization.
In G. Squillero and P. Burelli, editors, Applications of
Evolutionary Computation, volume 9597 of Lecture Notes in Computer
Science, pages 123–137. Springer, Heidelberg, 2016.
[ bib 
DOI ]

[1692]

Mihai Oltean.
Evolving Evolutionary Algorithms Using Linear Genetic
Programming.
Evolutionary Computation, 13(3):387–410, 2005.
[ bib 
DOI ]

[1693]

Michael O'Neill and Conor Ryan.
Grammatical Evolution.
IEEE Transactions on Evolutionary Computation, 5(4):349–358,
2001.
[ bib ]

[1694]

Lindell E. Ormsbee, Thomas M. Walski, Donald V. Chase, and W. W. Sharp.
Methodology for improving pump operation efficiency.
Journal of Water Resources Planning and Management, ASCE,
115(2):148–164, 1989.
[ bib ]

[1695]

Lindell E. Ormsbee and Kevin E. Lansey.
Optimal Control of Water Supply Pumping Systems.
Journal of Water Resources Planning and Management, ASCE,
120(2):237–252, 1994.
[ bib ]

[1696]

Lindell E. Ormsbee and Srinivasa L. Reddy.
Nonlinear Heuristic for Pump Operations.
Journal of Water Resources Planning and Management, ASCE,
121(4):302–309, July / August 1995.
[ bib ]

[1697]

Jeffrey E. Orosz and Sheldon H. Jacobson.
Analysis of Static Simulated Annealing Algorithms.
Journal of Optimization Theory and Applications,
115(1):165–182, 2002.
[ bib ]

[1698]

Ibrahim H. Osman and Chris N. Potts.
Simulated Annealing for Permutation FlowShop Scheduling.
Omega, 17(6):551–557, 1989.
[ bib ]

[1699]

Avi Ostfeld and Elad Salomons.
Optimal Scheduling of Pumping and Chlorine Injections under
Unsteady Hydraulics.
In G. Sehlke, D. F. Hayes, and D. K. Stevens, editors, Critical
Transitions In Water And Environmental Resources Management, pages 1–9,
July 2004.
[ bib ]
This paper describes the methodology and application
of a genetic algorithm (GA) scheme, tailormade to
EPANET for simultaneously optimizing the scheduling
of existing pumping and booster disinfection units,
as well as the design of new disinfection booster
chlorination stations, under unsteady
hydraulics. The objective is to minimize the total
cost of operating the pumping units and the chlorine
booster operation and design for a selected
operational time horizon, while delivering the
consumers required water quantities, at acceptable
pressures and chlorine residual concentrations. The
decision variables, for each of the time steps that
encompass the total operational time horizon,
include: the scheduling of the pumping units,
settings of the water distribution system control
valves, and the mass injection rates at each of the
booster chlorination stations. The constraints are
domain heads and chlorine concentrations at the
consumer nodes, maximum injection rates at the
chlorine injection stations, maximum allowable
amounts of water withdraws at the sources, and
returning at the end of the operational time horizon
to a prescribed total volume in the tanks. The model
is demonstrated through an example application.

[1700]

P. S. Ow and T. E. Morton.
Filtered Beam Search in Scheduling.
International Journal of Production Research, 26:297–307,
1988.
[ bib ]

[1701]

Gül Özerol and Esra Karasakal.
Interactive outranking approaches for multicriteria
decisionmaking problems with imprecise information.
Journal of the Operational Research Society, 59:1253–1268,
2007.
[ bib ]

[1702]

Meltem Öztürk, Alexis Tsoukiàs, and Philippe Vincke.
Preference Modelling.
In J. R. Figueira, S. Greco, and M. Ehrgott, editors, Multiple
Criteria Decision Analysis, State of the Art Surveys, chapter 2, pages
27–72. Springer, 2005.
[ bib ]

[1703]

Manfred Padberg and Giovanni Rinaldi.
A branchandcut algorithm for the resolution of largescale
symmetric traveling salesman problems.
SIAM Review, 33(1):60–100, 1991.
[ bib ]

[1704]

Federico Pagnozzi and Thomas Stützle.
Speeding up Local Search for the Insert Neighborhood in the
Weighted Tardiness Permutation Flowshop Problem.
Optimization Letters, 11:1283–1292, 2017.
[ bib 
DOI ]

[1705]

Federico Pagnozzi and Thomas Stützle.
Automatic Design of Hybrid Stochastic Local Search Algorithms
for Permutation Flowshop Problems.
Technical Report TR/IRIDIA/2018005, IRIDIA, Université Libre de
Bruxelles, Belgium, April 2018.
[ bib 
http ]

[1706]

Federico Pagnozzi and Thomas Stützle.
Automatic Design of Hybrid Stochastic Local Search Algorithms
for Permutation Flowshop Problems: Supplementary Material.
http://iridia.ulb.ac.be/supp/IridiaSupp2018002/, 2018.
[ bib ]

[1707]

Federico Pagnozzi and Thomas Stützle.
Automatic Design of Hybrid Stochastic Local Search Algorithms
for Permutation Flowshop Problems.
European Journal of Operational Research, 276:409–421, 2019.
[ bib 
DOI ]

[1708]

Federico Pagnozzi and Thomas Stützle.
Automatic design of hybrid stochastic local search algorithms
for permutation flowshop problems with additional constraints.
http://iridia.ulb.ac.be/supp/IridiaSupp2018002/, 2019.
[ bib ]

[1709]

Federico Pagnozzi and Thomas Stützle.
Evaluating the impact of grammar complexity in automatic
algorithm design.
International Transactions in Operational Research, pages
1–26, 2020.
[ bib 
DOI ]

[1710]

Federico Pagnozzi and Thomas Stützle.
Automatic design of hybrid stochastic local search algorithms
for permutation flowshop problems with additional constraints.
Operations Research Perspectives, 8, 2021.
[ bib 
DOI ]

[1711]

Federico Pagnozzi.
Automatic Design of Hybrid Stochastic Local Search Algorithms.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2019.
[ bib ]
Supervised by Thomas Stützle

[1712]

Daniel Palhazi Cuervo, Peter Goos, Kenneth Sörensen, and Emely
Arráiz.
An Iterated Local Search Algorithm for the Vehicle Routing
Problem with Backhauls.
European Journal of Operational Research, 237(2):454–464,
2014.
[ bib ]

[1713]

Gintaras Palubeckis.
Iterated tabu search for the unconstrained binary quadratic
optimization problem.
Informatica, 17(2):279–296, 2006.
[ bib 
DOI ]

[1714]

QuanKe Pan and Rubén Ruiz.
Local Search Methods for the Flowshop Scheduling Problem with
Flowtime Minimization.
European Journal of Operational Research, 222(1):31–43, 2012.
[ bib ]

[1715]

QuanKe Pan and Rubén Ruiz.
A Comprehensive Review and Evaluation of Permutation Flowshop
Heuristics to Minimize Flowtime.
Computers & Operations Research, 40(1):117–128, 2013.
[ bib ]

[1716]

QuanKe Pan, Rubén Ruiz, and Pedro AlfaroFernández.
Iterated Search Methods for Earliness and Tardiness Minimization
in Hybrid Flowshops with Due Windows.
Computers & Operations Research, 80:50–60, 2017.
[ bib ]

[1717]

QuanKe Pan, Mehmet Fatih Tasgetiren, and YunChia Liang.
A Discrete Differential Evolution Algorithm for the Permutation
Flowshop Scheduling Problem.
Computers and Industrial Engineering, 55(4):795 – 816, 2008.
[ bib ]

[1718]

QuanKe Pan, Ling Wang, and BaoHua Zhao.
An improved iterated greedy algorithm for the nowait flow shop
scheduling problem with makespan criterion.
International Journal of Advanced Manufacturing Technology,
38(78):778–786, 2008.
[ bib ]

[1719]

Sinno Jialin Pan and Qiang Yang.
A survey on transfer learning.
IEEE Transactions on Knowledge and Data Engineering,
22(10):1345–1359, 2009.
[ bib ]

[1720]

Christos H. Papadimitriou and K. Steiglitz.
Combinatorial Optimization – Algorithms and Complexity.
Prentice Hall, Englewood Cliffs, NJ, 1982.
[ bib ]

[1721]

Christos H. Papadimitriou and M. Yannakakis.
On the Approximability of Tradeoffs and Optimal Access of Web
Sources.
In A. Blum, editor, 41st Annual Symposium on Foundations of
Computer Science, pages 86–92. IEEE Computer Society Press, 2000.
[ bib 
DOI ]

[1722]

Luís Paquete.
Algoritmos Evolutivos Multiobjectivo para Afectação de
Recursos e sua Aplicação à Geração de Horários em
Universidades (Multiobjective Evolutionary Algorithms for Resource
Allocation and their Application to University Timetabling).
Master's thesis, University of Algarve, 2001.
In Portuguese.
[ bib ]
The aim of this study is the application of
multiobjective evolutionary algorithms to resource
allocation problems, such as university examination
timetabling and course timetabling
problems. Usually, these problems are characterized
by multiple conflicting objectives. A multiobjective
formalization of these problems is presented, based
on goals and priorities. Various aspects of
evolutionary algorithms are proposed and studied for
these problems, particulary, selection methods and
types and parameters of mutation operator. The
choice of both representation and operators is made
so as not to favour excessively certain objectives
with respect to others at the level of the
exploration mechanism. A comparative study of
performance is presented for the proposed algorithms
by means of statistical inference, based on real
problems of the University of Algarve. The notion of
attainment functions is used as a base for the
assessment of performance of multiobjective
evolutionary algorithms. Finally, the evolution of
the solution cost during the runs is analysed by
means of attainment functions, as well.

[1723]

Luís Paquete.
Stochastic Local Search Algorithms for Multiobjective
Combinatorial Optimization: Methods and Analysis.
PhD thesis, FB Informatik, TU Darmstadt, Germany, 2005.
[ bib ]

[1724]

Luís Paquete, Marco Chiarandini, and Thomas Stützle.
Pareto Local Optimum Sets in the Biobjective Traveling
Salesman Problem: An Experimental Study.
In X. Gandibleux, M. Sevaux, K. Sörensen, and V. T'Kindt,
editors, Metaheuristics for Multiobjective Optimisation, volume 535 of
Lecture Notes in Economics and Mathematical Systems, pages 177–199.
Springer, Berlin, Germany, 2004.
[ bib 
DOI ]
In this article, we study Pareto local optimum sets for the
biobjective Traveling Salesman Problem applying
straightforward extensions of local search algorithms for the
single objective case. The performance of the local search
algorithms is illustrated by experimental results obtained
for well known benchmark instances and comparisons to methods
from literature. In fact, a 3opt local search is able to
compete with the best performing metaheuristics in terms of
solution quality. Finally, we also present an empirical study
of the features of the solutions found by 3opt on a set of
randomly generated instances. The results indicate the
existence of several clusters of nearoptimal solutions that
are separated by only a few edges.
Keywords: Pareto local search, PLS

[1725]

Luís Paquete, Carlos M. Fonseca, and Manuel LópezIbáñez.
An optimal algorithm for a special case of Klee's measure
problem in three dimensions.
Technical Report CSIRTI01/2006, CSI, Universidade do Algarve,
2006.
Superseded by paper in IEEE Transactions on Evolutionary
Computation [209].
[ bib ]
The measure of the region dominated by (the maxima
of) a set of n points in the positive dorthant
has been proposed as an indicator of performance in
multiobjective optimization, known as the
hypervolume indicator, and the problem of computing
it efficiently is attracting increasing
attention. In this report, this problem is
formulated as a special case of Klee's measure
problem in d dimensions, which immediately
establishes O(n^{d/2}logn) as a, possibly
conservative, upper bound on the required
computation time. Then, an O(n log n) algorithm
for the 3dimensional version of this special case
is constructed, based on an existing dimensionsweep
algorithm for the related maxima problem. Finally,
O(n log n) is shown to remain a lower bound on the
time required by the hypervolume indicator for
d>1, which attests the optimality of the algorithm
proposed.
Proof of Theorem 3.1 is incorrect

[1726]

Luís Paquete, Tommaso Schiavinotto, and Thomas Stützle.
On Local Optima in Multiobjective Combinatorial Optimization
Problems.
Annals of Operations Research, 156:83–97, 2007.
[ bib 
DOI ]
In this article, local optimality in multiobjective
combinatorial optimization is used as a baseline for
the design and analysis of two iterative improvement
algorithms. Both algorithms search in a neighborhood
that is defined on a collection of sets of feasible
solutions and their acceptance criterion is based on
outperformance relations. Proofs of the soundness
and completeness of these algorithms are given.
Keywords: Pareto local search, PLS

[1727]

Luís Paquete and Thomas Stützle.
A study of stochastic local search algorithms for the
biobjective QAP with correlated flow matrices.
European Journal of Operational Research, 169(3):943–959,
2006.
[ bib ]

[1728]

Luís Paquete and Thomas Stützle.
Clusters of nondominated solutions in multiobjective
combinatorial optimization: An experimental analysis.
In V. Barichard, M. Ehrgott, X. Gandibleux, and V. T'Kindt, editors,
Multiobjective Programming and Goal Programming: Theoretical Results and
Practical Applications, volume 618 of Lecture Notes in Economics and
Mathematical Systems, pages 69–77. Springer, Berlin, Germany, 2009.
[ bib 
DOI ]

[1729]

Luís Paquete and Thomas Stützle.
Design and analysis of stochastic local search for the
multiobjective traveling salesman problem.
Computers & Operations Research, 36(9):2619–2631, 2009.
[ bib 
DOI ]

[1730]

Luís Paquete and Thomas Stützle.
An Experimental Investigation of Iterated Local Search for
Coloring Graphs.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2002, volume 2279 of Lecture
Notes in Computer Science, pages 122–131. Springer, Heidelberg, 2002.
[ bib ]

[1731]

Luís Paquete and Thomas Stützle.
A TwoPhase Local Search for the Biobjective Traveling Salesman
Problem.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 479–493. Springer,
Heidelberg, 2003.
[ bib ]

[1732]

Luís Paquete, Thomas Stützle, and Manuel LópezIbáñez.
On the design and analysis of SLS algorithms for
multiobjective combinatorial optimization problems.
Technical Report TR/IRIDIA/2005029, IRIDIA, Université Libre de
Bruxelles, Belgium, 2005.
[ bib 
http ]
Effective Stochastic Local Search (SLS) algorithms
can be seen as being composed of several algorithmic
components, each of which plays some specific role
with respect to overall performance. In this
article, we explore the application of experimental
design techniques to analyze the effect of different
choices for these algorithmic components on SLS
algorithms applied to Multiobjective Combinatorial
Optimization Problems that are solved in terms of
Pareto optimality. This analysis is done using the
example application of SLS algorithms to the
biobjective Quadratic Assignment Problem and we show
also that the same choices for algorithmic
components can lead to different behavior in
dependence of various instance features, such as the
structure of input data and the correlation between
objectives.

[1733]

Luís Paquete, Thomas Stützle, and Manuel LópezIbáñez.
Towards the Empirical Analysis of SLS Algorithms for
Multiobjective Combinatorial Optimization Problems through Experimental
Design.
In K. F. Doerner, M. Gendreau, P. Greistorfer, W. J. Gutjahr, R. F.
Hartl, and M. Reimann, editors, 6th Metaheuristics International
Conference (MIC 2005), pages 739–746, Vienna, Austria, 2005.
[ bib ]
Stochastic Local Search (SLS) algorithms for
Multiobjective Combinatorial Optimization Problems
(MCOPs) typically involve the selection and
parameterization of many algorithm components whose
role with respect to their overall performance and
relation to certain instance features is often not
clear. In this abstract, we use a modular approach
for the design of SLS algorithms for MCOPs defined
in terms of Pareto optimality and we present an
extensive analysis of SLS algorithms through
experimental design techniques, where each algorithm
component is considered a factor. The experimental
analysis is based on a sound experimental
methodology for analyzing the output of algorithms
for MCOPs. We show that different choices for
algorithm components can lead to different behavior
in dependence of various instance features.

[1734]

Luís Paquete, Thomas Stützle, and Manuel LópezIbáñez.
Using experimental design to analyze stochastic local search
algorithms for multiobjective problems.
In K. F. Doerner, M. Gendreau, P. Greistorfer, W. J. Gutjahr, R. F.
Hartl, and M. Reimann, editors, Metaheuristics: Progress in Complex
Systems Optimization, volume 39 of Operations Research / Computer
Science Interfaces, pages 325–344. Springer, New York, NY, 2007.
[ bib 
DOI ]
Stochastic Local Search (SLS) algorithms can be seen as being
composed of several algorithmic components, each playing some
specific role with respect to overall performance. This
article explores the application of experimental design
techniques to analyze the effect of components of SLS
algorithms for Multiobjective Combinatorial Optimization
problems, in particular for the Biobjective Quadratic
Assignment Problem. The analysis shows that there exists a
strong dependence between the choices for these components
and various instance features, such as the structure of the
input data and the correlation between the objectives.
PostConference Proceedings of the 6th Metaheuristics
International Conference (MIC 2005)

[1735]

S. N. Parragh, Karl F. Doerner, Richard F. Hartl, and Xavier Gandibleux.
A heuristic twophase solution approach for the multiobjective
dialaride problem.
Networks, 54(4):227–242, 2009.
[ bib ]

[1736]

Rebecca Parsons and Mark Johnson.
A Case Study in Experimental Design Applied to Genetic
Algorithms with Applications to DNA Sequence Assembly.
American Journal of Mathematical and Management Sciences,
17(34):369–396, 1997.
[ bib 
DOI ]

[1737]

MoonWon Park and YeongDae Kim.
A systematic procedure for setting parameters in simulated
annealing algorithms.
Computers & Operations Research, 25(3):207–217, 1998.
[ bib 
DOI ]

[1738]

R. S. Parpinelli, H. S. Lopes, and A. A. Freitas.
Data Mining with an Ant Colony Optimization Algorithm.
IEEE Transactions on Evolutionary Computation, 6(4):321–332,
2002.
[ bib ]

[1739]

R. O. Parreiras and J. A. Vascocelos.
A multiplicative version of PROMETHEE II applied to
multiobjective optimization problems.
European Journal of Operational Research, 183:729–740, 2007.
[ bib ]

[1740]

Gerald Paul.
Comparative performance of tabu search and simulated annealing
heuristics for the quadratic assignment problem.
Operations Research Letters, 38(6):577–581, 2010.
[ bib ]

[1741]

J Paulli.
A computational comparison of simulated annealing and tabu
search applied to the quadratic assignment problem.
In R. V. V. Vidal, editor, Applied Simulated Annealing, pages
85–102. Springer, 1993.
[ bib ]

[1742]

Lucas Marcondes Pavelski, Myriam Regattieri Delgado, and MarieEléonore
Kessaci.
MetaLearning on Flowshop Using Fitness Landscape Analysis.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pages 925–933, New York, NY, 2019. ACM Press.
[ bib 
DOI ]

[1743]

Judea Pearl.
Heuristics: Intelligent Search Strategies for Computer Problem
Solving.
AddisonWesley, Reading, MA, 1984.
[ bib ]

[1744]

Glen S. Peace.
Taguchi Methods: A HandsOn Approach.
AddisonWesley, 1993.
[ bib ]

[1745]

Judea Pearl and Elias Bareinboim.
Transportability of causal and statistical relations: A formal
approach.
In W. Burgard and D. Roth, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pages 247–254. AAAI Press, 2011.
[ bib ]

[1746]

Judea Pearl and Dana Mackenzie.
The book of why: the new science of cause and effect.
Basic books, 2018.
[ bib ]

[1747]

Judea Pearl.
Causality: Models, Reasoning and Inference.
Cambridge University Press, 2nd edition, 2009.
[ bib ]

[1748]

Judea Pearl.
The seven tools of causal inference, with reflections on machine
learning.
Communications of the ACM, 62(3):54–60, 2019.
[ bib ]

[1749]

Juan A. Pedraza, Carlos GarcíaMartínez, Alberto Cano, and Sebastián
Ventura.
Classification Rule Mining with Iterated Greedy.
In M. M. Polycarpou, A. C. P. L. F. de Carvalho, J. Pan, M. Wozniak,
H. Quintián, and E. Corchado, editors, Hybrid Artificial
Intelligence Systems  9th International Conference, HAIS 2014, Salamanca,
Spain, June 1113, 2014. Proceedings, volume 8480 of Lecture Notes in
Computer Science, pages 585–596. Springer, Heidelberg, 2014.
[ bib ]

[1750]

Martín Pedemonte, Sergio Nesmachnow, and Héctor Cancela.
A survey on parallel ant colony optimization.
Applied Soft Computing, 11(8):5181–5197, 2011.
[ bib ]

[1751]

Paola Pellegrini and Mauro Birattari.
Implementation Effort and Performance.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2007, volume 4638 of Lecture Notes
in Computer Science, pages 31–45. Springer, Heidelberg, 2007.
[ bib ]

[1752]

Paola Pellegrini, Mauro Birattari, and Thomas Stützle.
A Critical Analysis of Parameter Adaptation in Ant Colony
Optimization.
Swarm Intelligence, 6(1):23–48, 2012.
[ bib 
DOI ]

[1753]

Paola Pellegrini, L. Castelli, and R. Pesenti.
Metaheuristic algorithms for the simultaneous slot allocation
problem.
IET Intelligent Transport Systems, 6(4):453–462, December
2012.
[ bib 
DOI ]

[1754]

Paola Pellegrini, D. Favaretto, and E. Moretti.
On MaxMin Ant System's Parameters.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 5th International Workshop, ANTS 2006, volume 4150 of
Lecture Notes in Computer Science, pages 203–214. Springer, Heidelberg,
2006.
[ bib ]

[1755]

Paola Pellegrini, D. Favaretto, and E. Moretti.
Exploration in stochastic algorithms: An application on
MaxMin Ant System.
In N. Krasnogor, B. MeliÃ¡nBatista, J. A. MorenoPÃ©rez, J. M.
MorenoVega, and D. A. Pelta, editors, Nature Inspired Cooperative
Strategies for Optimization (NICSO 2008), volume 236 of Studies in
Computational Intelligence, pages 1–13. Springer, Berlin, Germany, 2009.
[ bib 
DOI ]

[1756]

Paola Pellegrini, Franco Mascia, Thomas Stützle, and Mauro Birattari.
On the Sensitivity of Reactive Tabu Search to its
Metaparameters.
Soft Computing, 18(11):2177–2190, 2014.
[ bib 
DOI ]

[1757]

Paola Pellegrini, Thomas Stützle, and Mauro Birattari.
Offline vs. Online Tuning: A Study on MaxMin Ant System for
the TSP.
In M. Dorigo et al., editors, Swarm Intelligence, 7th
International Conference, ANTS 2010, volume 6234 of Lecture Notes in
Computer Science, pages 239–250. Springer, Heidelberg, 2010.
[ bib 
DOI ]

[1758]

Puca Huachi Vaz Penna, Anand Subramanian, and Luiz Satoru Ochi.
An Iterated Local Search Heuristic for the Heterogeneous Fleet
Vehicle Routing Problem.
Journal of Heuristics, 19(2):201–232, 2013.
[ bib ]

[1759]

Jeffrey M. Perkel.
Challenge to scientists: does your tenyearold code still run?
Nature, 584:556–658, 2020.
[ bib 
DOI ]
https://www.nature.com/articles/d41586020024627
Keywords: reproducibility; software engineering; ReScience C; Ten Years
Reproducibility Challenge; code reusability

[1760]

Leslie Pérez Cáceres, Bernd Bischl, and Thomas Stützle.
Evaluating random forest models for irace.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference Companion, GECCO Companion 2017, pages
1146–1153, New York, NY, 2017. ACM Press.
[ bib 
DOI ]

[1761]

Leslie Pérez Cáceres, Manuel LópezIbáñez, Holger H. Hoos,
and Thomas Stützle.
An Experimental Study of Adaptive Capping in
irace.
In R. Battiti, D. E. Kvasov, and Y. D. Sergeyev, editors,
Learning and Intelligent Optimization, 11th International Conference, LION
11, volume 10556 of Lecture Notes in Computer Science, pages 235–250.
Springer, Cham, Switzerland, 2017.
[ bib 
DOI 
supplementary material ]

[1762]

Leslie Pérez Cáceres, Manuel LópezIbáñez, Holger H. Hoos,
and Thomas Stützle.
An experimental study of adaptive capping in irace:
Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2016007/, 2017.
[ bib ]

[1763]

Leslie Pérez Cáceres, Manuel LópezIbáñez, and Thomas
Stützle.
Ant Colony Optimization on a Budget of 1000.
In M. Dorigo et al., editors, Swarm Intelligence, 9th
International Conference, ANTS 2014, volume 8667 of Lecture Notes in
Computer Science, pages 50–61. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1764]

Leslie Pérez Cáceres, Manuel LópezIbáñez, and Thomas
Stützle.
An Analysis of Parameters of irace.
In C. Blum and G. Ochoa, editors, Proceedings of EvoCOP 2014 –
14th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 8600 of Lecture Notes in Computer Science, pages
37–48. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1765]

Leslie Pérez Cáceres, Manuel LópezIbáñez, and Thomas
Stützle.
Ant Colony Optimization on a Budget of 1000: Supplementary
material, 2015.
[ bib 
http ]

[1766]

Leslie Pérez Cáceres, Manuel LópezIbáñez, and Thomas
Stützle.
Ant colony optimization on a limited budget of evaluations.
Swarm Intelligence, 9(23):103–124, 2015.
[ bib 
DOI 
supplementary material ]

[1767]

Leslie Pérez Cáceres, Federico Pagnozzi, Alberto Franzin, and Thomas
Stützle.
Automatic Configuration of GCC Using irace.
In E. Lutton, P. Legrand, P. Parrend, N. Monmarché, and
M. Schoenauer, editors, EA 2017: Artificial Evolution, volume 10764 of
Lecture Notes in Computer Science, pages 202–216. Springer,
Heidelberg, 2017.
[ bib 
DOI ]
Automatic algorithm configuration techniques have proved to
be successful in finding performanceoptimizing parameter
settings of many searchbased decision and optimization
algorithms. A recurrent, important step in software
development is the compilation of source code written in some
programming language into machineexecutable code. The
generation of performanceoptimized machine code itself is a
difficult task that can be parametrized in many different
possible ways. While modern compilers usually offer different
levels of optimization as possible defaults, they have a
larger number of other flags and numerical parameters that
impact properties of the generated machinecode. While the
generation of performanceoptimized machine code has received
large attention and is dealt with in the research area of
autotuning, the usage of standard automatic algorithm
configuration software has not been explored, even though, as
we show in this article, the performance of the compiled code
has significant stochasticity, just as standard optimization
algorithms. As a practical case study, we consider the
configuration of the wellknown GNU compiler collection (GCC)
for minimizing the runtime of machine code for various
heuristic search methods. Our experimental results show that,
depending on the specific code to be optimized, improvements
of up to 40% of execution time when compared to the O2
and O3 optimization flags is possible.

[1768]

Leslie Pérez Cáceres, Federico Pagnozzi, Alberto Franzin, and Thomas
Stützle.
Automatic configuration of GCC using irace: Supplementary
material.
http://iridia.ulb.ac.be/supp/IridiaSupp2017009/, 2017.
[ bib ]

[1769]

Matias Péres, Germán Ruiz, Sergio Nesmachnow, and Ana C. Olivera.
Multiobjective evolutionary optimization of traffic flow and
pollution in Montevideo, Uruguay.
Applied Soft Computing, 70:472–485, 2018.
[ bib ]
Keywords: Multiobjective evolutionary
algorithms,Pollution,Simulation,Traffic flow

[1770]

Leslie Pérez Cáceres and Thomas Stützle.
Automatic Algorithm Configuration: Analysis, Improvements and
Applications.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2017.
[ bib 
eprint ]
Supervised by Thomas Stützle and Manuel LópezIbáñez

[1771]

A. Pessoa, E. Uchoa, M. Aragão, and R. Rodrigues.
Exact Algorithm over an Arctimeindexed formulation for
Parallel Machine Scheduling Problems.
Mathematical Programming Computation, 2(3–4):259–290, 2010.
[ bib ]

[1772]

Gilles Pesant, Michel Gendreau, JeanYves Potvin, and J.M. Rousseau.
An Exact Constraint Logic Programming Algorithm for the
Traveling Salesman Problem with Time Windows.
Transportation Science, 32:12–29, 1998.
[ bib ]

[1773]

Charles W. Petit.
Touched by nature: putting evolution to work on the assembly
line.
U.S. News & World Report, 125(4):43–45, July 1998.
[ bib 
http ]
Evolutionary optimization of turbine design of the
Boeing 777 GE

[1774]

James E. Pettinger and Richard M. Everson.
Controlling genetic algorithms with reinforcement learning.
In W. B. Langdon et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2002, pages 692–692. Morgan
Kaufmann Publishers, San Francisco, CA, 2002.
[ bib ]

[1775]

S. Pezeshk and O. J. Helweg.
Adaptative Search Optimisation in reducing pump operation
costs.
Journal of Water Resources Planning and Management, ASCE,
122(1):57–63, January / February 1996.
[ bib ]

[1776]

Selcen Phelps and Murat Köksalan.
An interactive evolutionary metaheuristic for multiobjective
combinatorial optimization.
Management Science, 49(12):1726–1738, 2003.
[ bib ]

[1777]

Francesco di Pierro, SoonThiam Khu, and Dragan A. Savic.
An investigation on preference order ranking scheme for
multiobjective evolutionary optimization.
IEEE Transactions on Evolutionary Computation, 11(1):17–45,
2007.
[ bib ]

[1778]

Josef Pihera and Nysret Musliu.
Application of Machine Learning to Algorithm Selection for
TSP.
In G. A. Papadopoulos, editor, 26th IEEE International
Conference on Tools with Artificial Intelligence, ICTAI 2014, Limassol,
Cyprus, November 1012, 2014, pages 47–54. IEEE Press, 2014.
[ bib ]

[1779]

M. L. Pilat and T. White.
Using Genetic Algorithms to optimize ACSTSP.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 282–287. Springer, Heidelberg, 2002.
[ bib ]

[1780]

Michael L. Pinedo.
Scheduling: Theory, Algorithms, and Systems.
Springer, New York, NY, 4th edition, 2012.
[ bib ]

[1781]

Pedro Pinto, Thomas Runkler, and João Sousa.
Ant Colony Optimization and its Application to Regular and
Dynamic MAXSAT Problems.
In Advances in Biologically Inspired Information Systems,
volume 69 of Studies in Computational Intelligence, pages 285–304.
Springer, Berlin, Germany, 2007.
[ bib 
DOI ]
In this chapter we discuss the ant colony
optimization metaheuristic (ACO) and its
application to static and dynamic constraint
satisfaction optimization problems, in particular
the static and dynamic maximum satisfiability
problems (MAXSAT). In the first part of the
chapter we give an introduction to metaheuristics
in general and ant colony optimization in
particular, followed by an introduction to
constraint satisfaction and static and dynamic
constraint satisfaction optimization problems.
Then, we describe how to apply the ACO algorithm
to the problems, and do an analysis of the results
obtained for several benchmarks. The adapted ant
colony optimization accomplishes very well the task
of dealing with systematic changes of dynamic
MAXSAT instances derived from static problems.

[1782]

Joelle Pineau and Koustuv Sinha.
The Machine Learning Reproducibility Checklist (v2.0).
https://www.cs.mcgill.ca/~jpineau/ReproducibilityChecklistv2.0.pdf,
2020.
[ bib ]
Used in NeurIPS 2020

[1783]

Joelle Pineau, Philippe VincentLamarre, Koustuv Sinha, Vincent LariviÃ¨re,
Alina Beygelzimer, Florence d'AlchÃ© Buc, Emily Fox, and Hugo Larochelle.
Improving Reproducibility in Machine Learning Research (A Report
from the NeurIPS 2019 Reproducibility Program).
Arxiv preprint arXiv:2003.12206 [cs.LG], 2020.
[ bib 
http ]

[1784]

David Pisinger and Stefan Ropke.
A General Heuristic for Vehicle Routing Problems.
Computers & Operations Research, 34(8):2403–2435, 2007.
[ bib ]

[1785]

David Pisinger and Stefan Ropke.
Large Neighborhood Search.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, pages 399–419. Springer, New York, NY, 2nd
edition, 2010.
[ bib ]

[1786]

Rapeepan Pitakaso, Christian Almeder, Karl F. Doerner, and Richard F. Hartl.
Combining exact and populationbased methods for the Constrained
Multilevel Lot Sizing Problem.
International Journal of Production Research,
44(22):4755–4771, 2006.
[ bib ]

[1787]

Rapeepan Pitakaso, Christian Almeder, Karl F. Doerner, and Richard F. Hartl.
A MaxMin Ant System for unconstrained multilevel lotsizing
problems.
Computers & Operations Research, 34(9):2533–2552, 2007.
[ bib 
DOI ]
In this paper, we present an antbased algorithm
for solving unconstrained multilevel lotsizing
problems called ant system for multilevel
lotsizing algorithm (ASMLLS). We apply a hybrid
approach where we use ant colony optimization in
order to find a good lotsizing sequence, i.e. a
sequence of the different items in the product
structure in which we apply a modified
WagnerWhitin algorithm for each item
separately. Based on the setup costs each ant
generates a sequence of items. Afterwards a simple
singlestage lotsizing rule is applied with
modified setup costs. This modification of the setup
costs depends on the position of the item in the
lotsizing sequence, on the items which have been
lotsized before, and on two further parameters,
which are tried to be improved by a systematic
search. For smallsized problems ASMLLS is among
the best algorithms, but for most medium and
largesized problems it outperforms all other
approaches regarding solution quality as well as
computational time.
Keywords: Ant colony optimization, Material requirements
planning, Multilevel lotsizing, WagnerWhitin
algorithm

[1788]

Erik Pitzer, Andreas Beham, and Michael Affenzeller.
Automatic Algorithm Selection for the Quadratic Assignment
Problem Using Fitness Landscape Analysis.
In M. Middendorf and C. Blum, editors, Proceedings of EvoCOP
2013 – 13th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7832 of Lecture Notes in Computer Science, pages
109–120, Heidelberg, 2013. Springer.
[ bib ]

[1789]

Hans E. Plesser.
Reproducibility vs. Replicability: A Brief History of a
Confused Terminology.
Frontiers in Neuroinformatics, 11, January 2018.
[ bib 
DOI ]

[1790]

Dmitry Plotnikov, Dmitry Melnik, Mamikon Vardanyan, Ruben Buchatskiy, Roman
Zhuykov, and JeHyung Lee.
Automatic Tuning of Compiler Optimizations and Analysis of their
Impact.
In V. Alexandrov, M. Lees, V. Krzhizhanovskaya, J. Dongarra, and
P. M. Sloot, editors, 2013 International Conference on Computational
Science, volume 18 of Procedia Computer Science, pages 1312–1321.
Elsevier, 2013.
[ bib 
DOI ]

[1791]

Robert Plotnick.
The Genie in the Machine: How ComputerAutomated Inventing Is
Revolutionizing Law and Business.
Stanford Law Books, 2009.
[ bib ]
Mentions evolutionary optimization of OralB toothbrushes

[1792]

Daniel Porumbel, Gilles Goncalves, Hamid Allaoui, and Tienté Hsu.
Iterated Local Search and Column Generation to solve ArcRouting
as a Permutation SetCovering Problem.
European Journal of Operational Research, 256(2):349–367,
2017.
[ bib ]

[1793]

Juan Porta, Jorge Parapar, Ramón Doallo, Vasco Barbosa, Inés Santé,
Rafael Crecente, and Carlos Díaz.
A Populationbased Iterated Greedy Algorithm for the
Delimitation and Zoning of Rural Settlements.
Computers, Environment and Urban Systems, 39:12–26, 2013.
[ bib ]

[1794]

JeanYves Potvin and S. Bengio.
The Vehicle Routing Problem with Time Windows Part II: Genetic
Search.
INFORMS Journal on Computing, 8:165–172, 1996.
[ bib ]

[1795]

M. Powell.
The BOBYQA algorithm for bound constrained optimization
without derivatives.
Technical Report Cambridge NA Report NA2009/06, University of
Cambridge, UK, 2009.
[ bib ]
http://www6.cityu.edu.hk/rcms/publications/preprint26.pdf

[1796]

T. Devi Prasad.
Design of pumped water distribution networks with storage.
Journal of Water Resources Planning and Management, ASCE,
136(4):129–136, 2009.
[ bib ]

[1797]

Marco Pranzo and D. Pacciarelli.
An Iterated Greedy Metaheuristic for the Blocking Job Shop
Scheduling Problem.
Journal of Heuristics, 22(4):587–611, 2016.
[ bib 
DOI ]

[1798]

Raphael Patrick Prager, Heike Trautmann, Hao Wang, Thomas Bäck, and Pascal
Kerschke.
PerInstance Configuration of the Modularized CMAES by Means
of Classifier Chains and Exploratory Landscape Analysis.
In C. A. Coello Coello, editor, 2020 IEEE Symposium Series
on Computational Intelligence, SSCI 2020, Canberra, Australia, December
14, 2020, pages 996–1003. IEEE Press, 2020.
[ bib ]

[1799]

Kata Praditwong and Xin Yao.
A new multiobjective evolutionary optimisation algorithm: the
twoarchive algorithm.
In Computational intelligence and security, 2006 international
conference on, volume 1, pages 286–291. IEEE, 2006.
[ bib ]

[1800]

T. Devi Prasad and Godfrey A. Walters.
Optimal rerouting to minimise residence times in water
distribution networks.
In C. Maksimović, D. Butler, and F. A. Memon, editors,
Advances in Water Supply Management, pages 299–306. CRC Press, 2003.
[ bib ]

[1801]

F. P. Preparata and M. I. Shamos.
Computational Geometry. An Introduction.
Springer, Berlin, Germany, 2nd edition, 1988.
[ bib ]

[1802]

Kenneth Price, Rainer M. Storn, and Jouni A. Lampinen.
Differential Evolution: A Practical Approach to Global
Optimization.
Springer, New York, NY, 2005.
[ bib 
DOI ]

[1803]

Robert Clay Prim.
Shortest connection networks and some generalizations.
Bell System Technical Journal, 36(6):1389–1401, 1957.
[ bib ]

[1804]

Philipp Probst, Bernd Bischl, and AnneLaure Boulesteix.
Tunability: Importance of Hyperparameters of Machine Learning
Algorithms.
Arxiv preprint arXiv:1802.09596, 2018.
[ bib 
http ]
Keywords: parameter importance

[1805]

Philipp Probst, Bernd Bischl, and AnneLaure Boulesteix.
Tunability: Importance of Hyperparameters of Machine Learning
Algorithms.
Journal of Machine Learning Research, 20(53):1–32, 2019.
[ bib ]

[1806]

Luc Pronzato and Werner G. Müller.
Design of computer experiments: space filling and beyond.
Statistics and Computing, 22(3):681–701, 2012.
[ bib ]
Keywords: Kriging; Entropy; Design of experiments; Spacefilling;
Sphere packing; Maximin design; Minimax design

[1807]

Andy Pryke, Sanaz Mostaghim, and Alireza Nazemi.
Heatmap visualization of population based multi objective
algorithms.
In S. Obayashi et al., editors, Evolutionary Multicriterion
Optimization, EMO 2007, volume 4403 of Lecture Notes in Computer
Science, pages 361–375. Springer, Heidelberg, 2007.
[ bib ]

[1808]

Harilaos N. Psaraftis.
Dynamic Vehicle Routing: Status and Prospects.
Annals of Operations Research, 61:143–164, 1995.
[ bib ]

[1809]

Timo Pukkala and Tero Heinonen.
Optimizing heuristic search in forest planning.
Nonlinear Analysis: Real World Applications, 7(5):1284–1297,
2006.
[ bib ]

[1810]

Gregorio Toscano Pulido and Carlos A. Coello Coello.
The Micro Genetic Algorithm 2: Towards Online Adaptation in
Evolutionary Multiobjective Optimization.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pages 252–266. Springer,
Heidelberg, 2003.
[ bib 
DOI ]

[1811]

Luca Pulina and Armando Tacchella.
A selfadaptive multiengine solver for quantified Boolean
formulas.
Constraints, 14(1):80–116, 2009.
[ bib ]

[1812]

Robin C. Purshouse, Kalyanmoy Deb, Maszatul M. Mansor, Sanaz Mostaghim, and Rui
Wang.
A review of hybrid evolutionary multiple criteria decision
making methods.
COIN Report 2014005, Computational Optimization and Innovation (COIN)
Laboratory, University of Michigan, USA, January 2014.
[ bib ]

[1813]

Robin C. Purshouse and Peter J. Fleming.
Evolutionary manyobjective optimisation: an exploratory
analysis.
In Proceedings of the 2003 Congress on Evolutionary Computation
(CEC'03), pages 2066–2073, Piscataway, NJ, December 2003. IEEE Press.
[ bib 
DOI ]
First to mention exponential number of nondominated solutions
with respect to many objectives? First to mention NSGAII failure to deal with manyobjectives.

[1814]

Robin C. Purshouse and Peter J. Fleming.
On the Evolutionary Optimization of Many Conflicting
Objectives.
IEEE Transactions on Evolutionary Computation, 11(6):770–784,
2007.
[ bib 
DOI ]

[1815]

Markus Püschel, Franz Franchetti, and Yevgen Voronenko.
Spiral.
In D. Padua, editor, Encyclopedia of Parallel Computing, pages
1920–1933. Springer, US, 2011.
[ bib 
DOI ]

[1816]

Yasha Pushak and Holger H. Hoos.
Algorithm Configuration Landscapes: More Benign Than Expected?
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature  PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pages 271–283.
Springer, Cham, Switzerland, 2018.
[ bib 
DOI 
supplementary material ]
Best paper award at PPSN2018

[1817]

Yasha Pushak and Holger H. Hoos.
Golden parameter search: exploiting structure to quickly
configure parameters in parallel.
In C. A. Coello Coello, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2020, pages 245–253. ACM Press,
New York, NY, 2020.
[ bib 
DOI ]
Keywords: algorithm configuration

[1818]

Julianne D. Quinn, Patrick M. Reed, and Klaus Keller.
Direct policy search for robust multiobjective management of
deeply uncertain socioecological tipping points.
Environmental Modelling & Software, 92:125–141, 2017.
[ bib ]

[1819]

Bernd Bischl, Michel Lang, Jakob Bossek, Daniel Horn, Karin Schork, Jakob
Richter, and Pascal Kerschke.
ParamHelpers : Helpers for Parameters in BlackBox
Optimization, Tuning and Machine Learning, 2017.
R package version 1.10.
[ bib 
http ]

[1820]

Hao Yu.
Rmpi: Interface (Wrapper) to MPI (MessagePassing
Interface), 2010.
R package version 0.58.
[ bib 
http ]

[1821]

Thomas BartzBeielstein, J. Ziegenhirt, W. Konen, O. Flasch, P. Koch, and
Martin Zaefferer.
SPOT: Sequential Parameter Optimization, 2011.
R package.
[ bib 
http ]

[1822]

Heike Trautmann, Olaf Mersmann, and David Arnu.
cmaes: Covariance Matrix Adapting Evolutionary
Strategy, 2011.
R package.
[ bib 
http ]

[1823]

Rob Carnell.
lhs: Latin Hypercube Samples, 2016.
R package version 0.14.
[ bib 
http ]

[1824]

Olaf Mersmann.
mco: Multiple Criteria Optimization Algorithms and
Related Functions, 2014.
R package version 1.015.1.
[ bib 
http ]

[1825]

Bernd Bischl, Michel Lang, Jakob Bossek, Leonard Judt, Jakob Richter, Tobias
Kuehn, and Erich Studerus.
mlr: Machine Learning in R, 2013.
R package.
[ bib 
http ]

[1826]

Simon Urbanek.
multicore: Parallel Processing of R Code
on Machines with Multiple Cores or CPUs, 2010.
R package version 0.13.
[ bib 
http ]

[1827]

Jakob Bossek.
smoof: Single and MultiObjective Optimization Test
Functions, 2016.
R package version 1.2.
[ bib 
http ]

[1828]

L. Rachmawati and D. Srinivasan.
Preference incorporation in multiobjective evolutionary
algorithms: A survey.
In Proceedings of the 2006 Congress on Evolutionary Computation
(CEC 2006), pages 3385–3391. IEEE Press, Piscataway, NJ, July 2006.
[ bib ]

[1829]

Andreea Radulescu, Manuel LópezIbáñez, and Thomas Stützle.
Automatically Improving the Anytime Behaviour of Multiobjective
Evolutionary Algorithms.
In R. C. Purshouse, P. J. Fleming, C. M. Fonseca, S. Greco, and
J. Shaw, editors, Evolutionary Multicriterion Optimization, EMO 2013,
volume 7811 of Lecture Notes in Computer Science, pages 825–840.
Springer, Heidelberg, 2013.
[ bib 
DOI ]

[1830]

Shahriar Farahmand Rad, Rubén Ruiz, and Naser Boroojerdian.
New High Performing Heuristics for Minimizing Makespan in
Permutation Flowshops.
Omega, 37(2):331–345, 2009.
[ bib ]

[1831]

Alma A. M. Rahat, Richard M. Everson, and Jonathan E. Fieldsend.
Alternative infill strategies for expensive multiobjective
optimisation.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pages 873–880, New York,
NY, 2017. ACM Press.
[ bib ]

[1832]

C. Rajendran.
Heuristic algorithm for scheduling in a flowshop to minimize
total flowtime.
International Journal of Production Economics, 29(1):65–73,
1993.
[ bib ]

[1833]

C. Rajendran and H. Ziegler.
Antcolony algorithms for permutation flowshop scheduling to
minimize makespan/total flowtime of jobs.
European Journal of Operational Research, 155(2):426–438,
2004.
[ bib ]

[1834]

C. Rajendran and H. Ziegler.
An efficient heuristic for scheduling in a flowshop to minimize
total weighted flowtime of jobs.
European Journal of Operational Research, 103(1):129–138,
1997.
[ bib 
DOI ]

[1835]

David Garzón Ramos and Mauro Birattari.
Automatic Design of Collective Behaviors for Robots that Can
Display and Perceive Colors.
Applied Sciences, 10(13):4654, 2020.
[ bib ]

[1836]

Camelia Ram, Gilberto Montibeller, and Alec Morton.
Extending the use of scenario planning and MCDA for the
evaluation of strategic options.
Journal of the Operational Research Society, 62(5):817–829,
2011.
[ bib ]

[1837]

Marcus Randall.
Near Parameter Free Ant Colony Optimisation.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pages 374–381. Springer, Heidelberg,
2004.
[ bib ]

[1838]

Marcus Randall and James Montgomery.
Candidate Set Strategies for Ant Colony Optimisation.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pages 243–249. Springer, Heidelberg, 2002.
[ bib ]

[1839]

Zhengfu Rao, Jon Wicks, and Sue West.
ENCOMS  An Energy Cost Minimisation System for RealTime,
Operational Control of Water Distribution Networks.
In D. A. Savic, G. A. Walters, R. King, and S. ThiamKhu, editors,
Proceedings of the Eighth International Conference on Computing and
Control for the Water Industry (CCWI 2005), volume 1, pages 85–90,
University of Exeter, UK, September 2005.
[ bib ]

[1840]

Zhengfu Rao and Elad Salomons.
Development of a realtime, nearoptimal control process for
waterdistribution networks.
Journal of Hydroinformatics, 9(1):25–37, 2007.
[ bib 
DOI ]

[1841]

Ronald L. Rardin and Reha Uzsoy.
Experimental Evaluation of Heuristic Optimization Algorithms: A
Tutorial.
Journal of Heuristics, 7(3):261–304, 2001.
[ bib ]

[1842]

Jussi Rasku, Nysret Musliu, and Tommi Kärkkäinen.
Automating the Parameter Selection in VRP: An Offline
Parameter Tuning Tool Comparison.
In W. Fitzgibbon, Y. A. Kuznetsov, P. Neittaanmäki, and
O. Pironneau, editors, Modeling, Simulation and Optimization for Science
and Technology, volume 34 of Computational Methods in Applied
Sciences, pages 191–209. Springer, 2014.
[ bib 
DOI ]
Keywords: irace

[1843]

Jussi Rasku, Nysret Musliu, and Tommi Kärkkäinen.
On automatic algorithm configuration of vehicle routing problem
solvers.
Journal on Vehicle Routing Algorithms, 2(14):1–22, February
2019.
[ bib 
DOI ]
Keywords: irace, SMAC, GGA, REVAC, VRP

[1844]

Carl Edward Rasmussen and Christopher K. I. Williams.
Gaussian Processes for Machine Learning.
MIT Press, Cambridge, MA, 2006.
[ bib ]
Keywords: Gaussian processes, data processing

[1845]

N. Rayner.
Maverick Research: Judgment Day, or Why We Should Let Machines
Automate Decision Making.
Gartner research note, Gartner, Inc, October 2011.
[ bib ]

[1846]

Ingo Rechenberg.
Evolutionsstrategie: Optimierung technischer Systeme nach
Prinzipien der biologischen Evolution.
PhD thesis, Department of Process Engineering, Technical University
of Berlin, 1971.
[ bib ]

[1847]

Ingo Rechenberg.
Evolutionsstrategie: Optimierung technischer Systeme nach
Prinzipien der biologischen Evolution.
FrommannHolzboog, Stuttgart, Germany, 1973.
[ bib ]

[1848]

Ingo Rechenberg.
Case studies in evolutionary experimentation and computation.
Computer Methods in Applied Mechanics and Engineering,
186(24):125–140, 2000.
[ bib 
DOI ]

[1849]

Colin R. Reeves.
Genetic algorithms.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, chapter 5, pages 109–140. Springer, New
York, NY, 2nd edition, 2010.
[ bib ]

[1850]

Patrick M. Reed.
ManyObjective Visual Analytics: Rethinking the Design of
Complex Engineered Systems.
In R. C. Purshouse, P. J. Fleming, C. M. Fonseca, S. Greco, and
J. Shaw, editors, Evolutionary Multicriterion Optimization, EMO 2013,
volume 7811 of Lecture Notes in Computer Science, pages 1–1. Springer,
Heidelberg, 2013.
[ bib ]

[1851]

Colin R. Reeves and A. V. Eremeev.
Statistical analysis of local search landscapes.
Journal of the Operational Research Society, 55(7):687–693,
2004.
[ bib 
eprint ]

[1852]

Gary R. Reeves and Juan J. Gonzalez.
A comparison of two interactive MCDM procedures.
European Journal of Operational Research, 41(2):203–209, 1989.
[ bib 
DOI ]
Keywords: artificial DM, interactive

[1853]

Patrick M. Reed, David Hadka, Jonathan D. Herman, Joseph R. Kasprzyk, and
Joshua B. Kollat.
Evolutionary multiobjective optimization in water resources: The
past, present, and future.
Advances in Water Resources, 51:438–456, 2013.
[ bib ]

[1854]

Marc Reimann.
Guiding ACO by Problem Relaxation: A Case Study on the
Symmetric TSP.
In T. BartzBeielstein, M. J. Blesa, C. Blum, B. Naujoks, A. Roli,
G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics, volume 4771
of Lecture Notes in Computer Science, pages 45–56. Springer,
Heidelberg, 2007.
[ bib ]

[1855]

Gerhard Reinelt.
TSPLIB — A Traveling Salesman Problem Library.
ORSA Journal on Computing, 3(4):376–384, 1991.
[ bib ]

[1856]

Gerhard Reinelt.
The Traveling Salesman: Computational Solutions for TSP
Applications, volume 840 of Lecture Notes in Computer Science.
Springer, Heidelberg, 1994.
[ bib ]

[1857]

Marc Reimann, Karl F. Doerner, and Richard F. Hartl.
Dants: Savings based ants divide and conquer the vehicle
routing problems.
Computers & Operations Research, 31(4):563–591, 2004.
[ bib ]

[1858]

Marc Reimann and Marco Laumanns.
Savings based ant colony optimization for the capacitated
minimum spanning tree problem.
Computers & Operations Research, 33(6):1794–1822, 2006.
[ bib 
DOI ]
The problem of connecting a set of client nodes
with known demands to a root node through a minimum
cost tree network, subject to capacity constraints
on all links is known as the capacitated minimum
spanning tree (CMST) problem. As the problem is
NPhard, we propose a hybrid ant colony
optimization (ACO) algorithm to tackle it
heuristically. The algorithm exploits two important
problem characteristics: (i) the CMST problem is
closely related to the capacitated vehicle routing
problem (CVRP), and (ii) given a clustering of
client nodes that satisfies capacity constraints,
the solution is to find a MST for each cluster,
which can be done exactly in polynomial time. Our
ACO exploits these two characteristics of the
CMST by a solution construction originally
developed for the CVRP. Given the CVRP solution,
we then apply an implementation of Prim's algorithm
to each cluster to obtain a feasible CMST
solution. Results from a comprehensive computational
study indicate the efficiency and effectiveness of
the proposed approach.
Keywords: Ant colony Optimization, Capacitated minimum
spanning tree problem

[1859]

ZhiGang Ren, ZuRen Feng, LiangJun Ke, and ZhaoJun Zhang.
New Ideas for Applying Ant Colony Optimization to the Set
Covering Problem.
Computers and Industrial Engineering, 58(4):774–784, 2010.
[ bib ]

[1860]

Mauricio G. C. Resende and Celso C. Ribeiro.
Greedy Randomized Adaptive Search Procedures.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pages 219–249. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib ]

[1861]

Mauricio G. C. Resende and Celso C. Ribeiro.
Greedy Randomized Adaptive Search Procedures: Advances,
Hybridizations, and Applications.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, pages 283–319. Springer, New York, NY, 2nd
edition, 2010.
[ bib ]

[1862]

M. ReyesSierra and Carlos A. Coello Coello.
Multiobjective particle swarm optimizers: A survey of the
stateoftheart.
International Journal of Computational Intelligence Research,
2(3):287–308, 2006.
[ bib ]

[1863]

Craig W. Reynolds.
Flocks, Herds, and Schools: A Distributed Behavioral Model.
ACM Computer Graphics, 21(4):25–34, 1987.
[ bib ]

[1864]

S. Reza Hejazi and S. Saghafian.
Flowshopscheduling Problems with Makespan Criterion: A Review.
International Journal of Production Research,
43(14):2895–2929, 2005.
[ bib ]

[1865]

Mona Riabacke, Mats Danielson, Love Ekenberg, and Aron Larsson.
A Prescriptive Approach for Eliciting Imprecise Weight
Statements in an MCDA Process.
In F. Rossi and A. Tsoukiàs, editors, Algorithmic Decision
Theory, First International Conference, ADT 2009, volume 5783 of
Lecture Notes in Computer Science, pages 168–179. Springer, Heidelberg,
2009.
[ bib ]

[1866]

Imma Ribas, Ramon Companys, and Xavier TortMartorell.
An iterated greedy algorithm for the flowshop scheduling problem
with blocking.
Omega, 39(3):293 – 301, 2011.
[ bib ]

[1867]

Imma Ribas, Ramon Companys, and Xavier TortMartorell.
An Efficient Iterated Local Search Algorithm for the Total
Tardiness Blocking Flow Shop Problem.
International Journal of Production Research,
51(17):5238–5252, 2013.
[ bib ]

[1868]

Celso C. Ribeiro and Sebastián Urrutia.
Heuristics for the Mirrored Traveling Tournament Problem.
European Journal of Operational Research, 179(3):775–787,
2007.
[ bib ]

[1869]

A. J. Richmond and John E. Beasley.
An Iterative Construction Heuristic for the Ore Selection
Problem.
Journal of Heuristics, 10(2):153–167, 2004.
[ bib ]

[1870]

John R. Rice.
The Algorithm Selection Problem.
Advances in Computers, 15:65–118, 1976.
[ bib ]

[1871]

Enda Ridge and Daniel Kudenko.
Tuning the Performance of the MMAS Heuristic.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2007, volume 4638 of Lecture Notes
in Computer Science, pages 46–60. Springer, Heidelberg, 2007.
[ bib ]

[1872]

Enda Ridge and Daniel Kudenko.
Tuning an Algorithm Using Design of Experiments.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pages 265–286. Springer, Berlin, Germany, 2010.
[ bib ]

[1873]

Sander van Rijn, Hao Wang, Matthijs van Leeuwen, and Thomas Bäck.
Evolving the structure of evolution strategies.
In X. Chen and A. Stafylopatis, editors, Computational
Intelligence (SSCI), 2016 IEEE Symposium Series on, pages 1–8, 2016.
[ bib ]
Keywords: automated design, automatic configuration, cmaes

[1874]

Juan Carlos Rivera, H. Murat Afsar, and Christian Prins.
A Multistart Iterated Local Search for the Multitrip Cumulative
Capacitated Vehicle Routing Problem.
Computational Optimization and Applications, 61(1):159–187,
2015.
[ bib ]

[1875]

Lucía Rivadeneira, JianBo Yang, and Manuel LópezIbáñez.
Predicting tweet impact using a novel evidential reasoning
prediction method.
Expert Systems with Applications, 2021.
[ bib 
DOI ]
This study presents a novel evidential reasoning (ER)
prediction model called MAKERRIMER to examine how different
features embedded in Twitter posts (tweets) can predict the
number of retweets achieved during an electoral campaign. The
tweets posted by the two most voted candidates during the
official campaign for the 2017 Ecuadorian Presidential
election were used for this research. For each tweet, five
features including type of tweet, emotion, URL, hashtag, and
date are identified and coded to predict if tweets are of
either high or low impact. The main contributions of the new
proposed model include its suitability to analyse tweet
datasets based on likelihood analysis of data. The model is
interpretable, and the prediction process relies only on the
use of available data. The experimental results show that
MAKERRIMER performed better, in terms of misclassification
error, when compared against other predictive machine
learning approaches. In addition, the model allows observing
which features of the candidates' tweets are linked to high
and low impact. Tweets containing allusions to the contender
candidate, either with positive or negative connotations,
without hashtags, and written towards the end of the
campaign, were persistently those with the highest
impact. URLs, on the other hand, is the only variable that
performs differently for the two candidates in terms of
achieving high impact. MAKERRIMER can provide campaigners of
political parties or candidates with a tool to measure how
features of tweets are predictors of their impact, which can
be useful to tailor Twitter content during electoral
campaigns.
Keywords: Evidential reasoning rule,Belief rulebased inference,Maximum
likelihood data analysis,Twitter,Retweet,Prediction

[1876]

R Development Core Team.
R: A Language and Environment for Statistical
Computing.
R Foundation for Statistical Computing, Vienna, Austria,
2008.
[ bib 
http ]

[1877]

C. P. Robert.
Simulation of truncated normal variables.
Statistics and Computing, 5(2):121–125, June 1995.
[ bib ]

[1878]

Tea Robič and Bogdan Filipič.
DEMO: Differential Evolution for Multiobjective Optimization.
In C. A. Coello Coello, A. H. Aguirre, and E. Zitzler, editors,
Evolutionary Multicriterion Optimization, EMO 2005, volume 3410 of
Lecture Notes in Computer Science, pages 520–533, Heidelberg, 2005.
Springer.
[ bib ]

[1879]

Francisco J. Rodríguez, Christian Blum, Manuel Lozano, and Carlos
GarcíaMartínez.
Iterated Greedy Algorithms for the Maximal Covering Location
Problem.
In J.K. Hao and M. Middendorf, editors, Proceedings of EvoCOP
2012 – 12th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7245 of Lecture Notes in Computer Science, pages
172–181. Springer, Heidelberg, 2012.
[ bib ]

[1880]

Cynthia A. Rodríguez Villalobos and Carlos A. Coello Coello.
A new multiobjective evolutionary algorithm based on a
performance assessment indicator.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pages 505–512. ACM
Press, New York, NY, 2012.
[ bib ]

[1881]

P. A. Romero, A. Krause, and F. H. Arnold.
Navigating the Protein Fitness Landscape with Gaussian
Processes.
Proceedings of the National Academy of Sciences,
110(3):E193–E201, December 2012.
[ bib 
DOI ]
Keywords: Combinatorial Blackbox Expensive

[1882]

Fabio Romeo and Alberto SangiovanniVincentelli.
A Theoretical Framework for Simulated Annealing.
Algorithmica, 6(16):302–345, 1991.
[ bib ]

[1883]

David S. Roos.
Bioinformatics–trying to swim in a sea of data.
Science, 291(5507):1260–1261, 2001.
[ bib ]

[1884]

Stefan Ropke and David Pisinger.
A Unified Heuristic for a Large Class of Vehicle Routing
Problems with Backhauls.
European Journal of Operational Research, 171(3):750–775,
2006.
[ bib ]

[1885]

Stefan Ropke and David Pisinger.
An Adaptive Large Neighborhood Search Heuristic for the Pickup
and Delivery Problme with Time Windows.
Transportation Science, 40(4):455–472, 2006.
[ bib ]

[1886]

Peter Ross.
HyperHeuristics.
In E. K. Burke and G. Kendall, editors, Search Methodologies,
pages 529–556. Springer, Boston, MA, 2005.
[ bib 
DOI ]

[1887]

Jonathan Rose, Wolfgang Klebsch, and Jürgen Wolf.
Temperature measurement and equilibrium dynamics of simulated
annealing placements.
IEEE Transactions on ComputerAided Design of Integrated
Circuits and Systems, 9(3):253–259, 1990.
[ bib ]

[1888]

Edward Rothberg.
An evolutionary algorithm for polishing mixed integer
programming solutions.
INFORMS Journal on Computing, 19(4):534–541, 2007.
[ bib ]

[1889]

Daniel H. Rothman.
Nonlinear inversion, statistical mechanics, and residual statics
estimation.
Geophysics, 50(12):2784–2796, 1985.
[ bib ]

[1890]

Daniel H. Rothman.
Automatic estimation of large residual statics corrections.
Geophysics, 51(2):332–346, 1986.
[ bib ]

[1891]

Bernard Roy.
Robustness in operational research and decision aiding: A
multifaceted issue.
European Journal of Operational Research, 200(3):629–638,
2010.
[ bib 
DOI 
http ]

[1892]

Frank Rubin.
An Iterative Technique for Printed Wire Routing.
In DAC'74, Proceedings of the 11th Design Automation Workshop,
pages 308–313. IEEE Press, 1974.
[ bib ]

[1893]

Günther Rudolph and Alexandru Agapie.
Convergence Properties of Some MultiObjective Evolutionary
Algorithms.
In Proceedings of the 2000 Congress on Evolutionary Computation
(CEC'00), volume 2, pages 1010–1016, Piscataway, NJ, July 2000. IEEE Press.
[ bib ]

[1894]

Günther Rudolph, Oliver Schütze, Christian Grimme, Christian
DomínguezMedina, and Heike Trautmann.
Optimal averaged Hausdorff archives for biobjective problems:
theoretical and numerical results.
Computational Optimization and Applications, 64(2):589–618,
2016.
[ bib ]

[1895]

Günther Rudolph.
On Correlated Mutations in Evolution Strategies.
In R. Männer and B. Manderick, editors, Parallel Problem
Solving from Nature – PPSN II, pages 107–116. Elsevier, 1992.
[ bib ]

[1896]

Günther Rudolph.
Convergence of nonelitist strategies.
In Z. Michalewicz, editor, Proceedings of the First IEEE
International Conference on Evolutionary Computation (ICEC'94), pages
63–66, Piscataway, NJ, 1994. IEEE Press.
[ bib ]

[1897]

Günther Rudolph.
Convergence analysis of canonical genetic algorithms.
tnn, 5(1):96–101, 1994.
[ bib ]

[1898]

Ana B. Ruiz, Mariano Luque, Kaisa Miettinen, and Rubén Saborido.
An Interactive Evolutionary Multiobjective Optimization Method:
Interactive WASFGA.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part II,
volume 9019 of Lecture Notes in Computer Science, pages 249–263.
Springer International Publishing, Heidelberg, 2015.
[ bib 
DOI ]

[1899]

Rubén Ruiz and C. Maroto.
A Comprehensive Review and Evaluation of Permutation Flowshop
Heuristics.
European Journal of Operational Research, 165(2):479–494,
2005.
[ bib ]

[1900]

Rubén Ruiz, C. Maroto, and Javier Alcaraz.
Two new robust genetic algorithms for the flowshop scheduling
problem.
Omega, 34(5):461–476, 2006.
[ bib 
DOI ]

[1901]

Rubén Ruiz and Thomas Stützle.
A Simple and Effective Iterated Greedy Algorithm for the
Permutation Flowshop Scheduling Problem.
European Journal of Operational Research, 177(3):2033–2049,
2007.
[ bib ]

[1902]

Rubén Ruiz and Thomas Stützle.
An Iterated Greedy heuristic for the sequence dependent
setup times flowshop problem with makespan and weighted tardiness
objectives.
European Journal of Operational Research, 187(3):1143 – 1159,
2008.
[ bib ]

[1903]

Rubén Ruiz, Eva Vallada, and Carlos FernándezMartínez.
Scheduling in flowshops with noidle machines.
In Computational intelligence in flow shop and job shop
scheduling, pages 21–51. Springer, 2009.
[ bib ]

[1904]

W. Ruml.
Incomplete Tree Search using Adaptive Probing.
In B. Nebel, editor, Proceedings of the Seventeenth
International Joint Conference on Artificial Intelligence (IJCAI01), pages
235–241. IEEE Press, 2001.
[ bib ]

[1905]

Stuart J. Russell and Peter Norvig.
Artificial Intelligence: A Modern Approach, volume 2.
Prentice Hall, Englewood Cliffs, NJ, 2003.
[ bib ]

[1906]

Robert A. Russell.
Hybrid Heuristics for the Vehicle Routing Problem with Time
Windows.
Transportation Science, 29(2):156–166, 1995.
[ bib ]

[1907]

John Rust.
Structural estimation of Markov decision processes.
In Handbook of Econometrics, volume 4, pages 3081–3143.
Elsevier, 1994.
[ bib 
DOI ]

[1908]

Michael Behrisch, Laura Bieker, Jakob Erdmann, and Daniel Krajzewicz.
SUMO  Simulation of Urban MObility: An Overview.
In SIMUL 2011, The Third International Conference on Advances in
System Simulation, pages 63–68, Barcelona, Spain, 2011. ThinkMind.
[ bib ]

[1909]

N. R. Sabar, M. Ayob, Graham Kendall, and R. Qu.
Grammatical Evolution HyperHeuristic for Combinatorial
Optimization Problems.
IEEE Transactions on Evolutionary Computation, 17(6):840–861,
2013.
[ bib ]

[1910]

N. R. Sabar, M. Ayob, Graham Kendall, and R. Qu.
A Dynamic Multiarmed BanditGene Expression Programming
HyperHeuristic for Combinatorial Optimization Problems.
IEEE Transactions on Cybernetics, 45(2):217–228, 2015.
[ bib ]

[1911]

N. R. Sabar, M. Ayob, Graham Kendall, and R. Qu.
Automatic Design of a HyperHeuristic Framework With Gene
Expression Programming for Combinatorial Optimization Problems.
IEEE Transactions on Evolutionary Computation, 19(3):309–325,
2015.
[ bib ]

[1912]

Matthieu Sacher, Régis Duvigneau, Olivier Le Maitre, Mathieu Durand, Elisa
Berrini, Frédéric Hauville, and JacquesAndré Astolfi.
A classification approach to efficient global optimization in
presence of noncomputable domains.
Structural and Multidisciplinary Optimization,
58(4):1537–1557, 2018.
[ bib 
DOI ]
Proposed EGOLSSVM
Keywords: Safe optimization; CMAES, Gaussian processes; LeastSquares
Support Vector Machine

[1913]

Pramod J. Sadalage and Martin Fowler.
NoSQL distilled.
AddisonWesley Professional, 2012.
[ bib ]

[1914]

Bhupinder Singh Saini, Manuel LópezIbáñez, and Kaisa Miettinen.
Automatic Surrogate Modelling Technique Selection based on
Features of Optimization Problems.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference Companion, GECCO Companion 2019, pages 1765–1772. ACM Press, New
York, NY, 2019.
[ bib 
DOI ]
A typical scenario when solving industrial single or
multiobjective optimization problems is that no explicit
formulation of the problem is available. Instead, a dataset
containing vectors of decision variables together with their
objective function value(s) is given and a surrogate model
(or metamodel) is build from the data and used for
optimization and decisionmaking. This datadriven
optimization process strongly depends on the ability of the
surrogate model to predict the objective value of decision
variables not present in the original dataset. Therefore, the
choice of surrogate modelling technique is crucial. While
many surrogate modelling techniques have been discussed in
the literature, there is no standard procedure that will
select the best technique for a given problem. In this work,
we propose the automatic selection of a surrogate modelling
technique based on exploratory landscape features of the
optimization problem that underlies the given dataset. The
overall idea is to learn offline from a large pool of
benchmark problems, on which we can evaluate a large number
of surrogate modelling techniques. When given a new dataset,
features are used to select the most appropriate surrogate
modelling technique. The preliminary experiments reported
here suggest that the proposed automatic selector is able to
identify highaccuracy surrogate models as long as an
appropriate classifier is used for selection.

[1915]

Yoshitaka Sakurai, Kouhei Takada, Takashi Kawabe, and Setsuo Tsuruta.
A method to control parameters of evolutionary algorithms by
using reinforcement learning.
In 2010 Sixth International Conference on SignalImage
Technology and Internet Based Systems, pages 74–79. IEEE, 2010.
[ bib ]

[1916]

A. Burcu Altan Sakarya and Larry W. Mays.
Optimal Operation of Water Distribution Pumps Considering Water
Quality.
Journal of Water Resources Planning and Management, ASCE,
126(4):210–220, July / August 2000.
[ bib ]

[1917]

A. Burcu Altan Sakarya, Fred E. Goldman, and Larry W. Mays.
Models for the optimal scheduling of pumps to meet water
quality.
In D. A. Savic and G. A. Walters, editors, Water Industry
Systems: Modelling and Optimization Applications, volume 2, pages 379–391.
Research Studies Press Ltd., Baldock, United Kingdom, 1999.
[ bib ]

[1918]

Francesco Sambo, Barbara Di Camillo, Alberto Franzin, Andrea Facchinetti, Liisa
Hakaste, Jasmina Kravic, Giuseppe Fico, Jaakko Tuomilehto, Leif Groop, Rafael
Gabriel, Tiinamaija Tuomi, and Claudio Cobelli.
A Bayesian Network analysis of the probabilistic relations
between risk factors in the predisposition to type 2 diabetes.
In N. Lovell and L. Mainardi, editors, 37th Annual International
Conference of the IEEE Engineering in Medicine and Biology Society, EMBC
2015, Proceedings, Annual International Conference of the IEEE Engineering
in Medicine and Biology, pages 2119–2122. IEEE Press, 2015.
[ bib ]

[1919]

Marcela Samà, Paola Pellegrini, Andrea D'Ariano, Joaquin Rodriguez, and Dario
Pacciarelli.
Ant colony optimization for the realtime train routing
selection problem.
Transportation Research Part B: Methodological, 85:89–108,
2016.
[ bib 
DOI ]
Keywords: irace

[1920]

Malcolm Sambridge.
Geophysical inversion with a neighbourhood algorithm–I.
Searching a parameter space.
Geophysical Journal International, 138(2):479–494, 1999.
[ bib ]

[1921]

Javier Sánchez, Manuel Galán, and Enrique Rubio.
Applying a traffic lights evolutionary optimization technique to
a real case: “Las Ramblas” area in Santa Cruz de Tenerife.
IEEE Transactions on Evolutionary Computation, 12(1):25–40,
2008.
[ bib ]
Keywords: Cellular automata (CA),Combinatorial optimization,Ggenetic
algorithms (GAs),Microscopic traffic simulator,Traffic lights
optimization

[1922]

J. J. SánchezMedina, M. J. GalánMoreno, and E. RubioRoyo.
Traffic Signal Optimization in “La Almozara” District in
Saragossa Under Congestion Conditions, Using Genetic Algorithms, Traffic
Microsimulation, and Cluster Computing.
IEEE Transactions on Intelligent Transportation Systems,
11(1):132–141, March 2010.
[ bib 
DOI ]
Keywords: cellular automata;genetic algorithms;road traffic;road
vehicles;traffic engineering computing;Beowulf cluster;La
Almozara district;Saragossa;cellular automata;cluster
computing;genetic algorithm;multipleinstruction multiple
data;traffic light programming;traffic
microsimulation;traffic signal optimization;urban traffic
congestion;Cellular automata (CA);genetic algorithms
(GAs);intelligent transportation
systems;microsimulation;traffic congestion;traffic modeling

[1923]

Nathan Sankary and Avi Ostfeld.
Stochastic Scenario Evaluation in Evolutionary Algorithms Used
for Robust ScenarioBased Optimization.
Water Resources Research, 54(4):2813–2833, 2018.
[ bib ]

[1924]

Alberto Santini, Stefan Ropke, and Lars Magnus Hvattum.
A compar