IRIDIA BibTeX Repository
What is this?
This list of references in automatically generated from a collection of BibTeX files organized in a way that tries to avoid redundancy, minimise mistakes and facilitate customization.
You only need to fork (or link) the git repository in your papers and sync with the main copy to send/receive updates.
Most customisations, such as shorter journal or conference names, do not require changing the existing .bib
files.
You should not need to edit the entries directly unless you find mistakes. See the README for more details.
References

[1]

David Abramson, Mohan Krishna Amoorthy, and Henry Dang.
Simulated annealing cooling schedules for the school timetabling
problem.
AsiaPacific Journal of Operational Research, 16(1):1–22,
1999.
[ bib ]

[2]

David Abramson.
Constructing School Timetables Using Simulated Annealing:
Sequential and Parallel Algorithms.
Management Science, 37(1):98–113, 1991.
[ bib ]

[3]

Tobias Achterberg.
SCIP: Solving constraint integer programs.
Mathematical Programming Computation, 1(1):1–41, July 2009.
[ bib 
epub ]

[4]

Tobias Achterberg and Timo Berthold.
Improving the feasibility pump.
Discrete Optimization, 4(1):77–86, 2007.
[ bib ]

[5]

HéctorGabriel AcostaMesa, Fernando RechyRamírez, Efrén
MezuraMontes, Nicandro CruzRamírez, and Rodolfo
Hernández Jiménez.
Application of time series discretization using evolutionary
programming for classification of precancerous cervical lesions.
Journal of Biomedical Informatics, 49:73–83, 2014.
[ bib 
DOI ]
Keywords: irace

[6]

Bernardetta Addis, Marco Locatelli, and Fabio Schoen.
Disk Packing in a Square: A New Global Optimization Approach.
INFORMS Journal on Computing, 20(4):516–524, 2008.
[ bib 
DOI ]

[7]

B. AdensoDíaz.
Restricted Neighborhood in the Tabu Search for the Flowshop
Problem.
European Journal of Operational Research, 62(1):27–37, 1992.
[ bib ]

[8]

B. AdensoDíaz and Manuel Laguna.
FineTuning of Algorithms Using Fractional Experimental Design
and Local Search.
Operations Research, 54(1):99–114, 2006.
[ bib ]
Keywords: Calibra

[9]

Bekir Afsar, Kaisa Miettinen, and Francisco Ruiz.
Assessing the Performance of Interactive Multiobjective
Optimization Methods: A Survey.
ACM Computing Surveys, 54(4), 2021.
[ bib 
DOI ]
Interactive methods are useful decisionmaking tools for
multiobjective optimization problems, because they allow a
decisionmaker to provide her/his preference information
iteratively in a comfortable way at the same time as (s)he
learns about all different aspects of the problem. A wide
variety of interactive methods is nowadays available, and
they differ from each other in both technical aspects and
type of preference information employed. Therefore, assessing
the performance of interactive methods can help users to
choose the most appropriate one for a given problem. This is
a challenging task, which has been tackled from different
perspectives in the published literature. We present a
bibliographic survey of papers where interactive
multiobjective optimization methods have been assessed
(either individually or compared to other methods). Besides
other features, we collect information about the type of
decisionmaker involved (utility or value functions,
artificial or human decisionmaker), the type of preference
information provided, and aspects of interactive methods that
were somehow measured. Based on the survey and on our own
experiences, we identify a series of desirable properties of
interactive methods that we believe should be assessed.
Keywords: decisionmakers, Interactive methods, performance assessment,
preference information, multiobjective optimization problems

[10]

Ralph D'Agostino and E. S. Pearson.
Tests for Departure from Normality. Empirical Results for the
Distributions of b_{2} and √b_{1}.
Biometrika, 60(3):613–622, December 1973.
[ bib 
DOI ]

[11]

Per J. Agrell.
On redundancy in multi criteria decision making.
European Journal of Operational Research, 98(3):571–586, 1997.
[ bib 
DOI ]
The concept of redundancy is accepted in Operations Research
and Information Theory. In Linear Programming, a constraint
is said to be redundant if the feasible decision space is
identical with or without the constraint. In Information
Theory, redundancy is used as a measure of the stability
against noise in transmission. Analogies with Multi Criteria
Decision Making (MCDM) are indicated and it is argued that
the redundancy concept should be used as a regular feature in
conditioning and analysis of Multi Criteria
Programs. Properties of a proposed conflictbased
characterisation are stated and some existence results are
derived. Redundancy is here intended for interactive methods,
when the efficient set is progressively explored. A new
redundancy test for the linear case is formulated from the
framework. A probabilistic method based on correlation is
proposed and tested for the nonlinear case. Finally, some
general guidelines are given concerning the redundancy
problem.
Keywords: Multi criteria decision making, Redundancy, objective
reduction, Vector optimisation

[12]

Hernán E. Aguirre and Kiyoshi Tanaka.
Working principles, behavior, and performance of MOEAs on
MNKlandscapes.
European Journal of Operational Research, 181(3):1670–1690,
2007.
[ bib 
DOI ]

[13]

Samad Ahmadi and Ibrahim H. Osman.
Density Based Problem Space Search for the Capacitated
Clustering pMedian Problem.
Annals of Operations Research, 131:21–43, 2004.
[ bib ]

[14]

Ali Ahrari, Saber Elsayed, Ruhul Sarker, Daryl Essam, and Carlos A. Coello
Coello.
Weighted pointwise prediction method for dynamic multiobjective
optimization.
Information Sciences, 546:349–367, 2021.
[ bib ]

[15]

R. K. Ahuja, O. Ergun, and A. P. Punnen.
A Survey of Very Largescale Neighborhood Search Techniques.
Discrete Applied Mathematics, 123(1–3):75–102, 2002.
[ bib ]

[16]

Sandip Aine, Rajeev Kumar, and P. P. Chakrabarti.
Adaptive parameter control of evolutionary algorithms to improve
qualitytime tradeoff.
Applied Soft Computing, 9(2):527–540, 2009.
[ bib 
DOI ]
Keywords: anytime

[17]

A. A. Albrecht, P. C. R. Lane, and K. Steinhöfel.
Analysis of Local Search Landscapes for kSAT Instances.
Mathematics in Computer Science, 3(4):465–488, 2010.
[ bib 
DOI ]

[18]

Susanne Albers.
Online Algorithms: A Survey.
Mathematical Programming, 97(1):3–26, 2003.
[ bib ]

[19]

Aldeida Aleti and Irene Moser.
A systematic literature review of adaptive parameter control
methods for evolutionary algorithms.
ACM Computing Surveys, 49(3, Article 56):35, October 2016.
[ bib 
DOI ]

[20]

Pedro AlfaroFernández, Rubén Ruiz, Federico Pagnozzi, and Thomas
Stützle.
Automatic Algorithm Design for Hybrid Flowshop Scheduling
Problems.
European Journal of Operational Research, 282(3):835–845,
2020.
[ bib 
DOI ]
Industrial production scheduling problems are challenges that
researchers have been trying to solve for decades. Many
practical scheduling problems such as the hybrid flowshop are
NPhard. As a result, researchers resort to metaheuristics to
obtain effective and efficient solutions. The traditional
design process of metaheuristics is mainly manual, often
metaphorbased, biased by previous experience and prone to
producing overly tailored methods that only work well on the
tested problems and objectives. In this paper, we use an
Automatic Algorithm Design (AAD) methodology to eliminate
these limitations. AAD is capable of composing algorithms
from components with minimal human intervention. We test the
proposed AAD for three different optimization objectives in
the hybrid flowshop. Comprehensive computational and
statistical testing demonstrates that automatically designed
algorithms outperform specifically tailored stateoftheart
methods for the tested objectives in most cases.
Keywords: Scheduling, Hybrid flowshop, Automatic algorithm
configuration, Automatic Algorithm Design

[21]

Alnur Ali and Marina Meilă.
Experiments with Kemeny ranking: What Works When?
Mathematical Social Science, 64(1):28–40, July 2012.
[ bib 
DOI ]
Computational Foundations of Social Choice
Keywords: Borda ranking, Kemeny ranking

[22]

Ali Allahverdi and Harun Aydilek.
Algorithms for nowait flowshops with total completion time
subject to makespan.
International Journal of Advanced Manufacturing Technology,
pp. 1–15, 2013.
[ bib ]

[23]

Richard Allmendinger, Andrzej Jaszkiewicz, Arnaud Liefooghe, and Christiane
Tammer.
What if we increase the number of objectives? Theoretical and
empirical implications for manyobjective combinatorial optimization.
Computers & Operations Research, 145:105857, 2022.
[ bib 
DOI ]

[24]

Richard Allmendinger and Joshua D. Knowles.
On Handling Ephemeral Resource Constraints in Evolutionary
Search.
Evolutionary Computation, 21(3):497–531, September 2013.
[ bib 
DOI ]
We consider optimization problems where the set of solutions
available for evaluation at any given time t during
optimization is some subset of the feasible space. This model
is appropriate to describe many closedloop optimization
settings (i.e. where physical processes or experiments are
used to evaluate solutions) where, due to resource
limitations, it may be impossible to evaluate particular
solutions at particular times (despite the solutions being
part of the feasible space). We call the constraints
determining which solutions are nonevaluable ephemeral
resource constraints (ERCs). In this paper, we investigate
two specific types of ERC: one encodes periodic resource
availabilities, the other models `commitment' constraints
that make the evaluable part of the space a function of
earlier evaluations conducted. In an experimental study, both
types of constraint are seen to impact the performance of an
evolutionary algorithm significantly. To deal with the
effects of the ERCs, we propose and test five different
constrainthandling policies (adapted from those used to
handle `standard' constraints), using a number of different
test functions including a fitness landscape from a real
closedloop problem. We show that knowing information about
the type of resource constraint in advance may be sufficient
to select an effective policy for dealing with it, even when
advance knowledge of the fitness landscape is limited.

[25]

Christian Almeder.
A hybrid optimization approach for multilevel capacitated
lotsizing problems.
European Journal of Operational Research, 200(2):599–606,
2010.
[ bib 
DOI ]
Solving multilevel capacitated lotsizing problems
is still a challenging task, in spite of increasing
computational power and faster algorithms. In this
paper a new approach combining an antbased
algorithm with an exact solver for (mixedinteger)
linear programs is presented. A MAXMIN ant system
is developed to determine the principal production
decisions, a LP/MIP solver is used to calculate
the corresponding production quantities and
inventory levels. Two different local search methods
and an improvement strategy based on reduced
mixedinteger problems are developed and integrated
into the ant algorithm. This hybrid approach
provides superior results for small and mediumsized
problems in comparison to the existing approaches in
the literature. For largescale problems the
performance of this method is among the best
Keywords: Ant colony optimization, Manufacturing, Material
requirements planning, Mixedinteger programming

[26]

S. Alupoaei and S. Katkoori.
Ant Colony System Application to Marcocell Overlap Removal.
IEEE Transactions on Very Large Scale Integration (VLSI)
Systems, 12(10):1118–1122, 2004.
[ bib ]

[27]

C. Amir, A. Badr, and I Farag.
A Fuzzy Logic Controller for Ant Algorithms.
Computing and Information Systems, 11(2):26–34, 2007.
[ bib ]

[28]

Christophe Andrieu, Nando de Freitas, Arnaud Doucet, and Michael I. Jordan.
An Introduction to MCMC for Machine Learning.
Machine Learning, 50(12):5–43, 2003.
[ bib ]

[29]

K. A. Andersen, K. Jörnsten, and M. Lind.
On bicriterion minimal spanning trees: An approximation.
Computers & Operations Research, 23(12):1171–1182, 1996.
[ bib ]

[30]

Y. P. Aneja and K. P. K. Nair.
Bicriteria Transportation Problem.
Management Science, 25(1):73–78, 1979.
[ bib ]

[31]

Eric Angel, Evripidis Bampis, and Laurent Gourvés.
Approximating the Pareto curve with local search for the
bicriteria TSP(1,2) problem.
Theoretical Computer Science, 310(13):135–146, 2004.
[ bib 
DOI ]
Keywords: Archiving, Local search, Multicriteria TSP,
Approximation algorithms

[32]

Daniel Angus and Clinton Woodward.
Multiple Objective Ant Colony Optimisation.
Swarm Intelligence, 3(1):69–85, 2009.
[ bib 
DOI ]

[33]

Kurt Anstreicher, Nathan Brixius, JeanPierre Goux, and Jeff Linderoth.
Solving large quadratic assignment problems on computational
grids.
Mathematical Programming Series B, 91(3):563–588, February
2002.
[ bib 
DOI ]
The quadratic assignment problem (QAP) is among the hardest
combinatorial optimization problems. Some instances of size
n = 30 have remained unsolved for decades. The solution of
these problems requires both improvements in mathematical
programming algorithms and the utilization of powerful
computational platforms. In this article we describe a novel
approach to solve QAPs using a stateoftheart
branchandbound algorithm running on a federation of
geographically distributed resources known as a computational
grid. Solution of QAPs of unprecedented complexity, including
the nug30, kra30b, and tho30 instances, is reported.

[34]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Implementing the DantzigFulkersonJohnson Algorithm for
Large Traveling Salesman Problems.
Mathematical Programming Series B, 97(1–2):91–153, 2003.
[ bib 
DOI ]

[35]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
On the Solution of Traveling Salesman Problems.
Documenta Mathematica, Extra Volume ICM III:645–656, 1998.
[ bib ]

[36]

J. S. Appleby, D. V. Blake, and E. A. Newman.
Techniques for producing school timetables on a computer and
their application to other scheduling problems.
The Computer Journal, 3(4):237–245, 1961.
[ bib 
DOI ]

[37]

David Applegate and William J. Cook.
A Computational Study of the JobShop Scheduling Problem.
ORSA Journal on Computing, 3(2):149–156, 1991.
[ bib ]

[38]

David Applegate, William J. Cook, and André Rohe.
Chained LinKernighan for Large Traveling Salesman
Problems.
INFORMS Journal on Computing, 15(1):82–92, 2003.
[ bib 
DOI ]

[39]

David Applegate, Robert E. Bixby, Vašek Chvátal, William J. Cook,
D. Espinoza, M. Goycoolea, and Keld Helsgaun.
Certification of an Optimal TSP Tour Through 85,900 Cities.
Operations Research Letters, 37(1):11–15, 2009.
[ bib ]

[40]

Claus Aranha, Christian Leonardo CamachoVillalón, Felipe Campelo, Marco
Dorigo, Rubén Ruiz, Marc Sevaux, Kenneth Sörensen, and Thomas
Stützle.
Metaphorbased Metaheuristics, a Call for Action: the Elephant
in the Room.
Swarm Intelligence, 16(1):1–6, 2022.
[ bib 
DOI ]

[41]

Florian Arnold, Ítalo Santana, Kenneth Sörensen, and Thibaut Vidal.
PILS: Exploring highorder neighborhoods by pattern mining and
injection.
Arxiv preprint arXiv:1912.11462, 2019.
[ bib 
http ]

[42]

Florian Arnold and Kenneth Sörensen.
Knowledgeguided local search for the vehicle routing problem.
Computers & Operations Research, 105:32–46, 2019.
[ bib 
DOI ]

[43]

Florian Arnold and Kenneth Sörensen.
What makes a VRP solution good? The generation of
problemspecific knowledge for heuristics.
Computers & Operations Research, 106:280–288, 2019.
[ bib 
DOI ]

[44]

Marvin A. Arostegui Jr, Sukran N. Kadipasaoglu, and Basheer M. Khumawala.
An empirical comparison of tabu search, simulated annealing, and
genetic algorithms for facilities location problems.
International Journal of Production Economics, 103(2):742–754,
2006.
[ bib ]

[45]

José Elias C. Arroyo and V. A. Armentano.
A partial enumeration heuristic for multiobjective flowshop
scheduling problems.
Journal of the Operational Research Society, 55(9):1000–1007,
2004.
[ bib ]

[46]

José Elias C. Arroyo and V. A. Armentano.
Genetic local search for multiobjective flowshop scheduling
problems.
European Journal of Operational Research, 167(3):717–738,
2005.
[ bib ]
Keywords: Multicriteria Scheduling

[47]

José Elias C. Arroyo and Joseph Y.T. Leung.
An Effective Iterated Greedy Algorithm for Scheduling Unrelated
Parallel Batch Machines with Nonidentical Capacities and Unequal Ready
Times.
Computers and Industrial Engineering, 105:84–100, 2017.
[ bib ]

[48]

N. Ascheuer, Matteo Fischetti, and M. Grötschel.
Solving asymmetric travelling salesman problem with time windows
by branchandcut.
Mathematical Programming, 90:475–506, 2001.
[ bib ]

[49]

JohnAlexander M. Assael, Ziyu Wang, and Nando de Freitas.
Heteroscedastic Treed Bayesian Optimisation.
Arxiv preprint arXiv:1410.7172, 2014.
[ bib 
DOI ]
Keywords: TreedGP

[50]

Alper Atamtürk.
On the facets of the mixed–integer knapsack polyhedron.
Mathematical Programming, 98(1):145–175, 2003.
[ bib 
DOI ]

[51]

Charles Audet, CongKien Dang, and Dominique Orban.
Optimization of Algorithms with OPAL.
Mathematical Programming Computation, 6(3):233–254, 2014.
[ bib ]

[52]

P. Audze and Vilnis Eglãjs.
New approach to the design of multifactor experiments.
Problems of Dynamics and Strengths, 35:104–107, 1977.
(in Russian).
[ bib ]

[53]

Charles Audet and Dominique Orban.
Finding Optimal Algorithmic Parameters Using DerivativeFree
Optimization.
SIAM Journal on Optimization, 17(3):642–664, 2006.
[ bib 
DOI ]
Keywords: mesh adaptive direct search; pattern search

[54]

Peter Auer.
Using Confidence Bounds for ExploitationExploration
Tradeoffs.
Journal of Machine Learning Research, 3:397–422, November
2002.
[ bib ]
We show how a standard tool from statistics — namely
confidence bounds — can be used to elegantly deal with
situations which exhibit an exploitationexploration
tradeoff. Our technique for designing and analyzing
algorithms for such situations is general and can be applied
when an algorithm has to make exploitationversusexploration
decisions based on uncertain information provided by a random
process. We apply our technique to two models with such an
exploitationexploration tradeoff. For the adversarial
bandit problem with shifting our new algorithm suffers only
O((ST)^{1/2}) regret with high probability over T trials
with S shifts. Such a regret bound was previously known
only in expectation. The second model we consider is
associative reinforcement learning with linear value
functions. For this model our technique improves the regret
from O(T^{3/4}) to O(T^{1/2}).

[55]

Peter Auer, Nicolo CesaBianchi, and Paul Fischer.
Finitetime analysis of the multiarmed bandit problem.
Machine Learning, 47(23):235–256, 2002.
[ bib ]

[56]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Hypervolumebased multiobjective optimization: Theoretical
foundations and practical implications.
Theoretical Computer Science, 425:75–103, 2012.
[ bib 
DOI ]

[57]

Mustafa Avci and Seyda Topaloglu.
A Multistart Iterated Local Search Algorithm for the
Generalized Quadratic Multiple Knapsack Problem.
Computers & Operations Research, 83:54–65, 2017.
[ bib ]

[58]

Andreea Avramescu, Richard Allmendinger, and Manuel LópezIbáñez.
Managing Manufacturing and Delivery of Personalised Medicine:
Current and Future Models.
Arxiv preprint arXiv:2105.12699 [econ.GN], 2021.
[ bib 
http ]

[59]

Doǧan Aydın, Gürcan Yavuz, and Thomas Stützle.
ABCX: A Generalized, Automatically Configurable Artificial
Bee Colony Framework.
Swarm Intelligence, 11(1):1–38, 2017.
[ bib ]

[60]

Mahdi Aziz and MohammadH. TayaraniN.
An adaptive memetic Particle Swarm Optimization algorithm for
finding largescale Latin hypercube designs.
Engineering Applications of Artificial Intelligence,
36:222–237, 2014.
[ bib 
DOI ]
Keywords: Frace

[61]

François Bachoc, Céline Helbert, and Victor Picheny.
Gaussian process optimization with failures: Classification and
convergence proof.
Journal of Global Optimization, 2020.
[ bib 
DOI 
epub ]
We consider the optimization of a computer model where each
simulation either fails or returns a valid output
performance. We first propose a new joint Gaussian process
model for classification of the inputs (computation failure
or success) and for regression of the performance
function. We provide results that allow for a computationally
efficient maximum likelihood estimation of the covariance
parameters, with a stochastic approximation of the likelihood
gradient. We then extend the classical improvement criterion
to our setting of joint classification and regression. We
provide an efficient computation procedure for the extended
criterion and its gradient. We prove the almost sure
convergence of the global optimization algorithm following
from this extended criterion. We also study the practical
performances of this algorithm, both on simulated data and on
a real computer model in the context of automotive fan
design.
Keywords: crashed simulation; latent gaussian process; automotive fan
design; industrial application; GP classification; Expected
Feasible Improvement with Gaussian Process Classification
with signs; EFI GPC sign

[62]

Johannes Bader and Eckart Zitzler.
HypE: An Algorithm for Fast HypervolumeBased ManyObjective
Optimization.
Evolutionary Computation, 19(1):45–76, 2011.
[ bib 
DOI ]

[63]

Hossein Baharmand, Tina Comes, and Matthieu Lauras.
Biobjective multilayer location– allocation model
for the immediate aftermath of suddenonset disasters.
Transportation Research Part E: Logistics and Transportation
Review, 127:86–110, 2019.
[ bib 
DOI ]
Locating distribution centers is critical for humanitarians
in the immediate aftermath of a suddenonset disaster. A
major challenge lies in balancing the complexity and
uncertainty of the problem with time and resource
constraints. To address this problem, we propose a
locationallocation model that divides the topography of
affected areas into multiple layers; considers constrained
number and capacity of facilities and fleets; and allows
decisionmakers to explore tradeoffs between response time
and logistics costs. To illustrate our theoretical work, we
apply the model to a real dataset from the 2015 Nepal
earthquake response. For this case, our method results in a
considerable reduction of logistics costs.

[64]

Monya Baker.
Is there a reproducibility crisis?
Nature, 533:452–454, 2016.
[ bib ]

[65]

Edward K. Baker.
An Exact Algorithm for the TimeConstrained Traveling Salesman
Problem.
Operations Research, 31(5):938–945, 1983.
[ bib 
DOI ]

[66]

Burcu Balcik and Benita M. Beamon.
Facility location in humanitarian relief.
International Journal of Logistics, 11(2):101–121, 2008.
[ bib ]

[67]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Adaptive Sampling Size and Importance Sampling in
Estimationbased Local Search for the Probabilistic Traveling Salesman
Problem.
European Journal of Operational Research, 199(1):98–110, 2009.
[ bib ]

[68]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Estimationbased Metaheuristics for the Probabilistic Travelling
Salesman Problem.
Computers & Operations Research, 37(11):1939–1951, 2010.
[ bib 
DOI ]

[69]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Estimationbased Metaheuristics for the Single Vehicle Routing
Problem with Stochastic Demands and Customers.
Computational Optimization and Applications, 61(2):463–487,
2015.
[ bib 
DOI ]

[70]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, Zhi Yuan, and Marco
Dorigo.
Estimationbased Ant Colony Optimization Algorithms for the
Probabilistic Travelling Salesman Problem.
Swarm Intelligence, 3(3):223–242, 2009.
[ bib ]

[71]

Egon Balas and M. C. Carrera.
A Dynamic Subgradientbased Branch and Bound Procedure for Set
Covering.
Operations Research, 44(6):875–890, 1996.
[ bib ]

[72]

Egon Balas and C. Martin.
Pivot and Complement–A Heuristic for 0–1 Programming.
Management Science, 26(1):86–96, 1980.
[ bib ]

[73]

Egon Balas and M. W. Padberg.
Set Partitioning: A Survey.
SIAM Review, 18:710–760, 1976.
[ bib ]

[74]

Egon Balas and Neil Simonetti.
Linear Time DynamicProgramming Algorithms for New Classes of
Restricted TSPs: A Computational Study.
INFORMS Journal on Computing, 13(1):56–75, 2001.
[ bib 
DOI ]
Consider the following restricted (symmetric or
asymmetric) travelingsalesman problem (TSP):
given an initial ordering of the n cities and an
integer k > 0, find a minimumcost
feasible tour, where a feasible tour is one in which
city i precedes city j whenever j >= i + k in the
initial ordering. Balas (1996) has proposed a
dynamicprogramming algorithm that solves this
problem in time linear in n, though exponential in
k. Some important realworld problems are amenable
to this model or some of its close relatives. The
algorithm of Balas (1996) constructs a layered
network with a layer of nodes for each position in
the tour, such that sourcesink paths in this
network are in onetoone correspondence with tours
that satisfy the postulated precedence
constraints. In this paper we discuss an
implementation of the dynamicprogramming algorithm
for the general case when the integer k is replaced
with cityspecific integers k(j), j = 1, . . .,
n. We discuss applications to, and computational
experience with, TSPs with time windows, a model
frequently used in vehicle routing as well as in
scheduling with setup, release and delivery
times. We also introduce a new model, the TSP with
target times, applicable to JustinTime
scheduling problems. Finally for TSPs that have no
precedence restrictions, we use the algorithm as a
heuristic that finds in linear time a local optimum
over an exponentialsize neighborhood. For this
case, we implement an iterated version of our
procedure, based on contracting some arcs of the
tour produced by a first application of the
algorithm, then reapplying the algorithm to the
shrunken graph with the same k.
Keywords: tsptw

[75]

Egon Balas and A. Vazacopoulos.
Guided Local Search with Shifting Bottleneck for Job Shop
Scheduling.
Management Science, 44(2):262–275, 1998.
[ bib ]

[76]

Steven C. Bankes.
Tools and techniques for developing policies for complex and
uncertain systems.
Proceedings of the National Academy of Sciences, 99(suppl
3):7263–7266, 2002.
[ bib 
DOI ]
Agentbased models (ABM) are examples of complex adaptive
systems, which can be characterized as those systems for
which no model less complex than the system itself can
accurately predict in detail how the system will behave at
future times. Consequently, the standard tools of policy
analysis, based as they are on devising policies that perform
well on some best estimate model of the system, cannot be
reliably used for ABM. This paper argues that policy analysis
by using ABM requires an alternative approach to decision
theory. The general characteristics of such an approach are
described, and examples are provided of its application to
policy analysis.ABM, agentbased model

[77]

Eduardo Batista de Moraes Barbosa, Edson Luiz Francça Senne, and
Messias Borges Silva.
Improving the Performance of Metaheuristics: An Approach
Combining Response Surface Methodology and Racing Algorithms.
International Journal of Engineering Mathematics, 2015:Article
ID 167031, 2015.
[ bib 
DOI ]
Keywords: Frace

[78]

Alejandro Barredo Arrieta, Natalia DíazRodríguez, Javier Del Ser,
Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio
GilLopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco
Herrera.
Explainable Artificial Intelligence (XAI): Concepts,
taxonomies, opportunities and challenges toward responsible AI.
Information Fusion, 58:82–115, June 2020.
[ bib 
DOI ]

[79]

Thomas BartzBeielstein, Carola Doerr, Daan van den Berg, Jakob Bossek, Sowmya
Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, William La
Cava, Manuel LópezIbáñez, Katherine M. Malan, Jason H. Moore,
Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas
Weise.
Benchmarking in Optimization: Best Practice and Open Issues.
Arxiv preprint arXiv:2007.03488 [cs.NE], 2020.
[ bib 
http ]

[80]

Richard S. Barr, Bruce L. Golden, James P. Kelly, Mauricio G. C. Resende, and
Jr. William R. Stewart.
Designing and Reporting on Computational Experiments with
Heuristic Methods.
Journal of Heuristics, 1(1):9–32, 1995.
[ bib 
DOI ]

[81]

Cynthia Barnhart, Ellis L. Johnson, George L. Nemhauser, Martin W. P.
Savelsbergh, and Pamela H. Vance.
Branchandprice: Column generation for solving huge integer
programs.
Operations Research, 46(3):316–329, 1998.
[ bib ]

[82]

Erin Bartholomew and Jan H. Kwakkel.
On considering robustness in the search phase of Robust Decision
Making: A comparison of ManyObjective Robust Decision Making, multiscenario
ManyObjective Robust Decision Making, and Many Objective Robust
Optimization.
Environmental Modelling & Software, 127:104699, 2020.
[ bib 
DOI ]

[83]

Elias Bareinboim and Judea Pearl.
Causal inference and the datafusion problem.
Proceedings of the National Academy of Sciences,
113(27):7345–7352, 2016.
[ bib 
DOI ]

[84]

Thomas BartzBeielstein and Martin Zaefferer.
Modelbased methods for continuous and discrete global
optimization.
Applied Soft Computing, 55:154–167, June 2017.
[ bib 
DOI ]

[85]

Atanu Basu and L. Neil Frazer.
Rapid Determination of the Critical Temperature in Simulated
Annealing Inversion.
Science, 249(4975):1409–1412, 1990.
[ bib ]

[86]

Roberto Battiti and Andrea Passerini.
BrainComputer Evolutionary Multiobjective Optimization: A
Genetic Algorithm Adapting to the Decision Maker.
IEEE Transactions on Evolutionary Computation, 14(5):671–687,
2010.
[ bib 
DOI ]
Errata: DTLZ6 and DTLZ7 in the paper are actually DTLZ7 and
DTLZ8 in [1650]
Keywords: BCEMOA

[87]

Roberto Battiti and M. Protasi.
Reactive Search, A HistoryBased Heuristic for MAXSAT.
ACM Journal of Experimental Algorithmics, 2, 1997.
[ bib ]

[88]

Michele Battistutta, Andrea Schaerf, and Tommaso Urli.
Featurebased Tuning of Singlestage Simulated Annealing for
Examination Timetabling.
Annals of Operations Research, 252(2):239–254, 2017.
[ bib ]

[89]

Roberto Battiti and Giampietro Tecchiolli.
Simulated annealing and Tabu search in the long run: A
comparison on QAP tasks.
Computer and Mathematics with Applications, 28(6):1–8, 1994.
[ bib 
DOI ]

[90]

Roberto Battiti and Giampietro Tecchiolli.
The Reactive Tabu Search.
ORSA Journal on Computing, 6(2):126–140, 1994.
[ bib ]

[91]

Roberto Battiti and Giampietro Tecchiolli.
The continuous reactive tabu search: blending combinatorial
optimization and stochastic search for global optimization.
Annals of Operations Research, 63(2):151–188, 1996.
[ bib ]

[92]

J. Bautista and J. Pereira.
Ant algorithms for a time and space constrained assembly line
balancing problem.
European Journal of Operational Research, 177(3):2016–2032,
2007.
[ bib 
DOI ]

[93]

William J. Baumol.
Management models and industrial applications of linear
programming.
Naval Research Logistics Quarterly, 9(1):63–64, 1962.
[ bib 
DOI ]

[94]

John Baxter.
Local Optima Avoidance in Depot Location.
Journal of the Operational Research Society, 32(9):815–819,
1981.
[ bib ]

[95]

John E. Beasley and P. C. Chu.
A Genetic Algorithm for the Set Covering Problem.
European Journal of Operational Research, 94(2):392–404, 1996.
[ bib ]

[96]

John E. Beasley and P. C. Chu.
A Genetic Algorithm for the Multidimensional Knapsack Problem.
Journal of Heuristics, 4(1):63–86, 1998.
[ bib ]

[97]

Jennifer Bealt, Duncan Shaw, Chris M. Smith, and Manuel
LópezIbáñez.
Peer Reviews for Making Cities Resilient: A Systematic
Literature Review.
International Journal of Emergency Management, 15(4):334–359,
2019.
[ bib 
DOI ]
Peer reviews are a unique governance tool that use expertise
from one city or country to assess and strengthen the
capabilities of another. Peer review tools are gaining
momentum in disaster management and remain an important but
understudied topic in risk governance. Methodologies to
conduct a peer review are still in their infancy. To enhance
these, a systematic literature review (SLR) of academic and
nonacademic literature was conducted on city resilience peer
reviews. Thirtythree attributes of resilience are
identified, which provides useful insights into how research
and practice can inform risk governance, and utilise peer
reviews, to drive meaningful change. Moreover, it situates
the challenges associated with resilience building tools
within risk governance to support the development of
interdisciplinary perspectives for integrated city resilience
frameworks. Results of this research have been used to
develop a peer review methodology and an international
standard on conducting peer reviews for disaster risk
reduction.
Keywords: city resilience, city peer review, disaster risk governance

[98]

John E. Beasley.
ORLibrary: distributing test problems by electronic mail.
Journal of the Operational Research Society, pp. 1069–1072,
1990.
Currently available from
http://people.brunel.ac.uk/~mastjjb/jeb/info.html.
[ bib ]

[99]

J. Behnamian and S.M.T. Fatemi Ghomi.
Hybrid Flowshop Scheduling with Machine and Resourcedependent
Processing Times.
Applied Mathematical Modelling, 35(3):1107–1123, 2011.
[ bib ]

[100]

Richard Bellman.
The theory of dynamic programming.
Bulletin of the American Mathematical Society, 60:503–515,
1954.
[ bib ]

[101]

Ruggero Bellio, Sara Ceschia, Luca Di Gaspero, Andrea Schaerf, and Tommaso
Urli.
Featurebased tuning of simulated annealing applied to the
curriculumbased course timetabling problem.
Computers & Operations Research, 65:83–92, 2016.
[ bib ]

[102]

Jon Louis Bentley.
Fast Algorithms for Geometric Traveling Salesman Problems.
ORSA Journal on Computing, 4(4):387–411, 1992.
[ bib ]

[103]

Una Benlic and JinKao Hao.
Breakout Local Search for the Quadratic Assignment Problem.
Applied Mathematics and Computation, 219(9):4800–4815, 2013.
[ bib ]

[104]

Calem J. Bendell, Shalon Liu, Tristan AumentadoArmstrong, Bogdan Istrate,
Paul T. Cernek, Samuel Khan, Sergiu Picioreanu, Michael Zhao, and Robert A.
Murgita.
Transient proteinprotein interface prediction: datasets,
features, algorithms, and the RADT predictor.
BMC Bioinformatics, 15:82, 2014.
[ bib ]

[105]

Alexander Javier Benavides and Marcus Ritt.
Two Simple and Effective Heuristics for Minimizing the Makespan
in Nonpermutation Flow Shops.
Computers & Operations Research, 66:160–169, 2016.
[ bib 
DOI ]

[106]

J. F. Benders.
Partitioning Procedures for Solving Mixedvariables Programming
Problems.
Numerische Mathematik, 4(3):238–252, 1962.
[ bib ]

[107]

Jon Louis Bentley.
Multidimensional Divideandconquer.
Communications of the ACM, 23(4):214–229, 1980.
[ bib 
DOI ]
Most results in the field of algorithm design are single
algorithms that solve single problems. In this paper we
discuss multidimensional divideandconquer, an algorithmic
paradigm that can be instantiated in many different ways to
yield a number of algorithms and data structures for
multidimensional problems. We use this paradigm to give
bestknown solutions to such problems as the ECDF, maxima,
range searching, closest pair, and all nearest neighbor
problems. The contributions of the paper are on two
levels. On the first level are the particular algorithms and
data structures given by applying the paradigm. On the
second level is the more novel contribution of this paper: a
detailed study of an algorithmic paradigm that is specific
enough to be described precisely yet general enough to solve
a wide variety of problems.

[108]

James S. Bergstra and Yoshua Bengio.
Random Search for HyperParameter Optimization.
Journal of Machine Learning Research, 13:281–305, 2012.
[ bib 
epub ]
Grid search and manual search are the most widely
used strategies for hyperparameter
optimization. This paper shows empirically and
theoretically that randomly chosen trials are more
efficient for hyperparameter optimization than
trials on a grid. Empirical evidence comes from a
comparison with a large previous study that used
grid search and manual search to configure neural
networks and deep belief networks. Compared with
neural networks configured by a pure grid search, we
find that random search over the same domain is able
to find models that are as good or better within a
small fraction of the computation time. Granting
random search the same computational budget, random
search finds better models by effectively searching
a larger, less promising configuration
space. Compared with deep belief networks configured
by a thoughtful combination of manual search and
grid search, purely random search over the same
32dimensional configuration space found
statistically equal performance on four of seven
data sets, and superior performance on one of
seven. A Gaussian process analysis of the function
from hyperparameters to validation set performance
reveals that for most data sets only a few of the
hyperparameters really matter, but that different
hyperparameters are important on different data
sets. This phenomenon makes grid search a poor
choice for configuring algorithms for new data
sets. Our analysis casts some light on why recent
"High Throughput" methods achieve surprising
success: they appear to search through a large number
of hyperparameters because most hyperparameters do
not matter much. We anticipate that growing interest
in large hierarchical models will place an
increasing burden on techniques for hyperparameter
optimization; this work shows that random search is
a natural baseline against which to judge progress
in the development of adaptive (sequential)
hyperparameter optimization algorithms.

[109]

Loïc Berger, Johannes Emmerling, and Massimo Tavoni.
Managing catastrophic climate risks under model uncertainty
aversion.
Management Science, 63(3):749–765, 2017.
[ bib ]

[110]

Livio Bertacco, Matteo Fischetti, and Andrea Lodi.
A feasibility pump heuristic for general mixedinteger
problems.
Discrete Optimization, 4(1):63–76, 2007.
[ bib ]

[111]

Dimitris Bertsimas and Nathan Kallus.
From predictive to prescriptive analytics.
Management Science, 66(3):1025–1044, 2020.
[ bib ]

[112]

Felix Berkenkamp, Andreas Krause, and Angela P. Schoellig.
Bayesian Optimization with Safety Constraints: Safe and
Automatic Parameter Tuning in Robotics.
Arxiv preprint arXiv:1602.04450, 2016.
[ bib 
http ]

[113]

Felix Berkenkamp, Andreas Krause, and Angela P. Schoellig.
Bayesian optimization with safety constraints: safe and
automatic parameter tuning in robotics.
Machine Learning, June 2021.
[ bib 
DOI ]
Selecting the right tuning parameters for algorithms is a
pravelent problem in machine learning that can significantly
affect the performance of algorithms. Dataefficient
optimization algorithms, such as Bayesian optimization, have
been used to automate this process. During experiments on
realworld systems such as robotic platforms these methods
can evaluate unsafe parameters that lead to safetycritical
system failures and can destroy the system. Recently, a safe
Bayesian optimization algorithm, called SafeOpt, has been
developed, which guarantees that the performance of the
system never falls below a critical value; that is, safety is
defined based on the performance function. However, coupling
performance and safety is often not desirable in practice,
since they are often opposing objectives. In this paper, we
present a generalized algorithm that allows for multiple
safety constraints separate from the objective. Given an
initial set of safe parameters, the algorithm maximizes
performance but only evaluates parameters that satisfy safety
for all constraints with high probability. To this end, it
carefully explores the parameter space by exploiting
regularity assumptions in terms of a Gaussian process
prior. Moreover, we show how context variables can be used to
safely transfer knowledge to new situations and tasks. We
provide a theoretical analysis and demonstrate that the
proposed algorithm enables fast, automatic, and safe
optimization of tuning parameters in experiments on a
quadrotor vehicle.
Preprint: http://arxiv.org/abs/1602.04450

[114]

Dimitri P. Bertsekas, John N. Tsitsiklis, and Cynara Wu.
Rollout Algorithms for Combinatorial Optimization.
Journal of Heuristics, 3(3):245–262, 1997.
[ bib ]

[115]

Judith O. Berkey and Pearl Y. Wang.
Twodimensional finite binpacking algorithms.
Journal of the Operational Research Society, 38(5):423–429,
1987.
[ bib 
DOI ]

[116]

Nicola Beume, Carlos M. Fonseca, Manuel LópezIbáñez, Luís
Paquete, and Jan Vahrenhold.
On the complexity of computing the hypervolume indicator.
IEEE Transactions on Evolutionary Computation,
13(5):1075–1082, 2009.
[ bib 
DOI ]
The goal of multiobjective optimization is to find
a set of best compromise solutions for typically
conflicting objectives. Due to the complex nature of
most reallife problems, only an approximation to
such an optimal set can be obtained within
reasonable (computing) time. To compare such
approximations, and thereby the performance of
multiobjective optimizers providing them, unary
quality measures are usually applied. Among these,
the hypervolume indicator (or
Smetric) is of particular relevance due to
its favorable properties. Moreover, this indicator
has been successfully integrated into stochastic
optimizers, such as evolutionary algorithms, where
it serves as a guidance criterion for finding good
approximations to the Pareto front. Recent results
show that computing the hypervolume indicator can be
seen as solving a specialized version of Klee's
Measure Problem. In general, Klee's Measure Problem
can be solved with O(n log n +
n^{d/2}log n) comparisons for an input instance of
size n in d dimensions; as of this writing, it
is unknown whether a lower bound higher than
Ω(n log n) can be proven. In this article,
we derive a lower bound of Ω(nlog n) for the
complexity of computing the hypervolume indicator in
any number of dimensions d>1 by reducing the
socalled UniformGap problem to it. For
the three dimensional case, we also present a
matching upper bound of O(nlog n)
comparisons that is obtained by extending an
algorithm for finding the maxima of a point set.

[117]

Nicola Beume, Boris Naujoks, and Michael T. M. Emmerich.
SMSEMOA: Multiobjective selection based on dominated
hypervolume.
European Journal of Operational Research, 181(3):1653–1669,
2007.
[ bib 
DOI ]

[118]

HansGeorg Beyer and HansPaul Schwefel.
Evolution Strategies: A Comprehensive Introduction.
Natural Computing, 1:3–52, 2002.
[ bib ]

[119]

HansGeorg Beyer, HansPaul Schwefel, and Ingo Wegener.
How to analyse evolutionary algorithms.
Theoretical Computer Science, 287(1):101–130, 2002.
[ bib ]

[120]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic ComponentWise Design of MultiObjective Evolutionary
Algorithms.
IEEE Transactions on Evolutionary Computation, 20(3):403–417,
2016.
[ bib 
DOI 
supplementary material ]

[121]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
Evolutionary Computation, 26(4):621–656, 2018.
[ bib 
DOI 
supplementary material ]
Research on multiobjective evolutionary algorithms (MOEAs)
has produced over the past decades a large number of
algorithms and a rich literature on performance assessment
tools to evaluate and compare them. Yet, newly proposed MOEAs
are typically compared against very few, often a decade older
MOEAs. One reason for this apparent contradiction is the lack
of a common baseline for comparison, with each subsequent
study often devising its own experimental scenario, slightly
different from other studies. As a result, the state of the
art in MOEAs is a disputed topic. This article reports a
systematic, comprehensive evaluation of a large number of
MOEAs that covers a wide range of experimental scenarios. A
novelty of this study is the separation between the
higherlevel algorithmic components related to
multiobjective optimization (MO), which characterize each
particular MOEA, and the underlying parameterssuch as
evolutionary operators, population size, etc.whose
configuration may be tuned for each scenario. Instead of
relying on a common or "default" parameter configuration that
may be lowperforming for particular MOEAs or scenarios and
unintentionally biased, we tune the parameters of each MOEA
for each scenario using automatic algorithm configuration
methods. Our results confirm some of the assumed knowledge in
the field, while at the same time they provide new insights
on the relative performance of MOEAs for manyobjective
problems. For example, under certain conditions,
indicatorbased MOEAs are more competitive for such problems
than previously assumed. We also analyze problemspecific
features affecting performance, the agreement between
performance metrics, and the improvement of tuned
configurations over the default configurations used in the
literature. Finally, the data produced is made publicly
available to motivate further analysis and a baseline for
future comparisons.

[122]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatically Designing StateoftheArt Multi and
ManyObjective Evolutionary Algorithms.
Evolutionary Computation, 28(2):195–226, 2020.
[ bib 
DOI 
supplementary material ]
A recent comparison of wellestablished multiobjective
evolutionary algorithms (MOEAs) has helped better identify
the current stateoftheart by considering (i) parameter
tuning through automatic configuration, (ii) a wide range of
different setups, and (iii) various performance
metrics. Here, we automatically devise MOEAs with verified
stateoftheart performance for multi and manyobjective
continuous optimization. Our work is based on two main
considerations. The first is that highperforming algorithms
can be obtained from a configurable algorithmic framework in
an automated way. The second is that multiple performance
metrics may be required to guide this automatic design
process. In the first part of this work, we extend our
previously proposed algorithmic framework, increasing the
number of MOEAs, underlying evolutionary algorithms, and
search paradigms that it comprises. These components can be
combined following a general MOEA template, and an automatic
configuration method is used to instantiate highperforming
MOEA designs that optimize a given performance metric and
present stateoftheart performance. In the second part, we
propose a multiobjective formulation for the automatic MOEA
design, which proves critical for the context of
manyobjective optimization due to the disagreement of
established performance metrics. Our proposed formulation
leads to an automatically designed MOEA that presents
stateoftheart performance according to a set of metrics,
rather than a single one.

[123]

Leonora Bianchi, Mauro Birattari, M. Manfrin, M. Mastrolilli, Luís Paquete,
O. RossiDoria, and Tommaso Schiavinotto.
Hybrid Metaheuristics for the Vehicle Routing Problem with
Stochastic Demands.
Journal of Mathematical Modelling and Algorithms, 5(1):91–110,
2006.
[ bib ]

[124]

Leonora Bianchi, Marco Dorigo, L. M. Gambardella, and Walter J. Gutjahr.
A survey on metaheuristics for stochastic combinatorial
optimization.
Natural Computing, 8(2):239–287, 2009.
[ bib ]

[125]

M. Binois, D. Ginsbourger, and O. Roustant.
Quantifying uncertainty on Pareto fronts with Gaussian
process conditional simulations.
European Journal of Operational Research, 243(2):386–394,
2015.
[ bib 
DOI ]
Multiobjective optimization algorithms aim at finding
Paretooptimal solutions. Recovering Pareto fronts or Pareto
sets from a limited number of function evaluations are
challenging problems. A popular approach in the case of
expensivetoevaluate functions is to appeal to
metamodels. Kriging has been shown efficient as a base for
sequential multiobjective optimization, notably through
infill sampling criteria balancing exploitation and
exploration such as the Expected Hypervolume
Improvement. Here we consider Kriging metamodels not only for
selecting new points, but as a tool for estimating the whole
Pareto front and quantifying how much uncertainty remains on
it at any stage of Krigingbased multiobjective optimization
algorithms. Our approach relies on the Gaussian process
interpretation of Kriging, and bases upon conditional
simulations. Using concepts from random set theory, we
propose to adapt the Vorob'ev expectation and deviation to
capture the variability of the set of nondominated
points. Numerical experiments illustrate the potential of the
proposed workflow, and it is shown on examples how Gaussian
process simulations and the estimated Vorob'ev deviation can
be used to monitor the ability of Krigingbased
multiobjective optimization algorithms to accurately learn
the Pareto front.
Keywords: Attainment function, Expected Hypervolume Improvement,
Kriging, Multiobjective optimization, Vorob'ev expectation

[126]

Mauro Birattari, Prasanna Balaprakash, Thomas Stützle, and Marco Dorigo.
Estimation Based Local Search for Stochastic Combinatorial
Optimization.
INFORMS Journal on Computing, 20(4):644–658, 2008.
[ bib ]

[127]

Mauro Birattari, Paola Pellegrini, and Marco Dorigo.
On the invariance of ant colony optimization.
IEEE Transactions on Evolutionary Computation, 11(6):732–742,
2007.
[ bib 
DOI ]

[128]

Mauro Birattari, M. Zlochin, and Marco Dorigo.
Towards a theory of practice in metaheuristics design: A machine
learning perspective.
Theoretical Informatics and Applications, 40(2):353–369, 2006.
[ bib ]

[129]

Francesco Biscani, Dario Izzo, and Chit Hong Yam.
A Global Optimisation Toolbox for Massively Parallel Engineering
Optimisation.
Arxiv preprint arXiv:1004.3824, 2010.
[ bib 
http ]
A software platform for global optimisation, called PaGMO,
has been developed within the Advanced Concepts Team (ACT) at
the European Space Agency, and was recently released as an
opensource project. PaGMO is built to tackle
highdimensional global optimisation problems, and it has
been successfully used to find solutions to reallife
engineering problems among which the preliminary design of
interplanetary spacecraft trajectories  both chemical
(including multiple flybys and deepspace maneuvers) and
lowthrust (limited, at the moment, to single phase
trajectories), the inverse design of nanostructured
radiators and the design of nonreactive controllers for
planetary rovers. Featuring an arsenal of global and local
optimisation algorithms (including genetic algorithms,
differential evolution, simulated annealing, particle swarm
optimisation, compass search, improved harmony search, and
various interfaces to libraries for local optimisation such
as SNOPT, IPOPT, GSL and NLopt), PaGMO is at its core a C++
library which employs an objectoriented architecture
providing a clean and easilyextensible optimisation
framework. Adoption of multithreaded programming ensures the
efficient exploitation of modern multicore architectures and
allows for a straightforward implementation of the island
model paradigm, in which multiple populations of candidate
solutions asynchronously exchange information in order to
speedup and improve the optimisation process. In addition to
the C++ interface, PaGMO's capabilities are exposed to the
highlevel language Python, so that it is possible to easily
use PaGMO in an interactive session and take advantage of the
numerous scientific Python libraries available.
Keywords: PaGMO

[130]

Bernd Bischl, Pascal Kerschke, Lars Kotthoff, Marius Thomas Lindauer, Yuri
Malitsky, Alexandre Fréchette, Holger H. Hoos, Frank Hutter, Kevin
LeytonBrown, Kevin Tierney, and Joaquin Vanschoren.
ASlib: A Benchmark Library for Algorithm Selection.
Artificial Intelligence, 237:41–58, 2016.
[ bib ]

[131]

Bernd Bischl, Michel Lang, Lars Kotthoff, Julia Schiffner, Jakob Richter, Erich
Studerus, Giuseppe Casalicchio, and Zachary M. Jones.
mlr: Machine Learning in R.
Journal of Machine Learning Research, 17(170):1–5, 2016.
[ bib 
epub ]

[132]

Xavier Blasco, Juan M. Herrero, Javier Sanchis, and Manuel Martínez.
A new graphical visualization of ndimensional Pareto front
for decisionmaking in multiobjective optimization.
Information Sciences, 178(20):3908–3924, 2008.
[ bib ]

[133]

Craig Blackmore, Oliver Ray, and Kerstin Eder.
Automatically Tuning the GCC Compiler to Optimize the
Performance of Applications Running on Embedded Systems.
Arxiv preprint arXiv:1703.08228, 2017.
[ bib 
http ]

[134]

María J. Blesa and Christian Blum.
Finding edgedisjoint paths in networks by means of artificial
ant colonies.
Journal of Mathematical Modelling and Algorithms,
6(3):361–391, 2007.
[ bib ]

[135]

Christian Blum.
BeamACO—Hybridizing Ant Colony Optimization with Beam
Search: An Application to Open Shop Scheduling.
Computers & Operations Research, 32(6):1565–1591, 2005.
[ bib ]

[136]

Christian Blum.
BeamACO for simple assembly line balancing.
INFORMS Journal on Computing, 20(4):618–627, 2008.
[ bib 
DOI ]

[137]

Christian Blum, María J. Blesa, and Manuel LópezIbáñez.
Beam search for the longest common subsequence problem.
Computers & Operations Research, 36(12):3178–3186, 2009.
[ bib 
DOI ]
The longest common subsequence problem is a classical string
problem that concerns finding the common part of a set of
strings. It has several important applications, for example,
pattern recognition or computational biology. Most research
efforts up to now have focused on solving this problem
optimally. In comparison, only few works exist dealing with
heuristic approaches. In this work we present a deterministic
beam search algorithm. The results show that our algorithm
outperforms the current stateoftheart approaches not only
in solution quality but often also in computation time.

[138]

Christian Blum, Borja Calvo, and María J. Blesa.
FrogCOL and FrogMIS: New Decentralized Algorithms for
Finding Large Independent Sets in Graphs.
Swarm Intelligence, 9(23):205–227, 2015.
[ bib 
DOI ]
Keywords: irace

[139]

Christian Blum and Marco Dorigo.
The hypercube framework for ant colony optimization.
IEEE Transactions on Systems, Man, and Cybernetics – Part B,
34(2):1161–1172, 2004.
[ bib ]

[140]

Christian Blum and Marco Dorigo.
Search Bias in Ant Colony Optimization: On the Role of
CompetitionBalanced Systems.
IEEE Transactions on Evolutionary Computation, 9(2):159–174,
2005.
[ bib ]

[141]

Christian Blum and Gabriela Ochoa.
A comparative analysis of two matheuristics by means of merged
local optima networks.
European Journal of Operational Research, 290(1):36–56, 2021.
[ bib ]

[142]

Christian Blum, Pedro Pinacho, Manuel LópezIbáñez, and José A.
Lozano.
Construct, Merge, Solve & Adapt: A New General Algorithm for
Combinatorial Optimization.
Computers & Operations Research, 68:75–88, 2016.
[ bib 
DOI ]
Keywords: irace, CMSA

[143]

Christian Blum, Jakob Puchinger, Günther R. Raidl, and Andrea Roli.
Hybrid Metaheuristics in Combinatorial Optimization: A Survey.
Applied Soft Computing, 11(6):4135–4151, 2011.
[ bib ]

[144]

Christian Blum and Andrea Roli.
Metaheuristics in Combinatorial Optimization: Overview and
Conceptual Comparison.
ACM Computing Surveys, 35(3):268–308, 2003.
[ bib ]

[145]

Christian Blum and M. Sampels.
An Ant Colony Optimization Algorithm for Shop Scheduling
Problems.
Journal of Mathematical Modelling and Algorithms,
3(3):285–308, 2004.
[ bib 
DOI ]

[146]

Christian Blum, M. Yábar Vallès, and María J. Blesa.
An ant colony optimization algorithm for DNA sequencing by
hybridization.
Computers & Operations Research, 35(11):3620–3635, 2008.
[ bib ]

[147]

Andrea F. Bocchese, Chris Fawcett, Mauro Vallati, Alfonso E. Gerevini, and
Holger H. Hoos.
Performance robustness of AI planners in the 2014
International Planning Competition.
AI Communications, 31(6):445–463, December 2018.
[ bib 
DOI ]
Solver competitions have been used in many areas of AI to
assess the current state of the art and guide future research
and development. AI planning is no exception, and the
International Planning Competition (IPC) has been frequently
run for nearly two decades. Due to the organisational and
computational burden involved in running these competitions,
solvers are generally compared using a single homogeneous
hardware and software environment for all competitors. To
what extent does the specific choice of hardware and software
environment have an effect on solver performance, and is that
effect distributed equally across the competing solvers? In
this work, we use the competing planners and benchmark
instance sets from the 2014 IPC to investigate these two
questions. We recreate the 2014 IPC Optimal and Agile tracks
on two distinct hardware environments and eight distinct
software environments. We show that solver performance varies
significantly based on the hardware and software environment,
and that this variation is not equal for all
planners. Furthermore, the observed variation is sufficient
to change the competition rankings, including the topranked
planners for some tracks.

[148]

Kenneth D. Boese, Andrew B. Kahng, and Sudhakar Muddu.
A New Adaptive MultiStart Technique for Combinatorial Global
Optimization.
Operations Research Letters, 16(2):101–113, 1994.
[ bib ]
Keywords: bigvalley hypothesis, TSP, landscape analysis

[149]

Marko Bohanec.
Decision making: a computerscience and informationtechnology
viewpoint.
Interdisciplinary Description of Complex Systems, 7(2):22–37,
2009.
[ bib ]

[150]

Ihor O. Bohachevsky, Mark E. Johnson, and Myron L. Stein.
Generalized Simulated Annealing for Function Optimization.
Technometrics, 28(3):209–217, 1986.
[ bib ]

[151]

P. C. Borges.
CHESS  Changing Horizon Efficient Set Search: A simple
principle for multiobjective optimization.
Journal of Heuristics, 6(3):405–418, 2000.
[ bib ]

[152]

Endre Boros, Peter L. Hammer, and Gabriel Tavares.
Local search heuristics for Quadratic Unconstrained Binary
Optimization (QUBO).
Journal of Heuristics, 13(2):99–132, 2007.
[ bib ]

[153]

JeanCharles de Borda.
Mémoire sur les Élections au Scrutin.
Histoire de l'Académie Royal des Sciences, 1781.
[ bib ]
Keywords: ranking

[154]

Hozefa M. Botee and Eric Bonabeau.
Evolving Ant Colony Optimization.
Advances in Complex Systems, 1:149–159, 1998.
[ bib ]

[155]

Marco Botte and Anita Schöbel.
Dominance for multiobjective robust optimization concepts.
European Journal of Operational Research, 273(2):430–440,
2019.
[ bib ]

[156]

Salim Bouamama, Christian Blum, and Abdellah Boukerram.
A Populationbased Iterated Greedy Algorithm for the Minimum
Weight Vertex Cover Problem.
Applied Soft Computing, 12(6):1632–1639, 2012.
[ bib ]

[157]

Géraldine Bous, Philippe Fortemps, François Glineur, and Marc Pirlot.
ACUTA: A novel method for eliciting additive value functions
on the basis of holistic preference statements.
European Journal of Operational Research, 206(2):435–444,
2010.
[ bib ]

[158]

K. Bouleimen and H. Lecocq.
A new efficient simulated annealing algorithm for the
resourceconstrained project scheduling problem and its multiple mode
version.
European Journal of Operational Research, 149(2):268–281,
2003.
[ bib 
DOI ]
This paper describes new simulated annealing (SA)
algorithms for the resourceconstrained project
scheduling problem (RCPSP) and its multiple mode
version (MRCPSP). The objective function
considered is minimisation of the makespan. The
conventional SA search scheme is replaced by a new
design that takes into account the specificity of
the solution space of project scheduling
problems. For RCPSP, the search was based on an
alternated activity and time incrementing process,
and all parameters were set after preliminary
statistical experiments done on test instances. For
MRCPSP, we introduced an original approach using
two embedded search loops alternating activity and
mode neighbourhood exploration. The performance
evaluation done on the benchmark instances available
in the literature proved the efficiency of both
adaptations that are currently among the most
competitive algorithms for these problems.
Keywords: multimode resourceconstrained project scheduling,
project scheduling, simulated annealing

[159]

B. Bozkurt, J. W. Fowler, E. S. Gel, B. Kim, Murat Köksalan, and Jyrki
Wallenius.
Quantitative comparison of approximate solution sets for
multicriteria optimization problems with weighted Tchebycheff preference
function.
Operations Research, 58(3):650–659, 2010.
[ bib ]
Proposed IPF indicator

[160]

Jürgen Branke, Salvatore Greco, Roman Slowiński, and P Zielniewicz.
Interactive evolutionary multiobjective optimization driven by
robust ordinal regression.
Bulletin of the Polish Academy of Sciences: Technical Sciences,
58(3):347–358, 2010.
[ bib 
DOI ]

[161]

S. C. Brailsford, Walter J. Gutjahr, M. S. Rauner, and W. Zeppelzauer.
Combined Discreteevent Simulation and Ant Colony Optimisation
Approach for Selecting Optimal Screening Policies for Diabetic Retinopathy.
Computational Management Science, 4(1):59–83, 2006.
[ bib ]

[162]

Jürgen Branke, T. Kaussler, and H. Schmeck.
Guidance in evolutionary multiobjective optimization.
Advances in Engineering Software, 32:499–507, 2001.
[ bib ]

[163]

Jürgen Branke, S. Nguyen, C. W. Pickardt, and M. Zhang.
Automated Design of Production Scheduling Heuristics: A Review.
IEEE Transactions on Evolutionary Computation, 20(1):110–124,
2016.
[ bib ]

[164]

Jürgen Branke and C. Schmidt.
Faster Convergence by Means of Fitness Estimation.
Soft Computing, 9(1):13–20, January 2005.
[ bib 
DOI ]

[165]

Roland Braune and G. Zäpfel.
Shifting Bottleneck Scheduling for Total Weighted Tardiness
Minimization—A Computational Evaluation of Subproblem and Reoptimization
Heuristics.
Computers & Operations Research, 66:130–140, 2016.
[ bib ]

[166]

Jürgen Branke, Salvatore Corrente, Salvatore Greco, Roman Slowiński,
and P. Zielniewicz.
Using Choquet integral as preference model in interactive
evolutionary multiobjective optimization.
European Journal of Operational Research, 250(3):884–901,
2016.
[ bib 
DOI ]

[167]

Jürgen Branke, S. S. Farid, and N. Shah.
Industry 4.0: a vision for personalized medicine supply chains?
Cell and Gene Therapy Insights, 2(2):263–270, 2016.
[ bib 
DOI ]

[168]

Jürgen Branke, Salvatore Greco, Roman Slowiński, and Piotr
Zielniewicz.
Learning Value Functions in Interactive Evolutionary
Multiobjective Optimization.
IEEE Transactions on Evolutionary Computation, 19(1):88–102,
2015.
[ bib ]

[169]

Yaochu Jin and Jürgen Branke.
Evolutionary Optimization in Uncertain Environments—A Survey.
IEEE Transactions on Evolutionary Computation, 9(5):303–317,
2005.
[ bib ]

[170]

Leo Breiman.
Random Forests.
Machine Learning, 45(1):5–32, 2001.
[ bib 
DOI ]

[171]

Karl Bringmann, Sergio Cabello, and Michael T. M. Emmerich.
Maximum volume subset selection for anchored boxes.
Arxiv preprint arXiv:1803.00849, 2018.
[ bib 
DOI ]
Let B be a set of n axisparallel boxes in R^{d}
such that each box has a corner at the origin and the other
corner in the positive quadrant of R^{d}, and let
k be a positive integer. We study the problem of selecting
k boxes in B that maximize the volume of the union of the
selected boxes. This research is motivated by applications
in skyline queries for databases and in multicriteria
optimization, where the problem is known as the
hypervolume subset selection problem. It is known
that the problem can be solved in polynomial time in the
plane, while the best known running time in any dimension d
≥3 is Ω(nk). We show that: The
problem is NPhard already in 3 dimensions. In 3 dimensions,
we break the bound Ω(nk), by
providing an n^{O(√(k))} algorithm. For any constant
dimension d, we present an efficient polynomialtime
approximation scheme.
Keywords: hypervolume subset selection

[172]

Karl Bringmann and Tobias Friedrich.
Approximating the Least Hypervolume Contributor: NPHard in
General, But Fast in Practice.
Theoretical Computer Science, 425:104–116, 2012.
[ bib 
DOI ]

[173]

Karl Bringmann and Tobias Friedrich.
An efficient algorithm for computing hypervolume contributions.
Evolutionary Computation, 18(3):383–402, 2010.
[ bib ]

[174]

Karl Bringmann and Tobias Friedrich.
Convergence of hypervolumebased archiving algorithms.
IEEE Transactions on Evolutionary Computation, 18(5):643–657,
2014.
[ bib 
DOI ]
Proof that all nondecreasing (μ+ λ) archiving algorithms with
λ< μ are ineffective.
Keywords: competitive ratio

[175]

Dimo Brockhoff, Johannes Bader, Lothar Thiele, and Eckart Zitzler.
Directed Multiobjective Optimization Based on the Weighted
Hypervolume Indicator.
Journal of MultiCriteria Decision Analysis, 20(56):291–317,
2013.
[ bib 
DOI ]
Recently, there has been a large interest in setbased
evolutionary algorithms for multi objective
optimization. They are based on the definition of indicators
that characterize the quality of the current population while
being compliant with the concept of Paretooptimality. It has
been shown that the hypervolume indicator, which measures the
dominated volume in the objective space, enables the design
of efficient search algorithms and, at the same time, opens
up opportunities to express user preferences in the search by
means of weight functions. The present paper contains the
necessary theoretical foundations and corresponding
algorithms to (i) select appropriate weight functions, to
(ii) transform user preferences into weight functions and to
(iii) efficiently evaluate the weighted hypervolume indicator
through Monte Carlo sampling. The algorithm WHypE, which
implements the previous concepts, is introduced, and the
effectiveness of the search, directed towards the user's
preferred solutions, is shown using an extensive set of
experiments including the necessary statistical performance
assessment.
Keywords: hypervolume, preferencebased search, multi objective
optimization, evolutionary algorithm

[176]

Eric Brochu, Vlad Cora, and Nando de Freitas.
A Tutorial on Bayesian Optimization of Expensive Cost
Functions, with Application to Active User Modeling and Hierarchical
Reinforcement Learning.
Arxiv preprint arXiv:1012.2599, December 2010.
[ bib 
http ]

[177]

Dimo Brockhoff, T. Wagner, and Heike Trautmann.
R2 indicatorbased multiobjective search.
Evolutionary Computation, 23(3):369–395, 2015.
[ bib ]

[178]

Dimo Brockhoff and Eckart Zitzler.
Objective Reduction in Evolutionary Multiobjective Optimization:
Theory and Applications.
Evolutionary Computation, 17(2):135–166, 2009.
[ bib 
DOI ]
Manyobjective problems represent a major challenge in the
field of evolutionary multiobjective optimization, in terms of
search efficiency, computational cost, decision making,
visualization, and so on. This leads to various research
questions, in particular whether certain objectives can be
omitted in order to overcome or at least diminish the
difficulties that arise when many, that is, more than three,
objective functions are involved. This study addresses this
question from different perspectives. First, we investigate
how adding or omitting objectives affects the problem
characteristics and propose a general notion of conflict
between objective sets as a theoretical foundation for
objective reduction. Second, we present both exact and
heuristic algorithms to systematically reduce the number of
objectives, while preserving as much as possible of the
dominance structure of the underlying optimization
problem. Third, we demonstrate the usefulness of the proposed
objective reduction method in the context of both decision
making and search for a radar waveform application as well as
for wellknown test functions.

[179]

Peter Brucker, Johann Hurink, and Frank Werner.
Improving Local Search Heuristics for some Scheduling Problems
— Part I.
Discrete Applied Mathematics, 65(1–3):97–122, 1996.
[ bib ]

[180]

Peter Brucker, Johann Hurink, and Frank Werner.
Improving Local Search Heuristics for some Scheduling Problems
— Part II.
Discrete Applied Mathematics, 72(1–2):47–69, 1997.
[ bib ]

[181]

M. J. Brusco, L. W. Jacobs, and G. M. Thompson.
A Morphing Procedure to Supplement a Simulated Annealing
Heuristic for Cost and Coveragecorrelated Set Covering Problems.
Annals of Operations Research, 86:611–627, 1999.
[ bib ]

[182]

John T. Buchanan.
An experimental evaluation of interactive MCDM methods and the
decision making process.
Journal of the Operational Research Society, 45(9):1050–1059,
1994.
[ bib ]

[183]

John T. Buchanan and James Corner.
The effects of anchoring in interactive MCDM solution
methods.
Computers & Operations Research, 24(10):907–918, October
1997.
[ bib 
DOI ]

[184]

A. L. Buchsbaum and M. T. Goodrich.
ThreeDimensional Layers of Maxima.
Algorithmica, 39:275–289, 2004.
[ bib ]

[185]

B. Bullnheimer, Richard F. Hartl, and Christine Strauss.
An Improved Ant System Algorithm for the Vehicle Routing
Problem.
Annals of Operations Research, 89:319–328, 1999.
[ bib ]

[186]

B. Bullnheimer, Richard F. Hartl, and Christine Strauss.
A new rankbased version of the Ant System: A
computational study.
Central European Journal for Operations Research and Economics,
7(1):25–38, 1999.
[ bib ]

[187]

Edmund K. Burke and Yuri Bykov.
The Late Acceptance HillClimbing Heuristic.
European Journal of Operational Research, 258(1):70–78, 2017.
[ bib ]

[188]

Rainer E. Burkard and Ulrich Fincke.
The asymptotic probabilistic behaviour of quadratic sum
assignment problems.
Zeitschrift für Operations Research, 27(1):73–81, 1983.
[ bib ]

[189]

Luciana Buriol, Paulo M. França, and Pablo Moscato.
A New Memetic Algorithm for the Asymmetric Traveling Salesman
Problem.
Journal of Heuristics, 10(5):483–506, 2004.
[ bib ]

[190]

Edmund K. Burke, Michel Gendreau, Matthew R. Hyde, Graham Kendall, Gabriela
Ochoa, Ender Özcan, and Rong Qu.
Hyperheuristics: A Survey of the State of the Art.
Journal of the Operational Research Society, 64(12):1695–1724,
2013.
[ bib 
DOI ]

[191]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, and John R. Woodward.
A Genetic Programming HyperHeuristic Approach for Evolving
2D Strip Packing Heuristics.
IEEE Transactions on Evolutionary Computation, 14(6):942–958,
2010.
[ bib 
DOI ]

[192]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, and John R. Woodward.
Automating the Packing Heuristic Design Process with Genetic
Programming.
Evolutionary Computation, 20(1):63–89, 2012.
[ bib 
DOI ]
Keywords: one, two, or threedimensional knapsack and bin packing
problems

[193]

Edmund K. Burke, Matthew R. Hyde, and Graham Kendall.
Grammatical Evolution of Local Search Heuristics.
IEEE Transactions on Evolutionary Computation, 16(7):406–417,
2012.
[ bib 
DOI ]

[194]

Rainer E. Burkard, Stefan E. Karisch, and Franz Rendl.
QAPLIB–a Quadratic Assignment Problem Library.
Journal of Global Optimization, 10(4):391–403, 1997.
[ bib ]

[195]

Rainer E. Burkard and Franz Rendl.
A Thermodynamically Motivated Simulation Procedure for
Combinatorial Optimization Problems.
European Journal of Operational Research, 17(2):169–174, 1984.
[ bib 
DOI ]
Keywords: 2exchange delta evaluation for QAP

[196]

Erika Buson, Roberto Roberti, and Paolo Toth.
A ReducedCost Iterated Local Search Heuristic for the
FixedCharge Transportation Problem.
Operations Research, 62(5):1095–1106, 2014.
[ bib ]

[197]

R. Caballero, Mariano Luque, Julián Molina, and Francisco Ruiz.
PROMOIN: An Interactive System for Multiobjective
Programming.
Information Technologies and Decision Making, 1:635–656, 2002.
[ bib ]
Keywords: preferences, multi interactive methods framework

[198]

Leslie Pérez Cáceres and Thomas Stützle.
Exploring Variable Neighborhood Search for Automatic Algorithm
Configuration.
Electronic Notes in Discrete Mathematics, 58:167–174, 2017.
[ bib 
DOI ]

[199]

Sebastien Cahon, Nordine Melab, and ElGhazali Talbi.
ParadisEO: A Framework for the Reusable Design of Parallel and
Distributed Metaheuristics.
Journal of Heuristics, 10(3):357–380, 2004.
[ bib 
DOI ]

[200]

Zhaoquan Cai, Han Huang, Yong Qin, and Xianheng Ma.
Ant Colony Optimization Based on Adaptive Volatility Rate of
Pheromone Trail.
International Journal of Communications, Network and System
Sciences, 2(8):792–796, 2009.
[ bib ]

[201]

Xinye Cai, Yexing Li, Zhun Fan, and Qingfu Zhang.
An external archive guided multiobjective evolutionary algorithm
based on decomposition for combinatorial optimization.
IEEE Transactions on Evolutionary Computation, 19(4):508–523,
2015.
[ bib ]

[202]

Xinye Cai, Yushun Xiao, Miqing Li, Han Hu, Hisao Ishibuchi, and Xiaoping Li.
A gridbased inverted generational distance for
multi/manyobjective optimization.
IEEE Transactions on Evolutionary Computation, 25(1):21–34,
2021.
[ bib ]
weakly Paretocompliant indicator

[203]

Xinye Cai, Yushun Xiao, Zhenhua Li, Qi Sun, Hanchuan Xu, Miqing Li, and Hisao
Ishibuchi.
A kernelbased indicator for multi/manyobjective optimization.
IEEE Transactions on Evolutionary Computation, 2021.
[ bib ]

[204]

Roberto Wolfler Calvo.
A New Heuristic for the Traveling Salesman Problem with Time
Windows.
Transportation Science, 34(1):113–124, 2000.
[ bib 
DOI ]

[205]

Felipe Campelo, Lucas S. Batista, and Claus Aranha.
The MOEADr Package: A ComponentBased Framework for
Multiobjective Evolutionary Algorithms Based on Decomposition.
Journal of Statistical Software, 92, 2020.
[ bib 
DOI ]

[206]

Christian Leonardo CamachoVillalón, Marco Dorigo, and Thomas Stützle.
The intelligent water drops algorithm: why it cannot be
considered a novel algorithm.
Swarm Intelligence, 13:173–192, 2019.
[ bib ]

[207]

Christian Leonardo CamachoVillalón, Marco Dorigo, and Thomas Stützle.
An analysis of why cuckoo search does not bring any novel ideas
to optimization.
Computers & Operations Research, p. 105747, 2022.
[ bib ]

[208]

Christian Leonardo CamachoVillalón, Marco Dorigo, and Thomas Stützle.
Exposing the grey wolf, mothflame, whale, firefly, bat, and
antlion algorithms: six misleading optimization techniques inspired by
bestial metaphors.
International Transactions in Operational Research, 2022.
[ bib 
DOI ]

[209]

Ann Melissa Campbell and Philip C. Jones.
Prepositioning supplies in preparation for disasters.
European Journal of Operational Research, 209(2):156–165,
2011.
[ bib ]

[210]

E Cambria, B Schuller, Y Xia, and C Havasi.
New avenues in opinion mining and sentiment analysis.
IEEE Intelligent Systems, 28(2):15–21, 2013.
[ bib 
DOI ]

[211]

Christian Leonardo CamachoVillalón, Thomas Stützle, and Marco Dorigo.
PSOX: A ComponentBased Framework for the Automatic Design of
Particle Swarm Optimization Algorithms.
IEEE Transactions on Evolutionary Computation, 26(3):402–416,
2021.
[ bib 
DOI ]

[212]

Felipe Campelo and Elizabeth F. Wanner.
Sample size calculations for the experimental comparison of
multiple algorithms on multiple problem instances.
Journal of Heuristics, 26(6):851–883, 2020.
[ bib 
DOI ]

[213]

Z. Cao, S. Jiang, J. Zhang, and H. Guo.
A unified framework for vehicle rerouting and traffic light
control to reduce traffic congestion.
IEEE Transactions on Intelligent Transportation Systems,
18(7):1958–1973, 2017.
[ bib ]

[214]

Gilles Caporossi.
Variable Neighborhood Search for Extremal Vertices : The
AutoGraphiXIII System.
Computers & Operations Research, 78:431–438, 2017.
[ bib ]

[215]

J. Carlier.
The Onemachine Sequencing Problem.
European Journal of Operational Research, 11(1):42–47, 1982.
[ bib ]

[216]

William B. Carlton and J. Wesley Barnes.
Solving the travelingsalesman problem with time windows using
tabu search.
IIE Transactions, 28:617–629, 1996.
[ bib ]

[217]

Fabio Caraffini, Anna V. Kononova, and David Corne.
Infeasibility and structural bias in differential evolution.
Information Sciences, 496:161–179, 2019.
[ bib 
DOI ]

[218]

Yves Caseau and François Laburthe.
Heuristics for large constrained vehicle routing problems.
Journal of Heuristics, 5(3):281–303, 1999.
[ bib ]

[219]

Yves Caseau, Glenn Silverstein, and François Laburthe.
Learning Hybrid Algorithms for Vehicle Routing Problems.
Theory and Practice of Logic Programming, 1(6):779–806, 2001.
[ bib 
epub ]

[220]

Diego Cattaruzza, Nabil Absi, Dominique Feillet, and Daniele Vigo.
An Iterated Local Search for the Multicommodity Multitrip
Vehicle Routing Problem with Time Windows.
Computers & Operations Research, 51:257–267, 2014.
[ bib ]

[221]

Aakil M. Caunhye, Xiaofeng Nie, and Shaligram Pokharel.
Optimization models in emergency logistics: A literature
review.
SocioEconomic Planning Sciences, 46(1):4–13, 2012.
[ bib ]

[222]

Josu Ceberio, Ekhine Irurozki, Alexander Mendiburu, and José A. Lozano.
A distancebased ranking model estimation of distribution
algorithm for the flowshop scheduling problem.
IEEE Transactions on Evolutionary Computation, 18(2):286–300,
2014.
[ bib 
DOI ]
The aim of this paper is twofold. First, we introduce a
novel general estimation of distribution algorithm to deal
with permutationbased optimization problems. The algorithm
is based on the use of a probabilistic model for permutations
called the generalized Mallows model. In order to prove the
potential of the proposed algorithm, our second aim is to
solve the permutation flowshop scheduling problem. A hybrid
approach consisting of the new estimation of distribution
algorithm and a variable neighborhood search is
proposed. Conducted experiments demonstrate that the proposed
algorithm is able to outperform the stateoftheart
approaches. Moreover, from the 220 benchmark instances
tested, the proposed hybrid approach obtains new best known
results in 152 cases. An indepth study of the results
suggests that the successful performance of the introduced
approach is due to the ability of the generalized Mallows
estimation of distribution algorithm to discover promising
regions in the search space.
Keywords: Estimation of distribution algorithms,Generalized Mallows
model,Permutation flowshop scheduling
problem,Permutationsbased optimization problems

[223]

Vladimír Černý.
A Thermodynamical Approach to the Traveling Salesman Problem: An
Efficient Simulation Algorithm.
Journal of Optimization Theory and Applications, 45(1):41–51,
1985.
[ bib ]

[224]

Sara Ceschia, Luca Di Gaspero, and Andrea Schaerf.
Design, Engineering, and Experimental Analysis of a Simulated
Annealing Approach to the PostEnrolment Course Timetabling Problem.
Computers & Operations Research, 39(7):1615–1624, 2012.
[ bib ]

[225]

Sara Ceschia and Andrea Schaerf.
Modeling and solving the dynamic patient admission scheduling
problem under uncertainty.
Artificial Intelligence in Medicine, 56(3):199–205, 2012.
[ bib 
DOI ]
Keywords: Frace

[226]

Sara Ceschia, Andrea Schaerf, and Thomas Stützle.
Local Search Techniques for a Routingpacking Problem.
Computers and Industrial Engineering, 66(4):1138–1149, 2013.
[ bib ]

[227]

Shelvin Chand and Markus Wagner.
Evolutionary manyobjective optimization: A quickstart guide.
Surveys in Operations Research and Management Science,
20(2):35–42, 2015.
[ bib 
DOI ]

[228]

Donald V. Chase and Lindell E. Ormsbee.
Computergenerated pumping schedules for satisfying operation
objectives.
J. Am. Water Works Assoc., 85(7):54–61, 1993.
[ bib ]

[229]

Shamik Chaudhuri and Kalyanmoy Deb.
An interactive evolutionary multiobjective optimization and
decision making procedure.
Applied Soft Computing, 10(2):496–511, 2010.
[ bib ]

[230]

Hsinchun Chen, Roger H. L. Chiang, and Veda C. Storey.
Business Intelligence and Analytics: From Big Data to Big
Impact.
MIS Quarterly, 36(4):1165–1188, 2012.
[ bib ]

[231]

Yuning Chen, JinKao Hao, and Fred Glover.
A hybrid metaheuristic approach for the capacitated arc routing
problem.
European Journal of Operational Research, 553(1):25–39, 2016.
[ bib 
DOI ]
Keywords: irace

[232]

RueyMaw Chen and FuRen Hsieh.
An exchange local search heuristic based scheme for permutation
flow shop problems.
Applied Mathematics & Information Sciences, 8(1):209–215,
2014.
[ bib ]

[233]

F. Y. Cheng and X. S. Li.
Generalized center method for multiobjective engineering
optimization.
Engineering Optimization, 31(5):641–661, 1999.
[ bib 
DOI ]

[234]

Renzhi Chen, Ke Li, and Xin Yao.
Dynamic multiobjectives optimization with a changing number of
objectives.
IEEE Transactions on Evolutionary Computation, 22(1):157–171,
2017.
[ bib 
DOI ]
two coevolving populations (two archive)

[235]

Rachid Chelouah and Patrick Siarry.
Tabu search applied to global optimization.
European Journal of Operational Research, 123(2):256–270,
2000.
[ bib ]

[236]

Ni Chen, WeiNeng Chen, YueJiao Gong, ZhiHui Zhan, Jun Zhang, Yun Li, and
YuSong Tan.
An evolutionary algorithm with doublelevel archives for
multiobjective optimization.
IEEE Transactions on Cybernetics, 45(9):1851–1863, 2014.
[ bib ]

[237]

ChinBin Cheng and ChunPin Mao.
A modified ant colony system for solving the travelling salesman
problem with time windows.
Mathematical and Computer Modelling, 46:1225–1235, 2007.
[ bib 
DOI ]

[238]

Marco Chiarandini, Mauro Birattari, Krzysztof Socha, and O. RossiDoria.
An Effective Hybrid Algorithm for University Course
Timetabling.
Journal of Scheduling, 9(5):403–432, October 2006.
[ bib 
DOI ]
Keywords: 2003 international timetabling competition, Frace

[239]

Manuel Chica, Oscar Cordón, Sergio Damas, and Joaquín Bautista.
A New Diversity Induction Mechanism for a Multiobjective Ant
Colony Algorithm to Solve a Realworld time and Space Assembly Line Balancing
Problem.
Memetic Computing, 3(1):15–24, 2011.
[ bib ]

[240]

D. S. Chivilikhin, V. I. Ulyantsev, and A. A. Shalyto.
Modified ant colony algorithm for constructing finite state
machines from execution scenarios and temporal formulas.
Automation and Remote Control, 77(3):473–484, 2016.
[ bib 
DOI ]
Keywords: irace

[241]

Francisco Chicano, Darrell Whitley, and Enrique Alba.
A Methodology to Find the Elementary Landscape Decomposition of
Combinatorial Optimization Problems.
Evolutionary Computation, 19(4):597–637, 2011.
[ bib ]

[242]

Francisco Chicano, Gabriel J. Luque, and Enrique Alba.
Autocorrelation Measures for the Quadratic Assignment Problem.
Applied Mathematics Letters, 25:698–705, 2012.
[ bib 
DOI ]

[243]

Nicos Christofides, A. Mingozzi, and Paolo Toth.
Statespace relaxation procedures for the computation of bounds
to routing problems.
Networks, 11(2):145–164, 1981.
[ bib 
DOI ]

[244]

Tinkle Chugh, Yaochu Jin, Kaisa Miettinen, Jussi Hakanen, and Karthik Sindhya.
A SurrogateAssisted Reference Vector Guided Evolutionary
Algorithm for Computationally Expensive ManyObjective Optimization.
IEEE Transactions on Evolutionary Computation, 22(1):129–142,
February 2018.
[ bib ]

[245]

Tinkle Chugh, Karthik Sindhya, Jussi Hakanen, and Kaisa Miettinen.
A survey on handling computationally expensive multiobjective
optimization problems with evolutionary algorithms.
Soft Computing, 23(9):3137–3166, 2019.
[ bib 
DOI ]
Evolutionary algorithms are widely used for solving
multiobjective optimization problems but are often criticized
because of a large number of function evaluations
needed. Approximations, especially function approximations,
also referred to as surrogates or metamodels are commonly
used in the literature to reduce the computation time. This
paper presents a survey of 45 different recent algorithms
proposed in the literature between 2008 and 2016 to handle
computationally expensive multiobjective optimization
problems. Several algorithms are discussed based on what kind
of an approximation such as problem, function or fitness
approximation they use. Most emphasis is given to function
approximationbased algorithms. We also compare these
algorithms based on different criteria such as metamodeling
technique and evolutionary algorithm used, type and
dimensions of the problem solved, handling constraints,
training time and the type of evolution control. Furthermore,
we identify and discuss some promising elements and major
issues among algorithms in the literature related to using an
approximation and numerical settings used. In addition, we
discuss selecting an algorithm to solve a given
computationally expensive multiobjective optimization problem
based on the dimensions in both objective and decision spaces
and the computation budget available.

[246]

Christian Cintrano, Javier Ferrer, Manuel LópezIbáñez, and Enrique
Alba.
Hybridization of Evolutionary Operators with Elitist Iterated
Racing for the Simulation Optimization of Traffic Lights Programs.
Evolutionary Computation, 2022.
[ bib 
DOI ]
In the traffic light scheduling problem the evaluation of
candidate solutions requires the simulation of a process
under various (traffic) scenarios. Thus, good solutions
should not only achieve good objective function values, but
they must be robust (low variance) across all different
scenarios. Previous work has shown that combining IRACE with
evolutionary operators is effective for this task due to the
power of evolutionary operators in numerical optimization. In
this paper, we further explore the hybridization of
evolutionary operators and the elitist iterated racing of
IRACE for the simulationoptimization of traffic light
programs. We review previous works from the literature to
find the evolutionary operators performing the best when
facing this problem to propose new hybrid algorithms. We
evaluate our approach over a realistic case study derived
from the traffic network of MÃ¡laga (Spain) with 275 traffic
lights that should be scheduled optimally. The experimental
analysis reveals that the hybrid algorithm comprising IRACE
plus differential evolution offers statistically better
results than the other algorithms when the budget of
simulations is low. In contrast, IRACE performs better than
the hybrids for high simulations budget, although the
optimization time is much longer.
Keywords: irace, Simulation optimization, Uncertainty, Traffic light
planning

[247]

R. M. Clark, L. A. Rossman, and L. J. Wymer.
Modeling distribution system water quality: regulatory
implications.
Journal of Water Resources Planning and Management, ASCE,
121(6):423–428, 1995.
[ bib ]

[248]

Maurice Clerc and J. Kennedy.
The particle swarm  explosion, stability, and convergence in a
multidimensional complex space.
IEEE Transactions on Evolutionary Computation, 6(1):58–73,
February 2002.
[ bib 
DOI ]

[249]

Andy Cockburn, Pierre Dragicevic, Lonni Besançon, and Carl Gutwin.
Threats of a Replication Crisis in Empirical Computer Science.
Communications of the ACM, 63(8):70–79, July 2020.
[ bib 
DOI ]
Research replication only works if there is confidence built
into the results.

[250]

B. Codenotti, G. Manzini, L. Margara, and G. Resta.
Perturbation: An Efficient Technique for the Solution of Very
Large Instances of the Euclidean TSP.
INFORMS Journal on Computing, 8(2):125–133, 1996.
[ bib ]

[251]

Carlos A. Coello Coello.
Theoretical and numerical constrainthandling techniques used
with evolutionary algorithms: a survey of the state of the art.
Computer Methods in Applied Mechanics and Engineering,
191(1112):1245–1287, 2002.
[ bib 
DOI ]

[252]

Carlos A. Coello Coello.
Special Issue on Evolutionary Multiobjective
Optimization.
IEEE Transactions on Evolutionary Computation, 7(2), 2003.
[ bib ]

[253]

Carlos A. Coello Coello.
Evolutionary multiobjective optimization: a historical view of
the field.
IEEE Computational Intelligence Magazine, 1(1):28–36, 2006.
[ bib ]

[254]

Harry Cohn and Mark J. Fielding.
Simulated Annealing: Searching for an Optimal Temperature.
SIAM Journal on Optimization, 9(3):779–802, 1999.
[ bib ]

[255]

Andrew F. Colombo and Bryan W. Karney.
Impacts of Leaks on Energy Consumption in Pumped Systems with
Storage.
Journal of Water Resources Planning and Management, ASCE,
131(2):146–155, March 2005.
[ bib ]

[256]

Alberto Colorni, Marco Dorigo, Vittorio Maniezzo, and M. Trubian.
Ant System for Jobshop Scheduling.
JORBEL — Belgian Journal of Operations Research, Statistics
and Computer Science, 34(1):39–53, 1994.
[ bib ]

[257]

Barry McCollum, Andrea Schaerf, Ben Paechter, Paul McMullan, Rhyd M. R. Lewis,
Andrew J. Parkes, Luca Di Gaspero, Rong Qu, and Edmund K. Burke.
Setting the Research Agenda in Automated Timetabling: The Second
International Timetabling Competition.
INFORMS, 22(1):120–130, February 2010.
[ bib 
DOI ]

[258]

Richard K. Congram, Chris N. Potts, and Steve van de Velde.
An Iterated Dynasearch Algorithm for the SingleMachine Total
Weighted Tardiness Scheduling Problem.
INFORMS Journal on Computing, 14(1):52–67, 2002.
[ bib ]

[259]

David T. Connolly.
An Improved Annealing Scheme for the QAP.
European Journal of Operational Research, 46(1):93–100, 1990.
[ bib ]

[260]

Richard J. Cook and Vern T. Farewell.
Multiplicity Considerations in the Design and Analysis of
Clinical Trials.
Journal of the Royal Statistical Society: Series A,
159:93–110, 1996.
[ bib ]
multiplicity; multiple endpoints; multiple treatments;
pvalue adjustment; type I error; argues that if results are
intended to be interpreted marginally, there may be no need
for controlling experimentwise error rate

[261]

Don Coppersmith, Lisa K. Fleischer, and Atri Rurda.
Ordering by Weighted Number of Wins Gives a Good Ranking for
Weighted Tournaments.
ACM Transactions on Algorithms, 6(3):1–13, July 2010.
[ bib 
DOI ]
Keywords: Approximation algorithms,Borda's method,feedback arc set
problem,rank aggregation,tournaments

[262]

Oscar Cordón and Sergio Damas.
Image Registration with Iterated Local Search.
Journal of Heuristics, 12(1–2):73–94, 2006.
[ bib ]

[263]

Jeroen Corstjens, Nguyen Dang, Benoît Depaire, An Caris, and Patrick De
Causmaecker.
A combined approach for analysing heuristic algorithms.
Journal of Heuristics, 25(4):591–628, 2019.
[ bib 
DOI ]

[264]

Jeroen Corstjens, Benoît Depaire, An Caris, and Kenneth Sörensen.
A multilevel evaluation method for heuristics with an
application to the VRPTW.
International Transactions in Operational Research,
27(1):168–196, 2020.
[ bib 
DOI ]

[265]

P. Corry and E. Kozan.
Ant Colony Optimisation for Machine Layout Problems.
Computational Optimization and Applications, 28(3):287–310,
2004.
[ bib ]

[266]

JeanFrançois Cordeau, Gilbert Laporte, and A. Mercier.
A unified tabu search heuristic for vehicle routing problems
with time windows.
Journal of the Operational Research Society, 52(8):928–936,
2001.
[ bib ]

[267]

JeanFrançois Cordeau and Mirko Maischberger.
A Parallel Iterated Tabu Search Heuristic for Vehicle Routing
Problems.
Computers & Operations Research, 39(9):2033–2050, 2012.
[ bib ]

[268]

Wagner Emanoel Costa, Marco Cesar Goldbarg, and Elizabeth Ferreira Gouvêa
Goldbarg.
Hybridizing VNS and pathrelinking on a particle swarm
framework to minimize total flowtime.
Expert Systems with Applications, 39(18):13118–13126, 2012.
[ bib ]

[269]

D. Costa and A. Hertz.
Ants can color graphs.
Journal of the Operational Research Society, 48:295–305, 1997.
[ bib ]

[270]

S. P. Coy, B. L. Golden, G. C. Runger, and E. A. Wasil.
Using Experimental Design to Find Effective Parameter Settings
for Heuristics.
Journal of Heuristics, 7(1):77–97, 2001.
[ bib ]

[271]

I. Barry Crabtree.
Resource Scheduling: Comparing Simulated Annealing with
Constraint Programming.
BT Technology Journal, 13(1):121–127, 1995.
[ bib ]

[272]

Douglas Edward Critchlow, Michael A. Fligner, and Joseph S. Verducci.
Probability Models on Rankings.
Journal of Mathematical Psychology, 35:294–318, 1991.
[ bib ]

[273]

G. A. Croes.
A Method for Solving Traveling Salesman Problems.
Operations Research, 6:791–812, 1958.
[ bib ]

[274]

Harlan P. Crowder, Ron S. Dembo, and John M. Mulvey.
Reporting computational experiments in mathematical
programming.
Mathematical Programming, 15(1):316–329, 1978.
[ bib 
DOI ]
Keywords: reproducibility

[275]

Carlos Cruz, Juan Ramón González, and David A. Pelta.
Optimization in Dynamic Environments: A Survey on Problems,
Methods and Measures.
Soft Computing, 15(7):1427–1448, 2011.
[ bib ]

[276]

Fábio Cruz, Anand Subramanian, Bruno P. Bruck, and Manuel Iori.
A Heuristic Algorithm for a Single Vehicle Static Bike Sharing
Rebalancing Problem.
Computers & Operations Research, 79:19–33, 2017.
[ bib ]

[277]

Joseph C. Culberson.
On the Futility of Blind Search: An Algorithmic View of “No
Free Lunch”.
Evolutionary Computation, 6(2):109–127, 1998.
[ bib 
DOI ]
Keywords: NFL

[278]

P. Czyzżak and Andrzej Jaszkiewicz.
Pareto simulated annealing – a metaheuristic technique for
multipleobjective combinatorial optimization.
Journal of MultiCriteria Decision Analysis, 7(1):34–47, 1998.
[ bib ]

[279]

Steven B. Damelin, Fred J. Hickernell, David L. Ragozin, and Xiaoyan Zeng.
On Energy, Discrepancy and Group Invariant Measures on
Measurable Subsets of Euclidean Space.
Journal of Fourier Analysis and Applications, 16(6):813–839,
2010.
[ bib ]
Keywords: Capacity; Cubature; Discrepancy; Distribution; Group
invariant kernel; Group invariant measure; Energy minimizer;
Equilibrium measure; Numerical integration; Positive
definite; Potential field; Riesz kernel; Reproducing Hilbert
space; Signed measure

[280]

M. Damas, M. Salmerón, J. Ortega, G. Olivares, and H. Pomares.
Parallel Dynamic Water Supply Scheduling in a Cluster of
Computers.
Concurrency and Computation: Practice and Experience,
13(15):1281–1302, December 2001.
[ bib ]

[281]

Augusto Dantas and Aurora Pozo.
On the use of fitness landscape features in metalearning based
algorithm selection for the quadratic assignment problem.
Theoretical Computer Science, 805:62–75, 2020.
[ bib 
DOI ]

[282]

Emilie Danna, Edward Rothberg, and Claude Le Pape.
Exploring relaxation induced neighborhoods to improve MIP
solutions.
Mathematical Programming, 102(1):71–90, 2005.
[ bib ]

[283]

George B. Dantzig and Philip Wolfe.
Decomposition Principle for Linear Programs.
Operations Research, 8(1):101–111, 1960.
[ bib ]

[284]

Fabio Daolio, Arnaud Liefooghe, Sébastien Verel, Hernán E. Aguirre, and
Kiyoshi Tanaka.
Problem Features versus Algorithm Performance on Rugged
Multiobjective Combinatorial Fitness Landscapes.
Evolutionary Computation, 25(4):555–585, 2017.
[ bib 
DOI ]

[285]

Indraneel Das and John E. Dennis.
A closer look at drawbacks of minimizing weighted sums of
objectives for Pareto set generation in multicriteria optimization
problems.
Structural Optimization, 14(1):63–69, 1997.
[ bib 
DOI ]

[286]

Swagatam Das, Sankha Subhra Mullick, and Ponnuthurai N. Suganthan.
Recent advances in differential evolution–An updated survey.
Swarm and Evolutionary Computation, 27:1–30, 2016.
[ bib ]

[287]

Swagatam Das and Ponnuthurai N. Suganthan.
Differential Evolution: A Survey of the Stateoftheart.
IEEE Transactions on Evolutionary Computation, 15(1), February
2011.
[ bib ]

[288]

Sanjeeb Dash.
Exponential Lower Bounds on the Lengths of Some Classes of
BranchandCut Proofs.
Mathematics of Operations Research, 30(3):678–700, 2005.
[ bib ]

[289]

Constantinos Daskalakis, Ilias Diakonikolas, and Mihalis Yannakakis.
How good is the Chord algorithm?
SIAM Journal on Computing, 45(3):811–858, 2016.
[ bib ]

[290]

Jean Daunizeau, Hanneke E. M. den Ouden, Matthias Pessiglione, Stefan J.
Kiebel, Karl J. Friston, and Klaas E. Stephan.
Observing the observer (II): deciding when to decide.
PLoS One, 5(12):e15555, 2010.
[ bib 
DOI ]

[291]

Jean Daunizeau, Hanneke E. M. den Ouden, Matthias Pessiglione, Klaas E.
Stephan, Stefan J. Kiebel, and Karl J. Friston.
Observing the observer (I): metaBayesian models of learning
and decisionmaking.
PLoS One, 5(12):e15554, 2010.
[ bib 
DOI ]

[292]

Kalyanmoy Deb.
An efficient constraint handling method for genetic algorithms.
Computer Methods in Applied Mechanics and Engineering,
186(2/4):311–338, 2000.
[ bib 
DOI ]

[293]

Kalyanmoy Deb, A. Pratap, S. Agarwal, and T. Meyarivan.
A fast and elitist multiobjective genetic algorithm:
NSGAII.
IEEE Transactions on Evolutionary Computation, 6(2):182–197,
2002.
[ bib 
DOI ]

[294]

Kalyanmoy Deb.
Multiobjective genetic algorithms: problem difficulties and
construction of test problems.
Evolutionary Computation, 7(3):205–230, 1999.
[ bib ]
Naive definition of PLOset

[295]

Kalyanmoy Deb and Ram Bhushan Agrawal.
Simulated binary crossover for continuous search spaces.
Complex Systems, 9(2):115–148, 1995.
[ bib 
epub ]
Keywords: SBX

[296]

Kalyanmoy Deb and Debayan Deb.
Analysing mutation schemes for realparameter genetic
algorithms.
International Journal of Artificial Intelligence and Soft
Computing, 4(1):1–28, 2014.
[ bib ]

[297]

Kalyanmoy Deb, S. Gupta, D. Daum, Jürgen Branke, A. Mall, and
D. Padmanabhan.
Reliabilitybased optimization using evolutionary algorithms.
IEEE Transactions on Evolutionary Computation,
13(5):1054–1074, October 2009.
[ bib 
DOI ]

[298]

Kalyanmoy Deb and Himanshu Jain.
An Evolutionary ManyObjective Optimization Algorithm Using
ReferencePointBased Nondominated Sorting Approach, Part I: Solving
Problems With Box Constraints.
IEEE Transactions on Evolutionary Computation, 18(4):577–601,
2014.
[ bib ]
Proposed NSGAIII

[299]

Kalyanmoy Deb and Murat Köksalan.
Guest Editorial: Special Issue on Preferencebased
Multiobjective Evolutionary Algorithms.
IEEE Transactions on Evolutionary Computation, 14(5):669–670,
October 2010.
[ bib 
DOI ]

[300]

Kalyanmoy Deb, Manikanth Mohan, and Shikhar Mishra.
Evaluating the εdomination based multiobjective
evolutionary algorithm for a quick computation of Paretooptimal
solutions.
Evolutionary Computation, 13(4):501–525, December 2005.
[ bib 
DOI ]
Keywords: εdominance, εMOEA

[301]

Kalyanmoy Deb, Ling Zhu, and Sandeep Kulkarni.
Handling Multiple Scenarios in Evolutionary MultiObjective
Numerical Optimization.
IEEE Transactions on Evolutionary Computation, 22(6):920–933,
2018.
[ bib 
DOI ]
Solutions to most practical numerical optimization problems
must be evaluated for their performance over a number of
different loading or operating conditions, which we refer
here as scenarios. Therefore, a meaningful and resilient
optimal solution must be such that it remains feasible under
all scenarios and performs close to an individual optimal
solution corresponding to each scenario. Despite its
practical importance, multiscenario consideration has
received a lukewarm attention, particularly in the context of
multiobjective optimization. The usual practice is to
optimize for the worstcase scenario. In this paper, we
review existing methodologies in this direction and set our
goal to suggest a new and potential populationbased method
for handling multiple scenarios by defining scenariowise
domination principle and scenariowise diversitypreserving
operators. To evaluate, the proposed method is applied to a
number of numerical test problems and engineering design
problems with a detail explanation of the obtained results
and compared with an existing method. This first systematic
evolutionary based multiscenario, multiobjective,
optimization study on numerical problems indicates that
multiple scenarios can be handled in an integrated manner
using an EMO framework to find a wellbalanced compromise set
of solutions to multiple scenarios and maintain a tradeoff
among multiple objectives. In comparison to an existing
serial multiple optimization approach, the proposed approach
finds a set of compromised tradeoff solutions
simultaneously. An achievement of multiobjective tradeoff
and multiscenario tradeoff is algorithmically challenging,
but due to its practical appeal, further research and
application must be spent.
Keywords: scenariobased

[302]

Annelies De Corte and Kenneth Sörensen.
Optimisation of gravityfed water distribution network design: A
critical review.
European Journal of Operational Research, 228(1):1–10, 2013.
[ bib 
DOI ]

[303]

Annelies De Corte and Kenneth Sörensen.
An Iterated Local Search Algorithm for Water Distribution
Network Design Optimization.
Networks, 67(3):187–198, 2016.
[ bib ]

[304]

Annelies De Corte and Kenneth Sörensen.
An Iterated Local Search Algorithm for multiperiod water
distribution network design optimization.
Water, 8(8):359, 2016.
[ bib 
DOI ]

[305]

V. Dekhtyarenko.
Verification of weight coefficients in multicriteria
optimization problems.
ComputerAided Design, 13(6):339–344, 1981.
[ bib ]

[306]

X. Delorme, Xavier Gandibleux, and F. Degoutin.
Evolutionary, constructive and hybrid procedures for the
biobjective set packing problem.
European Journal of Operational Research, 204(2):206–217,
2010.
[ bib ]
This paper cannot be found on internet!! Does it exist?

[307]

Federico Della Croce, Thierry Garaix, and Andrea Grosso.
Iterated Local Search and Very Large Neighborhoods for the
Parallelmachines Total Tardiness Problem.
Computers & Operations Research, 39(6):1213–1217, 2012.
[ bib ]

[308]

Maxence Delorme, Manuel Iori, and Silvano Martello.
Bin packing and cutting stock problems: Mathematical models and
exact algorithms.
European Journal of Operational Research, 255(1):1–20, 2016.
[ bib 
DOI ]

[309]

Mauro Dell'Amico, Manuel Iori, Silvano Martello, and Michele Monaci.
Heuristic and Exact Algorithms for the Identical Parallel
Machine Scheduling Problem.
INFORMS Journal on Computing, 20(3):333–344, 2016.
[ bib ]

[310]

Maxence Delorme, Manuel Iori, and Silvano Martello.
BPPLIB: a library for bin packing and cutting stock problems.
Optimization Letters, 12(2):235–250, 2018.
[ bib 
DOI ]

[311]

Mauro Dell'Amico, Manuel Iori, Stefano Novellani, and Thomas Stützle.
A destroy and repair algorithm for the Bike sharing Rebalancing
Problem.
Computers & Operations Research, 71:146–162, 2016.
[ bib 
DOI ]
Keywords: irace

[312]

Robert F. Dell and Mark H. Karwan.
An interactive MCDM weight space reduction method utilizing a
Tchebycheff utility function.
Naval Research Logistics, 37(2):263–277, 1990.
[ bib ]

[313]

Mauro Dell'Amico and Marco Trubian.
Applying Tabu Search to the Job Shop Scheduling Problem.
Annals of Operations Research, 41:231–252, 1993.
[ bib ]

[314]

Stephan Dempe, Gabriele Eichfelder, and Jörg Fliege.
On the effects of combining objectives in multiobjective
optimization.
Mathematical Methods of Operations Research, 82(1):1–18, 2015.
[ bib ]

[315]

JeanLouis Deneubourg, S. Aron, S. Goss, and J.M. Pasteels.
The SelfOrganizing Exploratory Pattern of the Argentine Ant.
Journal of Insect Behavior, 3(2):159–168, 1990.
[ bib 
DOI ]

[316]

Joaquín Derrac, Salvador García, Daniel Molina, and Francisco Herrera.
A practical tutorial on the use of nonparametric statistical
tests as a methodology for comparing evolutionary and swarm intelligence
algorithms.
Swarm and Evolutionary Computation, 1(1):3–18, 2011.
[ bib ]

[317]

Ulrich Derigs and Ulrich Vogel.
Experience with a Framework for Developing Heuristics for
Solving Rich Vehicle Routing Problems.
Journal of Heuristics, 20(1):75–106, 2014.
[ bib ]

[318]

Aryan Deshwal, Syrine Belakaria, Janardhan Rao Doppa, and Dae Hyun Kim.
Bayesian Optimization over Permutation Spaces.
Arxiv preprint arXiv:2112.01049, 2021.
[ bib 
DOI ]
Keywords: BOPS, CEGO

[319]

Marcelo De Souza, Marcus Ritt, Manuel LópezIbáñez, and Leslie
Pérez Cáceres.
ACVIZ: A Tool for the Visual Analysis of the
Configuration of Algorithms with irace.
Operations Research Perspectives, 8:100186, 2021.
[ bib 
DOI 
supplementary material ]
This paper introduces acviz, a tool that helps to analyze the
automatic configuration of algorithms with irace. It provides
a visual representation of the configuration process,
allowing users to extract useful information, e.g. how the
configurations evolve over time. When test data is available,
acviz also shows the performance of each configuration on the
test instances. Using this visualization, users can analyze
and compare the quality of the resulting configurations and
observe the performance differences on training and test
instances.

[320]

Paolo Detti, Francesco Papalini, and Garazi Zabalo Manrique de Lara.
A multidepot dialaride problem with heterogeneous vehicles
and compatibility constraints in healthcare.
Omega, 70:1–14, 2017.
[ bib 
DOI ]

[321]

Sven De Vries and Rakesh V. Vohra.
Combinatorial Auctions: A Survey.
INFORMS Journal on Computing, 15(3):284–309, 2003.
[ bib ]

[322]

Juan Esteban Diaz, Julia Handl, and DongLing Xu.
Evolutionary robust optimization in production planning:
interactions between number of objectives, sample size and choice of
robustness measure.
Computers & Operations Research, 79:266–278, 2017.
[ bib 
DOI ]
Keywords: Evolutionary multiobjective optimization, Production
planning, Robust optimization, Simulationbased optimization,
Uncertainty modelling

[323]

Juan Esteban Diaz, Julia Handl, and DongLing Xu.
Integrating metaheuristics, simulation and exact techniques for
production planning of a failureprone manufacturing system.
European Journal of Operational Research, 266(3):976–989,
2018.
[ bib 
DOI ]
Keywords: Genetic algorithms, Combinatorial optimization, Production
planning, Simulationbased optimization, Uncertainty
modelling

[324]

Juan Esteban Diaz and Manuel LópezIbáñez.
Incorporating DecisionMaker's Preferences into the Automatic
Configuration of BiObjective Optimisation Algorithms.
European Journal of Operational Research, 289(3):1209–1222,
2021.
[ bib 
DOI 
supplementary material ]
Automatic configuration (AC) methods are increasingly used to
tune and design optimisation algorithms for problems with
multiple objectives. Most AC methods use unary quality
indicators, which assign a single scalar value to an
approximation to the Pareto front, to compare the performance
of different optimisers. These quality indicators, however,
imply preferences beyond Paretooptimality that may differ
from those of the decision maker (DM). Although it is
possible to incorporate DM's preferences into quality
indicators, e.g., by means of the weighted hypervolume
indicator (HV^{w}), expressing preferences in terms of weight
function is not always intuitive nor an easy task for a DM,
in particular, when comparing the stochastic outcomes of
several algorithm configurations. A more visual approach to
compare such outcomes is the visualisation of their empirical
attainment functions (EAFs) differences. This paper proposes
using such visualisations as a way of eliciting information
about regions of the objective space that are preferred by
the DM. We present a method to convert the information about
EAF differences into a HV^{w} that will assign higher quality
values to approximation fronts that result in EAF differences
preferred by the DM. We show that the resulting HV^{w} may be
used by an AC method to guide the configuration of
multiobjective optimisers according to the preferences of
the DM. We evaluate the proposed approach on a wellknown
benchmark problem. Finally, we apply our approach to
reconfiguring, according to different DM's preferences, a
multiobjective optimiser tackling a realworld production
planning problem arising in the manufacturing industry.

[325]

L. C. Dias, Vincent Mousseau, José Rui Figueira, and J. N. Clímaco.
An aggregation/disaggregation approach to obtain robust
conclusions with ELECTRE TRI.
European Journal of Operational Research, 138(2):332–348,
April 2002.
[ bib ]

[326]

Ilias Diakonikolas and Mihalis Yannakakis.
Small approximate Pareto sets for biobjective shortest paths
and other problems.
SIAM Journal on Computing, 39(4):1340–1371, 2009.
[ bib ]

[327]

Gianni A. Di Caro and Marco Dorigo.
AntNet: Distributed Stigmergetic Control for Communications
Networks.
Journal of Artificial Intelligence Research, 9:317–365, 1998.
[ bib ]

[328]

Gianni A. Di Caro, F. Ducatelle, and L. M. Gambardella.
AntHocNet: An adaptive natureinspired algorithm for routing
in mobile ad hoc networks.
European Transactions on Telecommunications, 16(5):443–455,
2005.
[ bib ]

[329]

Luca Di Gaspero and Andrea Schaerf.
EasyLocal++: An objectoriented framework for flexible
design of local search algorithms.
Software — Practice & Experience, 33(8):733–765, July 2003.
[ bib 
epub ]
Keywords: software engineering, local search, easylocal

[]

Bistra Dilkina, Elias B. Khalil, and George L. Nemhauser.
Comments on: On learning and branching: a survey.
TOP, 25:242–246, 2017.
[ bib ]
Comments on [805].

[331]

Rui Ding, Hongbin Dong, Jun He, and Tao Li.
A novel twoarchive strategy for evolutionary manyobjective
optimization algorithm based on reference points.
Applied Soft Computing, 78:447–464, 2019.
[ bib 
DOI ]

[332]

J.Y. Ding, S. Song, J. N. D. Gupta, R. Zhang, R. Chiong, and C. Wu.
An Improved Iterated Greedy Algorithm with a Tabubased
Reconstruction Strategy for the Nowait Flowshop Scheduling Problem.
Applied Soft Computing, 30:604–613, 2015.
[ bib ]

[333]

Benjamin Doerr, Carola Doerr, and Franziska Ebel.
From blackbox complexity to designing new genetic algorithms.
Theoretical Computer Science, 567:87–104, 2015.
[ bib 
DOI ]

[334]

Benjamin Doerr, Carola Doerr, and Jing Yang.
Optimal parameter choices via precise blackbox analysis.
Theoretical Computer Science, 801:1–34, 2020.
[ bib 
DOI ]

[335]

Karl F. Doerner, Guenther Fuellerer, Manfred Gronalt, Richard F. Hartl, and
Manuel Iori.
Metaheuristics for the Vehicle Routing Problem with Loading
Constraints.
Networks, 49(4):294–307, 2006.
[ bib ]

[336]

Benjamin Doerr, Christian Gießen, Carsten Witt, and Jing Yang.
The (1+λ) evolutionary algorithm with selfadjusting
mutation rate.
Algorithmica, 81(2):593–631, 2019.
[ bib ]

[337]

Karl F. Doerner, Walter J. Gutjahr, Richard F. Hartl, Christine Strauss, and
Christian Stummer.
NatureInspired Metaheuristics in Multiobjective Activity
Crashing.
Omega, 36(6):1019–1037, 2008.
[ bib ]

[338]

Karl F. Doerner, Walter J. Gutjahr, Richard F. Hartl, Christine Strauss, and
Christian Stummer.
Pareto Ant Colony Optimization: A Metaheuristic Approach to
Multiobjective Portfolio Selection.
Annals of Operations Research, 131:79–99, 2004.
[ bib ]

[339]

Karl F. Doerner, Walter J. Gutjahr, Richard F. Hartl, Christine Strauss, and
Christian Stummer.
Pareto ant colony optimization with ILP preprocessing in
multiobjective project portfolio selection.
European Journal of Operational Research, 171:830–841, 2006.
[ bib ]

[340]

Karl F. Doerner, Richard F. Hartl, and Marc Reimann.
Are COMPETants more competent for problem solving? The case
of a multiple objective transportation problem.
Central European Journal for Operations Research and Economics,
11(2):115–141, 2003.
[ bib ]

[341]

Benjamin Doerr, Daniel Johannsen, and Carola Winzen.
Multiplicative drift analysis.
Algorithmica, 64(4):673–697, 2012.
[ bib ]

[342]

Benjamin Doerr, Timo Kötzing, Johannes Lengler, and Carola Winzen.
Blackbox complexities of combinatorial problems.
Theoretical Computer Science, 471:84–106, 2013.
[ bib ]

[343]

Karl F. Doerner, D. Merkle, and Thomas Stützle.
Special issue on Ant Colony Optimization.
Swarm Intelligence, 3(1), 2009.
[ bib ]

[344]

Benjamin Doerr, Frank Neumann, Dirk Sudholt, and Carsten Witt.
Runtime analysis of the 1ANT ant colony optimizer.
Theoretical Computer Science, 412(1):1629–1644, 2011.
[ bib ]

[345]

Doǧan Aydın.
Composite artificial bee colony algorithms: From componentbased
analysis to highperforming algorithms.
Applied Soft Computing, 32:266–285, 2015.
[ bib 
DOI ]
Keywords: irace

[346]

JeanPaul Doignon, Aleksandar Pekeč, and Michel Regenwetter.
The repeated insertion model for rankings: Missing link between
two subset choice models.
Psychometrika, 69(1):33–54, March 2004.
[ bib 
DOI ]
Several probabilistic models for subset choice have been
proposed in the literature, for example, to explain approval
voting data. We show that Marley et al.'s latent scale model
is subsumed by Falmagne and Regenwetter's sizeindependent
model, in the sense that every choice probability
distribution generated by the former can also be explained by
the latter. Our proof relies on the construction of a
probabilistic ranking model which we label the “repeated
insertion model”. This model is a special case of Marden's
orthogonal contrast model class and, in turn, includes the
classical Mallows φmodel as a special case. We
explore its basic properties as well as its relationship to
Fligner and Verducci's multistage ranking model.

[347]

Elizabeth D. Dolan and Jorge J. Moré.
Benchmarking optimization software with performance profiles.
Mathematical Programming, 91(2):201–213, 2002.
[ bib ]
Keywords: performance profiles; convergence

[348]

Xingye Dong, Ping, Houkuan Huang, and Maciek Nowak.
A Multirestart Iterated Local Search Algorithm for the
Permutation Flow Shop Problem Minimizing Total Flow Time.
Computers & Operations Research, 40(2):627–632, 2013.
[ bib ]

[349]

X. Dong, H. Huang, and P. Chen.
An Iterated Local Search Algorithm for the Permutation Flowshop
Problem with Total Flowtime Criterion.
Computers & Operations Research, 36(5):1664–1669, 2009.
[ bib ]

[350]

A. V. Donati, Roberto Montemanni, N. Casagrande, A. E. Rizzoli, and L. M.
Gambardella.
Time dependent vehicle routing problem with a multi ant colony
system.
European Journal of Operational Research, 185(3):1174–1191,
2008.
[ bib ]

[351]

Marco Dorigo.
Ant Colony Optimization.
Scholarpedia, 2(3):1461, 2007.
[ bib 
DOI ]

[352]

Marco Dorigo.
Swarm intelligence: A few things you need to know if you want to
publish in this journal.
Swarm Intelligence, November 2016.
[ bib 
http ]

[353]

Marco Dorigo, Mauro Birattari, Xiaodong Li, Manuel LópezIbáñez,
Kazuhiro Ohkura, Carlo Pinciroli, and Thomas Stützle.
ANTS 2016 Special Issue: Editorial.
Swarm Intelligence, November 2017.
[ bib 
DOI ]

[354]

Marco Dorigo, Mauro Birattari, and Thomas Stützle.
Ant Colony Optimization: Artificial Ants as a Computational
Intelligence Technique.
IEEE Computational Intelligence Magazine, 1(4):28–39, 2006.
[ bib ]

[355]

Marco Dorigo and Christian Blum.
Ant colony optimization theory: A survey.
Theoretical Computer Science, 344(23):243–278, 2005.
[ bib ]

[356]

Marco Dorigo, Gianni A. Di Caro, and L. M. Gambardella.
Ant Algorithms for Discrete Optimization.
Artificial Life, 5(2):137–172, 1999.
[ bib ]

[357]

Marco Dorigo and L. M. Gambardella.
Ant Colonies for the Traveling Salesman Problem.
BioSystems, 43(2):73–81, 1997.
[ bib 
DOI ]

[358]

Marco Dorigo and L. M. Gambardella.
Ant Colony System: A Cooperative Learning Approach to the
Traveling Salesman Problem.
IEEE Transactions on Evolutionary Computation, 1(1):53–66,
1997.
[ bib ]
Keywords: Ant Colony System

[359]

Marco Dorigo, L. M. Gambardella, Martin Middendorf, and Thomas Stützle.
Guest Editorial: Special Section on Ant Colony Optimization.
IEEE Transactions on Evolutionary Computation, 6(4):317–320,
2002.
[ bib 
DOI ]
Keywords: ant colony optimization, swarm intelligence

[360]

Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni.
Ant System: Optimization by a Colony of Cooperating Agents.
IEEE Transactions on Systems, Man, and Cybernetics – Part B,
26(1):29–41, 1996.
[ bib ]

[361]

Marco Dorigo, Thomas Stützle, and Gianni A. Di Caro.
Special Issue on “Ant Algorithms”.
Future Generation Computer Systems, 16(8), 2000.
[ bib ]
Keywords: swarm intelligence, ant colony optimization

[362]

Michael Doumpos and Constantin Zopounidis.
Preference disaggregation and statistical learning for
multicriteria decision support: A review.
European Journal of Operational Research, 209(3):203–214,
2011.
[ bib ]

[363]

Erik Dovgan, Tea Tušar, and Bogdan Filipič.
Parameter tuning in an evolutionary algorithm for commodity
transportation optimization.
Evolutionary Computation, pp. 1–8, 2010.
[ bib ]

[364]

Johann Dréo and P. Siarry.
Continuous interacting ant colony algorithm based on dense
heterarchy.
Future Generation Computer Systems, 20(5):841–856, 2004.
[ bib ]

[365]

Stefan Droste, Thomas Jansen, and Ingo Wegener.
Upper and lower bounds for randomized search heuristics in
blackbox optimization.
Theory of Computing Systems, 39(4):525–544, 2006.
[ bib ]

[366]

Mădălina M. Drugan and Dirk Thierens.
Stochastic Pareto local search: Pareto neighbourhood
exploration and perturbation strategies.
Journal of Heuristics, 18(5):727–766, 2012.
[ bib ]

[367]

J. Du and Joseph Y.T. Leung.
Minimizing Total Tardiness on One Machine is NPHard.
Mathematics of Operations Research, 15(3):483–495, 1990.
[ bib ]

[368]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Improving the Anytime Behavior of TwoPhase Local Search.
Annals of Mathematics and Artificial Intelligence,
61(2):125–154, 2011.
[ bib 
DOI ]

[369]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
A Hybrid TP+PLS Algorithm for Biobjective FlowShop
Scheduling Problems.
Computers & Operations Research, 38(8):1219–1236, 2011.
[ bib 
DOI 
supplementary material ]

[370]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Anytime Pareto Local Search.
European Journal of Operational Research, 243(2):369–385,
2015.
[ bib 
DOI ]
Keywords: Pareto local search

[371]

Jérémie DuboisLacoste, Federico Pagnozzi, and Thomas Stützle.
An Iterated Greedy Algorithm with Optimization of Partial
Solutions for the Permutation Flowshop Problem.
Computers & Operations Research, 81:160–166, 2017.
[ bib 
DOI 
supplementary material ]

[372]

Fabian Duddeck.
Multidisciplinary optimization of car bodies.
Structural and Multidisciplinary Optimization, 35(4):375–389,
2008.
[ bib 
DOI ]
Evolutionary optimization of car bodies at General Motors

[373]

Gunter Dueck and T. Scheuer.
Threshold Accepting: A General Purpose Optimization Algorithm
Appearing Superior to Simulated Annealing.
Journal of Computational Physics, 90(1):161–175, 1990.
[ bib ]

[374]

Gunter Dueck.
New Optimization Heuristics: the Great Deluge Algorithm and the
RecordToRecord Travel.
Journal of Computational Physics, 104(1):86–92, 1993.
[ bib ]

[375]

Rikky R. P. R. Duivenvoorden, Felix Berkenkamp, Nicolas Carion, Andreas Krause,
and Angela P. Schoellig.
Constrained Bayesian Optimization with Particle Swarms for
Safe Adaptive Controller Tuning.
IFACPapersOnLine, 50(1):11800–11807, 2017.
[ bib 
DOI ]
Tuning controller parameters is a recurring and
timeconsuming problem in control. This is especially true in
the field of adaptive control, where good performance is
typically only achieved after significant tuning. Recently,
it has been shown that constrained Bayesian optimization is a
promising approach to automate the tuning process without
risking system failures during the optimization
process. However, this approach is computationally too
expensive for tuning more than a couple of parameters. In
this paper, we provide a heuristic in order to efficiently
perform constrained Bayesian optimization in highdimensional
parameter spaces by using an adaptive discretization based on
particle swarms. We apply the method to the tuning problem of
an L1 adaptive controller on a quadrotor vehicle and show
that we can reliably and automatically tune parameters in
experiments.
20th IFAC World Congress
Keywords: Adaptive Control, Constrained Bayesian Optimization, Safety,
Gaussian Process, Particle Swarm Optimization, Policy Search,
Reinforcement learning

[376]

Cees Duin and Stefan Voß.
The Pilot Method: A Strategy for Heuristic Repetition with
Application to the Steiner Problem in Graphs.
Networks, 34(3):181–191, 1999.
[ bib ]

[377]

Y. Dumas, J. Desrosiers, E. Gelinas, and M. M. Solomon.
An Optimal Algorithm for the Traveling Salesman Problem with
Time Windows.
Operations Research, 43(2):367–371, 1995.
[ bib 
DOI ]

[378]

Olive Jean Dunn.
Multiple Comparisons Using Rank Sums.
Technometrics, 6(3):241–252, 1964.
[ bib ]

[379]

Olive Jean Dunn.
Multiple Comparisons Among Means.
Journal of the American Statistical Association,
56(293):52–64, 1961.
[ bib ]

[380]

Juan J. Durillo and Antonio J. Nebro.
jMetal: A Java framework for multiobjective optimization.
Advances in Engineering Software, 42(10):760–771, 2011.
[ bib 
DOI ]

[381]

Richard W. Eglese.
Simulated Annealing: a Tool for Operational Research.
European Journal of Operational Research, 46(3):271–281, 1990.
[ bib ]

[382]

Matthias Ehrgott.
A discussion of scalarization techniques for multiple objective
integer programming.
Annals of Operations Research, 147(1):343–360, 2006.
[ bib ]

[383]

Matthias Ehrgott and Xavier Gandibleux.
Approximative Solution Methods for Combinatorial Multicriteria
Optimization.
TOP, 12(1):1–88, 2004.
[ bib ]

[384]

Matthias Ehrgott and Kathrin Klamroth.
Connectedness of Efficient Solutions in Multiple Criteria
Combinatorial Optimization.
European Journal of Operational Research, 97(1):159–166, 1997.
[ bib 
DOI ]

[385]

Agoston E. Eiben, Robert Hinterding, and Zbigniew Michalewicz.
Parameter Control in Evolutionary Algorithms.
IEEE Transactions on Evolutionary Computation, 3(2):124–141,
1999.
[ bib ]

[386]

Agoston E. Eiben and Günther Rudolph.
Theory of evolutionary algorithms: A bird's eye view.
Theoretical Computer Science, 229(12):3–9, 1999.
[ bib ]

[387]

Agoston E. Eiben and Selmar K. Smit.
Parameter Tuning for Configuring and Analyzing Evolutionary
Algorithms.
Swarm and Evolutionary Computation, 1(1):19–31, 2011.
[ bib 
DOI ]

[388]

Sibel Eker and Jan H. Kwakkel.
Including robustness considerations in the search phase of
ManyObjective Robust Decision Making.
Environmental Modelling & Software, 105:201–216, 2018.
[ bib ]
Keywords: scenariobased

[389]

Jeffrey L Elman.
Distributed representations, simple recurrent networks, and
grammatical structure.
Machine Learning, 7(23):195–225, 1991.
[ bib ]

[390]

V. A. Emelichev and V. A. Perepelitsa.
Complexity of Vector Optimization Problems on Graphs.
Optimization, 22(6):906–918, 1991.
[ bib 
DOI ]

[391]

V. A. Emelichev and V. A. Perepelitsa.
On the Cardinality of the Set of Alternatives in Discrete
Manycriterion Problems.
Discrete Mathematics and Applications, 2(5):461–471, 1992.
[ bib ]

[392]

Michael T. M. Emmerich and André H. Deutz.
A tutorial on multiobjective optimization: Fundamentals and
evolutionary methods.
Natural Computing, 17(3):585–609, 2018.
[ bib ]

[393]

Michael T. M. Emmerich, K. C. Giannakoglou, and Boris Naujoks.
Single and multiobjective evolutionary optimization assisted by
Gaussian random field metamodels.
IEEE Transactions on Evolutionary Computation, 10(4):421–439,
2006.
[ bib 
DOI ]

[394]

Alexander Engau and Margaret M. Wiecek.
2D decisionmaking for multicriteria design optimization.
Structural and Multidisciplinary Optimization, 34:301–315,
2007.
[ bib 
DOI ]

[395]

Alexander Engau and Margaret M. Wiecek.
Interactive coordination of objective decompositions in
multiobjective programming.
Management Science, 54(7):1350–1363, 2008.
[ bib ]

[396]

Imen Essafi, Yazid Mati, and Stéphane DauzèrePèrés.
A Genetic Local Search Algorithm for Minimizing Total Weighted
Tardiness in the Jobshop Scheduling Problem.
Computers & Operations Research, 35(8):2599–2616, 2008.
[ bib ]

[397]

Wei Fan and Albert Bifet.
Mining big data: current status, and forecast to the future.
ACM sIGKDD Explorations Newsletter, 14(2):1–5, 2013.
[ bib ]

[398]

Daniele Fanelli.
Negative results are disappearing from most disciplines and
countries.
Scientometrics, 90(3):891–904, 2012.
[ bib 
DOI ]
Concerns that the growing competition for funding and
citations might distort science are frequently discussed, but
have not been verified directly. Of the hypothesized
problems, perhaps the most worrying is a worsening of
positiveoutcome bias. A system that disfavours negative
results not only distorts the scientific literature directly,
but might also discourage highrisk projects and pressure
scientists to fabricate and falsify their data. This study
analysed over 4,600 papers published in all disciplines
between 1990 and 2007, measuring the frequency of papers
that, having declared to have “tested” a hypothesis,
reported a positive support for it. The overall frequency of
positive supports has grown by over 22% between 1990 and
2007, with significant differences between disciplines and
countries. The increase was stronger in the social and some
biomedical disciplines. The United States had published, over
the years, significantly fewer positive results than Asian
countries (and particularly Japan) but more than European
countries (and in particular the United
Kingdom). Methodological artefacts cannot explain away these
patterns, which support the hypotheses that research is
becoming less pioneering and/or that the objectivity with
which results are produced and published is decreasing.

[399]

H. Faria, Jr, S. Binato, Mauricio G. C. Resende, and D. J. Falcão.
Power transmission network design by a greedy randomized
adaptive path relinking approach.
IEEE Transactions on Power Systems, 20(1):43–49, 2005.
[ bib ]

[400]

R. Farmani, Godfrey A. Walters, and Dragan A. Savic.
Evolutionary multiobjective optimization of the design and
operation of water distribution network: total cost vs. reliability vs. water
quality.
Journal of Hydroinformatics, 8(3):165–179, 2006.
[ bib ]

[401]

D. Favaretto, E. Moretti, and Paola Pellegrini.
Ant colony system approach for variants of the traveling
salesman problem with time windows.
Journal of Information and Optimization Sciences, 27(1):35–54,
2006.
[ bib ]

[402]

D. Favaretto, E. Moretti, and Paola Pellegrini.
Ant Colony System for a VRP with Multiple Time Windows and
Multiple Visits.
Journal of Interdisciplinary Mathematics, 10(2):263–284, 2007.
[ bib ]

[403]

Chris Fawcett and Holger H. Hoos.
Analysing Differences Between Algorithm Configurations through
Ablation.
Journal of Heuristics, 22(4):431–458, 2016.
[ bib ]

[404]

T. A. Feo and Mauricio G. C. Resende.
A Probabilistic Heuristic for a Computationally Difficult Set
Covering Problem.
Operations Research Letters, 8(2):67–71, 1989.
[ bib ]

[405]

T. A. Feo and Mauricio G. C. Resende.
Greedy Randomized Adaptive Search Procedures.
Journal of Global Optimization, 6(2):109–113, 1995.
[ bib ]

[406]

T. A. Feo, Mauricio G. C. Resende, and S. H. Smith.
A Greedy Randomized Adaptive Search Procedure for Maximum
Independent Set.
Operations Research, 42:860–878, October 1994.
[ bib ]

[407]

Victor FernandezViagas and Jose M. Framiñán.
On Insertion Tiebreaking Rules in Heuristics for the
Permutation Flowshop Scheduling Problem.
Computers & Operations Research, 45:60–67, 2014.
[ bib ]

[408]

Victor FernandezViagas and Jose M. Framiñán.
A Beamsearchbased Constructive Heuristic for the PFSP to
Minimise Total Flowtime.
Computers & Operations Research, 81:167–177, 2017.
[ bib ]

[409]

Victor FernandezViagas and Jose M. Framiñán.
Iteratedgreedybased algorithms with beam search initialization
for the permutation flowshop to minimise total tardiness.
Expert Systems with Applications, 94:58–69, 2018.
[ bib ]

[410]

Javier Ferrer, José GarcíaNieto, Enrique Alba, and Francisco Chicano.
Intelligent Testing of Traffic Light Programs: Validation in
Smart Mobility Scenarios.
Mathematical Problems in Engineering, 2016:1–19, 2016.
[ bib 
DOI ]

[411]

Alberto Ferrer, Daniel Guimarans, Helena Ramalhinho Lourenço, and
Angel A. Juan.
A BRILS Metaheuristic for Nonsmooth Flowshop Problems with
Failurerisk Costs.
Expert Systems with Applications, 44:177–186, 2016.
[ bib ]

[412]

Javier Ferrer, Manuel LópezIbáñez, and Enrique Alba.
Reliable SimulationOptimization of Traffic Lights in a
RealWorld City.
Applied Soft Computing, 78:697–711, 2019.
[ bib 
DOI 
supplementary material ]

[413]

Eduardo Fernandez, Jorge Navarro, and Sergio Bernal.
Multicriteria Sorting Using a Valued Indifference Relation Under
a Preference Disaggregation Paradigm.
European Journal of Operational Research, 198(2):602–609,
2009.
[ bib ]

[414]

Victor FernandezViagas, Rubén Ruiz, and Jose M. Framiñán.
A New Vision of Approximate Methods for the Permutation Flowshop
to Minimise Makespan: Stateoftheart and Computational Evaluation.
European Journal of Operational Research, 257(3):707–721,
2017.
[ bib ]

[415]

R. Ferreira da Silva and S. Urrutia.
A General VNS Heuristic for the Traveling Salesman Problem
with Time Windows.
Discrete Optimization, 7(4):203–211, 2010.
[ bib 
DOI ]
Keywords: TSPTW, GVNS

[416]

Victor FernandezViagas, Jorge M. S. Valente, and Jose M. Framiñán.
Iteratedgreedybased algorithms with Beam Search Initialization
for the Permutation Flowshop to Minimise Total Tardiness.
Expert Systems with Applications, 94:58–69, 2018.
[ bib ]

[417]

Álvaro Fialho, Luis Da Costa, Marc Schoenauer, and Michèle Sebag.
Analyzing Banditbased Adaptive Operator Selection Mechanisms.
Annals of Mathematics and Artificial Intelligence,
60(1–2):25–64, 2010.
[ bib ]

[418]

Mark J. Fielding.
Simulated Annealing with an Optimal Fixed Temperature.
SIAM Journal on Optimization, 11(2):289–307, 2000.
[ bib ]

[419]

Jonathan E. Fieldsend, Richard M. Everson, and Sameer Singh.
Using unconstrained elite archives for multiobjective
optimization.
IEEE Transactions on Evolutionary Computation, 7(3):305–323,
2003.
[ bib 
DOI ]

[420]

José Rui Figueira, Carlos M. Fonseca, Pascal Halffmann, Kathrin Klamroth,
Luís Paquete, Stefan Ruzika, Britta Schulze, Michael Stiglmayr, and David
Willems.
Easy to say they are Hard, but Hard to see they are EasyTowards
a Categorization of Tractable Multiobjective Combinatorial Optimization
Problems.
Journal of MultiCriteria Decision Analysis, 24(12):82–98,
2017.
[ bib 
DOI ]

[421]

Matteo Fischetti, Fred Glover, and Andrea Lodi.
The feasibility pump.
Mathematical Programming, 104(1):91–104, 2005.
[ bib ]

[422]

Matteo Fischetti and Andrea Lodi.
Local Branching.
Mathematical Programming Series B, 98:23–47, 2003.
[ bib ]

[423]

Matteo Fischetti and Michele Monaci.
Proximity search for 01 mixedinteger convex programming.
Journal of Heuristics, 20(6):709–731, 2014.
[ bib ]

[424]

Matteo Fischetti and Michele Monaci.
Exploiting Erraticism in Search.
Operations Research, 62(1):114–122, 2014.
[ bib 
DOI ]
High sensitivity to initial conditions is generally viewed
as a drawback of tree search methods because it leads to
erratic behavior to be mitigated somehow. In this paper we
investigate the opposite viewpoint and consider this behavior
as an opportunity to exploit. Our working hypothesis is that
erraticism is in fact just a consequence of the exponential
nature of tree search that acts as a chaotic amplifier, so it
is largely unavoidable. We propose a betandrun approach to
actually turn erraticism to one's advantage. The idea is to
make a number of short sample runs with randomized initial
conditions, to bet on the "most promising" run selected
according to certain simple criteria, and to bring it to
completion. Computational results on a large testbed of mixed
integer linear programs from the literature are presented,
showing the potential of this approach even when embedded in
a proofofconcept implementation.
http://mat.tepper.cmu.edu/blog/?p=1695

[425]

Matteo Fischetti, Michele Monaci, and Domenico Salvagnin.
Three Ideas for the Quadratic Assignment Problem.
Operations Research, 60(4):954–964, 2012.
[ bib ]

[426]

Matteo Fischetti and Domenico Salvagnin.
Feasibility pump 2.0.
Mathematical Programming Computation, 1(2–3):201–222, 2009.
[ bib ]

[427]

Charles Fleurent and Fred Glover.
Improved constructive multistart strategies for the quadratic
assignment problem using adaptive memory.
INFORMS Journal on Computing, 11(2):198–204, 1999.
[ bib ]

[428]

Jörg Fliege.
The effects of adding objectives to an optimisation problem on
the solution set.
Operations Research Letters, 35(6):782–790, 2007.
[ bib ]

[429]

Michael A. Fligner and Joseph S. Verducci.
Distance Based Ranking Models.
Journal of the Royal Statistical Society: Series B
(Methodological), 48(3):359–369, 1986.
[ bib 
DOI ]
Keywords: Mallows model, ranking, probabilistic models

[430]

M. M. Flood.
The Travelling Salesman Problem.
Operations Research, 4:61–75, 1956.
[ bib ]

[431]

D. Floreano and L. Keller.
Evolution of Adaptive Behaviour in Robots by Means of
Darwinian Selection.
PLoS Biology, 8(1):e1000292, 2010.
[ bib 
DOI ]

[432]

D. Floreano and J. Urzelai.
Evolutionary robots with online selforganization and
behavioral fitness.
Neural Networks, 13(45):431–443, 2000.
[ bib ]

[433]

Filippo Focacci, Andrea Lodi, and Michela Milano.
A Hybrid Exact Algorithm for the TSPTW.
INFORMS Journal on Computing, 14:403–417, 2002.
[ bib ]

[434]

Carlos M. Fonseca and Peter J. Fleming.
An overview of evolutionary algorithms in multiobjective
optimization.
Evolutionary Computation, 3(1):1–16, 1995.
[ bib ]
Proposed FON benchmark problem

[435]

Carlos M. Fonseca and Peter J. Fleming.
Multiobjective Optimization and Multiple Constraint Handling
with Evolutionary Algorithms (II): Application Example.
IEEE Transactions on Systems, Man, and Cybernetics – Part A,
28(1):38–44, January 1998.
[ bib 
DOI ]

[436]

Carlos M. Fonseca and Peter J. Fleming.
Multiobjective Optimization and Multiple Constraint Handling
with Evolutionary Algorithms (I): A Unified Formulation.
IEEE Transactions on Systems, Man, and Cybernetics – Part A,
28(1):26–37, January 1998.
[ bib 
DOI ]

[437]

Alexander I. J. Forrester and Andy J. Keane.
Recent advances in surrogatebased optimization.
Progress in Aerospace Sciences, 45(13):50–79, 2009.
[ bib 
DOI ]
Keywords: Kriging; Gaussian Process; EGO; Design of Experiments

[438]

John W. Fowler, Esma S. Gel, Murat Köksalan, Pekka Korhonen, Jon L.
Marquis, and Jyrki Wallenius.
Interactive evolutionary multiobjective optimization for
quasiconcave preference functions.
European Journal of Operational Research, 206(2):417–425,
2010.
[ bib 
DOI ]
We present a new hybrid approach to interactive evolutionary
multiobjective optimization that uses a partial preference
order to act as the fitness function in a customized genetic
algorithm. We periodically send solutions to the decision
maker (DM) for her evaluation and use the resulting
preference information to form preference cones consisting of
inferior solutions. The cones allow us to implicitly rank
solutions that the DM has not considered. This technique
avoids assuming an exact form for the preference function,
but does assume that the preference function is
quasiconcave. This paper describes the genetic algorithm and
demonstrates its performance on the multiobjective knapsack
problem.
Keywords: Interactive optimization, Multiobjective optimization,
Evolutionary optimization, Knapsack problem

[439]

Bennett L. Fox.
Integrating and accelerating tabu search, simulated annealing,
and genetic algorithms.
Annals of Operations Research, 41(2):47–67, 1993.
[ bib ]

[440]

Peter I. Frazier.
A Tutorial on Bayesian Optimization.
Arxiv preprint arXiv:1807.02811, 2018.
[ bib 
http ]

[441]

Alberto Franzin.
Empirical Analysis of Stochastic Local Search Behaviour:
Connecting Structure, Components and Landscape.
4OR: A Quarterly Journal of Operations Research, 2022.
[ bib 
DOI ]

[442]

G. Francesca, M. Brambilla, A. Brutschy, Vito Trianni, and Mauro Birattari.
AutoMoDe: A Novel Approach to the Automatic Design of Control
Software for Robot Swarms.
Swarm Intelligence, 8(2):89–112, 2014.
[ bib 
DOI ]

[443]

Gianpiero Francesca, Manuele Brambilla, Arne Brutschy, Lorenzo Garattoni, Roman
Miletitch, Gaetan Podevijn, Andreagiovanni Reina, Touraj Soleymani, Mattia
Salvaro, Carlo Pinciroli, Franco Mascia, Vito Trianni, and Mauro Birattari.
AutoMoDeChocolate: Automatic Design of Control Software for
Robot Swarms.
Swarm Intelligence, 2015.
[ bib 
DOI ]
Keywords: Swarm robotics; Automatic design; AutoMoDe

[444]

Jose M. Framiñán, Jatinder N.D. Gupta, and Rainer Leisten.
A Review and Classification of Heuristics for Permutation
Flowshop Scheduling with Makespan Objective.
Journal of the Operational Research Society, 55(12):1243–1255,
2004.
[ bib ]

[445]

Alberto Franzin, Leslie Pérez Cáceres, and Thomas Stützle.
Effect of Transformations of Numerical Parameters in Automatic
Algorithm Configuration.
Optimization Letters, 12(8):1741–1753, 2018.
[ bib 
DOI ]

[446]

Alberto Franzin, Francesco Sambo, and Barbara Di Camillo.
bnstruct: an R package for Bayesian Network
structure learning in the presence of missing data.
Bioinformatics, 33(8):1250–1252, 2016.
[ bib ]

[447]

Alberto Franzin and Thomas Stützle.
Revisiting Simulated Annealing: A ComponentBased Analysis.
Computers & Operations Research, 104:191–206, 2019.
[ bib 
DOI ]

[448]

Alberto Franzin and Thomas Stützle.
A Landscapebased Analysis of Fixed Temperature and Simulated
Annealing.
European Journal of Operational Research, In press, 2022.
[ bib 
DOI ]

[449]

Brendan J. Frey and Delbert Dueck.
Clustering by Passing Messages Between Data Points.
Science, 315(5814):972–976, February 2007.
[ bib 
DOI ]
Keywords: clustering; affinity propagation

[450]

Alan R. R. de Freitas, Peter J. Fleming, and Frederico G. Guimarães.
Aggregation trees for visualization and dimension reduction in
manyobjective optimization.
Information Sciences, 298:288–314, 2015.
[ bib ]

[451]

Hela Frikha, Habib Chabchoub, and JeanMarc Martel.
Inferring criteria's relative importance coefficients in
PROMETHEE II.
International Journal of Operational Research, 7(2):257–275,
2010.
[ bib ]

[452]

Matteo Frigo and Steven G. Johnson.
The Design and Implementation of FFTW3.
Proceedings of the IEEE, 93(2):216–231, 2005.
Special issue on “Program Generation, Optimization, and Platform
Adaptation”.
[ bib ]

[453]

Milton Friedman.
The use of ranks to avoid the assumption of normality implicit
in the analysis of variance.
Journal of the American Statistical Association,
32(200):675–701, 1937.
[ bib ]

[454]

Z Fu, R Eglese, and L Y O Li.
A unified tabu search algorithm for vehicle routing problems
with soft time windows.
Journal of the Operational Research Society, 59(5):663–673,
2008.
[ bib ]

[455]

Guenther Fuellerer, Karl F. Doerner, Richard F. Hartl, and Manuel Iori.
Metaheuristics for vehicle routing problems with
threedimensional loading constraints.
European Journal of Operational Research, 201(3):751–759,
2009.
[ bib 
DOI ]

[456]

Guenther Fuellerer, Karl F. Doerner, Richard F. Hartl, and Manuel Iori.
Ant colony optimization for the twodimensional loading vehicle
routing problem.
Computers & Operations Research, 36(3):655–673, 2009.
[ bib ]

[457]

Alex S. Fukunaga.
Automated Discovery of Local Search Heuristics for
Satisfiability Testing.
Evolutionary Computation, 16(1):31–61, March 2008.
[ bib 
DOI ]
The development of successful metaheuristic
algorithms such as local search for a difficult
problem such as satisfiability testing (SAT) is a
challenging task. We investigate an evolutionary
approach to automating the discovery of new local
search heuristics for SAT. We show that several
wellknown SAT local search algorithms such as
Walksat and Novelty are composite heuristics that
are derived from novel combinations of a set of
building blocks. Based on this observation, we
developed CLASS, a genetic programming system that
uses a simple composition operator to automatically
discover SAT local search heuristics. New
heuristics discovered by CLASS are shown to be
competitive with the best Walksat variants,
including Novelty+. Evolutionary algorithms have
previously been applied to directly evolve a
solution for a particular SAT instance. We show
that the heuristics discovered by CLASS are also
competitive with these previous, direct evolutionary
approaches for SAT. We also analyze the local
search behavior of the learned heuristics using the
depth, mobility, and coverage metrics proposed by
Schuurmans and Southey.

[458]

Grigori Fursin, Yuriy Kashnikov, Abdul Wahid Memon, Zbigniew Chamski, Olivier
Temam, Mircea Namolaru, Elad YomTov, Bilha Mendelson, Ayal Zaks, Eric
Courtois, Francois Bodin, Phil Barnard, Elton Ashton, Edwin Bonilla, John
Thomson, Christopher K. I. Williams, and Michael O'Boyle.
Milepost GCC: Machine Learning Enabled Selftuning Compiler.
International Journal of Parallel Programming, 39(3):296–327,
2011.
[ bib 
DOI ]

[459]

Caroline Gagné, W. L. Price, and M. Gravel.
Comparing an ACO algorithm with other heuristics for the
single machine scheduling problem with sequencedependent setup times.
Journal of the Operational Research Society, 53:895–906, 2002.
[ bib ]

[460]

Matteo Gagliolo and J. Schmidhuber.
Learning dynamic algorithm portfolios.
Annals of Mathematics and Artificial Intelligence,
47(34):295–328, 2007.
[ bib 
DOI ]
fully dynamic and online algorithm selection technique, with
no separate training phase: all candidate algorithms are run
in parallel, while a model incrementally learns their runtime
distributions.

[461]

Philippe Galinier and JinKao Hao.
Hybrid evolutionary algorithms for graph coloring.
Journal of Combinatorial Optimization, 3(4):379–397, 1999.
[ bib 
DOI ]

[462]

Tomas Gal and Heiner Leberling.
Redundant objective functions in linear vector maximum problems
and their determination.
European Journal of Operational Research, 1(3):176–184, 1977.
[ bib 
DOI ]
Suppose that in a multicriteria linear programming problem
among the given objective functions there are some which can
be deleted without influencing the set E of all efficient
solutions. Such objectives are said to be
redundant. Introducing systems of objective functions which
realize their individual optimum in a single vertex of the
polyhedron generated by the restriction set, the notion of
relative or absolute redundant objectives is defined. A
theory which describes properties of absolute and relative
redundant objectives is developed. A method for determining
all the relative and absolute redundant objectives, based on
this theory, is given. Illustrative examples demonstrate the
procedure.

[463]

L. M. Gambardella and Marco Dorigo.
Ant Colony System Hybridized with a New Local Search for the
Sequential Ordering Problem.
INFORMS Journal on Computing, 12(3):237–255, 2000.
[ bib ]

[464]

L. M. Gambardella, Roberto Montemanni, and Dennis Weyland.
Coupling Ant Colony Systems with Strong Local Searches.
European Journal of Operational Research, 220(3):831–843,
2012.
[ bib 
DOI ]

[465]

Xavier Gandibleux, Andrzej Jaszkiewicz, A. Fréville, and Roman
Slowiński.
Special Issue on Multiple Objective Metaheuristics.
Journal of Heuristics, 6(3), 2000.
[ bib ]

[466]

Kaizhou Gao, Yicheng Zhang, Ali Sadollah, and Rong Su.
Optimizing urban traffic light scheduling problem using harmony
search with ensemble of local search.
Applied Soft Computing, 48:359–372, November 2016.
[ bib 
DOI ]
Keywords: harmony search algorithm,traffic light scheduling

[467]

Huiru Gao, Haifeng Nie, and Ke Li.
Visualisation of Pareto Front Approximation: A Short Survey
and Empirical Comparisons.
Arxiv preprint arXiv:1903.01768, 2019.
[ bib 
http ]

[468]

José GarcíaNieto, Enrique Alba, and Ana Carolina Olivera.
Swarm intelligence for traffic light scheduling: Application to
real urban areas.
Engineering Applications of Artificial Intelligence,
25(2):274–283, March 2012.
[ bib ]
Keywords: Cycle program optimization,Particle swarm
optimization,Realistic traffic instances,SUMO microscopic
simulator of urban mobility,Traffic light scheduling

[469]

Carlos GarcíaMartínez, Oscar Cordón, and Francisco Herrera.
A taxonomy and an empirical analysis of multiple objective ant
colony optimization algorithms for the bicriteria TSP.
European Journal of Operational Research, 180(1):116–148,
2007.
[ bib ]

[470]

Javier García and Fernando Fernández.
A comprehensive survey on safe reinforcement learning.
Journal of Machine Learning Research, 16(1):1437–1480, 2015.
[ bib 
epub ]

[471]

Salvador García, Alberto Fernández, Julián Luengo, and Francisco
Herrera.
Advanced nonparametric tests for multiple comparisons in the
design of experiments in computational intelligence and data mining:
Experimental analysis of power.
Information Sciences, 180(10):2044–2064, 2010.
[ bib ]

[472]

Carlos GarcíaMartínez, Fred Glover, Francisco J. Rodríguez, Manuel
Lozano, and Rafael Martí.
Strategic Oscillation for the Quadratic Multiple Knapsack
Problem.
Computational Optimization and Applications, 58(1):161–185,
2014.
[ bib ]

[473]

M. R. Garey, David S. Johnson, and R. Sethi.
The Complexity of Flowshop and Jobshop Scheduling.
Mathematics of Operations Research, 1:117–129, 1976.
[ bib ]

[474]

Josselin Garnier and Leila Kallel.
Efficiency of Local Search with Multiple Local Optima.
SIAM Journal Discrete Mathematics, 15(1):122–141, 2001.
[ bib 
DOI ]

[475]

Salvador García, Daniel Molina, Manuel Lozano, and Francisco Herrera.
A study on the use of nonparametric tests for analyzing the
evolutionary algorithms' behaviour: a case study on the CEC'2005 Special
Session on Real Parameter Optimization.
Journal of Heuristics, 15(617):617–644, 2009.
[ bib 
DOI ]

[476]

José GarcíaNieto, Ana Carolina Olivera, and Enrique Alba.
Optimal Cycle Program of Traffic Lights With Particle Swarm
Optimization.
IEEE Transactions on Evolutionary Computation, 17(6):823–839,
December 2013.
[ bib 
DOI ]

[477]

Carlos GarcíaMartínez, Francisco J. Rodríguez, and Manuel Lozano.
Arbitrary function optimisation with metaheuristics: No free
lunch and realworld problems.
Soft Computing, 16(12):2115–2133, 2012.
[ bib 
DOI ]

[478]

Carlos GarcíaMartínez, Francisco J. Rodríguez, and Manuel Lozano.
Tabuenhanced Iterated Greedy Algorithm: A Case Study in the
Quadratic Multiple Knapsack Problem.
European Journal of Operational Research, 232(3):454–463,
2014.
[ bib ]

[479]

Gauci Melvin, Tony J. Dodd, and Roderich Groß.
Why `GSA: a gravitational search algorithm' is not genuinely
based on the law of gravity.
Natural Computing, 11(4):719–720, 2012.
[ bib ]

[480]

Martin Josef Geiger.
Decision Support for Multiobjective Flow Shop Scheduling by the
Pareto Iterated Local Search Methodology.
Computers and Industrial Engineering, 61(3):805–812, 2011.
[ bib ]

[481]

Martin Josef Geiger.
A Multithreaded Local Search Algorithm and Computer
Implementation for the Multimode, Resourceconstrained Multiproject
Scheduling Problem.
European Journal of Operational Research, 256:729–741, 2017.
[ bib ]

[482]

Stuart Geman and Donald Geman.
Stochastic Relaxation, Gibbs Distributions, and the Bayesian
Restoration of Images.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
6(6):721–741, 1984.
[ bib ]

[483]

Michel Gendreau, Francois Guertin, JeanYves Potvin, and Éric D. Taillard.
Parallel tabu search for realtime vehicle routing and
dispatching.
Transportation Science, 33(4):381–390, 1999.
[ bib ]

[484]

Michel Gendreau, Francois Guertin, JeanYves Potvin, and René Séguin.
Neighborhood search heuristics for a dynamic vehicle dispatching
problem with pickups and deliveries.
Transportation Research Part C: Emerging Technologies,
14(3):157–174, 2006.
[ bib ]

[485]

Mitsuo Gen and Lin Lin.
Multiobjective evolutionary algorithm for manufacturing
scheduling problems: stateoftheart survey.
Journal of Intelligent Manufacturing, 25(5):849–866, 2014.
[ bib ]

[486]

Robin Genuer, JeanMichel Poggi, and Christine TuleauMalot.
Variable selection using random forests.
Pattern Recognition Letters, 31(14):2225–2236, 2010.
[ bib ]

[487]

Michel Gendreau, A. Hertz, Gilbert Laporte, and M. Stan.
A Generalized Insertion Heuristic for the Traveling Salesman
Problem with Time Windows.
Operations Research, 46:330–335, 1998.
[ bib ]

[488]

Samuel J. Gershman, Eric J. Horvitz, and Joshua B. Tenenbaum.
Computational rationality: A converging paradigm for
intelligence in brains, minds, and machines.
Science, 349(6245):273–278, 2015.
[ bib 
DOI 
epub ]
After growing up together, and mostly growing apart in the
second half of the 20th century, the fields of artificial
intelligence (AI), cognitive science, and neuroscience are
reconverging on a shared view of the computational
foundations of intelligence that promotes valuable
crossdisciplinary exchanges on questions, methods, and
results. We chart advances over the past several decades that
address challenges of perception and action under uncertainty
through the lens of computation. Advances include the
development of representations and inferential procedures for
largescale probabilistic inference and machinery for
enabling reflection and decisions about tradeoffs in effort,
precision, and timeliness of computations. These tools are
deployed toward the goal of computational rationality:
identifying decisions with highest expected utility, while
taking into consideration the costs of computation in complex
realworld problems in which most relevant calculations can
only be approximated. We highlight key concepts with examples
that show the potential for interchange between computer
science, cognitive science, and neuroscience.

[]

Pierre Geurts, Damien Ernst, and Louis Wehenkel.
Extremely randomized trees.
Machine Learning, 63(1):3–42, March 2006.
[ bib 
DOI ]
Proposed ExtraTrees

[490]

Sanjay Ghemawat, Howard Gobioff, and ShunTak Leung.
The Google File System.
SIGOPS Oper. Syst. Rev., 37(5):29–43, December 2003.
[ bib ]

[491]

K. Ghoseiri and B. Nadjari.
An ant colony optimization algorithm for the biobjective
shortest path problem.
Applied Soft Computing, 10(4):1237–1246, 2010.
[ bib ]

[492]

Nicolas Girerd, Muriel Rabilloud, Philippe Pibarot, Patrick Mathieu, and Pascal
Roy.
Quantification of Treatment Effect Modification on Both an
Additive and Multiplicative Scale.
PLoS One, 11(4):1–14, April 2016.
[ bib 
DOI ]

[493]

Fred Glover.
Heuristics for Integer Programming Using Surrogate Constraints.
Decision Sciences, 8:156–166, 1977.
[ bib ]

[494]

Fred Glover.
Future Paths for Integer Programming and Links to Artificial
Intelligence.
Computers & Operations Research, 13(5):533–549, 1986.
[ bib ]

[495]

Fred Glover.
Tabu Search – Part I.
INFORMS Journal on Computing, 1(3):190–206, 1989.
[ bib 
DOI ]

[496]

Fred Glover.
Tabu Search – Part II.
INFORMS Journal on Computing, 2(1):4–32, 1990.
[ bib ]

[497]

Fred Glover, Gary A. Kochenberger, and Bahram Alidaee.
Adaptive Memory Tabu Search for Binary Quadratic Programs.
Management Science, 44(3):336–345, 1998.
[ bib ]

[498]

Fred Glover, Zhipeng Lü, and JinKao Hao.
Diversificationdriven tabu search for unconstrained binary
quadratic problems.
4OR: A Quarterly Journal of Operations Research,
8(3):239–253, 2010.
[ bib 
DOI ]

[499]

Marc Goerigk and Anita Schöbel.
Recoverytooptimality: A new twostage approach to robustness
with an application to aperiodic timetabling.
Computers & Operations Research, 52:1–15, 2014.
[ bib ]

[500]

David E. Goldberg.
Probability matching, the magnitude of reinforcement, and
classifier system bidding.
Machine Learning, 5(4):407–425, 1990.
[ bib ]

[501]

Zaiwu Gong, Ning Zhang, and Francisco Chiclana.
The optimization ordering model for intuitionistic fuzzy
preference relations with utility functions.
KnowledgeBased Systems, 162:174–184, 2018.
[ bib 
DOI ]
Intuitionistic fuzzy sets describe information from the three
aspects of membership degree, nonmembership degree and
hesitation degree, which has more practical significance when
uncertainty pervades qualitative decision problems. In this
paper, we investigate the problem of ranking intuitionistic
fuzzy preference relations (IFPRs) based on various
nonlinear utility functions. First, we transform IFPRs into
their isomorphic intervalvalue fuzzy preference relations
(IVFPRs), and utilise nonlinear utility functions, such as
parabolic, Sshaped, and hyperbolic absolute risk aversion,
to fit the true value of a decisionmakerâ€™s
judgement. Ultimately, the optimization ordering models for
the membership and nonmembership of IVFPRs based on utility
function are constructed, with objective function aiming at
minimizing the distance deviation between the multiplicative
consistency ideal judgment and the actual judgment,
represented by utility function, subject to the
decisionmakerâ€™s utility constraints. The proposed models
ensure that more factual and optimal ranking of alternative
is acquired, avoiding information distortion caused by the
operations of intervals. Second, by introducing a
nonArchimedean infinitesimal, we establish the optimization
ordering model for IFPRs with the priority of utility or
deviation, which realises the goal of prioritising solutions
under multiobjective programming. Subsequently, we verify
that a close connection exists between the ranking for
membership and nonmembership degree IVFPRs. Comparison
analyses with existing approaches are summarized to
demonstrate that the proposed models have advantage in
dealing with group decision making problems with IFPRs.
Special Issue on intelligent decisionmaking and consensus
under uncertainty in inconsistent and dynamic environments
Keywords: Intuitionistic fuzzy preference relation, Utility function,
Ranking, Multiplicative consistency, Nonarchimedean
infinitesimal

[502]

Jochen Gorski, Kathrin Klamroth, and Stefan Ruzika.
Connectedness of Efficient Solutions in Multiple Objective
Combinatorial Optimization.
Journal of Optimization Theory and Applications,
150(3):475–497, 2011.
[ bib 
DOI ]

[503]

Abhijit Gosavi.
Reinforcement Learning: A Tutorial Survey and Recent Advances.
INFORMS Journal on Computing, 21(2):178–192, 2009.
[ bib 
DOI ]

[504]

N. I. M. Gould, D. Orban, and P. L. Toint.
CUTEr and SifDec: A constrained and unconstrained testing
environment, revisited.
ACM Transactions on Mathematical Software, 29:373–394, 2003.
[ bib ]

[505]

Robert B. Gramacy and Herbert K. H. Lee.
Bayesian Treed Gaussian Process Models With an Application to
Computer Modeling.
Journal of the American Statistical Association,
103:1119–1130, 2008.
[ bib 
DOI ]
Keywords: TreedGP

[506]

Alex Grasas, Angel A. Juan, and Helena Ramalhinho Lourenço.
SimILS: A Simulationbased Extension of the Iterated Local
Search Metaheuristic for Stochastic Combinatorial Optimization.
Journal of Simulation, 10(1):69–77, 2016.
[ bib ]

[507]

M. Gravel, W. L. Price, and Caroline Gagné.
Scheduling continuous casting of aluminum using a multiple
objective ant colony optimization metaheuristic.
European Journal of Operational Research, 143(1):218–229,
2002.
[ bib 
DOI ]

[508]

J. J. Grefenstette.
Optimization of Control Parameters for Genetic Algorithms.
IEEE Transactions on Systems, Man, and Cybernetics,
16(1):122–128, 1986.
[ bib ]
Keywords: parameter tuning

[509]

Salvatore Greco, Milosz Kadziński, Vincent Mousseau, and Roman
Slowiński.
ELECTRE^{}GKMS: Robust ordinal regression for
outranking methods.
European Journal of Operational Research, 214(1):118–135,
2011.
[ bib ]

[510]

Salvatore Greco, Vincent Mousseau, and Roman Slowiński.
Robust ordinal regression for value functions handling
interacting criteria.
European Journal of Operational Research, 239(3):711–730,
2014.
[ bib 
DOI ]

[511]

David R. Grimes, Chris T. Bauch, and John P. A. Ioannidis.
Modelling science trustworthiness under publish or perish
pressure.
Royal Society Open Science, 5:171511, 2018.
[ bib ]

[512]

Andrea Grosso, Federico Della Croce, and R. Tadei.
An Enhanced Dynasearch Neighborhood for the SingleMachine Total
Weighted Tardiness Scheduling Problem.
Operations Research Letters, 32(1):68–72, 2004.
[ bib ]

[513]

Andrea Grosso, A. R. M. J. U. Jamali, and Marco Locatelli.
Finding Maximin Latin Hypercube Designs by Iterated Local Search
Heuristics.
European Journal of Operational Research, 197(2):541–547,
2009.
[ bib ]

[514]

Peter Groves, Basel Kayyali, David Knott, and Steve Van Kuiken.
The "big data" revolution in healthcare.
McKinsey Quarterly, 2, 2013.
[ bib ]

[515]

Benoît Groz and Silviu Maniu.
Hypervolume subset selection with small subsets.
Evolutionary Computation, 27(4):611–637, 2019.
[ bib ]

[516]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
A link between the multivariate cumulative distribution function
and the hitting function for random closed sets.
Statistics & Probability Letters, 57(2):179–182, 2002.
[ bib 
DOI ]

[517]

Andreia P. Guerreiro, Carlos M. Fonseca, and Luís Paquete.
The Hypervolume Indicator: Computational Problems and
Algorithms.
ACM Computing Surveys, 54(6):1–42, 2021.
[ bib ]

[518]

Andreia P. Guerreiro, Vasco Manquinho, and José Rui Figueira.
Exact hypervolume subset selection through incremental
computations.
Computers & Operations Research, 136:105–471, December 2021.
[ bib 
DOI ]

[519]

Gonzalo GuillénGosálbez.
A novel MILPbased objective reduction method for
multiobjective optimization: Application to environmental problems.
Computers & Chemical Engineering, 35(8):1469–1477, 2011.
[ bib 
DOI ]
Multiobjective optimization has recently emerged as a useful
technique in sustainability analysis, as it can assist in the
study of optimal tradeoff solutions that balance several
criteria. The main limitation of multiobjective optimization
is that its computational burden grows in size with the
number of objectives. This computational barrier is critical
in environmental applications in which decisionmakers seek
to minimize simultaneously several environmental indicators
of concern. With the aim to overcome this limitation, this
paper introduces a systematic method for reducing the number
of objectives in multiobjective optimization with emphasis
on environmental problems. The approach presented relies on a
novel mixedinteger linear programming formulation that
minimizes the error of omitting objectives. We test the
capabilities of this technique through two environmental
problems of different nature in which we attempt to minimize
a set of life cycle assessment impacts. Numerical examples
demonstrate that certain environmental metrics tend to behave
in a nonconflicting manner, which makes it possible to
reduce the dimension of the problem without losing
information.
Keywords: Environmental engineering, Life cycle assessment,
Multiobjective optimization, Objective reduction

[520]

Odd Erik Gundersen, Yolanda Gil, and David W. Aha.
On Reproducible AI: Towards Reproducible Research, Open
Science, and Digital Scholarship in AI Publications.
AI Magazine, 39(3):56–68, September 2018.
[ bib 
DOI ]
The reproducibility guidelines can be found here:
https://folk.idi.ntnu.no/odderik/reproducibility_guidelines.pdf
and a short howto can be found here:
https://folk.idi.ntnu.no/odderik/reproducibility_guidelines_how_to.html

[521]

Aldy Gunawan, Kien Ming Ng, and Kim Leng Poh.
A Hybridized Lagrangian Relaxation and Simulated Annealing
Method for the Course Timetabling Problem.
Computers & Operations Research, 39(12):3074–3088, 2012.
[ bib ]

[522]

J. N. D. Gupta.
Flowshop schedules with sequence dependent setup times.
Journal of Operations Research Society of Japan, 29:206–219,
1986.
[ bib ]

[523]

Walter J. Gutjahr.
A Graphbased Ant System and its Convergence.
Future Generation Computer Systems, 16(8):873–888, 2000.
[ bib ]

[524]

Walter J. Gutjahr.
ACO Algorithms with Guaranteed Convergence to the Optimal
Solution.
Information Processing Letters, 82(3):145–153, 2002.
[ bib ]

[525]

Walter J. Gutjahr.
On the finitetime dynamics of ant colony optimization.
Methodology and Computing in Applied Probability,
8(1):105–133, 2006.
[ bib ]

[526]

Walter J. Gutjahr.
Mathematical runtime analysis of ACO algorithms: survey on an
emerging issue.
Swarm Intelligence, 1(1):59–79, 2007.
[ bib ]

[527]

Walter J. Gutjahr and Marion S. Rauner.
An ACO algorithm for a dynamic regional nursescheduling
problem in Austria.
Computers & Operations Research, 34(3):642–666, 2007.
[ bib 
DOI ]
To the best of our knowledge, this paper describes the first
ant colony optimization (ACO) approach applied to nurse
scheduling, analyzing a dynamic regional problem which is
currently under discussion at the Vienna hospital
compound. Each day, pool nurses have to be assigned for the
following days to public hospitals while taking into account
a variety of soft and hard constraints regarding working date
and time, working patterns, nurses qualifications, nurses
and hospitals preferences, as well as costs. Extensive
computational experiments based on a four week simulation
period were used to evaluate three different scenarios
varying the number of nurses and hospitals for six different
hospitals demand intensities. The results of our simulations
and optimizations reveal that the proposed ACO algorithm
achieves highly significant improvements compared to a greedy
assignment algorithm.

[528]

Walter J. Gutjahr.
First steps to the runtime complexity analysis of ant colony
optimization.
Computers & Operations Research, 35(9):2711–2727, 2008.
[ bib ]

[529]

Walter J. Gutjahr and G. Sebastiani.
Runtime analysis of ant colony optimization with bestsofar
reinforcement.
Methodology and Computing in Applied Probability,
10(3):409–433, 2008.
[ bib ]

[530]

Gregory Gutin, Anders Yeo, and Alexey Zverovich.
Traveling salesman should not be greedy: domination analysis of
greedytype heuristics for the TSP.
Discrete Applied Mathematics, 117(1–3), 2002.
[ bib ]

[531]

Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik.
Gene selection for cancer classification using support vector
machines.
Machine Learning, 46(1):389–422, 2002.
[ bib ]
Keywords: recursive feature elimination

[532]

Heikki Haario, Eero Saksman, and Johanna Tamminen.
An adaptive Metropolis algorithm.
Bernoulli, 7(2):223–242, 2001.
[ bib ]

[533]

David Hadka and Patrick M. Reed.
Borg: An AutoAdaptive ManyObjective Evolutionary Computing
Framework.
Evolutionary Computation, 21(2):231–259, 2013.
[ bib ]

[534]

David Hadka and Patrick M. Reed.
Diagnostic Assessment of Search Controls and Failure Modes in
ManyObjective Evolutionary Optimization.
Evolutionary Computation, 20(3):423–452, 2012.
[ bib ]

[535]

Josef Hadar and William R. Russell.
Rules for ordering uncertain prospects.
The American Economic Review, 59(1):25–34, 1969.
[ bib 
epub ]
Keywords: stochastic dominance

[536]

Y. Haimes, L. Lasdon, and D. Da Wismer.
On a bicriterion formation of the problems of integrated system
identification and system optimization.
IEEE Transactions on Systems, Man, and Cybernetics,
1(3):296–297, 1971.
[ bib 
DOI ]
Keywords: epsilonconstraint method

[537]

Prabhat Hajela and CY Lin.
Genetic search strategies in multicriterion optimal design.
Structural Optimization, 4(2):99–107, 1992.
[ bib ]

[538]

Bruce Hajek and Galen Sasaki.
Simulated annealing–to cool or not.
System & Control Letters, 12(5):443–447, 1989.
[ bib ]

[539]

Bruce Hajek.
Cooling Schedules for Optimal Annealing.
Mathematics of Operations Research, 13(2):311–329, 1988.
[ bib ]

[540]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
On the impact of the performance metric on efficient algorithm
configuration.
Artificial Intelligence, 303:103629, February 2022.
[ bib 
DOI ]
Keywords: irace

[541]

Raimo P. Hämäläinen and Tuomas J. Lahtinen.
Path dependence in Operational Research–How the modeling
process can influence the results.
Operations Research Perspectives, 3:14–20, January 2016.
[ bib 
DOI ]
In Operational Research practice there are almost always
alternative paths that can be followed in the modeling and
problem solving process. Path dependence refers to the impact
of the path on the outcome of the process. The steps of the
path include, e.g. forming the problem solving team, the
framing and structuring of the problem, the choice of model,
the order in which the different parts of the model are
specified and solved, and the way in which data or
preferences are collected. We identify and discuss seven
possibly interacting origins or drivers of path dependence:
systemic origins, learning, procedure, behavior, motivation,
uncertainty, and external environment. We provide several
ideas on how to cope with path dependence.
Keywords: Behavioral Biases, Behavioral Operational Research, Ethics in
modelling, OR practice, Path dependence, Systems perspective

[542]

Raimo P. Hämäläinen, Jukka Luoma, and Esa Saarinen.
On the importance of behavioral operational research: The case
of understanding and communicating about dynamic systems.
European Journal of Operational Research, 228(3):623–634,
August 2013.
[ bib 
DOI ]
We point out the need for Behavioral Operational Research
(BOR) in advancing the practice of OR. So far, in OR
behavioral phenomena have been acknowledged only in
behavioral decision theory but behavioral issues are always
present when supporting human problem solving by
modeling. Behavioral effects can relate to the group
interaction and communication when facilitating with OR
models as well as to the possibility of procedural mistakes
and cognitive biases. As an illustrative example we use well
known system dynamics studies related to the understanding of
accumulation. We show that one gets completely opposite
results depending on the way the phenomenon is described and
how the questions are phrased and graphs used. The results
suggest that OR processes are highly sensitive to various
behavioral effects. As a result, we need to pay attention to
the way we communicate about models as they are being
increasingly used in addressing important problems like
climate change.

[543]

Horst W. Hamacher and Günter Ruhe.
On spanning tree problems with multiple objectives.
Annals of Operations Research, 52(4):209–230, 1994.
[ bib ]

[544]

Nikolaus Hansen, Anne Auger, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff.
COCO: A platform for comparing continuous optimizers in a
blackbox setting.
Arxiv preprint arXiv:1603.08785, 2016.
Published as [545].
[ bib ]

[545]

Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and
Dimo Brockhoff.
COCO: A platform for comparing continuous optimizers in a
blackbox setting.
Optimization Methods and Software, 36(1):1–31, 2020.
[ bib 
DOI ]

[546]

Pierre Hansen and B. Jaumard.
Algorithms for the Maximum Satisfiability Problem.
Computing, 44:279–303, 1990.
[ bib ]

[547]

Pierre Hansen and Nenad Mladenović.
Variable neighborhood search: Principles and applications.
European Journal of Operational Research, 130(3):449–467,
2001.
[ bib ]

[548]

Nikolaus Hansen and A. Ostermeier.
Completely derandomized selfadaptation in evolution
strategies.
Evolutionary Computation, 9(2):159–195, 2001.
[ bib 
DOI ]
Keywords: CMAES

[549]

Nikolaus Hansen, Raymond Ros, Nikolaus Mauny, Marc Schoenauer, and Anne Auger.
Impacts of invariance in search: When CMAES and PSO face
illconditioned and nonseparable problems.
Applied Soft Computing, 11(8):5755–5769, 2011.
[ bib ]

[550]

Thomas Hanne.
On the convergence of multiobjective evolutionary algorithms.
European Journal of Operational Research, 117(3):553–564,
1999.
[ bib 
DOI ]
Keywords: archiving, efficiency presserving

[551]

Thomas Hanne.
A multiobjective evolutionary algorithm for approximating the
efficient set.
European Journal of Operational Research, 176(3):1723–1734,
2007.
[ bib ]

[552]

Douglas P. Hardin and Edward B. Saff.
Discretizing Manifolds via Minimum Energy Points.
Notices of the American Mathematical Society,
51(10):1186–1194, 2004.
[ bib ]

[553]

J. P. Hart and A. W. Shogan.
Semigreedy heuristics: An empirical study.
Operations Research Letters, 6(3):107–114, 1987.
[ bib ]

[554]

Emma Hart and Kevin Sim.
A HyperHeuristic Ensemble Method for Static JobShop
Scheduling.
Evolutionary Computation, 24(4):609–635, 2016.
[ bib 
DOI ]

[555]

Kazuya Haraguchi.
Iterated Local Search with TrellisNeighborhood for the
Partial Latin Square Extension Problem.
Journal of Heuristics, 22(5):727–757, 2016.
[ bib ]

[556]

Sameer Hasija and Chandrasekharan Rajendran.
Scheduling in flowshops to minimize total tardiness of jobs.
International Journal of Production Research,
42(11):2289–2301, 2004.
[ bib 
DOI ]

[557]

Hideki Hashimoto, Mutsunori Yagiura, and Toshihide Ibaraki.
An Iterated Local Search Algorithm for the Timedependent
Vehicle Routing Problem with Time Windows.
Discrete Optimization, 5(2):434–456, 2008.
[ bib ]

[558]

Simon Haykin.
A comprehensive foundation.
Neural Networks, 2:41, 2004.
[ bib ]

[559]

Öncü Hazir, Yavuz Günalay, and Erdal Erel.
Customer order scheduling problem: a comparative metaheuristics
study.
International Journal of Advanced Manufacturing Technology,
37(5):589–598, May 2008.
[ bib 
DOI ]
The customer order scheduling problem (COSP) is defined as
to determine the sequence of tasks to satisfy the demand of
customers who order several types of products produced on a
single machine. A setup is required whenever a product type
is launched. The objective of the scheduling problem is to
minimize the average customer order flow time. Since the
customer order scheduling problem is known to be strongly
NPhard, we solve it using four major metaheuristics and
compare the performance of these heuristics, namely,
simulated annealing, genetic algorithms, tabu search, and ant
colony optimization. These are selected to represent various
characteristics of metaheuristics: natureinspired
vs. artificially created, populationbased vs. local search,
etc. A set of problems is generated to compare the solution
quality and computational efforts of these heuristics.
Results of the experimentation show that tabu search and ant
colony perform better for large problems whereas simulated
annealing performs best in smallsize problems. Some
conclusions are also drawn on the interactions between
various problem parameters and the performance of the
heuristics.
Keywords: ACO,Customer order scheduling,Genetic
algorithms,Metaheuristics,Simulated annealing,Tabu
search

[560]

Zhenan He and Gary G. Yen.
ManyObjective Evolutionary Algorithm: Objective Space Reduction
and Diversity Improvement.
IEEE Transactions on Evolutionary Computation, 20(1):145–160,
2016.
[ bib ]

[561]

Xin He, Kaiyong Zhao, and Xiaowen Chu.
AutoML: A survey of the stateoftheart.
KnowledgeBased Systems, 212:106622, 2021.
[ bib 
DOI ]

[562]

Sabine Helwig, Jürgen Branke, and Sanaz Mostaghim.
Experimental Analysis of Bound Handling Techniques in Particle
Swarm Optimization.
IEEE Transactions on Evolutionary Computation, 17(2):259–271,
April 2013.
[ bib 
DOI ]

[563]

Michael Held and Richard M. Karp.
The TravelingSalesman Problem and Minimum Spanning Trees.
Operations Research, 18(6):1138–1162, 1970.
[ bib ]

[564]

Christoph Helmberg and Franz Rendl.
Solving quadratic (0,1)problems by semidefinite programs and
cutting planes.
Mathematical Programming, 82(3):291–315, 1998.
[ bib ]

[565]

Keld Helsgaun.
An Effective Implementation of the LinKernighan Traveling
Salesman Heuristic.
European Journal of Operational Research, 126:106–130, 2000.
[ bib ]

[566]

Keld Helsgaun.
General kopt Submoves for the LinKernighan TSP
Heuristic.
Mathematical Programming Computation, 1(2–3):119–163, 2009.
[ bib ]

[567]

Michael A. Heroux.
Editorial: ACM TOMS Replicated Computational Results
Initiative.
ACM Transactions on Mathematical Software, 41(3):1–5, June
2015.
[ bib 
DOI ]

[568]

H. Hernández and Christian Blum.
Ant colony optimization for multicasting in static wireless
adhoc networks.
Swarm Intelligence, 3(2):125–148, 2009.
[ bib ]

[569]

Francisco Herrera, Manuel Lozano, and A. M. Sánchez.
A taxonomy for the crossover operator for realcoded genetic
algorithms: An experimental study.
International Journal of Intelligent Systems, 18(3):309–338,
2003.
[ bib 
DOI ]

[570]

Francisco Herrera, Manuel Lozano, and J. L. Verdegay.
Tackling RealCoded Genetic Algorithms: Operators and Tools for
Behavioural Analysis.
Artificial Intelligence Review, 12:265–319, 1998.
[ bib ]
Keywords: genetic algorithms, real coding, continuous search
spaces, mutation, recombination

[571]

Carlos Ignacio Hernández Castellanos and Oliver Schütze.
A Bounded Archiver for Hausdorff Approximations of the
Pareto Front for MultiObjective Evolutionary Algorithms.
Mathematical and Computational Applications, 27(3):48, 2022.
[ bib 
DOI ]

[572]

Jano I. van Hemert.
Evolving Combinatorial Problem Instances That Are Difficult to
Solve.
Evolutionary Computation, 14(4):433–462, 2006.
[ bib 
DOI ]
This paper demonstrates how evolutionary computation can be
used to acquire difficult to solve combinatorial problem
instances. As a result of this technique, the corresponding
algorithms used to solve these instances are
stresstested. The technique is applied in three important
domains of combinatorial optimisation, binary constraint
satisfaction, Boolean satisfiability, and the travelling
salesman problem. The problem instances acquired through this
technique are more difficult than the ones found in popular
benchmarks. In this paper, these evolved instances are
analysed with the aim to explain their difficulty in terms of
structural properties, thereby exposing the weaknesses of
corresponding algorithms.

[573]

Robert Heumüller, Sebastian Nielebock, Jacob Krüger, and Frank
Ortmeier.
Publish or perish, but do not forget your software artifacts.
Empirical Software Engineering, 25(6):4585–4616, 2020.
[ bib 
DOI ]

[574]

Christian Hicks.
A Genetic Algorithm tool for optimising cellular or functional
layouts in the capital goods industry.
International Journal of Production Economics, 104(2):598–614,
2006.
[ bib 
DOI ]

[575]

Robert M. Hierons, Miqing Li, Xiaohui Liu, Jose Antonio Parejo, Sergio Segura,
and Xin Yao.
Manyobjective test suite generation for software product
lines.
ACM Transactions on Software Engineering and Methodology,
29(1):1–46, 2020.
[ bib ]

[576]

Geoffrey E. Hinton and Ruslan R. Salakhutdinov.
Reducing the dimensionality of data with neural networks.
Science, 313(5786):504–507, 2006.
[ bib ]

[577]

Wassily Hoeffding.
Probability inequalities for sums of bounded random variables.
Journal of the American Statistical Association,
58(301):13–30, 1963.
[ bib ]

[578]

I. Hong, A. B. Kahng, and B. R. Moon.
Improved largestep Markov chain variants for the symmetric
TSP.
Journal of Heuristics, 3(1):63–81, 1997.
[ bib ]

[579]

John N. Hooker.
Needed: An Empirical Science of Algorithms.
Operations Research, 42(2):201–212, 1994.
[ bib ]

[580]

John N. Hooker.
Testing Heuristics: We Have It All Wrong.
Journal of Heuristics, 1(1):33–42, 1996.
[ bib 
DOI ]

[581]

Giles Hooker.
Generalized functional ANOVA diagnostics for highdimensional
functions of dependent variables.
Journal of Computational and Graphical Statistics,
16(3):709–732, 2012.
[ bib 
DOI ]

[582]

Holger H. Hoos, Marius Thomas Lindauer, and Torsten Schaub.
Claspfolio 2: Advances in Algorithm Selection for Answer Set
Programming.
Theory and Practice of Logic Programming, 14(45):560–585,
2014.
[ bib ]

[583]

Holger H. Hoos and Thomas Stützle.
On the Empirical Scaling of Runtime for Finding Optimal
Solutions to the Traveling Salesman Problem.
European Journal of Operational Research, 238(1):87–94, 2014.
[ bib ]

[584]

Holger H. Hoos and Thomas Stützle.
On the Empirical Time Complexity of Finding Optimal Solutions
vs. Proving Optimality for Euclidean TSP Instances.
Optimization Letters, 9(6):1247–1254, 2015.
[ bib ]

[585]

Holger H. Hoos.
Programming by optimization.
Communications of the ACM, 55(2):70–80, February 2012.
[ bib 
DOI ]

[586]

André Hottung, Shunji Tanaka, and Kevin Tierney.
Deep learning assisted heuristic tree search for the container
premarshalling problem.
Computers & Operations Research, 113:104781, 2020.
[ bib ]

[587]

Stela Pudar Hozo, Benjamin Djulbegovic, and Iztok Hozo.
Estimating the mean and variance from the median, range, and the
size of a sample.
BMC Medical Research Methodology, 5(1):13, 2005.
[ bib ]

[588]

T. C. Hu, A. B. Kahng, and C.W. A. Tsao.
Old Bachelor Acceptance: A New Class of NonMonotone Threshold
Accepting Methods.
ORSA Journal on Computing, 7(4):417–425, 1995.
[ bib ]

[589]

Wenbin Hu, Huan Wang, Zhenyu Qiu, Cong Nie, and Liping Yan.
A quantum particle swarm optimization driven urban traffic light
scheduling model.
Neural Computing & Applications, 2018.
[ bib 
DOI ]
Keywords: BML,Optimization,Simulation,Traffic congestion,Updating
rules

[590]

Wenbin Hu, Liping Yan, Huan Wang, Bo Du, and Dacheng Tao.
Realtime traffic jams prediction inspired by Biham,
Middleton and Levine (BML) model.
Information Sciences, 2017.
[ bib ]
Keywords: BML model,Prediction,Realtime,Traffic jam,Urban traffic
network

[591]

Deng Huang, Theodore T. Allen, William I. Notz, and Ning Zeng.
Global Optimization of Stochastic BlackBox Systems via
Sequential Kriging MetaModels.
Journal of Global Optimization, 34(3):441–466, 2006.
[ bib 
DOI ]

[592]

Changwu Huang, Yuanxiang Li, and Xin Yao.
A Survey of Automatic Parameter Tuning Methods for
Metaheuristics.
IEEE Transactions on Evolutionary Computation, 24(2):201–216,
2020.
[ bib 
DOI ]

[593]

S. Huband, P. Hingston, L. Barone, and L. While.
A Review of Multiobjective Test Problems and a Scalable Test
Problem Toolkit.
IEEE Transactions on Evolutionary Computation, 10(5):477–506,
2006.
[ bib 
DOI ]
Proposed the WFG benchmark suite

[594]

B. Huberman, R. Lukose, and T. Hogg.
An Economic Approach to Hard Computational Problems.
Science, 275:51–54, 1997.
[ bib ]

[595]

D. L. HuertaMuñoz, R. Z. RíosMercado, and Rubén Ruiz.
An Iterated Greedy Heuristic for a Market Segmentation Problem
with Multiple Attributes.
European Journal of Operational Research, 261(1):75–87, 2017.
[ bib ]

[596]

Jérémie Humeau, Arnaud Liefooghe, ElGhazali Talbi, and Sébastien
Verel.
ParadisEOMO: From Fitness Landscape Analysis to Efficient
Local Search Algorithms.
Journal of Heuristics, 19(6):881–915, June 2013.
[ bib 
DOI ]

[597]

Ying Hung, V. Roshan Joseph, and Shreyes N. Melkote.
Design and Analysis of Computer Experiments With Branching and
Nested Factors.
Technometrics, 51(4):354–365, 2009.
[ bib 
DOI ]

[598]

M. Hurtgen and J.C. Maun.
Optimal PMU placement using Iterated Local Search.
International Journal of Electrical Power & Energy Systems,
32(8):857–860, 2010.
[ bib ]

[599]

S. H. Hurlbert.
Pseudoreplication and the Design of Ecological Field
Experiments.
Ecological Monographs, 54(2):187–211, 1984.
[ bib ]

[600]

Mohamed Saifullah Hussin and Thomas Stützle.
Tabu Search vs. Simulated Annealing for Solving Large Quadratic
Assignment Instances.
Computers & Operations Research, 43:286–291, 2014.
[ bib ]

[601]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Tradeoffs in the Empirical Evaluation of Competing Algorithm
Designs.
Annals of Mathematics and Artificial Intelligence,
60(1–2):65–89, 2010.
[ bib ]

[602]

Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Bayesian Optimization With Censored Response Data.
Arxiv preprint arXiv:1310.1947, 2013.
[ bib 
http ]

[603]

Frank Hutter, Holger H. Hoos, Kevin LeytonBrown, and Thomas Stützle.
ParamILS: An Automatic Algorithm
Configuration Framework.
Journal of Artificial Intelligence Research, 36:267–306,
October 2009.
[ bib 
DOI ]

[604]

Frank Hutter, Marius Thomas Lindauer, Adrian Balint, Sam Bayless, Holger H.
Hoos, and Kevin LeytonBrown.
The Configurable SAT Solver Challenge (CSSC).
Artificial Intelligence, 243:1–25, 2017.
[ bib 
DOI ]

[605]

Frank Hutter, Lin Xu, Holger H. Hoos, and Kevin LeytonBrown.
Algorithm runtime prediction: Methods & evaluation.
Artificial Intelligence, 206:79–111, 2014.
[ bib ]

[606]

Hao Wang, Diederick Vermetten, Furong Ye, Carola Doerr, and Thomas Bäck.
IOHanalyzer: Detailed Performance Analyses for Iterative
Optimization Heuristics.
ACM Transactions on Evolutionary Learning and Optimization,
2(1):3:1–3:29, 2022.
[ bib 
DOI ]

[607]

Carola Doerr, Hao Wang, Furong Ye, Sander van Rijn, and Thomas Bäck.
IOHprofiler: A Benchmarking and Profiling Tool for Iterative
Optimization Heuristics.
Arxiv preprint arXiv:1806.07555, October 2018.
[ bib 
DOI ]
Keywords: Benchmarking; Heuristics

[608]

Claudio Iacopino and Phil Palmer.
The Dynamics of Ant Colony Optimization Algorithms Applied to
Binary Chains.
Swarm Intelligence, 6(4):343–377, 2012.
[ bib ]

[609]

Claudio Iacopino, Phil Palmer, N. Policella, A. Donati, and A. Brewer.
How Ants Can Manage Your Satellites.
Acta Futura, 9:59–72, 2014.
[ bib 
DOI ]
Keywords: ACO, Space

[610]

Toshihide Ibaraki, Shinji Imahori, Koji Nonobe, Kensuke Sobue, Takeaki Uno, and
Mutsunori Yagiura.
An Iterated Local Search Algorithm for the Vehicle Routing
Problem with Convex Time Penalty Functions.
Discrete Applied Mathematics, 156(11):2050–2069, 2008.
[ bib ]

[611]

Toshihide Ibaraki.
A Personal Perspective on Problem Solving by General Purpose
Solvers.
International Transactions in Operational Research,
17(3):303–315, 2010.
[ bib ]

[612]

Jonas Ide and Anita Schöbel.
Robustness for uncertain multiobjective optimization: a survey
and analysis of different concepts.
OR Spectrum, 38(1):235–271, 2016.
[ bib 
DOI ]
In this paper, we discuss various concepts of robustness for
uncertain multiobjective optimization problems. We extend
the concepts of flimsily, highly, and lightly robust
efficiency and we collect different versions of minmax robust
efficiency and concepts based on set order relations from the
literature. Altogether, we compare and analyze ten different
concepts and point out their relations to each
other. Furthermore, we present reduction results for the
class of objectivewise uncertain multiobjective
optimization problems.

[613]

Christian Igel, Nikolaus Hansen, and S. Roth.
Covariance Matrix Adaptation for Multiobjective Optimization.
Evolutionary Computation, 15(1):1–28, 2007.
[ bib ]

[614]

Christian Igel, V. HeidrichMeisner, and T. Glasmachers.
Shark.
Journal of Machine Learning Research, 9:993–996, June 2008.
[ bib 
http ]

[615]

Nesa Ilich and Slobodan P. Simonovic.
Evolutionary Algorithm for minimization of pumping cost.
Journal of Computing in Civil Engineering, ASCE,
12(4):232–240, October 1998.
[ bib ]
This paper deals with minimizing the total cost of
pumping in a liquid pipeline. Previous experience
with the most common solution procedures in pipeline
optimization is discussed along with their strengths
and weaknesses. The proposed method is an
evolutionary algorithm with two distinct features:
(1) The search is restricted to feasible region
only; and (2) it utilizes a floating point decision
variable rather than integer or binary as is the
case with most other similar approaches. A numerical
example is presented as a basis for verification of
the proposed method and its comparison with the
existing solver that utilizes the nonlinear
Newtonian search. The proposed method provides
promising improvements in terms of optimality when
compared to the widespread gradient search methods
because it does not involve evaluation of the
gradient of the objective function. It also provides
potential to improve the performance of previous
evolutionary programs because it restricts the
search to the feasible region, thus eliminating
large overhead associated with generation and
inspection of solutions that are
infeasible. Comparison of the two solutions revealed
improvement of the solution in favor of the proposed
algorithm, which ranged up to 6% depending on the
initial values of the decision variables in the
Newtonian search. The proposed method was not
sensitive to the starting value of the decision
variables.

[616]

Takashi Imamichi, Mutsunori Yagiura, and Hiroshi Nagamochi.
An Iterated Local Search Algorithm Based on Nonlinear
Programming for the Irregular Strip Packing Problem.
Discrete Optimization, 6(4):345–361, 2009.
[ bib ]

[617]

Alfred Inselberg.
The Plane with Parallel Coordinates.
The Visual Computer, 1(2):69–91, 1985.
[ bib ]

[618]

John P. A. Ioannidis.
Why Most Published Research Findings Are False.
PLoS Medicine, 2(8):e124, 2005.
[ bib 
DOI ]

[619]

Stefan Irnich.
A Unified Modeling and Solution Framework for Vehicle Routing
and Local SearchBased Metaheuristics.
INFORMS Journal on Computing, 20(2):270–287, 2008.
[ bib ]

[620]

Ekhine Irurozki, Borja Calvo, and José A. Lozano.
Sampling and Learning Mallows and Generalized Mallows Models
Under the Cayley Distance.
Methodology and Computing in Applied Probability, 20(1):1–35,
June 2016.
[ bib 
DOI ]

[621]

Ekhine Irurozki, Borja Calvo, and José A. Lozano.
PerMallows: An R Package for Mallows
and Generalized Mallows Models.
Journal of Statistical Software, 71, 2019.
[ bib 
DOI ]
In this paper we present the R package PerMallows, which is a
complete toolbox to work with permutations, distances and
some of the most popular probability models for permutations:
Mallows and the Generalized Mallows models. The Mallows model
is an exponential location model, considered as analogous to
the Gaussian distribution. It is based on the definition of a
distance between permutations. The Generalized Mallows model
is its bestknown extension. The package includes functions
for making inference, sampling and learning such
distributions. The distances considered in PerMallows are
Kendall's τ, Cayley, Hamming and Ulam.
Keywords: Cayley,Generalized Mallows,Hamming,Kendall's
τ,Learning,Mallows,Permutation,R,Ranking,Sampling,Ulam

[622]

Ekhine Irurozki, Jesus Lobo, Aritz Perez, and Javier Del Ser.
Rank aggregation for nonstationary data streams.
Arxiv preprint arXiv:1910.08795 [stat.ML], 2020.
[ bib 
http ]
Keywords: uborda

[623]

Hisao Ishibuchi and T. Murata.
A multiobjective genetic local search algorithm and its
application to flowshop scheduling.
IEEE Transactions on Systems, Man, and Cybernetics – Part C,
28(3):392–403, 1998.
[ bib ]

[624]

Hisao Ishibuchi, N. Akedo, and Y. Nojima.
Behavior of Multiobjective Evolutionary Algorithms on
ManyObjective Knapsack Problems.
IEEE Transactions on Evolutionary Computation, 19(2):264–283,
2015.
[ bib 
DOI ]

[625]

Hisao Ishibuchi, Ryo Imada, Yu Setoguchi, and Yusuke Nojima.
How to specify a reference point in hypervolume calculation for
fair performance comparison.
Evolutionary Computation, 26(3):411–440, 2018.
[ bib ]

[626]

Hisao Ishibuchi, Shinta Misaki, and Hideo Tanaka.
Modified simulated annealing algorithms for the flow shop
sequencing problem.
European Journal of Operational Research, 81(2):388–398, 1995.
[ bib ]

[627]

Hisao Ishibuchi, Yu Setoguchi, Hiroyuki Masuda, and Yusuke Nojima.
Performance of decompositionbased manyobjective algorithms
strongly depends on Pareto front shapes.
IEEE Transactions on Evolutionary Computation, 21(2):169–190,
2017.
[ bib ]

[628]

Peter Ivie and Douglas Thain.
Reproducibility in Scientific Computing.
ACM Computing Surveys, 51(3):1–36, 2019.
[ bib 
DOI ]

[629]

Srikanth K. Iyer and Barkha Saxena.
Improved genetic algorithm for the permutation flowshop
scheduling problem.
Computers & Operations Research, 31(4):593–606, 2004.
[ bib 
DOI ]

[630]

Christopher H. Jackson.
MultiState Models for Panel Data: The msm Package
for R.
Journal of Statistical Software, 38(8):1–29, 2011.
[ bib 
http ]

[631]

Richard H. F. Jackson, Paul T. Boggs, Stephen G. Nash, and Susan Powell.
Guidelines for Reporting Results of Computational Experiments.
Report of the Ad Hoc Committee.
Mathematical Programming, 49(3):413–425, 1991.
[ bib ]

[632]

Larry W. Jacobs and Michael J. Brusco.
A Local Search Heuristic for Large SetCovering Problems.
Naval Research Logistics, 42(7):1129–1140, 1995.
[ bib ]

[633]

Warren G. Jackson, Ender Özcan, and Robert I. John.
Move acceptance in local search metaheuristics for crossdomain
search.
Expert Systems with Applications, 109:131–151, 2018.
[ bib ]

[634]

Daniel M Jaeggi, Geoffrey T Parks, Timoleon Kipouros, and P John Clarkson.
The development of a multiobjective Tabu Search algorithm for
continuous optimisation problems.
European Journal of Operational Research, 185(3):1192–1212,
2008.
[ bib ]

[635]

Satish Jajodia, Ioannis Minis, George Harhalakis, and JeanMarie Proth.
CLASS: computerized layout solutions using simulated
annealing.
International Journal of Production Research, 30(1):95–108,
1992.
[ bib ]

[636]

Andrzej Jaszkiewicz.
Genetic local search for multiobjective combinatorial
optimization.
European Journal of Operational Research, 137(1):50–71, 2002.
[ bib ]

[637]

Andrzej Jaszkiewicz.
ManyObjective Pareto Local Search.
European Journal of Operational Research, 271(3):1001–1013,
2018.
[ bib 
DOI ]

[638]

Andrzej Jaszkiewicz and Thibaut Lust.
NDtreebased update: a fast algorithm for the dynamic
nondominance problem.
IEEE Transactions on Evolutionary Computation, 22(5):778–791,
2018.
[ bib ]

[639]

Andrzej Jaszkiewicz.
On the performance of multipleobjective genetic local search on
the 0/1 knapsack problem – A comparative experiment.
IEEE Transactions on Evolutionary Computation, 6(4):402–412,
2002.
[ bib ]

[640]

M. T. Jensen.
Reducing the runtime complexity of multiobjective EAs: The
NSGAII and other algorithms.
IEEE Transactions on Evolutionary Computation, 7(5):503–515,
2003.
[ bib ]

[641]

M. T. Jensen.
HelperObjectives: Using MultiObjective Evolutionary
Algorithms for SingleObjective Optimisation.
Journal of Mathematical Modelling and Algorithms,
3(4):323–347, 2004.
[ bib ]
Keywords: multiobjectivization

[642]

Mark Jerrum and Gregory Sorkin.
The Metropolis algorithm for graph bisection.
Discrete Applied Mathematics, 82(1):155–175, 1998.
[ bib ]

[643]

Mark Jerrum.
Large cliques elude the Metropolis process.
Random Structures & Algorithms, 3(4):347–359, 1992.
[ bib ]

[644]

S. Jiang, Y. S. Ong, J. Zhang, and L. Feng.
Consistencies and Contradictions of Performance Metrics in
Multiobjective Optimization.
IEEE Transactions on Cybernetics, 44(12):2391–2404, 2014.
[ bib ]

[645]

Shouyong Jiang, Juan Zou, Shengxiang Yang, and Xin Yao.
Evolutionary Dynamic MultiObjective Optimisation: A Survey.
ACM Computing Surveys, 55(4), November 2022.
[ bib 
DOI ]
Keywords: evolutionary algorithm, evolutionary dynamic multiobjective
optimisation, dynamic environment, Multiobjective
optimisation

[646]

Yaochu Jin.
A Comprehensive Survey of Fitness Approximation in Evolutionary
Computation.
Soft Computing, 9(1):3–12, 2005.
[ bib ]

[647]

Yaochu Jin.
SurrogateAssisted Evolutionary Computation: Recent Advances and
Future Challenges.
Swarm and Evolutionary Computation, 1(2):61–70, June 2011.
[ bib 
DOI ]
Surrogateassisted, or metamodel based evolutionary
computation uses efficient computational models, often known
as surrogates or metamodels, for approximating the fitness
function in evolutionary algorithms. Research on
surrogateassisted evolutionary computation began over a
decade ago and has received considerably increasing interest
in recent years. Very interestingly, surrogateassisted
evolutionary computation has found successful applications
not only in solving computationally expensive single or
multiobjective optimization problems, but also in addressing
dynamic optimization problems, constrained optimization
problems and multimodal optimization problems. This paper
provides a concise overview of the history and recent
developments in surrogateassisted evolutionary computation
and suggests a few future trends in this research area.
Keywords: Evolutionary computation,Expensive optimization
problems,Machine learning,Metamodels,Model
management,Surrogates

[648]

Yaochu Jin, Handing Wang, Tinkle Chugh, Dan Guo, and Kaisa Miettinen.
DataDriven Evolutionary Optimization: An Overview and Case
Studies.
IEEE Transactions on Evolutionary Computation, 23(3):442–458,
June 2019.
[ bib 
DOI ]

[649]

Huidong Jin and ManLeung Wong.
Adaptive, convergent, and diversified archiving strategy for
multiobjective evolutionary algorithms.
Expert Systems with Applications, 37(12):8462–8470, 2010.
[ bib ]

[650]

David S. Johnson.
Optimal Two and Threestage Production Scheduling with Setup
Times Included.
Naval Research Logistics Quarterly, 1:61–68, 1954.
[ bib ]

[651]

David S. Johnson, Cecilia R. Aragon, Lyle A. McGeoch, and Catherine Schevon.
Optimization by Simulated Annealing: An Experimental Evaluation:
Part I, Graph Partitioning.
Operations Research, 37(6):865–892, 1989.
[ bib ]

[652]

David S. Johnson, Cecilia R. Aragon, Lyle A. McGeoch, and Catherine Schevon.
Optimization by Simulated Annealing: An Experimental Evaluation:
Part II, Graph Coloring and Number Partitioning.
Operations Research, 39(3):378–406, 1991.
[ bib ]

[653]

Alan W. Johnson and Sheldon H. Jacobson.
On the Convergence of Generalized Hill Climbing Algorithms.
Discrete Applied Mathematics, 119(1):37–57, 2002.
[ bib ]

[654]

Mark E. Johnson, Leslie M. Moore, and Donald Ylvisaker.
Minimax and maximin distance designs.
Journal of Statistical Planning and Inference, 26(2):131–148,
1990.
[ bib ]
Keywords: Bayesian design

[655]

David S. Johnson, Christos H. Papadimitriou, and Mihalis Yannakakis.
How Easy is Local Search?
Journal of Computer System Science, 37(1):79–100, 1988.
[ bib ]

[656]

C. Joncour, J. Kritter, S. Michel, and X. Schepler.
Generalized RelaxandFix Heuristic.
Computers & Operations Research, 149:106038, 2023.
[ bib ]

[657]

Donald R. Jones, Matthias Schonlau, and William J. Welch.
Efficient Global Optimization of Expensive BlackBox Functions.
Journal of Global Optimization, 13(4):455–492, 1998.
[ bib ]
Proposed EGO algorithm
Keywords: EGO

[658]

Kenneth A. De Jong and William M. Spears.
A formal analysis of the role of multipoint crossover in
genetic algorithms.
Annals of Mathematics and Artificial Intelligence, 5(1):1–26,
1992.
[ bib ]

[659]

D. E. Joslin and D. P. Clements.
Squeaky Wheel Optimization.
Journal of Artificial Intelligence Research, 10:353–373, 1999.
[ bib ]

[660]

P. W. Jowitt and G. Germanopoulos.
Optimal pump scheduling in water supply networks.
Journal of Water Resources Planning and Management, ASCE,
118(4):406–422, 1992.
[ bib ]
The electricity cost of pumping accounts for a large
part of the total operating cost for watersupply
networks. This study presents a method based on
linear programming for determining an optimal
(minimum cost) schedule of pumping on a 24hr
basis. Both unit and maximum demand electricity
charges are considered. Account is taken of the
relative efficiencies of the available pumps, the
structure of the electricity tariff, the
consumerdemand profile, and the hydraulic
characteristics and operational constraints of the
network. The use of extendedperiod simulation of
the network operation in determining the parameters
of the linearized network equations and constraints
and in studying the optimized network operation is
described. An application of the method to an
existing network in the United Kingdom is presented,
showing that considerable savings are possible. The
method was found to be robust and with low
computationtime requirements, and is therefore
suitable for realtime implementation.

[661]

Angel A. Juan, Javier Faulin, Scott E. Grasman, Markus Rabe, and Gonçalo
Figueira.
A review of simheuristics: Extending metaheuristics to deal with
stochastic combinatorial optimization problems.
Operations Research Perspectives, 2:62–72, 2015.
[ bib 
DOI ]
Keywords: Metaheuristics; Simulation; Combinatorial optimization;
Stochastic problems

[662]

Angel A. Juan, Helena R. Lourenço, Manuel Mateo, Rachel Luo, and Quim
Castellà.
Using Iterated Local Search for Solving the Flowshop Problem:
Parallelization, Parametrization, and Randomization Issues.
International Transactions in Operational Research,
21(1):103–126, 2014.
[ bib ]

[663]

M. Jünger, Gerhard Reinelt, and S. Thienel.
Provably Good Solutions for the Traveling Salesman Problem.
Zeitschrift für Operations Research, 40(2):183–217, 1994.
[ bib ]

[664]

Elena A. Kabova, Jason C. Cole, Oliver Korb, Manuel LópezIbáñez,
Adrian C. Williams, and Kenneth Shankland.
Improved performance of crystal structure solution from powder
diffraction data through parameter tuning of a simulated annealing
algorithm.
Journal of Applied Crystallography, 50(5):1411–1420, October
2017.
[ bib 
DOI ]
Significant gains in the performance of the simulated
annealing algorithm in the DASH software package have
been realized by using the irace automatic
configuration tool to optimize the values of three key
simulated annealing parameters. Specifically, the success
rate in finding the global minimum in intensity χ^{2}
space is improved by up to an order of magnitude. The general
applicability of these revised simulated annealing parameters
is demonstrated using the crystal structure determinations of
over 100 powder diffraction datasets.
Keywords: crystal structure determination, powder diffraction,
simulated annealing, parameter tuning, irace

[665]

Daniel Kahneman and Amos Tversky.
Prospect theory: An analysis of decision under risk.
Econometrica, 47(2):263–291, 1979.
[ bib 
DOI ]

[666]

Daniel Kahneman.
Maps of bounded rationality: Psychology for behavioral
economics.
The American Economic Review, 93(5):1449–1475, 2003.
[ bib ]

[667]

Qinma Kang, Hong He, and Jun Wei.
An Effective Iterated Greedy Algorithm for Reliabilityoriented
Task Allocation in Distributed Computing Systems.
Journal of Parallel and Distributed Computing,
73(8):1106–1115, 2013.
[ bib ]

[668]

Korhan Karabulut.
A hybrid iterated greedy algorithm for total tardiness
minimization in permutation flowshops.
Computers and Industrial Engineering, 98(Supplement C):300 –
307, 2016.
[ bib ]

[669]

Dervis Karaboga and Bahriye Akay.
A Survey: Algorithms Simulating Bee Swarm Intelligence.
Artificial Intelligence Review, 31(1–4):61–85, 2009.
[ bib ]

[670]

Giorgos Karafotias, Mark Hoogendoorn, and Agoston E. Eiben.
Parameter Control in Evolutionary Algorithms: Trends and
Challenges.
IEEE Transactions on Evolutionary Computation, 19(2):167–187,
2015.
[ bib ]

[671]

İbrahim Karahan and Murat Köksalan.
A territory defining multiobjective evolutionary algorithms and
preference incorporation.
IEEE Transactions on Evolutionary Computation, 14(4):636–664,
2010.
[ bib 
DOI ]
Keywords: TDEA

[672]

Oleksiy Karpenko, Jianming Shi, and Yang Dai.
Prediction of MHC class II binders using the ant colony
search strategy.
Artificial Intelligence in Medicine, 35(1):147–156, 2005.
[ bib ]

[673]

Korhan Karabulut and Fatih M. Tasgetiren.
A Variable Iterated Greedy Algorithm for the Traveling Salesman
Problem with Time Windows.
Information Sciences, 279:383–395, 2014.
[ bib ]

[674]

Joseph R. Kasprzyk, Shanthi Nataraj, Patrick M. Reed, and Robert J. Lempert.
Many objective robust decision making for complex environmental
systems undergoing change.
Environmental Modelling & Software, 42:55–71, 2013.
[ bib ]
Keywords: scenariobased

[675]

Joseph R. Kasprzyk, Patrick M. Reed, Gregory W. Characklis, and Brian R.
Kirsch.
Manyobjective de Novo water supply portfolio planning under
deep uncertainty.
Environmental Modelling & Software, 34:87–104, 2012.
[ bib ]
Keywords: scenariobased

[676]

Artem Kaznatcheev, David A. Cohen, and Peter Jeavons.
Representing Fitness Landscapes by Valued Constraints to
Understand the Complexity of Local Search.
Journal of Artificial Intelligence Research, 69:1077–1102,
2020.
[ bib 
DOI ]

[677]

Liangjun Ke, Claudia Archetti, and Zuren Feng.
Ants can solve the team orienteering problem.
Computers and Industrial Engineering, 54(3):648–665, 2008.
[ bib 
DOI ]
The team orienteering problem (TOP) involves
finding a set of paths from the starting point to
the ending point such that the total collected
reward received from visiting a subset of locations
is maximized and the length of each path is
restricted by a prespecified limit. In this paper,
an ant colony optimization (ACO) approach is
proposed for the team orienteering problem. Four
methods, i.e., the sequential,
deterministicconcurrent and randomconcurrent and
simultaneous methods, are proposed to construct
candidate solutions in the framework of ACO. We
compare these methods according to the results
obtained on wellknown problems from the
literature. Finally, we compare the algorithm with
several existing algorithms. The results show that
our algorithm is promising.
Keywords: Ant colony optimization, Ant system, Heuristics,
Team orienteering problem

[678]

R. L. Keeney.
Analysis of preference dependencies among objectives.
Operations Research, 29:1105–1120, 1981.
[ bib ]

[679]

Graham Kendall, Ruibin Bai, Jacek Blazewicz, Patrick De Causmaecker,
Michel Gendreau, Robert John, Jiawei Li, Barry McCollum, Erwin Pesch, Rong
Qu, Nasser Sabar, Greet Vanden Berghe, and Angelina Yee.
Good Laboratory Practice for Optimization Research.
Journal of the Operational Research Society, 67(4):676–689,
2016.
[ bib 
DOI ]

[680]

Pascal Kerschke, Holger H. Hoos, Frank Neumann, and Heike Trautmann.
Automated Algorithm Selection: Survey and Perspectives.
Evolutionary Computation, 27(1):3–45, March 2019.
[ bib 
DOI ]

[681]

B. W. Kernighan and S. Lin.
An Efficient Heuristic Procedure for Partitioning Graphs.
Bell Systems Technology Journal, 49(2):213–219, 1970.
[ bib ]

[682]

Pascal Kerschke and Heike Trautmann.
Automated Algorithm Selection on Continuous BlackBox Problems
by Combining Exploratory Landscape Analysis and Machine Learning.
Evolutionary Computation, 27(1):99–127, 2019.
[ bib 
DOI ]
In this article, we build upon previous work on designing
informative and efficient Exploratory Landscape Analysis
features for characterizing problems' landscapes and show
their effectiveness in automatically constructing algorithm
selection models in continuous blackbox optimization
problems. Focusing on algorithm performance results of the
COCO platform of several years, we construct a representative
set of highperforming complementary solvers and present an
algorithm selection model that, compared to the portfolio's
single best solver, on average requires less than half of the
resources for solving a given problem. Therefore, there is a
huge gain in efficiency compared to classical ensemble
methods combined with an increased insight into problem
characteristics and algorithm properties by using informative
features. The model acts on the assumption that the function
set of the BlackBox Optimization Benchmark is representative
enough for practical applications. The model allows for
selecting the best suited optimization algorithm within the
considered set for unseen problems prior to the optimization
itself based on a small sample of function evaluations. Note
that such a sample can even be reused for the initial
population of an evolutionary (optimization) algorithm so
that even the feature costs become negligible.

[683]

Pascal Kerschke, Hao Wang, Mike Preuss, Christian Grimme, André H. Deutz,
Heike Trautmann, and Michael T. M. Emmerich.
Search Dynamics on Multimodal Multiobjective Problems.
Evolutionary Computation, 27(4):577–609, 2019.
[ bib 
DOI ]

[684]

Norbert L. Kerr.
HARKing: Hypothesizing After the Results are Known.
Personality and Social Psychology Review, 2(3):196–217, August
1998.
[ bib 
DOI ]

[685]

A. R. KhudaBukhsh, Lin Xu, Holger H. Hoos, and Kevin LeytonBrown.
SATenstein: Automatically Building Local Search SAT
Solvers from Components.
Artificial Intelligence, 232:20–42, 2016.
[ bib 
DOI ]

[686]

Philip Kilby and Tommaso Urli.
Fleet design optimisation from historical data using constraint
programming and large neighbourhood search.
Constraints, pp. 1–20, 2015.
[ bib 
DOI ]
Keywords: Frace

[687]

YeongDae Kim.
Heuristics for Flowshop Scheduling Problems Minimizing Mean
Tardiness.
Journal of the Operational Research Society, 44(1):19–28,
1993.
[ bib 
DOI ]

[688]

Youngmin Kim, Richard Allmendinger, and Manuel LópezIbáñez.
Safe Learning and Optimization Techniques: Towards a Survey of
the State of the Art.
Arxiv preprint arXiv:2101.09505 [cs.LG], 2020.
[ bib 
http ]
Safe learning and optimization deals with learning and
optimization problems that avoid, as much as possible, the
evaluation of nonsafe input points, which are solutions,
policies, or strategies that cause an irrecoverable loss
(e.g., breakage of a machine or equipment, or life
threat). Although a comprehensive survey of safe
reinforcement learning algorithms was published in 2015, a
number of new algorithms have been proposed thereafter, and
related works in active learning and in optimization were not
considered. This paper reviews those algorithms from a number
of domains including reinforcement learning, Gaussian process
regression and classification, evolutionary algorithms, and
active learning. We provide the fundamental concepts on which
the reviewed algorithms are based and a characterization of
the individual algorithms. We conclude by explaining how the
algorithms are connected and suggestions for future
research.

[689]

Jungtaek Kim, Michael McCourt, Tackgeun You, Saehoon Kim, and Seungjin Choi.
Bayesian Optimization with Approximate Set Kernels.
Machine Learning, 2021.
[ bib 
DOI ]
We propose a practical Bayesian optimization method over
sets, to minimize a blackbox function that takes a set as a
single input. Because set inputs are permutationinvariant,
traditional Gaussian processbased Bayesian optimization
strategies which assume vector inputs can fall short. To
address this, we develop a Bayesian optimization method with
set kernel that is used to build surrogate
functions. This kernel accumulates similarity over set
elements to enforce permutationinvariance, but this comes at
a greater computational cost. To reduce this burden, we
propose two key components: (i) a more efficient approximate
set kernel which is still positivedefinite and is an
unbiased estimator of the true set kernel with upperbounded
variance in terms of the number of subsamples, (ii) a
constrained acquisition function optimization over sets,
which uses symmetry of the feasible region that defines a set
input. Finally, we present several numerical experiments
which demonstrate that our method outperforms other methods.

[690]

J.S. Kim, J.H. Park, and D.H. Lee.
Iterated Greedy Algorithms to Minimize the Total Family Flow
Time for Jobshop Scheduling with Job Families and Sequencedependent
Setups.
Engineering Optimization, 49(10):1719–1732, 2017.
[ bib ]

[691]

Diederik P. Kingma and Jimmy Ba.
Adam: A method for stochastic optimization.
Arxiv preprint arXiv:1412.6980 [cs.LG], 2014.
[ bib 
http ]
Published as a conference paper at the 3rd International
Conference for Learning Representations, San Diego, 2015

[692]

Scott Kirkpatrick and G. Toulouse.
Configuration Space Analysis of Travelling Salesman Problems.
Journal de Physique, 46(8):1277–1292, 1985.
[ bib ]

[693]

Scott Kirkpatrick.
Optimization by Simulated Annealing: Quantitative Studies.
Journal of Statistical Physics, 34(56):975–986, 1984.
[ bib ]

[694]

Scott Kirkpatrick, C. D. Gelatt, and M. P. Vecchi.
Optimization by Simulated Annealing.
Science, 220(4598):671–680, 1983.
[ bib 
DOI ]

[695]

Kathrin Klamroth, Sanaz Mostaghim, Boris Naujoks, Silvia Poles, Robin C.
Purshouse, Günther Rudolph, Stefan Ruzika, Serpil Sayın, Margaret M.
Wiecek, and Xin Yao.
Multiobjective optimization for interwoven systems.
Journal of MultiCriteria Decision Analysis, 24(12):71–81,
2017.
[ bib 
DOI ]

[696]

Anton J. Kleywegt, Alexander Shapiro, and Tito HomemdeMello.
The Sample Average Approximation Method for Stochastic Discrete
Optimization.
SIAM Journal on Optimization, 12(2):479–502, 2002.
[ bib ]

[697]

Joshua D. Knowles.
ParEGO: A hybrid algorithm with online landscape
approximation for expensive multiobjective optimization problems.
IEEE Transactions on Evolutionary Computation, 10(1):50–66,
2006.
[ bib 
DOI ]
Keywords: ParEGO, online, metamodel

[698]

Joshua D. Knowles.
Closedloop evolutionary multiobjective optimization.
IEEE Computational Intelligence Magazine, 4:77–91, 2009.
[ bib 
DOI ]
Artificial evolution has been used for more than 50 years as a method of optimization in engineering, operations research and computational intelligence. In closedloop evolution (a term used by the statistician, George Box) or, equivalently, evolutionary experimentation (Ingo Rechenberg's terminology), the “phenotypes” are evaluated in the real world by conducting a physical experiment, whilst selection and breeding is simulated. Wellknown early work on artificial evolution — design engineering problems in fluid dynamics, and chemical plant process optimization — was carried out in this experimental mode. More recently, the closedloop approach has been successfully used in much evolvable hardware and evolutionary robotics research, and in some microbiology and biochemistry applications. In this article, several further new targets for closedloop evolutionary and multiobjective optimization are considered. Four case studies from my own collaborative work are described: (i) instrument optimization in analytical biochemistry; (ii) finding effective drug combinations in vitro; (iii) onchip synthetic biomolecule design; and (iv) improving chocolate production processes. Accurate simulation in these applications is not possible due to complexity or a lack of adequate analytical models. In these and other applications discussed, optimizing experimentally brings with it several challenges: noise; nuisance factors; ephemeral resource constraints; expensive evaluations, and evaluations that must be done in (large) batches. Evolutionary algorithms (EAs) are largely equal to these vagaries, whilst modern multiobjective EAs also enable tradeoffs among conflicting optimization goals to be explored. Nevertheless, principles from other disciplines, such as statistics, Design of Experiments, machine learning and global optimization are also relevant to aspects of the closedloop problem, and may inspire futher development of multiobjective EAs.

[699]

Joshua D. Knowles and David Corne.
Approximating the Nondominated Front Using the Pareto Archived
Evolution Strategy.
Evolutionary Computation, 8(2):149–172, 2000.
[ bib 
DOI ]
Proposed PAES

[700]

Joshua D. Knowles and David Corne.
Properties of an Adaptive Archiving Algorithm for Storing
Nondominated Vectors.
IEEE Transactions on Evolutionary Computation, 7(2):100–116,
April 2003.
[ bib ]
Proposed to use Smetric (hypervolume metric) for
environmental selection
Keywords: Smetric, hypervolume

[701]

Mirjam J. Knol, Tyler J. VanderWeele, Rolf H. H. Groenwold, Olaf H. Klungel,
Maroeska M. Rovers, and Diederick E. Grobbee.
Estimating measures of interaction on an additive scale for
preventive exposures.
European Journal of Epidemiology, 26(6):433–438, 2011.
[ bib ]

[702]

Gary A. Kochenberger, Fred Glover, Bahram Alidaee, and Cesar Rego.
A unified modeling and solution framework for combinatorial
optimization problems.
OR Spektrum, 26(2):237–250, 2004.
[ bib ]

[703]

Gary A. Kochenberger, JinKao Hao, Fred Glover, Mark Lewis, Zhipeng Lü,
Haibo Wang, and Yang Wang.
The unconstrained binary quadratic programming problem: a
survey.
Journal of Combinatorial Optimization, 28(1):58–81, 2014.
[ bib 
DOI ]

[704]

Murat Köksalan.
Multiobjective Combinatorial Optimization: Some Approaches.
Journal of MultiCriteria Decision Analysis, 15:69–78, 2009.
[ bib 
DOI ]

[705]

Murat Köksalan and İbrahim Karahan.
An Interactive Territory Defining Evolutionary Algorithm:
iTDEA.
IEEE Transactions on Evolutionary Computation, 14(5):702–722,
October 2010.
[ bib 
DOI ]

[706]

Rainer Kolisch and Sönke Hartmann.
Experimental investigation of heuristics for
resourceconstrained project scheduling: An update.
European Journal of Operational Research, 174(1):23–37,
October 2006.
[ bib 
DOI ]
This paper considers heuristics for the wellknown
resourceconstrained project scheduling problem
(RCPSP). It provides an update of our survey which
was published in 2000. We summarize and categorize a
large number of heuristics that have recently been
proposed in the literature. Most of these heuristics
are then evaluated in a computational study and
compared on the basis of our standardized
experimental design. Based on the computational
results we discuss features of good heuristics. The
paper closes with some remarks on our test design
and a summary of the recent developments in research
on heuristics for the RCPSP.
Keywords: Computational evaluation, Heuristics, Project
scheduling, Resource constraints

[707]

Vladlen Koltun and Christos H. Papadimitriou.
Approximately dominating representatives.
Theoretical Computer Science, 371(3):148–154, 2007.
[ bib ]

[708]

A. Kolen and Erwin Pesch.
Genetic Local Search in Combinatorial Optimization.
Discrete Applied Mathematics, 48(3):273–284, 1994.
[ bib ]

[709]

Joshua B. Kollat and Patrick M. Reed.
A framework for visually interactive decisionmaking and design
using evolutionary multiobjective optimization (VIDEO).
Environmental Modelling & Software, 22(12):1691–1704, 2007.
[ bib ]
Keywords: glyph plot

[710]

Tjalling C. Koopmans and Martin J. Beckmann.
Assignment Problems and the Location of Economic Activities.
Econometrica, 25:53–76, 1957.
[ bib ]
Introduced the Quadratic Assignment Problem (QAP)

[711]

Jsh Kornbluth.
Sequential multicriterion decision making.
Omega, 13(6):569–574, 1985.
[ bib 
DOI ]
In this paper we consider a simple sequential
multicriterion decision making problem in which a
decision maker has to accept or reject a series of
multiattributed outcomes. We show that using very
simple programming techniques, a great deal of the
decision making can be automated. The method might
be applicable to situations in which a dealer is
having to consider sequential offers in a trading
market.
Keywords: machine decision making

[712]

Flip Korn, B.U. Pagel, and Christos Faloutsos.
On the “dimensionality curse” and the “selfsimilarity
blessing”.
IEEE Transactions on Knowledge and Data Engineering,
13(1):96–111, 2001.
[ bib 
DOI ]
Spatial queries in highdimensional spaces have been studied
extensively. Among them, nearest neighbor queries are
important in many settings, including spatial databases (Find
the k closest cities) and multimedia databases (Find the k
most similar images). Previous analyses have concluded that
nearestneighbor search is hopeless in high dimensions due to
the notorious "curse of dimensionality". We show that this
may be overpessimistic. We show that what determines the
search performance (at least for Rtreelike structures) is
the intrinsic dimensionality of the data set and not the
dimensionality of the address space (referred to as the
embedding dimensionality). The typical (and often implicit)
assumption in many previous studies is that the data is
uniformly distributed, with independence between
attributes. However, real data sets overwhelmingly disobey
these assumptions; rather, they typically are skewed and
exhibit intrinsic ("fractal") dimensionalities that are much
lower than their embedding dimension, e.g. due to subtle
dependencies between attributes. We show how the Hausdorff
and Correlation fractal dimensions of a data set can yield
extremely accurate formulas that can predict the I/O
performance to within one standard deviation on multiple real
and synthetic data sets.

[713]

P. Korošec, Jurij Šilc, and B. Robič.
Solving the meshpartitioning problem with an antcolony
algorithm.
Parallel Computing, 30:785–801, 2004.
[ bib ]

[714]

Oliver Korb, Thomas Stützle, and Thomas E. Exner.
An Ant Colony Optimization Approach to Flexible Protein–Ligand
Docking.
Swarm Intelligence, 1(2):115–134, 2007.
[ bib ]

[715]

Oliver Korb, Thomas Stützle, and Thomas E. Exner.
Empirical Scoring Functions for Advanced ProteinLigand Docking
with PLANTS.
Journal of Chemical Information and Modeling, 49(2):84–96,
2009.
[ bib ]

[716]

Oliver Korb, Peter Monecke, Gerhard Hessler, Thomas Stützle, and Thomas E.
Exner.
pharmACOphore: Multiple Flexible Ligand Alignment Based on Ant
Colony Optimization.
Journal of Chemical Information and Modeling, 50(9):1669–1681,
2010.
[ bib ]

[717]

Lars Kotthoff.
Algorithm Selection for Combinatorial Search Problems: A
Survey.
AI Magazine, 35(3):48–60, 2014.
[ bib ]

[718]

Timo Kötzing, Frank Neumann, Heiko Röglin, and Carsten Witt.
Theoretical Analysis of Two ACO Approaches for the Traveling
Salesman Problem.
Swarm Intelligence, 6(1):1–21, 2012.
[ bib 
DOI ]
Bioinspired algorithms, such as evolutionary algorithms and
ant colony optimization, are widely used for different
combinatorial optimization problems. These algorithms rely
heavily on the use of randomness and are hard to understand
from a theoretical point of view. This paper contributes to
the theoretical analysis of ant colony optimization and
studies this type of algorithm on one of the most prominent
combinatorial optimization problems, namely the traveling
salesperson problem (TSP). We present a new construction
graph and show that it has a stronger local property than one
commonly used for constructing solutions of the TSP. The
rigorous runtime analysis for two ant colony optimization
algorithms, based on these two construction procedures, shows
that they lead to good approximation in expected polynomial
time on random instances. Furthermore, we point out in which
situations our algorithms get trapped in local optima and
show where the use of the right amount of heuristic
information is provably beneficial.

[719]

Lars Kotthoff, Chris Thornton, Holger H. Hoos, Frank Hutter, and Kevin
LeytonBrown.
AutoWEKA 2.0: Automatic model selection and hyperparameter
optimization in WEKA.
Journal of Machine Learning Research, 17:1–5, 2016.
[ bib ]

[720]

Katharina Kowalski, Sigrid Stagl, Reinhard Madlener, and Ines Omann.
Sustainable energy futures: Methodological challenges in
combining scenarios and participatory multicriteria analysis.
European Journal of Operational Research, 197(3):1063–1074,
2009.
[ bib ]

[721]

Oliver Kramer.
Iterated Local Search with Powell's Method: A Memetic
Algorithm for Continuous Global Optimization.
Memetic Computing, 2(1):69–83, 2010.
[ bib 
DOI ]

[722]

Daniel Krajzewicz, Jakob Erdmann, Michael Behrisch, and Laura Bieker.
Recent development and applications of SUMO  Simulation of
Urban MObility.
International Journal On Advances in Systems and Measurements,
5(34):128–138, 2012.
[ bib ]

[723]

S. Kreipl.
A Large Step Random Walk for Minimizing Total Weighted Tardiness
in a Job Shop.
Journal of Scheduling, 3(3):125–138, 2000.
[ bib ]

[724]

Stefanie Kritzinger, Fabien Tricoire, Karl F. Doerner, Richard F. Hartl, and
Thomas Stützle.
A Unified Framework for Routing Problems with a Fixed Fleet
Size.
International Journal of Metaheuristics, 6(3):160–209, 2017.
[ bib ]

[725]

Joseph B Kruskal.
On the shortest spanning subtree of a graph and the traveling
salesman problem.
Proceedings of the American Mathematical Society, 7(1):48–50,
1956.
[ bib ]

[726]

J Kuhpfahl and Christian Bierwirth.
A Study on Local Search Neighborhoods for the Job Shop
Scheduling Problem with Total Weighted Tardiness Objective.
Computers & Operations Research, 66:44–57, 2016.
[ bib ]

[727]

Tobias Kuhn, Carlos M. Fonseca, Luís Paquete, Stefan Ruzika, Miguel M.
Duarte, and José Rui Figueira.
Hypervolume subset selection in two dimensions: Formulations and
algorithms.
Evolutionary Computation, 24(3):411–425, 2016.
[ bib ]

[728]

Harold W. Kuhn.
The hungarian method for the assignment problem.
Naval Research Logistics Quarterly, 2(1–2):83–97, 1955.
[ bib ]

[729]

Max Kuhn.
Building Predictive Models in R Using the
caret Package.
Journal of Statistical Software, 28(5):1–26, 2008.
[ bib ]

[730]

R. Kumar and P. K. Singh.
Pareto Evolutionary Algorithm Hybridized with Local Search for
Biobjective TSP.
Studies in Computational Intelligence, 75:361–398, 2007.
[ bib ]

[731]

H. T. Kung, F. Luccio, and F. P. Preparata.
On Finding the Maxima of a Set of Vectors.
Journal of the ACM, 22(4):469–476, 1975.
[ bib ]

[732]

I. Kurtulus and E. W. Davis.
MultiProject Scheduling: Categorization of Heuristic Rules
Performance.
Management Science, 28(2):161–172, 1982.
[ bib 
DOI ]
Application of heuristic solution procedures to the
practical problem of project scheduling has
previously been studied by numerous
researchers. However, there is little consensus
about their findings, and the practicing manager is
currently at a loss as to which scheduling rule to
use. Furthermore, since no categorization process
was developed, it is assumed that once a rule is
selected it must be used throughout the whole
project. This research breaks away from this
tradition by providing a categorization process
based on two powerful project summary measures. The
first measure identifies the location of the peak of
total resource requirements and the second measure
identifies the rate of utilization of each resource
type. The performance of the rules are classified
according to values of these two measures, and it is
shown that a rule introduced by this research
performs significantly better on most categories of
projects.
Keywords: project management, research and development

[733]

H. J. Kushner.
A New Method of Locating the Maximum Point of an Arbitrary
Multipeak Curve in the Presence of Noise.
Journal of Basic Engineering, 86(1):97–106, March 1964.
[ bib 
DOI 
epub ]
A versatile and practical method of searching a parameter
space is presented. Theoretical and experimental results
illustrate the usefulness of the method for such problems as
the experimental optimization of the performance of a system
with a very general multipeak performance function when the
only available information is noisedistributed samples of
the function. At present, its usefulness is restricted to
optimization with respect to one system parameter. The
observations are taken sequentially; but, as opposed to the
gradient method, the observation may be located anywhere on
the parameter interval. A sequence of estimates of the
location of the curve maximum is generated. The location of
the next observation may be interpreted as the location of
the most likely competitor (with the current best estimate)
for the location of the curve maximum. A Brownian motion
stochastic process is selected as a model for the unknown
function, and the observations are interpreted with respect
to the model. The model gives the results a simple intuitive
interpretation and allows the use of simple but efficient
sampling procedures. The resulting process possesses some
powerful convergence properties in the presence of noise; it
is nonparametric and, despite its generality, is efficient in
the use of observations. The approach seems quite promising
as a solution to many of the problems of experimental system
optimization.

[734]

Jan H. Kwakkel.
The Exploratory Modeling Workbench: An open source toolkit for
exploratory modeling, scenario discovery, and (multiobjective) robust
decision making.
Environmental Modelling & Software, 96:239–250, 2017.
[ bib ]

[735]

Marco Laumanns, Lothar Thiele, Kalyanmoy Deb, and Eckart Zitzler.
Combining Convergence and Diversity in Evolutionary
Multiobjective Optimization.
Evolutionary Computation, 10(3):263–282, 2002.
[ bib 
DOI ]
Proposed εapprox and εPareto archivers
Keywords: archiving, εdominance, εapproximation,
εPareto

[736]

Antonio LaTorre, Santiago Muelas, and JoséMaría Peña.
A MOSbased dynamic memetic differential evolution algorithm
for continuous optimization: a scalability test.
Soft Computing, 15(11):2187–2199, 2011.
[ bib ]

[737]

Peter J. M. van Laarhoven, Emile H. L. Aarts, and Jan Karel Lenstra.
Job Shop Scheduling by Simulated Annealing.
Operations Research, 40(1):113–125, 1992.
[ bib ]

[738]

Martine Labbé, Patrice Marcotte, and Gilles Savard.
A Bilevel Model of Taxation and Its Application to Optimal
Highway Pricing.
Management Science, 44(12):1608–1622, 1998.
[ bib 
DOI ]

[739]

Martine Labbé and Alessia Violin.
Bilevel programming and price setting problems.
4OR: A Quarterly Journal of Operations Research, 11(1):1–30,
2013.
[ bib 
DOI ]

[740]

Benjamin Lacroix, Daniel Molina, and Francisco Herrera.
Region based memetic algorithm for realparameter optimisation.
Information Sciences, 262:15–31, 2014.
[ bib 
DOI ]
Keywords: irace

[741]

Manuel Laguna.
Editor's Note on the MIC 2013 Special Issue of the Journal of
Heuristics (Volume 22, Issue 4, August 2016).
Journal of Heuristics, 22(5):665–666, 2016.
[ bib ]

[742]

Xiangjing Lai and JinKao Hao.
Iterated Maxima Search for the Maximally Diverse Grouping
Problem.
European Journal of Operational Research, 254(3):780–800,
2016.
[ bib ]

[743]

A. H. Land and A. G. Doig.
An Automatic Method of Solving Discrete Programming Problems.
Econometrica, 28(3):497–520, 1960.
[ bib ]

[744]

William B. Langdon and M. Harman.
Optimising Software with Genetic Programming.
IEEE Transactions on Evolutionary Computation, 19(1):118–135,
2015.
[ bib ]

[745]

M. Lang, H. Kotthaus, P. Marwedel, C. Weihs, J. Rahnenführer, and Bernd
Bischl.
Automatic Model Selection for HighDimensional Survival
Analysis.
Journal of Statistical Computation and Simulation,
85(1):62–76, 2014.
[ bib 
DOI ]

[746]

A. Langevin, F. Soumis, and J. Desrosiers.
Classification of travelling salesman problem formulations.
Operations Research Letters, 9(2):127–132, 1990.
[ bib ]

[747]

A. Langevin, M. Desrochers, J. Desrosiers, Sylvie Gélinas, and F. Soumis.
A TwoCommodity Flow Formulation for the Traveling Salesman and
Makespan Problems with Time Windows.
Networks, 23(7):631–640, 1993.
[ bib ]

[748]

Kevin E. Lansey and K. Awumah.
Optimal Pump Operations Considering Pump Switches.
Journal of Water Resources Planning and Management, ASCE,
120(1):17–35, January / February 1994.
[ bib ]

[749]

Gilbert Laporte.
Fifty Years of Vehicle Routing.
Transportation Science, 43(4):408–416, 2009.
[ bib ]

[750]

Marco Laumanns.
Stochastic convergence of random search to fixed size Pareto
set approximations.
Arxiv preprint arXiv:0711.2949, 2007.
[ bib 
http ]

[751]

Benoît Laurent and JinKao Hao.
Iterated Local Search for the Multiple Depot Vehicle Scheduling
Problem.
Computers and Industrial Engineering, 57(1):277–286, 2009.
[ bib ]

[752]

Marco Laumanns, Lothar Thiele, and Eckart Zitzler.
Running time analysis of multiobjective evolutionary algorithms
on pseudoboolean functions.
IEEE Transactions on Evolutionary Computation, 8(2):170–182,
2004.
[ bib ]

[753]

Marco Laumanns, Lothar Thiele, and Eckart Zitzler.
Running time analysis of evolutionary algorithms on a simplified
multiobjective knapsack problem.
Natural Computing, 3(1):37–51, 2004.
[ bib ]

[754]

Marco Laumanns and Rico Zenklusen.
Stochastic convergence of random search methods to fixed size
Pareto front approximations.
European Journal of Operational Research, 213(2):414–421,
2011.
[ bib 
DOI ]

[755]

E. L. Lawler and D. E. Wood.
BranchandBound Methods: A Survey.
Operations Research, 14(4):699–719, 1966.
[ bib 
DOI ]

[756]

S. E. Lazic.
The problem of pseudoreplication in neuroscientific studies: is
it affecting your analysis?
BMC Neuroscience, 11(5):397–407, 2004.
[ bib 
DOI ]

[757]

Yann LeCun, Yoshua Bengio, et al.
Convolutional networks for images, speech, and time series.
The handbook of brain theory and neural networks,
3361(10):255–258, 1995.
[ bib ]

[758]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.
Deep learning.
Nature, 521(7553):436–444, 2015.
[ bib ]

[759]

Vinícius Leal do Forte, Flávio Marcelo Tavares Montenegro,
José André de Moura Brito, and Nelson Maculan.
Iterated Local Search Algorithms for the Euclidean Steiner
Tree Problem in n Dimensions.
International Transactions in Operational Research,
23(6):1185–1199, 2016.
[ bib ]

[760]

Per Kristian Lehre and Carsten Witt.
Blackbox search by unbiased variation.
Algorithmica, 64(4):623–642, 2012.
[ bib ]

[761]

Frank Thomson Leighton.
A Graph Coloring Algorithm for Large Scheduling Problems.
Journal of Research of the National Bureau of Standards,
84(6):489–506, 1979.
[ bib ]

[762]

Robert J. Lempert, David G. Groves, Steven W. Popper, and Steven C. Bankes.
A general analytic method for generating robust strategies and
narrative scenarios.
Management Science, 52(4):514–528, 2006.
[ bib ]

[763]

C. Leon, S. Martin, J. M. Elena, and J. Luque.
EXPLORE: Hybrid expert system for water networks management.
Journal of Water Resources Planning and Management, ASCE,
126(2):65–74, 2000.
[ bib ]

[764]

Leonid Levin.
Universal'nyie perebornyie zadachi.
Problemy Peredachi Informatsii, 9:265–266, 1973.
[ bib ]

[765]

Jianjun David Li.
A twostep rejection procedure for testing multiple hypotheses.
Journal of Statistical Planning and Inference,
138(6):1521–1527, 2008.
[ bib ]

[766]

Miqing Li.
Is Our Archiving Reliable? Multiobjective Archiving Methods on
“Simple” Artificial Input Sequences.
ACM Transactions on Evolutionary Learning and Optimization,
1(3):1–19, 2021.
[ bib 
DOI ]

[767]

Ke Li, Renzhi Chen, Guangtao Fu, and Xin Yao.
Twoarchive evolutionary algorithm for constrained
multiobjective optimization.
IEEE Transactions on Evolutionary Computation, 23(2):303–315,
2018.
[ bib ]

[768]

Miqing Li, Tao Chen, and Xin Yao.
How to evaluate solutions in Paretobased searchbased
software engineering? A critical review and methodological guidance.
IEEE Transactions on Software Engineering, 2020.
[ bib 
DOI ]

[769]

Miqing Li, Crina Grosan, Shengxiang Yang, Xiaohui Liu, and Xin Yao.
Multiline distance minimization: A visualized manyobjective
test problem suite.
IEEE Transactions on Evolutionary Computation, 22(1):61–78,
2018.
[ bib ]
highly degenerate Pareto fronts

[770]

Zhiyi Li, Mohammad Shahidehpour, Shay Bahramirad, and Amin Khodaei.
Optimizing Traffic Signal Settings in Smart Cities.
IEEE Transactions on Smart Grid, 3053(4):1–1, 2016.
[ bib 
DOI ]
Traffic signals play a critical role in smart cities for
mitigating traffic congestions and reducing the emission in
metropolitan areas. This paper proposes a bilevel
optimization framework to settle the optimal traffic signal
setting problem. The upperlevel problem determines the
traffic signal settings to minimize the drivers' average
travel time, while the lowerlevel problem aims for achieving
the network equilibrium using the settings calculated at the
upper level. Genetic algorithm is employed with the
integration of microscopictrafficsimulation based dynamic
traffic assignment (DTA) to decouple the complex bilevel
problem into tractable singlelevel problems which are solved
sequentially. Case studies on a synthetic traffic network and
a realworld traffic subnetwork are conducted to examine the
effectiveness of the proposed model and relevant solution
methods. Additional strategies are provided for the extension
of the proposed model and the acceleration solution process
in largearea traffic network applications.

[771]

Xiaoping Li, Long Chen, Haiyan Xu, and Jatinder N.D. Gupta.
Trajectory Scheduling Methods for Minimizing Total Tardiness in
a Flowshop.
Operations Research Perspectives, 2:13–23, 2015.
[ bib 
DOI ]

[772]

Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet
Talwalkar.
Hyperband: A Novel BanditBased Approach to Hyperparameter
Optimization.
Journal of Machine Learning Research, 18(185):1–52, 2018.
[ bib 
epub ]
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and earlystopping. We formulate hyperparameter optimization as a pureexploration nonstochastic infinitearmed bandit problem where a predefined resource like iterations, data samples, or features is allocated to randomly sampled configurations. We introduce a novel algorithm, our algorithm, for this framework and analyze its theoretical properties, providing several desirable guarantees. Furthermore, we compare our algorithm with popular Bayesian optimization methods on a suite of hyperparameter optimization problems. We observe that our algorithm can provide over an orderofmagnitude speedup over our competitor set on a variety of deeplearning and kernelbased learning problems.
Keywords: racing

[773]

Y. Li and W. Li.
Adaptive Ant Colony Optimization Algorithm Based on Information
Entropy: Foundation and Application.
Fundamenta Informaticae, 77(3):229–242, 2007.
[ bib ]

[774]

Bingdong Li, Jinlong Li, Ke Tang, and Xin Yao.
ManyObjective Evolutionary Algorithms: A Survey.
ACM Computing Surveys, 48(1):1–35, 2015.
[ bib 
DOI ]

[775]

Bingdong Li, Ke Tang, Jinlong Li, and Xin Yao.
Stochastic ranking algorithm for manyobjective optimization
based on multiple indicators.
IEEE Transactions on Evolutionary Computation, 20(6):924–938,
2016.
[ bib ]

[776]

Miqing Li, Shengxiang Yang, and Xiaohui Liu.
Shiftbased density estimation for Paretobased algorithms in
manyobjective optimization.
IEEE Transactions on Evolutionary Computation, 18(3):348–365,
2014.
[ bib ]
Proposed SDE indicator algorithm

[777]

Miqing Li, Shengxiang Yang, and Xiaohui Liu.
Pareto or nonPareto: Bicriterion evolution in
multiobjective optimization.
IEEE Transactions on Evolutionary Computation, 20(5):645–665,
2016.
[ bib ]

[778]

Miqing Li and Xin Yao.
Quality Evaluation of Solution Sets in Multiobjective
Optimisation: A Survey.
ACM Computing Surveys, 52(2):1–38, 2019.
[ bib 
DOI ]

[779]

Miqing Li and Xin Yao.
Dominance Move: A Measure of Comparing Solution Sets in
Multiobjective Optimization.
arXiv preprint arXiv:1702.00477, 2017.
[ bib ]

[780]

Miqing Li and Xin Yao.
What weights work for you? Adapting weights for any Pareto
front shape in decompositionbased evolutionary multiobjective optimisation.
Evolutionary Computation, 28(2):227–253, 2020.
[ bib ]

[781]

Hui Li and Qingfu Zhang.
Multiobjective Optimization Problems with Complicated Pareto
sets, MOEA/D and NSGAII.
IEEE Transactions on Evolutionary Computation, 13(2):284–302,
2009.
[ bib ]

[782]

Zhipan Li, Juan Zou, Shengxiang Yang, and Jinhua Zheng.
A twoarchive algorithm with decomposition and fitness
allocation for multimodal multiobjective optimization.
Information Sciences, 574:413–430, 2021.
[ bib ]

[783]

Tianjun Liao, Doǧan Aydın, and Thomas Stützle.
Artificial Bee Colonies for Continuous Optimization:
Experimental Analysis and Improvements.
Swarm Intelligence, 7(4):327–356, 2013.
[ bib ]

[784]

Tianjun Liao, Daniel Molina, Marco A. Montes de Oca, and Thomas Stützle.
A Note on the Effects of Enforcing Bound Constraints on
Algorithm Comparisons using the IEEE CEC'05 Benchmark Function Suite.
Evolutionary Computation, 22(2):351–359, 2014.
[ bib ]

[785]

Tianjun Liao, Daniel Molina, and Thomas Stützle.
Performance Evaluation of Automatically Tuned Continuous
Optimizers on Different Benchmark Sets.
Applied Soft Computing, 27:490–503, 2015.
[ bib ]

[786]

Tianjun Liao, Marco A. Montes de Oca, and Thomas Stützle.
Computational results for an automatically tuned CMAES with
increasing population size on the CEC'05 benchmark set.
Soft Computing, 17(6):1031–1046, 2013.
[ bib 
DOI ]

[787]

Tianjun Liao, Krzysztof Socha, Marco A. Montes de Oca, Thomas Stützle,
and Marco Dorigo.
Ant Colony Optimization for MixedVariable Optimization
Problems.
IEEE Transactions on Evolutionary Computation, 18(4):503–518,
2014.
[ bib ]
Keywords: ACOR

[788]

Tianjun Liao, Thomas Stützle, Marco A. Montes de Oca, and Marco Dorigo.
A Unified Ant Colony Optimization Algorithm for Continuous
Optimization.
European Journal of Operational Research, 234(3):597–609,
2014.
[ bib ]

[789]

C.J. Liao, C.T. Tseng, and P. Luarn.
A Discrete Version of Particle Swarm Optimization for Flowshop
Scheduling Problems.
Computers & Operations Research, 34(10):3099–3111, 2007.
[ bib ]

[790]

Arnaud Liefooghe, Fabio Daolio, Bilel Derbel, Sébastien Verel,
Hernán E. Aguirre, and Kiyoshi Tanaka.
LandscapeAware Performance Prediction for Evolutionary
Multiobjective Optimization.
IEEE Transactions on Evolutionary Computation,
24(6):1063–1077, 2020.
[ bib ]

[791]

Arnaud Liefooghe, Jérémie Humeau, Salma Mesmoudi, Laetitia Jourdan, and
ElGhazali Talbi.
On dominancebased multiobjective local search: design,
implementation and experimental analysis on scheduling and traveling salesman
problems.
Journal of Heuristics, 18(2):317–352, 2012.
[ bib 
DOI ]
This paper discusses simple local search approaches for
approximating the efficient set of multiobjective
combinatorial optimization problems. We focus on algorithms
defined by a neighborhood structure and a dominance relation
that iteratively improve an archive of nondominated
solutions. Such methods are referred to as dominancebased
multiobjective local search. We first provide a concise
overview of existing algorithms, and we propose a model
trying to unify them through a finegrained
decomposition. The main problemindependent search components
of dominance relation, solution selection, neighborhood
exploration and archiving are largely discussed. Then, a
number of stateoftheart and original strategies are
experimented on solving a permutation flowshop scheduling
problem and a traveling salesman problem, both on a two and
a threeobjective formulation. Experimental results and a
statistical comparison are reported in the paper, and some
directions for future research are highlighted.

[792]

Arnaud Liefooghe, Laetitia Jourdan, and ElGhazali Talbi.
A Software Framework Based on a Conceptual Unified Model for
Evolutionary Multiobjective Optimization: ParadisEOMOEO.
European Journal of Operational Research, 209(2):104–112,
2011.
[ bib ]

[793]

Bojan Likar and Juš Kocijan.
Predictive control of a gas–liquid separation plant based on a
Gaussian process model.
Computers & Chemical Engineering, 31(3):142–152, 2007.
[ bib 
DOI ]

[794]

Marius Thomas Lindauer, Holger H. Hoos, Frank Hutter, and Torsten Schaub.
AutoFolio: An Automatically Configured Algorithm Selector.
Journal of Artificial Intelligence Research, 53:745–778, 2015.
[ bib ]

[795]

S. Lin and B. W. Kernighan.
An Effective Heuristic Algorithm for the Traveling Salesman
Problem.
Operations Research, 21(2):498–516, 1973.
[ bib ]

[796]

Marius Thomas Lindauer, Jan N. van Rijn, and Lars Kotthoff.
The algorithm selection competitions 2015 and 2017.
Artificial Intelligence, 272:86–100, 2019.
[ bib ]

[797]

Andrei Lissovoi and Carsten Witt.
Runtime Analysis of Ant Colony Optimization on Dynamic Shortest
Path Problems.
Theoretical Computer Science, 561(Part A):73–85, 2015.
[ bib 
DOI ]
A simple ACO algorithm called λMMAS for dynamic
variants of the singledestination shortest paths problem is
studied by rigorous runtime analyses. Building upon previous
results for the special case of 1MMAS, it is studied to what
extent an enlarged colony using λ ants per vertex
helps in tracking an oscillating optimum. It is shown that
easy cases of oscillations can be tracked by a constant
number of ants. However, the paper also identifies more
involved oscillations that with overwhelming probability
cannot be tracked with any polynomialsize colony. Finally,
parameters of dynamic shortestpath problems which make the
optimum difficult to track are discussed. Experiments
illustrate theoretical findings and conjectures.

[798]

J. D. C. Little, K. G. Murty, D. W. Sweeney, and C. Karel.
An Algorithm for the Traveling Salesman Problem.
Operations Research, 11:972–989, 1963.
[ bib ]

[799]

Shusen Liu, Dan Maljovec, Bei Wang, PeerTimo Bremer, and Valerio Pascucci.
Visualizing HighDimensional Data: Advances in the Past Decade.
IEEE Transactions on Visualization and Computer Graphics,
23(3), 2017.
[ bib 
DOI ]

[800]

Jiyin Liu and Colin R. Reeves.
Constructive and Composite Heuristic Solutions to the
P//ΣCi Scheduling Problem.
European Journal of Operational Research, 132(2):439–452,
2001.
[ bib 
DOI ]

[801]

Yiping Liu, Gary G. Yen, and Dunwei Gong.
A multimodal multiobjective evolutionary algorithm using
twoarchive and recombination strategies.
IEEE Transactions on Evolutionary Computation, 23(4):660–674,
2018.
[ bib ]

[802]

Andrea Lodi, Silvano Martello, and Michele Monaci.
Twodimensional packing problems: A survey.
European Journal of Operational Research, 141(2):241–252,
2002.
[ bib 
DOI ]

[803]

Andrea Lodi, Silvano Martello, and Daniele Vigo.
Heuristic and metaheuristic approaches for a class of
twodimensional bin packing problems.
INFORMS Journal on Computing, 11(4):345–357, 1999.
[ bib 
DOI ]

[804]

Andrea Lodi, Silvano Martello, and Daniele Vigo.
TSpack: a unified tabu search code for multidimensional bin
packing problems.
Annals of Operations Research, 131(14):203–213, 2004.
[ bib 
DOI ]

[805]

Andrea Lodi and Giulia Zarpellon.
On Learning and Branching: A Survey.
TOP, 25:207–236, 2017.
[ bib ]

[806]

Jason D. Lohn, Gregory S. Hornby, and Derek S. Linden.
Humancompetitive Evolved Antennas.
Artificial Intelligence for Engineering Design, Analysis and
Manufacturing, 22(3):235–247, 2008.
[ bib 
DOI ]
Evolutionary optimization of antennas for NASA

[807]

Manuel LópezIbáñez and Christian Blum.
BeamACO for the travelling salesman problem with time
windows.
Computers & Operations Research, 37(9):1570–1583, 2010.
[ bib 
DOI ]
The travelling salesman problem with time windows is
a difficult optimization problem that arises, for
example, in logistics. This paper deals with the
minimization of the travelcost. For solving this
problem, this paper proposes a BeamACO algorithm,
which is a hybrid method combining ant colony
optimization with beam search. In general, BeamACO
algorithms heavily rely on accurate and
computationally inexpensive bounding information for
differentiating between partial solutions. This work
uses stochastic sampling as a useful alternative. An
extensive experimental evaluation on seven benchmark
sets from the literature shows that the proposed
BeamACO algorithm is currently a stateoftheart
technique for the travelling salesman problem with
time windows when travelcost optimization is
concerned.
Keywords: Ant colony optimization, Travelling salesman problem with
time windows, Hybridization

[808]

Manuel LópezIbáñez, Christian Blum, Jeffrey W. Ohlmann, and
Barrett W. Thomas.
The Travelling Salesman Problem with Time Windows: Adapting
Algorithms from Traveltime to Makespan Optimization.
Applied Soft Computing, 13(9):3806–3815, 2013.
[ bib 
DOI 
epub ]

[809]

Manuel LópezIbáñez, Jürgen Branke, and Luís Paquete.
Reproducibility in Evolutionary Computation.
Arxiv preprint arXiv:20102.03380 [cs.AI], 2021.
[ bib 
http ]
Experimental studies are prevalent in Evolutionary
Computation (EC), and concerns about the reproducibility and
replicability of such studies have increased in recent times,
reflecting similar concerns in other scientific fields. In
this article, we suggest a classification of different types
of reproducibility that refines the badge system of the
Association of Computing Machinery (ACM) adopted by TELO. We
discuss, within the context of EC, the different types of
reproducibility as well as the concepts of artifact and
measurement, which are crucial for claiming
reproducibility. We identify cultural and technical obstacles
to reproducibility in the EC field. Finally, we provide
guidelines and suggest tools that may help to overcome some
of these reproducibility obstacles.
Keywords: Evolutionary Computation, Reproducibility, Empirical study,
Benchmarking

[810]

Manuel LópezIbáñez, Jürgen Branke, and Luís Paquete.
Reproducibility in Evolutionary Computation.
ACM Transactions on Evolutionary Learning and Optimization,
1(4):1–21, 2021.
[ bib 
DOI 
epub ]
Experimental studies are prevalent in Evolutionary
Computation (EC), and concerns about the reproducibility and
replicability of such studies have increased in recent times,
reflecting similar concerns in other scientific fields. In
this article, we suggest a classification of different types
of reproducibility that refines the badge system of the
Association of Computing Machinery (ACM) adopted by TELO. We
discuss, within the context of EC, the different types of
reproducibility as well as the concepts of artifact and
measurement, which are crucial for claiming
reproducibility. We identify cultural and technical obstacles
to reproducibility in the EC field. Finally, we provide
guidelines and suggest tools that may help to overcome some
of these reproducibility obstacles.
Keywords: Evolutionary Computation, Reproducibility, Empirical study,
Benchmarking

[811]

Manuel LópezIbáñez, Jérémie DuboisLacoste, Leslie
Pérez Cáceres, Thomas Stützle, and Mauro Birattari.
The irace Package: Iterated Racing for Automatic
Algorithm Configuration.
Operations Research Perspectives, 3:43–58, 2016.
[ bib 
DOI 
supplementary material ]

[812]

Manuel LópezIbáñez, MarieEléonore Kessaci, and Thomas
Stützle.
Automatic Design of Hybrid Metaheuristics from Algorithmic
Components.
Submitted, 2017.
[ bib ]

[813]

Manuel LópezIbáñez, Luís Paquete, and Thomas Stützle.
Hybrid Populationbased Algorithms for the Biobjective
Quadratic Assignment Problem.
Journal of Mathematical Modelling and Algorithms,
5(1):111–137, 2006.
[ bib 
DOI ]
We present variants of an ant colony optimization
(MOACO) algorithm and of an evolutionary algorithm
(SPEA2) for tackling multiobjective combinatorial
optimization problems, hybridized with an iterative
improvement algorithm and the robust tabu search
algorithm. The performance of the resulting hybrid
stochastic local search (SLS) algorithms is
experimentally investigated for the biobjective
quadratic assignment problem (bQAP) and compared
against repeated applications of the underlying
local search algorithms for several
scalarizations. The experiments consider structured
and unstructured bQAP instances with various degrees
of correlation between the flow matrices. We do a
systematic experimental analysis of the algorithms
using outperformance relations and the attainment
functions methodology to asses differences in the
performance of the algorithms. The experimental
results show the usefulness of the hybrid algorithms
if the available computation time is not too limited
and identify SPEA2 hybridized with very short tabu
search runs as the most promising variant.

[814]

Manuel LópezIbáñez, Leslie Pérez Cáceres, and Thomas
Stützle.
irace: A Tool for the Automatic Configuration of Algorithms.
International Federation of Operational Research Societies
(IFORS) News, 14(2):30–32, June 2020.
[ bib 
http ]

[815]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Ant Colony Optimisation for the Optimal Control of Pumps in
Water Distribution Networks.
Journal of Water Resources Planning and Management, ASCE,
134(4):337–346, 2008.
[ bib 
DOI 
epub ]
Reducing energy consumption of water distribution
networks has never had more significance than today. The greatest
energy savings can be obtained by careful scheduling of operation of
pumps. Schedules can be defined either implicitly, in terms of other
elements of the network such as tank levels, or explicitly by
specifying the time during which each pump is on/off. The
traditional representation of explicit schedules is a string of
binary values with each bit representing pump on/off status during a
particular time interval. In this paper a new explicit
representation is presented. It is based on time controlled
triggers, where the maximum number of pump switches is specified
beforehand. In this representation a pump schedule is divided into a
series of integers with each integer representing the number of
hours for which a pump is active/inactive. This reduces the number
of potential schedules (search space) compared to the binary
representation. Ant colony optimization (ACO) is a stochastic
metaheuristic for combinatorial optimization problems that is
inspired by the foraging behavior of some species of ants. In this
paper, an application of the ACO framework was developed for the
optimal scheduling of pumps. The proposed representation was adapted
to an ant colony Optimization framework and solved for the optimal
pump schedules. Minimization of electrical cost was considered as
the objective, while satisfying system constraints. Instead of using
a penalty function approach for constraint violations, constraint
violations were ordered according to their importance and solutions
were ranked based on this order. The proposed approach was tested on
a small test network and on a large realworld network. Results are
compared with those obtained using a simple genetic algorithm based
on binary representation and a hybrid genetic algorithm that uses
levelbased triggers.

[816]

Manuel LópezIbáñez, T. Devi Prasad, and Ben Paechter.
Representations and Evolutionary Operators for the Scheduling of
Pump Operations in Water Distribution Networks.
Evolutionary Computation, 19(3):429–467, 2011.
[ bib 
DOI ]
Reducing the energy consumption of water
distribution networks has never had more
significance. The greatest energy savings can be
obtained by carefully scheduling the operations of
pumps. Schedules can be defined either implicitly,
in terms of other elements of the network such as
tank levels, or explicitly by specifying the time
during which each pump is on/off. The traditional
representation of explicit schedules is a string of
binary values with each bit representing pump on/off
status during a particular time interval. In this
paper, we formally define and analyze two new
explicit representations based on timecontrolled
triggers, where the maximum number of pump switches
is established beforehand and the schedule may
contain less switches than the maximum. In these
representations, a pump schedule is divided into a
series of integers with each integer representing
the number of hours for which a pump is
active/inactive. This reduces the number of
potential schedules compared to the binary
representation, and allows the algorithm to operate
on the feasible region of the search space. We
propose evolutionary operators for these two new
representations. The new representations and their
corresponding operations are compared with the two
mostused representations in pump scheduling,
namely, binary representation and levelcontrolled
triggers. A detailed statistical analysis of the
results indicates which parameters have the greatest
effect on the performance of evolutionary
algorithms. The empirical results show that an
evolutionary algorithm using the proposed
representations improves over the results obtained
by a recent stateoftheart Hybrid Genetic
Algorithm for pump scheduling using levelcontrolled
triggers.

[817]

Manuel LópezIbáñez and Thomas Stützle.
An experimental analysis of design choices of multiobjective
ant colony optimization algorithms.
Swarm Intelligence, 6(3):207–232, 2012.
[ bib 
DOI 
supplementary material ]

[818]

Manuel LópezIbáñez and Thomas Stützle.
The Automatic Design of MultiObjective Ant Colony Optimization
Algorithms.
IEEE Transactions on Evolutionary Computation, 16(6):861–875,
2012.
[ bib 
DOI ]
Multiobjective optimization problems are problems with several,
typically conflicting criteria for evaluating solutions. Without
any a priori preference information, the Pareto optimality
principle establishes a partial order among solutions, and the
output of the algorithm becomes a set of nondominated solutions
rather than a single one. Various ant colony optimization (ACO)
algorithms have been proposed in recent years for solving such
problems. These multiobjective ACO (MOACO) algorithms exhibit
different design choices for dealing with the particularities of
the multiobjective context. This paper proposes a formulation of
algorithmic components that suffices to describe most MOACO
algorithms proposed so far. This formulation also shows that
existing MOACO algorithms often share equivalent design choices
but they are described in different terms. Moreover, this
formulation is synthesized into a flexible algorithmic framework,
from which not only existing MOACO algorithms may be
instantiated, but also combinations of components that were never
studied in the literature. In this sense, this paper goes beyond
proposing a new MOACO algorithm, but it rather introduces a
family of MOACO algorithms. The flexibility of the proposed MOACO
framework facilitates the application of automatic algorithm
configuration techniques. The experimental results presented in
this paper show that the automatically configured MOACO framework
outperforms the MOACO algorithms that inspired the framework
itself. This paper is also among the first to apply automatic
algorithm configuration techniques to multiobjective algorithms.

[819]

Manuel LópezIbáñez and Thomas Stützle.
Automatically Improving the Anytime Behaviour of Optimisation
Algorithms.
European Journal of Operational Research, 235(3):569–582,
2014.
[ bib 
DOI 
supplementary material ]
Optimisation algorithms with good anytime behaviour try to
return as highquality solutions as possible independently of
the computation time allowed. Designing algorithms with good
anytime behaviour is a difficult task, because performance is
often evaluated subjectively, by plotting the tradeoff curve
between computation time and solution quality. Yet, the
tradeoff curve may be modelled also as a set of mutually
nondominated, biobjective points. Using this model, we
propose to combine an automatic configuration tool and the
hypervolume measure, which assigns a single quality measure
to a nondominated set. This allows us to improve the anytime
behaviour of optimisation algorithms by means of
automatically finding algorithmic configurations that produce
the best nondominated sets. Moreover, the recently proposed
weighted hypervolume measure is used here to incorporate the
decisionmaker's preferences into the automatic tuning
procedure. We report on the improvements reached when
applying the proposed method to two relevant scenarios: (i)
the design of parameter variation strategies for MAXMIN Ant
System and (ii) the tuning of the anytime behaviour of SCIP,
an opensource mixed integer programming solver with more
than 200 parameters.

[820]

Eunice LÃ³pezCamacho, Hugo TerashimaMarin, Peter Ross, and Gabriela Ochoa.
A unified hyperheuristic framework for solving bin packing
problems.
Expert Systems with Applications, 41(15):6876–6889, 2014.
[ bib 
DOI ]

[821]

Samir Loudni and Patrice Boizumault.
Combining VNS with constraint programming for solving anytime
optimization problems.
European Journal of Operational Research, 191:705–735, 2008.
[ bib 
DOI ]

[822]

Helena R. Lourenço.
JobShop Scheduling: Computational Study of Local Search and
LargeStep Optimization Methods.
European Journal of Operational Research, 83(2):347–364, 1995.
[ bib ]

[823]

Antonio Lova and Pilar Tormos.
Analysis of Scheduling Schemes and Heuristic Rules Performance
in ResourceConstrained Multiproject Scheduling.
Annals of Operations Research, 102(14):263–286, February
2001.
[ bib 
DOI ]
Frequently, the availability of resources assigned
to a project is limited and not sufficient to
execute all the concurrent activities. In this
situation, decision making about their schedule is
necessary. Many times this schedule supposes an
increase in the project completion
time. Additionally, companies commonly manage
various projects simultaneously, sharing a pool of
renewable resources. Given these resource
constraints, we often can only apply heuristic
methods to solve the scheduling problem. In this
work the effect of the schedule generation schemes 
serial or parallel  and priority rules  MINLFT,
MINSLK, MAXTWK, SASP or FCFS  with two
approaches  multiproject and singleproject  are
analysed. The time criteria considered are the mean
project delay and the multiproject duration
increase. Through an extensive computational study,
results show that with the parallel scheduling
generation scheme and the multiproject approach the
project manager can obtain a good multiproject
schedule with the time criterion selected:
minimising mean project delay or minimising
multiproject duration increase. New heuristics 
based on priority rules with a twophase approach 
that outperform classical ones are proposed to
minimise mean project delay with a multiproject
approach. Finally, the best heuristics analysed are
evaluated together with a representative sample of
commercial project management software.
Keywords: Combinatorics, heuristic based on priority rules,
Multiproject scheduling, Operation
Research/Decision Theory, Project management,
project management software, Resource allocation,
Theory of Computation

[824]

Antonio Lova, Pilar Tormos, Mariamar Cervantes, and Federico Barber.
An efficient hybrid genetic algorithm for scheduling projects
with resource constraints and multiple execution modes.
International Journal of Production Economics, 117(2):302–316,
2009.
[ bib 
DOI ]
Multimode Resource Constrained Project Scheduling
Problem (MRCPSP) aims at finding the start times
and execution modes for the activities of a project
that optimize a given objective function while
verifying a set of precedence and resource
constraints. In this paper, we focus on this problem
and develop a hybrid Genetic Algorithm (MMHGA) to
solve it. Its main contributions are the mode
assignment procedure, the fitness function and the
use of a very efficient improving method. Its
performance is demonstrated by extensive
computational results obtained on a set of standard
instances and against the best currently available
algorithms.
Keywords: genetic algorithm, multimode resourceconstrained
project scheduling

[825]

Manuel Lozano, Fred Glover, Carlos GarcíaMartínez, Francisco J.
Rodríguez, and Rafael Martí.
Tabu Search with Strategic Oscillation for the Quadratic Minimum
Spanning Tree.
IIE Transactions, 46(4):414–428, 2014.
[ bib ]

[826]

Manuel Lozano, Daniel Molina, and Carlos GarcíaMartínez.
Iterated Greedy for the Maximum Diversity Problem.
European Journal of Operational Research, 214(1):31–38, 2011.
[ bib ]

[827]

Zhipeng Lü, Fred Glover, and JinKao Hao.
A hybrid metaheuristic approach to solving the UBQP problem.
European Journal of Operational Research, 207(3):1254–1262,
2010.
[ bib 
DOI ]

[828]

Andrew Lucas.
Ising formulations of many NP problems.
Frontiers in Physics, 2:5, 2014.
[ bib 
DOI ]

[829]

M. Lundy and A. Mees.
Convergence of an Annealing Algorithm.
Mathematical Programming, 34(1):111–124, 1986.
[ bib ]

[830]

Thibaut Lust and Jacques Teghem.
Twophase Pareto local search for the biobjective traveling
salesman problem.
Journal of Heuristics, 16(3):475–510, 2010.
[ bib 
DOI ]
In this work, we present a method, called TwoPhase
Pareto Local Search, to find a good approximation of the
efficient set of the biobjective traveling salesman
problem. In the first phase of the method, an initial
population composed of a good approximation of the extreme
supported efficient solutions is generated. We use as second
phase a Pareto Local Search method applied to each solution
of the initial population. We show that using the combination
of these two techniques: good initial population generation
plus Pareto Local Search gives better results than
stateoftheart algorithms. Two other points are introduced:
the notion of ideal set and a simple way to produce
nearefficient solutions of multiobjective problems, by using
an efficient singleobjective solver with a data perturbation
technique.

[831]

Thibaut Lust and Jacques Teghem.
The multiobjective multidimensional knapsack problem: a survey
and a new approach.
Arxiv preprint arXiv:1007.4063, 2010.
Published as [832].
[ bib ]

[832]

Thibaut Lust and Jacques Teghem.
The multiobjective multidimensional knapsack problem: a survey
and a new approach.
International Transactions in Operational Research,
19(4):495–520, 2012.
[ bib 
DOI ]

[833]

Thibaut Lust and Andrzej Jaszkiewicz.
Speedup techniques for solving largescale biobjective TSP.
Computers & Operations Research, 37(3):521–533, 2010.
[ bib 
DOI ]
Keywords: Multiobjective combinatorial optimization, Hybrid
metaheuristics, TSP, Local search, Speedup techniques

[834]

C. von Lücken, Benjamín Barán, and Carlos Brizuela.
A survey on multiobjective evolutionary algorithms for
manyobjective problems.
Computational Optimization and Applications, 58(3):707–756,
2014.
[ bib ]

[835]

Laurens van der Maaten and Geoffrey Hinton.
Visualizing Data using tSNE.
Journal of Machine Learning Research, 9(86):2579–2605, 2008.
[ bib 
epub ]

[836]

Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew
Hausknecht, and Michael Bowling.
Revisiting the Arcade Learning Environment: Evaluation
Protocols and Open Problems for General Agents.
Journal of Artificial Intelligence Research, 61(1):523–562,
January 2018.
[ bib ]
The Arcade Learning Environment (ALE) is an evaluation
platform that poses the challenge of building AI agents with
general competency across dozens of Atari 2600 games. It
supports a variety of different problem settings and it has
been receiving increasing attention from the scientific
community, leading to some highpro_le success stories such
as the much publicized Deep QNetworks (DQN). In this article
we take a big picture look at how the ALE is being used by
the research community. We show how diverse the evaluation
methodologies in the ALE have become with time, and highlight
some key concerns when evaluating agents in the ALE. We use
this discussion to present some methodological best practices
and provide new benchmark results using these best
practices. To further the progress in the field, we introduce
a new version of the ALE that supports multiple game modes
and provides a form of stochasticity we call sticky
actions. We conclude this big picture look by revisiting
challenges posed when the ALE was introduced, summarizing the
stateoftheart in various problems and highlighting
problems that remain open.

[837]

Sam Madden.
From Databases to Big Data.
IEEE Internet Computing, 16(3), 2012.
[ bib ]

[838]

M. Mahdavi, M. Fesanghary, and E. Damangir.
An improved harmony search algorithm for solving optimization
problems.
Applied Mathematics and Computation, 188(2):1567–1579, 2007.
[ bib 
DOI ]
This paper develops an Improved harmony search (IHS)
algorithm for solving optimization problems. IHS employs a
novel method for generating new solution vectors that
enhances accuracy and convergence rate of harmony search (HS)
algorithm. In this paper the impacts of constant parameters
on harmony search algorithm are discussed and a strategy for
tuning these parameters is presented. The IHS algorithm has
been successfully applied to various benchmarking and
standard engineering optimization problems. Numerical results
reveal that the proposed algorithm can find better solutions
when compared to HS and other heuristic or deterministic
methods and is a powerful search algorithm for various
engineering optimization problems.
Keywords: Global optimization, Heuristics, Harmony search algorithm,
Mathematical programming

[839]

Guilherme B. Mainieri and Débora P. Ronconi.
New heuristics for total tardiness minimization in a flexible
flowshop.
Optimization Letters, pp. 1–20, 2012.
[ bib ]

[840]

Holger R. Maier, Angus R. Simpson, Aaron C. Zecchin, Wai Kuan Foong, Kuang Yeow
Phang, Hsin Yeow Seah, and Chan Lim Tan.
Ant Colony Optimization for Design of Water Distribution
Systems.
Journal of Water Resources Planning and Management, ASCE,
129(3):200–209, May / June 2003.
[ bib ]

[841]

Sri Srinivasa Raju M, Rammohan Mallipeddi, and Kedar Nath Das.
A twinarchive guided decomposition based multi/manyobjective
evolutionary algorithm.
Swarm and Evolutionary Computation, 71:101082, 2022.
[ bib 
DOI ]

[842]

Katherine M. Malan and Andries Engelbrecht.
A survey of techniques for characterising fitness landscapes and
some possible ways forward.
Information Sciences, 241:148–163, 2013.
[ bib 
DOI ]

[843]

R. M. Males, R. M. Clark, P. J. Wehrman, and W. E. Gateset.
Algorithm for mixing problems in water systems.
Journal of Hydraulic Engineering, ASCE, 111(2):206–219,
1985.
[ bib ]

[844]

Vittorio Maniezzo.
Exact and Approximate Nondeterministic TreeSearch Procedures
for the Quadratic Assignment Problem.
INFORMS Journal on Computing, 11(4):358–369, 1999.
[ bib ]

[845]

Vittorio Maniezzo and A. Carbonaro.
An ANTS Heuristic for the Frequency Assignment Problem.
Future Generation Computer Systems, 16(8):927–935, 2000.
[ bib ]

[846]

Vittorio Maniezzo and Alberto Colorni.
The Ant System Applied to the Quadratic Assignment Problem.
IEEE Transactions on Knowledge and Data Engineering,
11(5):769–778, 1999.
[ bib ]

[847]

E. Q. V. Martins.
On a multicritera shortest path problem.
European Journal of Operational Research, 16:236–245, 1984.
[ bib ]

[848]

R. T. Marler and J. S. Arora.
Survey of multiobjective optimization methods for engineering.
Structural and Multidisciplinary Optimization, 26(6):369–395,
April 2004.
[ bib 
DOI ]
Discusses a priori (scalarized) methods.

[849]

D. Martens, M. De Backer, R. Haesen, J. Vanthienen, M. Snoeck, and B. Baesens.
Classification With Ant Colony Optimization.
IEEE Transactions on Evolutionary Computation, 11(5):651–665,
2007.
[ bib ]

[850]

Hugues Marchand, Alexander Martin, Robert Weismantel, and Laurence Wolsey.
Cutting planes in integer and mixed integer programming.
Discrete Applied Mathematics, 123(1–3):397–446, 2002.
[ bib ]

[851]

O. Maron and A. W. Moore.
The Racing Algorithm: Model Selection for Lazy Learners.
Artificial Intelligence Research, 11(1–5):193–225, 1997.
[ bib ]

[852]

Olivier Martin and S. W. Otto.
Partitioning of Unstructured Meshes for Load Balancing.
Concurrency: Practice and Experience, 7(4):303–314, 1995.
[ bib ]

[853]

Olivier Martin and S. W. Otto.
Combining Simulated Annealing with Local Search Heuristics.
Annals of Operations Research, 63:57–75, 1996.
[ bib ]

[854]

Olivier Martin, S. W. Otto, and E. W. Felten.
LargeStep Markov Chains for the Traveling Salesman Problem.
Complex Systems, 5(3):299–326, 1991.
[ bib ]

[855]

Olivier Martin, S. W. Otto, and E. W. Felten.
Largestep Markov Chains for the TSP Incorporating Local
Search Heuristics.
Operations Research Letters, 11(4):219–224, 1992.
[ bib ]

[856]

Rafael Martí, Gerhard Reinelt, and Abraham Duarte.
A Benchmark Library and a Comparison of Heuristic Methods for
the Linear Ordering Problem.
Computational Optimization and Applications, 51(3):1297–1317,
2012.
[ bib ]

[857]

Silvano Martello and Paolo Toth.
Lower bounds and reduction procedures for the bin packing
problem.
Discrete Applied Mathematics, 28(1):59–70, 1990.
[ bib 
DOI ]

[858]

Silvano Martello and Daniele Vigo.
Exact solution of the twodimensional finite bin packing
problem.
Management Science, 44(3):388–399, 1998.
[ bib 
DOI ]

[859]

Franco Mascia, Manuel LópezIbáñez, Jérémie DuboisLacoste,
and Thomas Stützle.
GrammarBased Generation of Stochastic Local Search Heuristics
through Automatic Algorithm Configuration Tools.
Computers & Operations Research, 51:190–199, 2014.
[ bib 
DOI ]

[860]

Franco Mascia, Paola Pellegrini, Thomas Stützle, and Mauro Birattari.
An Analysis of Parameter Adaptation in Reactive Tabu Search.
International Transactions in Operational Research,
21(1):127–152, 2014.
[ bib ]

[861]

Renaud Masson, Thibaut Vidal, Julien Michallet, Puca Huachi Vaz Penna,
Vinicius Petrucci, Anand Subramanian, and Hugues Dubedout.
An Iterated Local Search Heuristic for Multicapacity Bin
Packing and Machine Reassignment Problems.
Expert Systems with Applications, 40(13):5266–5275, 2013.
[ bib ]

[862]

Yazid Mati, Stéphane DauzèrePèrés, and Chams Lahlou.
A General Approach for Optimizing Regular Criteria in the
Jobshop Scheduling Problem.
European Journal of Operational Research, 212(1):33–42, 2011.
[ bib ]

[863]

Ross M. McConnell, Kurt Mehlhorn, Stefan Näher, and Pascal Schweitzer.
Certifying algorithms.
Computer Science Review, 5(2):119–161, 2011.
[ bib 
DOI ]
A certifying algorithm is an algorithm that produces, with
each output, a certificate or witness (easytoverify proof)
that the particular output has not been compromised by a
bug. A user of a certifying algorithm inputs x, receives the
output y and the certificate w, and then checks, either
manually or by use of a program, that w proves that y is a
correct output for input x. In this way, he/she can be sure
of the correctness of the output without having to trust the
algorithm. We put forward the thesis that certifying
algorithms are much superior to noncertifying algorithms,
and that for complex algorithmic tasks, only certifying
algorithms are satisfactory. Acceptance of this thesis would
lead to a change of how algorithms are taught and how
algorithms are researched. The widespread use of certifying
algorithms would greatly enhance the reliability of
algorithmic software. We survey the state of the art in
certifying algorithms and add to it. In particular, we start
a theory of certifying algorithms and prove that the concept
is universal.
Keywords: Algorithms, Software reliability, Certification

[864]

G. McCormick and R. S. Powell.
Optimal Pump Scheduling in Water Supply Systems with Maximum
Demand Charges.
Journal of Water Resources Planning and Management, ASCE,
129(5):372–379, 2003.
[ bib 
DOI 
epub ]
Keywords: water supply; pumps; Markov processes; cost optimal
control

[865]

G. McCormick and R. S. Powell.
Derivation of nearoptimal pump schedules for water distribution
by simulated annealing.
Journal of the Operational Research Society, 55(7):728–736,
July 2004.
[ bib 
DOI ]
The scheduling of pumps for clean water distribution
is a partially discrete nonlinear problem with many
variables. The scheduling method described in this
paper typically produces costs within 1% of a
linear programbased solution, and can incorporate
realistic nonlinear costs that may be hard to
incorporate in linear programming
formulations. These costs include pump switching and
maximum demand charges. A simplified model is
derived from a standard hydraulic simulator. An
initial schedule is produced by a descent
method. Twostage simulated annealing then produces
solutions in a few minutes. Iterative recalibration
ensures that the solution agrees closely with the
results from a full hydraulic simulation.

[866]

James McDermott.
When and Why Metaheuristics Researchers can Ignore "No Free
Lunch" Theorems.
SN Computer Science, 1(60):1–18, 2020.
[ bib 
DOI ]

[867]

Catherine C. McGeoch.
Analyzing Algorithms by Simulation: Variance Reduction
Techniques and Simulation Speedups.
ACM Computing Surveys, 24(2):195–212, 1992.
[ bib 
DOI ]
Although experimental studies have been widely applied to the
investigation of algorithm performance, very little attention
has been given to experimental method in this area. This is
unfortunate, since much can be done to improve the quality of
the data obtained; often, much improvement may be needed for
the data to be useful. This paper gives a tutorial discussion
of two aspects of good experimental technique: the use of
variance reduction techniques and simulation speedups in
algorithm studies. In an illustrative study, application of
variance reduction techniques produces a decrease in variance
by a factor 1000 in one case, giving a dramatic improvement
in the precision of experimental results. Furthermore, the
complexity of the simulation program is improved from
Θ(m n/H_{n}) to Θ(m + n log n) (where m is
typically much larger than n), giving a much faster
simulation program and therefore more data per unit of
computation time. The general application of variance
reduction techniques is also discussed for a variety of
algorithm problem domains.
Keywords: experimental analysis of algorithms, movetofront rule,
selforganizing sequential search, statistical analysis of
algorithms, transpose rule, variance reduction techniques

[868]

Catherine C. McGeoch.
Toward an Experimental Method for Algorithm Simulation.
INFORMS Journal on Computing, 8(1):1–15, 1996.
[ bib 
DOI ]

[869]

Michael D. McKay, Richard J. Beckman, and W. J. Conover.
A Comparison of Three Methods for Selecting Values of Input
Variables in the Analysis of Output from a Computer Code.
Technometrics, 21(2):239–245, 1979.
[ bib 
DOI ]
Two types of sampling plans are examined as alternatives to
simple random sampling in Monte Carlo studies. These plans
are shown to be improvements over simple random sampling with
respect to variance for a class of estimators which includes
the sample mean and the empirical distribution function.

[870]

Russell McKenna, Valentin Bertsch, Kai Mainzer, and Wolf Fichtner.
Combining local preferences with multicriteria decision
analysis and linear optimization to develop feasible energy concepts in small
communities.
European Journal of Operational Research, 268(3):1092–1110,
2018.
[ bib ]

[871]

Robert I. Mckay, Nguyen Xuan Hoai, Peter Alexander Whigham, Yin Shan, and
Michael O'Neill.
Grammarbased Genetic Programming: A Survey.
Genetic Programming and Evolvable Machines, 11(34):365–396,
September 2010.
[ bib 
DOI ]

[872]

Klaus Meer.
Simulated annealing versus Metropolis for a TSP instance.
Information Processing Letters, 104(6):216–219, 2007.
[ bib ]

[873]

Gábor Melis, Chris Dyer, and Phil Blunsom.
On the State of the Art of Evaluation in Neural Language
Models.
Arxiv preprint arXiv:1807.02811, 2017.
[ bib 
http ]

[874]

Ole J. Mengshoel.
Understanding the role of noise in stochastic local search:
Analysis and experiments.
Artificial Intelligence, 172(8):955–990, 2008.
[ bib ]

[875]

JuanJulián Merelo and Carlos Cotta.
Building bridges: the role of subfields in metaheuristics.
SIGEVOlution, 1(4):9–15, 2006.
[ bib 
DOI ]

[876]

Peter Merz and Bernd Freisleben.
Memetic Algorithms for the Traveling Salesman Problem.
Complex Systems, 13(4):297–345, 2001.
[ bib ]

[877]

Peter Merz and Bernd Freisleben.
Fitness Landscape Analysis and Memetic Algorithms for the
Quadratic Assignment Problem.
IEEE Transactions on Evolutionary Computation, 4(4):337–352,
2000.
[ bib ]

[878]

Peter Merz and Kengo Katayama.
Memetic algorithms for the unconstrained binary quadratic
programming problem.
BioSystems, 78(1):99–118, 2004.
[ bib 
DOI ]

[879]

D. Merkle and Martin Middendorf.
Ant Colony Optimization with Global Pheromone Evaluation for
Scheduling a Single Machine.
Applied Intelligence, 18(1):105–111, 2003.
[ bib ]

[880]

D. Merkle and Martin Middendorf.
Modeling the Dynamics of Ant Colony Optimization.
Evolutionary Computation, 10(3):235–262, 2002.
[ bib ]

[881]

D. Merkle, Martin Middendorf, and Hartmut Schmeck.
Ant Colony Optimization for ResourceConstrained Project
Scheduling.
IEEE Transactions on Evolutionary Computation, 6(4):333–346,
2002.
[ bib ]

[882]

Peter Merz and Bernd Freisleben.
Greedy and Local Search Heuristics for Unconstrained Binary
Quadratic Programming.
Journal of Heuristics, 8(2):197–213, 2002.
[ bib 
DOI ]

[883]

Rafael G. Mesquita, Ricardo M. A. Silva, Carlos A. B. Mello, and Péricles
B. C. Miranda.
Parameter tuning for document image binarization using a racing
algorithm.
Expert Systems with Applications, 42(5):2593–2603, 2015.
[ bib 
DOI ]
Keywords: irace

[884]

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. Teller, and E. Teller.
Equation of State Calculations by Fast Computing Machines.
Journal of Chemical Physics, 21:1087–1092, 1953.
[ bib ]

[885]

Nicolas Meuleau and Marco Dorigo.
Ant Colony Optimization and Stochastic Gradient Descent.
Artificial Life, 8(2):103–121, 2002.
[ bib ]

[886]

Laurent Meunier, Herilalaina Rakotoarison, PakKan Wong, Baptiste
Rozière, Jérémy Rapin, Olivier Teytaud, Antoine Moreau, and
Carola Doerr.
BlackBox Optimization Revisited: Improving Algorithm Selection
Wizards through Massive Benchmarking.
Arxiv preprint arXiv:2010.04542, 2020.
[ bib 
DOI ]
Keywords: Nevergrad, NGOpt

[887]

Laurent Meunier, Herilalaina Rakotoarison, PakKan Wong, Baptiste
Rozière, Jérémy Rapin, Olivier Teytaud, Antoine Moreau, and
Carola Doerr.
BlackBox Optimization Revisited: Improving Algorithm Selection
Wizards Through Massive Benchmarking.
IEEE Transactions on Evolutionary Computation, 26(3):490–500,
2022.
[ bib 
DOI ]
Keywords: nevergrad, NGOpt

[888]

R. M'Hallah.
An iterated local search variable neighborhood descent hybrid
heuristic for the total earliness tardiness permutation flow shop.
International Journal of Production Research,
52(13):3802–3819, 2014.
[ bib ]

[889]

Zbigniew Michalewicz, Dipankar Dasgupta, Rodolphe G. Le Riche, and Marc
Schoenauer.
Evolutionary algorithms for constrained engineering problems.
Computers and Industrial Engineering, 30(4):851–870, 1996.
[ bib 
DOI ]

[890]

Julien Michallet, Christian Prins, Farouk Yalaoui, and Grégoire Vitry.
Multistart Iterated Local Search for the Periodic Vehicle
Routing Problem with Time Windows and Time Spread Constraints on Services.
Computers & Operations Research, 41:196–207, 2014.
[ bib ]

[891]

Kaisa Miettinen.
Survey of methods to visualize alternatives in multiple criteria
decision making problems.
OR Spectrum, 36(1):3–37, 2014.
[ bib 
DOI ]

[892]

Kaisa Miettinen, Jyri Mustajoki, and T. J. Stewart.
Interactive multiobjective optimization with NIMBUS for
decision making under uncertainty.
OR Spectrum, 36(1):39–56, 2014.
[ bib ]

[893]

R. B. Millar and M. J. Anderson.
Remedies for pseudoreplication.
Fisheries Research, 70(2–3):397–407, 2004.
[ bib 
DOI ]

[894]

George A. Miller.
The magical number seven, plus or minus two: Some limits on our
capacity for processing information.
Psychological Review, 63(2):81–97, 1956.
[ bib 
DOI ]

[895]

Steven Minton.
Automatically configuring constraint satisfaction programs: A
case study.
Constraints, 1(1):7–43, 1996.
[ bib 
DOI ]

[896]

Gerardo Minella, Rubén Ruiz, and M. Ciavotta.
A Review and Evaluation of Multiobjective Algorithms for the
Flowshop Scheduling Problem.
INFORMS Journal on Computing, 20(3):451–471, 2008.
[ bib ]

[897]

Giovanni Misitano, Bekir Afsar, Giomara Larraga, and Kaisa Miettinen.
Towards explainable interactive multiobjective optimization:
RXIMO.
Autonomous Agents and MultiAgent Systems, 36(42), 2022.
[ bib 
DOI ]

[898]

Alfonsas Misevičius and Dovilė Kuznecovaitė.
Investigating some strategies for construction of initial
populations in genetic algorithms.
Computational Science and Techniques, 5(1):560–573, 2018.
[ bib ]

[899]

Alfonsas Misevičius.
Genetic Algorithm Hybridized with Ruin and Recreate Procedure:
Application to the Quadratic Assignment Problem.
KnowledgeBased Systems, 16(5–6):261–268, 2003.
[ bib ]

[900]

Alfonsas Misevičius.
A modified simulated annealing algorithm for the quadratic
assignment problem.
Informatica, 14(4):497–514, 2003.
[ bib ]

[901]

P. Mitra, C. A. Murthy, and S. K. Pal.
Unsupervised feature selection using feature similarity.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
24(3):301–312, 2002.
[ bib 
DOI ]

[902]

Alfonsas Misevičius, Dovilė Kuznecovaitė, and Jūratė
Platužienė.
Some Further Experiments with Crossover Operators for Genetic
Algorithms.
Informatica, 29(3):499–516, 2018.
[ bib ]

[903]

Nenad Mladenović and Pierre Hansen.
Variable Neighborhood Search.
Computers & Operations Research, 24(11):1097–1100, 1997.
[ bib ]

[904]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness,
Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland,
Georg Ostrovski, et al.
Humanlevel control through deep reinforcement learning.
Nature, 518(7540):529, 2015.
[ bib ]

[905]

Jonas Močkus, Vytautas Tiesis, and Antanas Zilinskas.
The application of bayesian methods for seeking the extremum.
Towards global optimization, pp. 117–129, 1978.
[ bib ]
Proposed Bayesian optimization (but later than [2177])

[906]

Julián Molina, Luis V. Santana, Alfredo G. HernándezDíaz,
Carlos A. Coello Coello, and Rafael Caballero.
gDominance: Reference point based dominance for
Multiobjective Metaheuristics.
European Journal of Operational Research, 197(2):685–692,
September 2009.
[ bib 
DOI ]
Proposed gNSGAII

[907]

Marco A. Montes de Oca, Doǧan Aydın, and Thomas Stützle.
An Incremental Particle Swarm for LargeScale Continuous
Optimization Problems: An Example of Tuningintheloop (Re)Design of
Optimization Algorithms.
Soft Computing, 15(11):2233–2255, 2011.
[ bib 
DOI ]

[908]

Alysson Mondoro, Dan M. Frangopol, and Liang Liu.
Multicriteria robust optimization framework for bridge
adaptation under climate change.
Structural Safety, 74:14–23, 2018.
[ bib ]

[909]

Roberto Montemanni, L. M. Gambardella, A. E. Rizzoli, and A. V. Donati.
Ant colony system for a dynamic vehicle routing problem.
Journal of Combinatorial Optimization, 10:327–343, 2005.
[ bib ]

[910]

James Montgomery, Marcus Randall, and Tim Hendtlass.
Solution bias in ant colony optimisation: Lessons for
selecting pheromone models.
Computers & Operations Research, 35(9):2728–2749, 2008.
[ bib 
DOI ]

[911]

Elizabeth Montero, MaríaCristina Riff, and Bertrand Neveu.
A Beginner's Buide to Tuning Methods.
Applied Soft Computing, 17:39–51, 2014.
[ bib 
DOI ]

[912]

Marco A. Montes de Oca, Thomas Stützle, Mauro Birattari, and Marco
Dorigo.
Frankenstein's PSO: A Composite Particle Swarm Optimization
Algorithm.
IEEE Transactions on Evolutionary Computation,
13(5):1120–1132, 2009.
[ bib 
DOI ]

[913]

Nicolas Monmarché, G. Venturini, and M. Slimane.
On how pachycondyla apicalis ants suggest a new search
algorithm.
Future Generation Computer Systems, 16(8):937–946, 2000.
[ bib ]

[914]

Peter D. Morgan.
Simulation of an adaptive behavior mechanism in an expert
decisionmaker.
IEEE Transactions on Systems, Man, and Cybernetics,
23(1):65–76, 1993.
[ bib ]

[915]

J. N. Morse.
Reducing the size of the nondominated set: Pruning by
clustering.
Computers & Operations Research, 7(12):55–66, 1980.
[ bib ]

[916]

Mouad Morabit, Guy Desaulniers, and Andrea Lodi.
Machinelearning–based column selection for column generation.
Transportation Science, 55(4):815–831, 2021.
[ bib ]
Keywords: graph neural networks

[917]

Sara Morin, Caroline Gagné, and Marc Gravel.
Ant colony optimization with a specialized pheromone trail for
the carsequencing problem.
European Journal of Operational Research, 197(3):1185–1191,
2009.
[ bib 
DOI ]
This paper studies the learning process in an ant
colony optimization algorithm designed to solve the
problem of ordering cars on an assembly line
(carsequencing problem). This problem has been
shown to be NPhard and evokes a great deal of
interest among practitioners. Learning in an ant
algorithm is achieved by using an artificial
pheromone trail, which is a central element of this
metaheuristic. Many versions of the algorithm are
found in literature, the main distinction among them
being the management of the pheromone
trail. Nevertheless, few of them seek to perfect
learning by modifying the internal structure of the
trail. In this paper, a new pheromone trail
structure is proposed that is specifically adapted
to the type of constraints in the carsequencing
problem. The quality of the results obtained when
solving three sets of benchmark problems is superior
to that of the best solutions found in literature
and shows the efficiency of the specialized trail.
Keywords: Ant colony optimization,Carsequencing
problem,Pheromone trail,Scheduling

[918]

A. M. Mora, JuanJulián Merelo, Juan Luis Jiménez Laredo, C. Millan,
and J. Torrecillas.
CHAC, a MOACO algorithm for computation of bicriteria
military unit path in the battlefield: Presentation and first results.
International Journal of Intelligent Systems, 24(7):818–843,
2009.
[ bib ]

[919]

Max D. Morris and Toby J. Mitchell.
Exploratory designs for computational experiments.
Journal of Statistical Planning and Inference, 43(3):381–402,
1995.
[ bib 
DOI ]
Keywords: Bayesian prediction

[920]

Pablo Moscato and José F. Fontanari.
Stochastic Versus Deterministic Update in Simulated Annealing.
Physics Letters A, 146(4):204–208, 1990.
[ bib ]

[921]

J. Mote, D. L. Olson, and M. A. Venkataramanan.
A comparative multiobjective programming study.
Mathematical and Computer Modelling, 10(10):719–729, 1988.
[ bib 
DOI ]
The purpose of this study was to systematically evaluate a
number of multiobjective programming concepts relative to
reflection of utility, assurance of nondominated solutions
and practicality for larger problems using conventional
software. In the problem used, the nonlinear simulated DM
utility function applied resulted in a nonextreme point
solution. Very often, the preferred solution could end up
being an extreme point solution, in which case the techniques
relying upon LP concepts would work as well if not better
than utilizing constrained objective attainments. The point
is that there is no reason to expect linear or near linear
utility.
Keywords: artificial DM, interactive

[922]

Sébastien Mouthuy, Yves Deville, and Pascal van Hentenryck.
Constraintbased Very LargeScale Neighborhood Search.
Constraints, 17(2):87–122, 2012.
[ bib 
DOI ]

[923]

Lucien Mousin, MarieEléonore Kessaci, and Clarisse Dhaenens.
Exploiting Promising SubSequences of Jobs to solve the NoWait
Flowshop Scheduling Problem.
Arxiv preprint arXiv:1903.09035, 2019.
[ bib 
http ]

[924]

Vincent Mousseau and Roman Slowiński.
Inferring an ELECTRE TRI model from assignment examples.
Journal of Global Optimization, 12(2):157–174, 1998.
[ bib ]

[925]

Christian L. Müller and Ivos F. Sbalzarini.
Energy Landscapes of Atomic Clusters as Black Box Optimization
Benchmarks.
Evolutionary Computation, 20(4):543–573, 2012.
[ bib 
DOI ]

[926]

H. Mühlenbein and D. SchlierkampVoosen.
Predictive models for the breeder genetic algorithm.
Evolutionary Computation, 1(1):25–49, 1993.
[ bib ]
Keywords: crossover, intermediate, line

[927]

Mario A. Muñoz and Kate SmithMiles.
Generating New SpaceFilling Test Instances for Continuous
BlackBox Optimization.
Evolutionary Computation, 28(3):379–404, September 2020.
[ bib 
DOI ]

[928]

Mario A. Muñoz, Yuan Sun, Michael Kirley, and Saman K. Halgamuge.
Algorithm selection for blackbox continuous optimization
problems: a survey on methods and challenges.
Information Sciences, 317:224–245, 2015.
[ bib ]

[929]

Yuichi Nagata and Shigenobu Kobayashi.
A Powerful Genetic Algorithm Using Edge Assembly Crossover for
the Traveling Salesman Problem.
INFORMS Journal on Computing, 25(2):346–363, 2013.
[ bib 
DOI ]
This paper presents a genetic algorithm (GA) for solving the
traveling salesman problem (TSP). To construct a powerful GA,
we use edge assembly crossover (EAX) and make substantial
enhancements to it: (i) localization of EAX together with its
efficient implementation and (ii) the use of a local search
procedure in EAX to determine good combinations of building
blocks of parent solutions for generating even better
offspring solutions from very highquality parent
solutions. In addition, we develop (iii) an innovative
selection model for maintaining population diversity at a
negligible computational cost. Experimental results on
wellstudied TSP benchmarks demonstrate that the proposed GA
outperforms stateoftheart heuristic algorithms in finding
very highquality solutions on instances with up to 200,000
cities. In contrast to the stateoftheart TSP heuristics,
which are all based on the LinKernighan (LK) algorithm, our
GA achieves top performance without using an LKbased
algorithm.
Keywords: TSP, EAX, evolutionary algorithms

[930]

Marcelo S. Nagano, Fernando L. Rossi, and Nádia J. Martarelli.
Highperforming heuristics to minimize flowtime in noidle
permutation flowshop.
Engineering Optimization, 51(2):185–198, 2019.
[ bib ]

[931]

Yuichi Nagata and David Soler.
A New Genetic Algorithm for the Asymmetric TSP.
Expert Systems with Applications, 39(10):8947–8953, 2012.
[ bib ]

[932]

Samadhi Nallaperuma, Pietro S. Oliveto, Jorge Pérez Heredia, and Dirk
Sudholt.
On the Analysis of TrajectoryBased Search Algorithms: When is
it Beneficial to Reject Improvements?
Algorithmica, 81(2):858–885, 2019.
[ bib ]

[933]

Yang Nan, Ke Shang, Hisao Ishibuchi, and Linjun He.
Reverse strategy for nondominated archiving.
IEEE Access, 8:119458–119469, 2020.
[ bib ]

[934]

Kaname Narukawa, Yu Setoguchi, Yuki Tanigaki, Markus Olhofer, Bernhard
Sendhoff, and Hisao Ishibuchi.
Preference representation using Gaussian functions on a
hyperplane in evolutionary multiobjective optimization.
Soft Computing, 20(7):2733–2757, July 2016.
[ bib 
DOI ]

[935]

John Nash and Ravi Varadhan.
Unifying Optimization Algorithms to Aid Software System Users:
optimx for R.
Journal of Statistical Software, 43(9):1–14, 2011.
[ bib ]

[936]

M. Nawaz, E. Enscore, Jr, and I. Ham.
A Heuristic Algorithm for the mMachine, nJob FlowShop
Sequencing Problem.
Omega, 11(1):91–95, 1983.
[ bib ]
Keywords: NEH heuristic

[937]

Antonio J. Nebro, F. Luna, Enrique Alba, Bernabé Dorronsoro, Juan J.
Durillo, and A. Beham.
AbYSS: Adapting Scatter Search to Multiobjective
Optimization.
IEEE Transactions on Evolutionary Computation, 12(4):439–457,
August 2008.
[ bib 
DOI ]

[938]

F. Nerri and Carlos Cotta.
Memetic algorithms and memetic computing optimization: A
literature review.
Swarm and Evolutionary Computation, 2:1–14, 2012.
[ bib 
DOI ]

[939]

Frank Neumann, Dirk Sudholt, and Carsten Witt.
Analysis of different MMAS ACO algorithms on unimodal
functions and plateaus.
Swarm Intelligence, 3(1):35–68, 2009.
[ bib ]

[940]

Frank Neumann and Carsten Witt.
Runtime Analysis of a Simple Ant Colony Optimization Algorithm.
Electronic Colloquium on Computational Complexity (ECCC),
13(084), 2006.
[ bib ]

[941]

Allen Newell and Herbert A. Simon.
Computer Science as Empirical Inquiry: Symbols and Search.
Communications of the ACM, 19(3):113–126, March 1976.
[ bib 
DOI ]
Computer science is the study of the phenomena surrounding
computers. The founders of this society understood this very
well when they called themselves the Association for
Computing Machinery. The machinenot just the hardware, but
the programmed, living machineis the organism we study.
Keywords: cognition, Turing, search, problem solving, symbols,
heuristics, list processing, computer science, artificial
intelligence, science, empirical

[942]

VietPhuong Nguyen, Christian Prins, and Caroline Prodhon.
A Multistart Iterated Local Search with Tabu List and Path
Relinking for the Twoechelon Locationrouting Problem.
Engineering Applications of Artificial Intelligence,
25(1):56–71, 2012.
[ bib ]

[943]

AnhTuan Nguyen, Sigrid Reiter, and Philippe Rigo.
A review on simulationbased optimization methods applied to
building performance analysis.
Applied Energy, 113:1043–1058, 2014.
[ bib 
DOI ]

[944]

Trung Thanh Nguyen, Shengxiang Yang, and Jürgen Branke.
Evolutionary Dynamic Optimization: A Survey of the State of the
Art.
Swarm and Evolutionary Computation, 6:1–24, 2012.
[ bib ]

[945]

Su Nguyen, Mengjie Zhang, Mark Johnston, and Kay Chen Tan.
Genetic Programming for Evolving DueDate Assignment Models in
Job Shop Environments.
Evolutionary Computation, 22(1):105–138, 2014.
[ bib ]

[946]

Su Nguyen, Mengjie Zhang, Mark Johnston, and Kay Chen Tan.
Automatic Design of Scheduling Policies for Dynamic
Multiobjective Job Shop Scheduling via Cooperative Coevolution Genetic
Programming.
IEEE Transactions on Evolutionary Computation, 18(2):193–208,
2014.
[ bib ]

[947]

Peter Nightingale, Özguür Akgün, Ian P. Gent, Christopher Jefferson, Ian
Miguel, and Patrick Spracklen.
Automatically Improving Constraint Models in Savile Row.
Artificial Intelligence, 251:35–61, 2017.
[ bib ]

[948]

Naoki Nishimura, Kotaro Tanahashi, Koji Suganuma, Masamichi J. Miyama, and
Masayuki Ohzeki.
Item listing optimization for ecommerce websites based on
diversity.
Frontiers in Computer Science, 1:2, 2019.
[ bib ]
Keywords: Quantum Annealing

[949]

Vilas Nitivattananon, Elaine C. Sadowski, and Rafael G. Quimpo.
Optimization of Water Supply System Operation.
Journal of Water Resources Planning and Management, ASCE,
122(5):374–384, September / October 1996.
[ bib ]

[950]

Bruno Nogueira, Rian G. S. Pinheiro, and Anand Subramanian.
A Hybrid Iterated Local Search Heuristic for the Maximum Weight
Independent Set Problem.
Optimization Letters, 12(3):567–583, 2018.
[ bib 
DOI ]

[951]

B. A. Nosek, G. Alter, G. C. Banks, D. Borsboom, S. D. Bowman, S. J. Breckler,
S. Buck, C. D. Chambers, G. Chin, G. Christensen, M. Contestabile, A. Dafoe,
E. Eich, J. Freese, R. Glennerster, D. Goroff, D. P. Green, B. Hesse,
M. Humphreys, J. Ishiyama, D. Karlan, A. Kraut, A. Lupia, P. Mabry, T. Madon,
N. Malhotra, E. MayoWilson, M. McNutt, E. Miguel, E. L. Paluck,
U. Simonsohn, C. Soderberg, B. A. Spellman, J. Turitto, G. VandenBos,
S. Vazire, E. J. Wagenmakers, R. Wilson, and T. Yarkoni.
Promoting an open research culture.
Science, 348(6242):1422–1425, June 2015.
[ bib 
DOI ]

[952]

Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor.
The Preregistration Revolution.
Proceedings of the National Academy of Sciences,
115(11):2600–2606, March 2018.
[ bib 
DOI ]
Progress in science relies in part on generating hypotheses
with existing observations and testing hypotheses with new
observations. This distinction between postdiction and
prediction is appreciated conceptually but is not respected
in practice. Mistaking generation of postdictions with
testing of predictions reduces the credibility of research
findings. However, ordinary biases in human reasoning, such
as hindsight bias, make it hard to avoid this mistake. An
effective solution is to define the research questions and
analysis plan before observing the research outcomes–a
process called preregistration. Preregistration distinguishes
analyses and outcomes that result from predictions from those
that result from postdictions. A variety of practical
strategies are available to make the best possible use of
preregistration in circumstances that fall short of the ideal
application, such as when the data are preexisting. Services
are now available for preregistration across all disciplines,
facilitating a rapid increase in the practice. Widespread
adoption of preregistration will increase distinctiveness
between hypothesis generation and hypothesis testing and will
improve the credibility of research findings.

[953]

Yaghout Nourani and Bjarne Andresen.
A Comparison of Simulated Annealing Cooling Strategies.
Journal of Physics A, 31(41):8373–8385, 1998.
[ bib ]

[954]

Eugeniusz Nowicki and Czeslaw Smutnicki.
A Fast Taboo Search Algorithm for the Job Shop Problem.
Management Science, 42(6):797–813, 1996.
[ bib ]

[955]

Eugeniusz Nowicki and Czeslaw Smutnicki.
A fast tabu search algorithm for the permutation flowshop
problem.
European Journal of Operational Research, 91(1):160–175, 1996.
[ bib ]

[956]

Open Science Collaboration.
Estimating the reproducibility of psychological science.
Science, 349(6251):aac4716, 2015.
[ bib 
DOI ]

[957]

Gabriela Ochoa and Nadarajen Veerapen.
Mapping the global structure of TSP fitness landscapes.
Journal of Heuristics, 24(3):265–294, 2018.
[ bib ]

[958]

Angelo Oddi, Amadeo Cesta, Nicola Policella, and Stephen F. Smith.
Combining Variants of Iterative Flattening Search.
Engineering Applications of Artificial Intelligence,
21(5):683–690, 2008.
[ bib ]

[959]

Angelo Oddi, Amadeo Cesta, Nicola Policella, and Stephen F. Smith.
Iterative Flattening Search for Resource Constrained
Scheduling.
Journal of Intelligent Manufacturing, 21(1):17–30, 2010.
[ bib ]

[960]

F. A. Ogbu and David K. Smith.
The Application of the Simulated Annealing Algorithm to the
Solution of the n/m/C Max Flowshop Problem.
Computers & Operations Research, 17(3):243–253, 1990.
[ bib ]

[961]

Jeffrey W. Ohlmann and Barrett W. Thomas.
A CompressedAnnealing Heuristic for the Traveling Salesman
Problem with Time Windows.
INFORMS Journal on Computing, 19(1):80–90, 2007.
[ bib 
DOI ]

[962]

Pietro S. Oliveto, Jun He, and Xin Yao.
Time complexity of evolutionary algorithms for combinatorial
optimization: A decade of results.
International Journal of Automation and Computing,
4(3):281–293, 2007.
[ bib ]

[963]

Pietro S. Oliveto and Carsten Witt.
Improved time complexity analysis of the Simple Genetic
Algorithm.
Theoretical Computer Science, 605:21–41, 2015.
[ bib 
DOI ]

[964]

David L. Olson.
Review of Empirical Studies in Multiobjective Mathematical
Programming: Subject Reflection of Nonlinear Utility and Learning.
Decision Sciences, 23(1):1–20, 1992.
[ bib 
DOI ]
Multiple objective programming provides a means of
aiding decision makers facing complex decisions where
tradeoffs among conflicting objectives must be
reconciled. Interactive multiobjective programming provides a
means for decision makers to learn what these tradeoffs
involve, while the mathematical program generates solutions
that seek improvement of the implied utility of the decision
maker. A variety of multiobjective programming techniques
have been presented in the multicriteria decisionmaking
literature. This study reviews published studies with human
subjects where some of these techniques were applied. While
all of the techniques have the ability to support decision
makers under conditions of multiple objectives, a number of
features in applying these systems have been tested by these
studies. A general evolution of techniques is traced,
starting with methods relying upon linear combinations of
value, to more recent methods capable of reflecting nonlinear
tradeoffs of value. Support of nonlinear utility and
enhancing decisionmaker learning are considered.
Keywords: Decision Analysis, Human Information Processing, Linear
Programming

[965]

Roland Olsson and Arne Løkketangen.
Using Automatic Programming to Generate Stateoftheart
Algorithms for Random 3SAT.
Journal of Heuristics, 19(5):819–844, 2013.
[ bib 
DOI ]
Uses evolution but it is not genetic programming, nor
grammatical evolution.

[966]

Mihai Oltean.
Evolving Evolutionary Algorithms Using Linear Genetic
Programming.
Evolutionary Computation, 13(3):387–410, 2005.
[ bib 
DOI ]

[967]

Michael O'Neill and Conor Ryan.
Grammatical Evolution.
IEEE Transactions on Evolutionary Computation, 5(4):349–358,
2001.
[ bib ]

[968]

Lindell E. Ormsbee, Thomas M. Walski, Donald V. Chase, and W. W. Sharp.
Methodology for improving pump operation efficiency.
Journal of Water Resources Planning and Management, ASCE,
115(2):148–164, 1989.
[ bib ]

[969]

Lindell E. Ormsbee and Kevin E. Lansey.
Optimal Control of Water Supply Pumping Systems.
Journal of Water Resources Planning and Management, ASCE,
120(2):237–252, 1994.
[ bib ]

[970]

Lindell E. Ormsbee and Srinivasa L. Reddy.
Nonlinear Heuristic for Pump Operations.
Journal of Water Resources Planning and Management, ASCE,
121(4):302–309, July / August 1995.
[ bib ]

[971]

Jeffrey E. Orosz and Sheldon H. Jacobson.
Analysis of Static Simulated Annealing Algorithms.
Journal of Optimization Theory and Applications,
115(1):165–182, 2002.
[ bib ]

[972]

Ibrahim H. Osman and Chris N. Potts.
Simulated Annealing for Permutation FlowShop Scheduling.
Omega, 17(6):551–557, 1989.
[ bib ]

[973]

P. S. Ow and T. E. Morton.
Filtered Beam Search in Scheduling.
International Journal of Production Research, 26:297–307,
1988.
[ bib ]

[974]

Gül Özerol and Esra Karasakal.
Interactive outranking approaches for multicriteria
decisionmaking problems with imprecise information.
Journal of the Operational Research Society, 59:1253–1268,
2007.
[ bib ]

[975]

Manfred Padberg and Giovanni Rinaldi.
A branchandcut algorithm for the resolution of largescale
symmetric traveling salesman problems.
SIAM Review, 33(1):60–100, 1991.
[ bib ]

[976]

Federico Pagnozzi and Thomas Stützle.
Speeding up Local Search for the Insert Neighborhood in the
Weighted Tardiness Permutation Flowshop Problem.
Optimization Letters, 11:1283–1292, 2017.
[ bib 
DOI ]

[977]

Federico Pagnozzi and Thomas Stützle.
Automatic Design of Hybrid Stochastic Local Search Algorithms
for Permutation Flowshop Problems.
European Journal of Operational Research, 276:409–421, 2019.
[ bib 
DOI ]
Keywords: EMILI

[978]

Federico Pagnozzi and Thomas Stützle.
Evaluating the impact of grammar complexity in automatic
algorithm design.
International Transactions in Operational Research, pp. 1–26,
2020.
[ bib 
DOI ]

[979]

Federico Pagnozzi and Thomas Stützle.
Automatic design of hybrid stochastic local search algorithms
for permutation flowshop problems with additional constraints.
Operations Research Perspectives, 8, 2021.
[ bib 
DOI ]

[980]

Alberto Pajares, Xavier Blasco, Juan Manuel Herrero, and Miguel A.
Martínez.
A Comparison of Archiving Strategies for Characterization of
Nearly Optimal Solutions under MultiObjective Optimization.
Mathematics, 9(9):999, 2021.
[ bib 
DOI ]
In a multiobjective optimization problem, in addition to
optimal solutions, multimodal and/or nearly optimal
alternatives can also provide additional useful information
for the decision maker. However, obtaining all nearly optimal
solutions entails an excessive number of
alternatives. Therefore, to consider the nearly optimal
solutions, it is convenient to obtain a reduced set, putting
the focus on the potentially useful alternatives. These
solutions are the alternatives that are close to the optimal
solutions in objective space, but which differ significantly
in the decision space. To characterize this set, it is
essential to simultaneously analyze the decision and
objective spaces. One of the crucial points in an
evolutionary multiobjective optimization algorithm is the
archiving strategy. This is in charge of keeping the solution
set, called the archive, updated during the optimization
process. The motivation of this work is to analyze the three
existing archiving strategies proposed in the literature
(ArchiveUpdateP_{Q,ε}D_{xy}, Archive_nevMOGA, and
targetSelect) that aim to characterize the potentially useful
solutions. The archivers are evaluated on two benchmarks and
in a real engineering example. The contribution clearly shows
the main differences between the three archivers. This
analysis is useful for the design of evolutionary algorithms
that consider nearly optimal solutions.
Keywords: multiobjective optimization; nearly optimal solutions;
nonepsilon dominance; multimodality; decision space
diversity; archiving strategy; evolutionary algorithm;
nonlinear parametric identification

[981]

Daniel Palhazi Cuervo, Peter Goos, Kenneth Sörensen, and Emely
Arráiz.
An Iterated Local Search Algorithm for the Vehicle Routing
Problem with Backhauls.
European Journal of Operational Research, 237(2):454–464,
2014.
[ bib ]

[982]

Gintaras Palubeckis.
Iterated tabu search for the unconstrained binary quadratic
optimization problem.
Informatica, 17(2):279–296, 2006.
[ bib 
DOI ]

[983]

QuanKe Pan and Rubén Ruiz.
Local Search Methods for the Flowshop Scheduling Problem with
Flowtime Minimization.
European Journal of Operational Research, 222(1):31–43, 2012.
[ bib ]

[984]

QuanKe Pan and Rubén Ruiz.
A Comprehensive Review and Evaluation of Permutation Flowshop
Heuristics to Minimize Flowtime.
Computers & Operations Research, 40(1):117–128, 2013.
[ bib ]

[985]

QuanKe Pan, Rubén Ruiz, and Pedro AlfaroFernández.
Iterated Search Methods for Earliness and Tardiness Minimization
in Hybrid Flowshops with Due Windows.
Computers & Operations Research, 80:50–60, 2017.
[ bib ]

[986]

QuanKe Pan, Mehmet Fatih Tasgetiren, and YunChia Liang.
A Discrete Differential Evolution Algorithm for the Permutation
Flowshop Scheduling Problem.
Computers and Industrial Engineering, 55(4):795 – 816, 2008.
[ bib ]

[987]

QuanKe Pan, Ling Wang, and BaoHua Zhao.
An improved iterated greedy algorithm for the nowait flow shop
scheduling problem with makespan criterion.
International Journal of Advanced Manufacturing Technology,
38(78):778–786, 2008.
[ bib ]

[988]

Sinno Jialin Pan and Qiang Yang.
A survey on transfer learning.
IEEE Transactions on Knowledge and Data Engineering,
22(10):1345–1359, 2009.
[ bib ]

[989]

Luís Paquete, Tommaso Schiavinotto, and Thomas Stützle.
On Local Optima in Multiobjective Combinatorial Optimization
Problems.
Annals of Operations Research, 156:83–97, 2007.
[ bib 
DOI ]
In this article, local optimality in multiobjective
combinatorial optimization is used as a baseline for
the design and analysis of two iterative improvement
algorithms. Both algorithms search in a neighborhood
that is defined on a collection of sets of feasible
solutions and their acceptance criterion is based on
outperformance relations. Proofs of the soundness
and completeness of these algorithms are given.
Keywords: Pareto local search, PLS

[990]

Luís Paquete and Thomas Stützle.
A study of stochastic local search algorithms for the
biobjective QAP with correlated flow matrices.
European Journal of Operational Research, 169(3):943–959,
2006.
[ bib ]

[991]

Luís Paquete and Thomas Stützle.
Design and analysis of stochastic local search for the
multiobjective traveling salesman problem.
Computers & Operations Research, 36(9):2619–2631, 2009.
[ bib 
DOI ]

[992]

S. N. Parragh, Karl F. Doerner, Richard F. Hartl, and Xavier Gandibleux.
A heuristic twophase solution approach for the multiobjective
dialaride problem.
Networks, 54(4):227–242, 2009.
[ bib ]

[993]

Rebecca Parsons and Mark Johnson.
A Case Study in Experimental Design Applied to Genetic
Algorithms with Applications to DNA Sequence Assembly.
American Journal of Mathematical and Management Sciences,
17(34):369–396, 1997.
[ bib 
DOI ]

[994]

MoonWon Park and YeongDae Kim.
A systematic procedure for setting parameters in simulated
annealing algorithms.
Computers & Operations Research, 25(3):207–217, 1998.
[ bib 
DOI ]

[995]

R. S. Parpinelli, H. S. Lopes, and A. A. Freitas.
Data Mining with an Ant Colony Optimization Algorithm.
IEEE Transactions on Evolutionary Computation, 6(4):321–332,
2002.
[ bib ]

[996]

R. O. Parreiras and J. A. Vascocelos.
A multiplicative version of PROMETHEE II applied to
multiobjective optimization problems.
European Journal of Operational Research, 183:729–740, 2007.
[ bib ]

[997]

Gerald Paul.
Comparative performance of tabu search and simulated annealing
heuristics for the quadratic assignment problem.
Operations Research Letters, 38(6):577–581, 2010.
[ bib ]

[998]

Judea Pearl.
The seven tools of causal inference, with reflections on machine
learning.
Communications of the ACM, 62(3):54–60, 2019.
[ bib ]

[999]

Martín Pedemonte, Sergio Nesmachnow, and Héctor Cancela.
A survey on parallel ant colony optimization.
Applied Soft Computing, 11(8):5181–5197, 2011.
[ bib ]

[1000]

Paola Pellegrini, Mauro Birattari, and Thomas Stützle.
A Critical Analysis of Parameter Adaptation in Ant Colony
Optimization.
Swarm Intelligence, 6(1):23–48, 2012.
[ bib 
DOI ]

[1001]

Paola Pellegrini, L. Castelli, and R. Pesenti.
Metaheuristic algorithms for the simultaneous slot allocation
problem.
IET Intelligent Transport Systems, 6(4):453–462, December
2012.
[ bib 
DOI ]

[1002]

Paola Pellegrini, Franco Mascia, Thomas Stützle, and Mauro Birattari.
On the Sensitivity of Reactive Tabu Search to its
Metaparameters.
Soft Computing, 18(11):2177–2190, 2014.
[ bib 
DOI ]

[1003]

Puca Huachi Vaz Penna, Anand Subramanian, and Luiz Satoru Ochi.
An Iterated Local Search Heuristic for the Heterogeneous Fleet
Vehicle Routing Problem.
Journal of Heuristics, 19(2):201–232, 2013.
[ bib ]

[1004]

Jeffrey M. Perkel.
Challenge to scientists: does your tenyearold code still run?
Nature, 584:556–658, 2020.
[ bib 
DOI ]
Keywords: reproducibility; software engineering; ReScience C; Ten Years
Reproducibility Challenge; code reusability

[1005]

Leslie Pérez Cáceres, Manuel LópezIbáñez, and Thomas
Stützle.
Ant colony optimization on a limited budget of evaluations.
Swarm Intelligence, 9(23):103–124, 2015.
[ bib 
DOI 
supplementary material ]

[1006]

Matias Péres, Germán Ruiz, Sergio Nesmachnow, and Ana C. Olivera.
Multiobjective evolutionary optimization of traffic flow and
pollution in Montevideo, Uruguay.
Applied Soft Computing, 70:472–485, 2018.
[ bib ]
Keywords: Multiobjective evolutionary
algorithms,Pollution,Simulation,Traffic flow

[1007]

A. Pessoa, E. Uchoa, M. Aragão, and R. Rodrigues.
Exact Algorithm over an Arctimeindexed formulation for
Parallel Machine Scheduling Problems.
Mathematical Programming Computation, 2(3–4):259–290, 2010.
[ bib ]

[1008]

Gilles Pesant, Michel Gendreau, JeanYves Potvin, and J.M. Rousseau.
An Exact Constraint Logic Programming Algorithm for the
Traveling Salesman Problem with Time Windows.
Transportation Science, 32:12–29, 1998.
[ bib ]

[1009]

Charles W. Petit.
Touched by nature: putting evolution to work on the assembly
line.
U.S. News & World Report, 125(4):43–45, July 1998.
[ bib 
http ]
Evolutionary optimization of turbine design of the
Boeing 777 GE

[1010]

Justyna Petke, Saemundur O. Haraldsson, Mark Harman, William B. Langdon,
David R. White, and John R. Woodward.
Genetic Improvement of Software: A Comprehensive Survey.
IEEE Transactions on Evolutionary Computation, 22(3):415–432,
2018.
[ bib 
DOI ]

[1011]

Marek Petrik and Shlomo Zilberstein.
Learning parallel portfolios of algorithms.
Annals of Mathematics and Artificial Intelligence,
48(1):85–106, 2006.
[ bib ]
Keywords: algorithm selection

[1012]

S. Pezeshk and O. J. Helweg.
Adaptative Search Optimisation in reducing pump operation
costs.
Journal of Water Resources Planning and Management, ASCE,
122(1):57–63, January / February 1996.
[ bib ]

[1013]

Selcen Phelps and Murat Köksalan.
An interactive evolutionary metaheuristic for multiobjective
combinatorial optimization.
Management Science, 49(12):1726–1738, 2003.
[ bib ]

[1014]

Francesco di Pierro, SoonThiam Khu, and Dragan A. Savic.
An investigation on preference order ranking scheme for
multiobjective evolutionary optimization.
IEEE Transactions on Evolutionary Computation, 11(1):17–45,
2007.
[ bib ]

[1015]

Joelle Pineau, Philippe VincentLamarre, Koustuv Sinha, Vincent LariviÃ¨re,
Alina Beygelzimer, Florence d'AlchÃ© Buc, Emily Fox, and Hugo Larochelle.
Improving Reproducibility in Machine Learning Research (A Report
from the NeurIPS 2019 Reproducibility Program).
Arxiv preprint arXiv:2003.12206 [cs.LG], 2020.
[ bib 
http ]

[1016]

David Pisinger.
Where are the hard knapsack problems?
Computers & Operations Research, 32(9):2271–2284, 2005.
[ bib ]

[1017]

David Pisinger and Stefan Ropke.
A General Heuristic for Vehicle Routing Problems.
Computers & Operations Research, 34(8):2403–2435, 2007.
[ bib ]

[1018]

Rapeepan Pitakaso, Christian Almeder, Karl F. Doerner, and Richard F. Hartl.
Combining exact and populationbased methods for the Constrained
Multilevel Lot Sizing Problem.
International Journal of Production Research,
44(22):4755–4771, 2006.
[ bib ]

[1019]

Rapeepan Pitakaso, Christian Almeder, Karl F. Doerner, and Richard F. Hartl.
A MaxMin Ant System for unconstrained multilevel lotsizing
problems.
Computers & Operations Research, 34(9):2533–2552, 2007.
[ bib 
DOI ]
In this paper, we present an antbased algorithm
for solving unconstrained multilevel lotsizing
problems called ant system for multilevel
lotsizing algorithm (ASMLLS). We apply a hybrid
approach where we use ant colony optimization in
order to find a good lotsizing sequence, i.e. a
sequence of the different items in the product
structure in which we apply a modified
WagnerWhitin algorithm for each item
separately. Based on the setup costs each ant
generates a sequence of items. Afterwards a simple
singlestage lotsizing rule is applied with
modified setup costs. This modification of the setup
costs depends on the position of the item in the
lotsizing sequence, on the items which have been
lotsized before, and on two further parameters,
which are tried to be improved by a systematic
search. For smallsized problems ASMLLS is among
the best algorithms, but for most medium and
largesized problems it outperforms all other
approaches regarding solution quality as well as
computational time.
Keywords: Ant colony optimization, Material requirements
planning, Multilevel lotsizing, WagnerWhitin
algorithm

[1020]

Hans E. Plesser.
Reproducibility vs. Replicability: A Brief History of a
Confused Terminology.
Frontiers in Neuroinformatics, 11, January 2018.
[ bib 
DOI ]

[1021]

Daniel Porumbel, Gilles Goncalves, Hamid Allaoui, and Tienté Hsu.
Iterated Local Search and Column Generation to solve ArcRouting
as a Permutation SetCovering Problem.
European Journal of Operational Research, 256(2):349–367,
2017.
[ bib ]

[1022]

Juan Porta, Jorge Parapar, Ramón Doallo, Vasco Barbosa, Inés Santé,
Rafael Crecente, and Carlos Díaz.
A Populationbased Iterated Greedy Algorithm for the
Delimitation and Zoning of Rural Settlements.
Computers, Environment and Urban Systems, 39:12–26, 2013.
[ bib ]

[1023]

JeanYves Potvin and S. Bengio.
The Vehicle Routing Problem with Time Windows Part II: Genetic
Search.
INFORMS Journal on Computing, 8:165–172, 1996.
[ bib ]

[1024]

T. Devi Prasad.
Design of pumped water distribution networks with storage.
Journal of Water Resources Planning and Management, ASCE,
136(4):129–136, 2009.
[ bib ]

[1025]

Marco Pranzo and D. Pacciarelli.
An Iterated Greedy Metaheuristic for the Blocking Job Shop
Scheduling Problem.
Journal of Heuristics, 22(4):587–611, 2016.
[ bib 
DOI ]

[1026]

Robert Clay Prim.
Shortest connection networks and some generalizations.
Bell System Technical Journal, 36(6):1389–1401, 1957.
[ bib ]

[1027]

Philipp Probst, Bernd Bischl, and AnneLaure Boulesteix.
Tunability: Importance of Hyperparameters of Machine Learning
Algorithms.
Arxiv preprint arXiv:1802.09596, 2018.
[ bib 
http ]
Keywords: parameter importance

[1028]

Philipp Probst, Bernd Bischl, and AnneLaure Boulesteix.
Tunability: Importance of Hyperparameters of Machine Learning
Algorithms.
Journal of Machine Learning Research, 20(53):1–32, 2019.
[ bib ]

[1029]

Luc Pronzato and Werner G. Müller.
Design of computer experiments: space filling and beyond.
Statistics and Computing, 22(3):681–701, 2012.
[ bib ]
Keywords: Kriging; Entropy; Design of experiments; Spacefilling;
Sphere packing; Maximin design; Minimax design

[1030]

Harilaos N. Psaraftis.
Dynamic Vehicle Routing: Status and Prospects.
Annals of Operations Research, 61:143–164, 1995.
[ bib ]

[1031]

Timo Pukkala and Tero Heinonen.
Optimizing heuristic search in forest planning.
Nonlinear Analysis: Real World Applications, 7(5):1284–1297,
2006.
[ bib ]

[1032]

Luca Pulina and Armando Tacchella.
A selfadaptive multiengine solver for quantified Boolean
formulas.
Constraints, 14(1):80–116, 2009.
[ bib ]

[1033]

Robin C. Purshouse and Peter J. Fleming.
On the Evolutionary Optimization of Many Conflicting
Objectives.
IEEE Transactions on Evolutionary Computation, 11(6):770–784,
2007.
[ bib 
DOI ]

[1034]

Yutao Qi, Xiaoliang Ma, Fang Liu, Licheng Jiao, Jianyong Sun, and Jianshe Wu.
MOEA/D with adaptive weight adjustment.
Evolutionary Computation, 22(2):231–264, 2014.
[ bib 
DOI ]
Uses an external population

[1035]

Julianne D. Quinn, Patrick M. Reed, and Klaus Keller.
Direct policy search for robust multiobjective management of
deeply uncertain socioecological tipping points.
Environmental Modelling & Software, 92:125–141, 2017.
[ bib ]

[1036]

Shahriar Farahmand Rad, Rubén Ruiz, and Naser Boroojerdian.
New High Performing Heuristics for Minimizing Makespan in
Permutation Flowshops.
Omega, 37(2):331–345, 2009.
[ bib ]

[1037]

C. Rajendran.
Heuristic algorithm for scheduling in a flowshop to minimize
total flowtime.
International Journal of Production Economics, 29(1):65–73,
1993.
[ bib ]

[1038]

C. Rajendran and H. Ziegler.
Antcolony algorithms for permutation flowshop scheduling to
minimize makespan/total flowtime of jobs.
European Journal of Operational Research, 155(2):426–438,
2004.
[ bib ]

[1039]

C. Rajendran and H. Ziegler.
An efficient heuristic for scheduling in a flowshop to minimize
total weighted flowtime of jobs.
European Journal of Operational Research, 103(1):129–138,
1997.
[ bib 
DOI ]

[1040]

David Garzón Ramos and Mauro Birattari.
Automatic Design of Collective Behaviors for Robots that Can
Display and Perceive Colors.
Applied Sciences, 10(13):4654, 2020.
[ bib ]

[1041]

JuanManuel RamosPÃ©rez, Gara Miranda, Eduardo Segredo, Coromoto LeÃ³n, and
Casiano RodrÃguezLeÃ³n.
Application of MultiObjective Evolutionary Algorithms for
Planning Healthy and Balanced School Lunches.
Mathematics, 9(1):80, December 2021.
[ bib 
DOI ]
A multiobjective formulation of the Menu Planning Problem,
which is termed the Multiobjective Menu Planning Problem, is
presented herein. Menu planning is of great interest in the
health field due to the importance of proper nutrition in
today's society, and particularly, in school
canteens. In addition to considering the cost of the meal
plan as the classic objective to be minimized, we also
introduce a second objective aimed at minimizing the degree
of repetition of courses and food groups that a particular
meal plan consists of. The motivation behind this particular
multiobjective formulation is to offer a meal plan that is
not only affordable but also varied and balanced from a
nutritional standpoint. The plan is designed for a given
number of days and ensures that the specific nutritional
requirements of schoolage children are satisfied. The main
goal of the current work is to demonstrate the
multiobjective nature of the said formulation, through a
comprehensive experimental assessment carried out over a set
of multiobjective evolutionary algorithms applied to
different instances. At the same time, we are also interested
in validating the multiobjective formulation by performing
quantitative and qualitative analyses of the solutions
attained when solving it. Computational results show the
multiobjective nature of the said formulation, as well as
that it allows suitable meal plans to be obtained.

[1042]

Camelia Ram, Gilberto Montibeller, and Alec Morton.
Extending the use of scenario planning and MCDA for the
evaluation of strategic options.
Journal of the Operational Research Society, 62(5):817–829,
2011.
[ bib ]

[1043]

Zhengfu Rao and Elad Salomons.
Development of a realtime, nearoptimal control process for
waterdistribution networks.
Journal of Hydroinformatics, 9(1):25–37, 2007.
[ bib 
DOI ]

[1044]

Ronald L. Rardin and Reha Uzsoy.
Experimental Evaluation of Heuristic Optimization Algorithms: A
Tutorial.
Journal of Heuristics, 7(3):261–304, 2001.
[ bib ]

[1045]

Jussi Rasku, Nysret Musliu, and Tommi Kärkkäinen.
On automatic algorithm configuration of vehicle routing problem
solvers.
Journal on Vehicle Routing Algorithms, 2(14):1–22, February
2019.
[ bib 
DOI ]
Keywords: irace, SMAC, GGA, REVAC, VRP

[1046]

Ingo Rechenberg.
Case studies in evolutionary experimentation and computation.
Computer Methods in Applied Mechanics and Engineering,
186(24):125–140, 2000.
[ bib 
DOI ]

[1047]

Colin R. Reeves and A. V. Eremeev.
Statistical analysis of local search landscapes.
Journal of the Operational Research Society, 55(7):687–693,
2004.
[ bib 
epub ]

[1048]

Gary R. Reeves and Juan J. Gonzalez.
A comparison of two interactive MCDM procedures.
European Journal of Operational Research, 41(2):203–209, 1989.
[ bib 
DOI ]
Keywords: artificial DM, interactive

[1049]

Patrick M. Reed, David Hadka, Jonathan D. Herman, Joseph R. Kasprzyk, and
Joshua B. Kollat.
Evolutionary multiobjective optimization in water resources: The
past, present, and future.
Advances in Water Resources, 51:438–456, 2013.
[ bib ]

[1050]

Tao Chen, Miqing Li, and Xin Yao.
Standing on the shoulders of giants: Seeding searchbased
multiobjective optimization with prior knowledge for software service
composition.
Information and Software Technology, 114:155–175, 2019.
[ bib ]
Example of deteroriation in archiving

[1051]

Gerhard Reinelt.
TSPLIB — A Traveling Salesman Problem Library.
ORSA Journal on Computing, 3(4):376–384, 1991.
[ bib ]

[1052]

Marc Reimann, Karl F. Doerner, and Richard F. Hartl.
Dants: Savings based ants divide and conquer the vehicle
routing problems.
Computers & Operations Research, 31(4):563–591, 2004.
[ bib ]

[1053]

Marc Reimann and Marco Laumanns.
Savings based ant colony optimization for the capacitated
minimum spanning tree problem.
Computers & Operations Research, 33(6):1794–1822, 2006.
[ bib 
DOI ]
The problem of connecting a set of client nodes
with known demands to a root node through a minimum
cost tree network, subject to capacity constraints
on all links is known as the capacitated minimum
spanning tree (CMST) problem. As the problem is
NPhard, we propose a hybrid ant colony
optimization (ACO) algorithm to tackle it
heuristically. The algorithm exploits two important
problem characteristics: (i) the CMST problem is
closely related to the capacitated vehicle routing
problem (CVRP), and (ii) given a clustering of
client nodes that satisfies capacity constraints,
the solution is to find a MST for each cluster,
which can be done exactly in polynomial time. Our
ACO exploits these two characteristics of the
CMST by a solution construction originally
developed for the CVRP. Given the CVRP solution,
we then apply an implementation of Prim's algorithm
to each cluster to obtain a feasible CMST
solution. Results from a comprehensive computational
study indicate the efficiency and effectiveness of
the proposed approach.
Keywords: Ant colony Optimization, Capacitated minimum
spanning tree problem

[1054]

ZhiGang Ren, ZuRen Feng, LiangJun Ke, and ZhaoJun Zhang.
New Ideas for Applying Ant Colony Optimization to the Set
Covering Problem.
Computers and Industrial Engineering, 58(4):774–784, 2010.
[ bib ]

[1055]

M. ReyesSierra and Carlos A. Coello Coello.
Multiobjective particle swarm optimizers: A survey of the
stateoftheart.
International Journal of Computational Intelligence Research,
2(3):287–308, 2006.
[ bib ]

[1056]

Craig W. Reynolds.
Flocks, Herds, and Schools: A Distributed Behavioral Model.
ACM Computer Graphics, 21(4):25–34, 1987.
[ bib ]

[1057]

Jafar Rezaei, Alireza Arab, and Mohammadreza Mehregan.
Analyzing anchoring bias in attribute weight elicitation of
SMART, Swing, and bestworst method.
International Transactions in Operational Research, 2022.
[ bib 
DOI ]
Keywords: anchoring bias, bestworst method, cognitive bias, MADM,
multiattribute weighting, SMART, Swing

[1058]

S. Reza Hejazi and S. Saghafian.
Flowshopscheduling Problems with Makespan Criterion: A Review.
International Journal of Production Research,
43(14):2895–2929, 2005.
[ bib ]

[1059]

Imma Ribas, Ramon Companys, and Xavier TortMartorell.
An iterated greedy algorithm for the flowshop scheduling problem
with blocking.
Omega, 39(3):293 – 301, 2011.
[ bib ]

[1060]

Imma Ribas, Ramon Companys, and Xavier TortMartorell.
An Efficient Iterated Local Search Algorithm for the Total
Tardiness Blocking Flow Shop Problem.
International Journal of Production Research,
51(17):5238–5252, 2013.
[ bib ]

[1061]

Celso C. Ribeiro and Sebastián Urrutia.
Heuristics for the Mirrored Traveling Tournament Problem.
European Journal of Operational Research, 179(3):775–787,
2007.
[ bib ]

[1062]

A. J. Richmond and John E. Beasley.
An Iterative Construction Heuristic for the Ore Selection
Problem.
Journal of Heuristics, 10(2):153–167, 2004.
[ bib ]

[1063]

John R. Rice.
The Algorithm Selection Problem.
Advances in Computers, 15:65–118, 1976.
[ bib 
DOI ]
The problem of selecting an effective algorithm arises in a
wide variety of situations. This chapter starts with a
discussion on abstract models: the basic model and associated
problems, the model with selection based on features, and the
model with variable performance criteria. One objective of
this chapter is to explore the applicability of the
approximation theory to the algorithm selection
problem. There is an intimate relationship here and that the
approximation theory forms an appropriate base upon which to
develop a theory of algorithm selection methods. The
approximation theory currently lacks much of the necessary
machinery for the algorithm selection problem. There is a
need to develop new results and apply known techniques to
these new circumstances. The final pages of this chapter form
a sort of appendix, which lists 15 specific open problems and
questions in this area. There is a close relationship between
the algorithm selection problem and the general optimization
theory. This is not surprising since the approximation
problem is a special form of the optimization problem. Most
realistic algorithm selection problems are of moderate to
high dimensionality and thus one should expect them to be
quite complex. One consequence of this is that most
straightforward approaches (even wellconceived ones) are
likely to lead to enormous computations for the best
selection. The single most important part of the solution of
a selection problem is the appropriate choice of the form for
selection mapping. It is here that theories give the least
guidance and that the art of problem solving is most
crucial.

[1064]

Juan Carlos Rivera, H. Murat Afsar, and Christian Prins.
A Multistart Iterated Local Search for the Multitrip Cumulative
Capacitated Vehicle Routing Problem.
Computational Optimization and Applications, 61(1):159–187,
2015.
[ bib ]

[]

Lucía Rivadeneira, JianBo Yang, and Manuel LópezIbáñez.
Predicting tweet impact using a novel evidential reasoning
prediction method.
Expert Systems with Applications, 2021.
[ bib 
DOI ]
This study presents a novel evidential reasoning (ER)
prediction model called MAKERRIMER to examine how different
features embedded in Twitter posts (tweets) can predict the
number of retweets achieved during an electoral campaign. The
tweets posted by the two most voted candidates during the
official campaign for the 2017 Ecuadorian Presidential
election were used for this research. For each tweet, five
features including type of tweet, emotion, URL, hashtag, and
date are identified and coded to predict if tweets are of
either high or low impact. The main contributions of the new
proposed model include its suitability to analyse tweet
datasets based on likelihood analysis of data. The model is
interpretable, and the prediction process relies only on the
use of available data. The experimental results show that
MAKERRIMER performed better, in terms of misclassification
error, when compared against other predictive machine
learning approaches. In addition, the model allows observing
which features of the candidates' tweets are linked to high
and low impact. Tweets containing allusions to the contender
candidate, either with positive or negative connotations,
without hashtags, and written towards the end of the
campaign, were persistently those with the highest
impact. URLs, on the other hand, is the only variable that
performs differently for the two candidates in terms of
achieving high impact. MAKERRIMER can provide campaigners of
political parties or candidates with a tool to measure how
features of tweets are predictors of their impact, which can
be useful to tailor Twitter content during electoral
campaigns.
Keywords: Evidential reasoning rule,Belief rulebased inference,Maximum
likelihood data analysis,Twitter,Retweet,Prediction

[1066]

C. P. Robert.
Simulation of truncated normal variables.
Statistics and Computing, 5(2):121–125, June 1995.
[ bib ]

[1067]

P. A. Romero, A. Krause, and F. H. Arnold.
Navigating the Protein Fitness Landscape with Gaussian
Processes.
Proceedings of the National Academy of Sciences,
110(3):E193–E201, December 2012.
[ bib 
DOI ]
Keywords: Combinatorial Blackbox Expensive

[1068]

Fabio Romeo and Alberto SangiovanniVincentelli.
A Theoretical Framework for Simulated Annealing.
Algorithmica, 6(16):302–345, 1991.
[ bib ]

[1069]

David S. Roos.
Bioinformatics–trying to swim in a sea of data.
Science, 291(5507):1260–1261, 2001.
[ bib ]

[1070]

Stefan Ropke and David Pisinger.
A Unified Heuristic for a Large Class of Vehicle Routing
Problems with Backhauls.
European Journal of Operational Research, 171(3):750–775,
2006.
[ bib ]

[1071]

Stefan Ropke and David Pisinger.
An Adaptive Large Neighborhood Search Heuristic for the Pickup
and Delivery Problme with Time Windows.
Transportation Science, 40(4):455–472, 2006.
[ bib ]

[1072]

Brian C. Ross.
Mutual Information between Discrete and Continuous Data Sets.
PLoS One, 9(2):1–5, February 2014.
[ bib 
DOI ]
Mutual information (MI) is a powerful method for detecting
relationships between data sets. There are accurate methods
for estimating MI that avoid problems with “binning” when
both data sets are discrete or when both data sets are
continuous. We present an accurate, nonbinning MI estimator
for the case of one discrete data set and one continuous data
set. This case applies when measuring, for example, the
relationship between base sequence and gene expression level,
or the effect of a cancer drug on patient survival time. We
also show how our method can be adapted to calculate the
JensenShannon divergence of two or more data sets.

[1073]

Jonathan Rose, Wolfgang Klebsch, and Jürgen Wolf.
Temperature measurement and equilibrium dynamics of simulated
annealing placements.
IEEE Transactions on ComputerAided Design of Integrated
Circuits and Systems, 9(3):253–259, 1990.
[ bib ]

[1074]

Edward Rothberg.
An evolutionary algorithm for polishing mixed integer
programming solutions.
INFORMS Journal on Computing, 19(4):534–541, 2007.
[ bib ]

[1075]

Daniel H. Rothman.
Nonlinear inversion, statistical mechanics, and residual statics
estimation.
Geophysics, 50(12):2784–2796, 1985.
[ bib ]

[1076]

Daniel H. Rothman.
Automatic estimation of large residual statics corrections.
Geophysics, 51(2):332–346, 1986.
[ bib ]

[1077]

Bernard Roy.
Robustness in operational research and decision aiding: A
multifaceted issue.
European Journal of Operational Research, 200(3):629–638,
2010.
[ bib 
DOI ]

[1078]

Günther Rudolph, Oliver Schütze, Christian Grimme, Christian
DomínguezMedina, and Heike Trautmann.
Optimal averaged Hausdorff archives for biobjective problems:
theoretical and numerical results.
Computational Optimization and Applications, 64(2):589–618,
2016.
[ bib ]

[1079]

Günther Rudolph.
Convergence analysis of canonical genetic algorithms.
IEEE Transactions on Neural Networks, 5(1):96–101, 1994.
[ bib ]

[1080]

Rubén Ruiz and C. Maroto.
A Comprehensive Review and Evaluation of Permutation Flowshop
Heuristics.
European Journal of Operational Research, 165(2):479–494,
2005.
[ bib ]

[1081]

Rubén Ruiz, C. Maroto, and Javier Alcaraz.
Two new robust genetic algorithms for the flowshop scheduling
problem.
Omega, 34(5):461–476, 2006.
[ bib 
DOI ]

[1082]

Ana Belén Ruiz, Rubén Saborido, and Mariano Luque.
A preferencebased evolutionary algorithm for multiobjective
optimization: the weighting achievement scalarizing function genetic
algorithm.
Journal of Global Optimization, 62(1):101–129, May 2015.
[ bib 
DOI ]
When solving multiobjective optimization problems,
preferencebased evolutionary multiobjective optimization
(EMO) algorithms introduce preference information into an
evolutionary algorithm in order to focus the search for
objective vectors towards the region of interest of the
Pareto optimal front. In this paper, we suggest a
preferencebased EMO algorithm called weighting achievement
scalarizing function genetic algorithm (WASFGA), which
considers the preferences of the decision maker (DM)
expressed by means of a reference point. The main purpose of
WASFGA is to approximate the region of interest of the
Pareto optimal front determined by the reference point, which
contains the Pareto optimal objective vectors that obey the
preferences expressed by the DM in the best possible way. The
proposed approach is based on the use of an achievement
scalarizing function (ASF) and on the classification of the
individuals into several fronts. At each generation of
WASFGA, this classification is done according to the values
that each solution takes on the ASF for the reference point
and using different weight vectors. These vectors of weights
are selected so that the vectors formed by their inverse
components constitute a welldistributed representation of
the weight vectors space. The efficiency and usefulness of
WASFGA is shown in several test problems in comparison to
other preferencebased EMO algorithms. Regarding a metric
based on the hypervolume, we can say that WASFGA has
outperformed the other algorithms considered in most of the
problems.
Proposed WASFGA

[1083]

Rubén Ruiz and Thomas Stützle.
A Simple and Effective Iterated Greedy Algorithm for the
Permutation Flowshop Scheduling Problem.
European Journal of Operational Research, 177(3):2033–2049,
2007.
[ bib ]

[1084]

Rubén Ruiz and Thomas Stützle.
An Iterated Greedy heuristic for the sequence dependent
setup times flowshop problem with makespan and weighted tardiness
objectives.
European Journal of Operational Research, 187(3):1143 – 1159,
2008.
[ bib ]

[1085]

Robert A. Russell.
Hybrid Heuristics for the Vehicle Routing Problem with Time
Windows.
Transportation Science, 29(2):156–166, 1995.
[ bib ]

[1086]

N. R. Sabar, M. Ayob, Graham Kendall, and R. Qu.
Grammatical Evolution HyperHeuristic for Combinatorial
Optimization Problems.
IEEE Transactions on Evolutionary Computation, 17(6):840–861,
2013.
[ bib ]

[1087]

N. R. Sabar, M. Ayob, Graham Kendall, and R. Qu.
A Dynamic Multiarmed BanditGene Expression Programming
HyperHeuristic for Combinatorial Optimization Problems.
IEEE Transactions on Cybernetics, 45(2):217–228, 2015.
[ bib ]

[1088]

N. R. Sabar, M. Ayob, Graham Kendall, and R. Qu.
Automatic Design of a HyperHeuristic Framework With Gene
Expression Programming for Combinatorial Optimization Problems.
IEEE Transactions on Evolutionary Computation, 19(3):309–325,
2015.
[ bib ]

[1089]

Matthieu Sacher, Régis Duvigneau, Olivier Le Maitre, Mathieu Durand, Elisa
Berrini, Frédéric Hauville, and JacquesAndré Astolfi.
A classification approach to efficient global optimization in
presence of noncomputable domains.
Structural and Multidisciplinary Optimization,
58(4):1537–1557, 2018.
[ bib 
DOI ]
Proposed EGOLSSVM
Keywords: Safe optimization; CMAES, Gaussian processes; LeastSquares
Support Vector Machine

[1090]

Pramod J. Sadalage and Martin Fowler.
NoSQL distilled.
AddisonWesley Professional, 2012.
[ bib ]

[1091]

A. Burcu Altan Sakarya and Larry W. Mays.
Optimal Operation of Water Distribution Pumps Considering Water
Quality.
Journal of Water Resources Planning and Management, ASCE,
126(4):210–220, July / August 2000.
[ bib ]

[1092]

Marcela Samà, Paola Pellegrini, Andrea D'Ariano, Joaquin Rodriguez, and Dario
Pacciarelli.
Ant colony optimization for the realtime train routing
selection problem.
Transportation Research Part B: Methodological, 85:89–108,
2016.
[ bib 
DOI ]
Keywords: irace

[1093]

Malcolm Sambridge.
Geophysical inversion with a neighbourhood algorithm–I.
Searching a parameter space.
Geophysical Journal International, 138(2):479–494, 1999.
[ bib ]

[1094]

Alejandro Santiago, Bernabé Dorronsoro, Antonio J. Nebro, Juan J. Durillo,
Oscar Castillo, and Héctor J. Fraire.
A novel multiobjective evolutionary algorithm with fuzzy logic
based adaptive selection of operators: FAME.
Information Sciences, 471:233–251, 2019.
[ bib 
DOI ]
Keywords: Multiobjective optimization, density estimation,
evolutionary algorithm, adaptive algorithm, fuzzy logic, spatial spread deviation

[1095]

Javier Sánchez, Manuel Galán, and Enrique Rubio.
Applying a traffic lights evolutionary optimization technique to
a real case: “Las Ramblas” area in Santa Cruz de Tenerife.
IEEE Transactions on Evolutionary Computation, 12(1):25–40,
2008.
[ bib ]
Keywords: Cellular automata (CA),Combinatorial optimization,Ggenetic
algorithms (GAs),Microscopic traffic simulator,Traffic lights
optimization

[1096]

J. J. SánchezMedina, M. J. GalánMoreno, and E. RubioRoyo.
Traffic Signal Optimization in “La Almozara” District in
Saragossa Under Congestion Conditions, Using Genetic Algorithms, Traffic
Microsimulation, and Cluster Computing.
IEEE Transactions on Intelligent Transportation Systems,
11(1):132–141, March 2010.
[ bib 
DOI ]
Keywords: cellular automata;genetic algorithms;road traffic;road
vehicles;traffic engineering computing;Beowulf cluster;La
Almozara district;Saragossa;cellular automata;cluster
computing;genetic algorithm;multipleinstruction multiple
data;traffic light programming;traffic
microsimulation;traffic signal optimization;urban traffic
congestion;Cellular automata (CA);genetic algorithms
(GAs);intelligent transportation
systems;microsimulation;traffic congestion;traffic modeling

[1097]

Nathan Sankary and Avi Ostfeld.
Stochastic Scenario Evaluation in Evolutionary Algorithms Used
for Robust ScenarioBased Optimization.
Water Resources Research, 54(4):2813–2833, 2018.
[ bib ]

[1098]

Alberto Santini, Stefan Ropke, and Lars Magnus Hvattum.
A comparison of acceptance criteria for the adaptive large
neighbourhood search metaheuristic.
Journal of Heuristics, 24:783–815, 2018.
[ bib 
DOI ]

[1099]

E. Sandgren.
Nonlinear integer and discrete programming in mechanical design
optimization.
Journal of Mechanical Design, 112(2):223–229, 1990.
[ bib 
DOI ]

[1100]

Martin W. P. Savelsbergh.
Local search in routing problems with time windows.
Annals of Operations Research, 4(1):285–305, December 1985.
[ bib 
DOI ]
We develop local search algorithms for routing
problems with time windows. The presented algorithms
are based on thekinterchange concept. The presence
of time windows introduces feasibility constraints,
the checking of which normally requires O(N)
time. Our method reduces this checking effort to
O(1) time. We also consider the problem of finding
initial solutions. A complexity result is given and
an insertion heuristic is described.

[1101]

Dhish Kumar Saxena, João A. Duro, Anish Tiwari, Kalyanmoy Deb, and Qingfu
Zhang.
Objective Reduction in ManyObjective Optimization: Linear and
Nonlinear Algorithms.
IEEE Transactions on Evolutionary Computation, 17(1):77–99,
2013.
[ bib 
DOI ]

[1102]

Michael Schilde, Karl F. Doerner, Richard F. Hartl, and Guenter Kiechle.
Metaheuristics for the biobjective orienteering problem.
Swarm Intelligence, 3(3):179–201, 2009.
[ bib 
DOI ]
In this paper, heuristic solution
techniques for the multiobjective orienteering
problem are developed. The motivation stems from the
problem of planning individual tourist routes in a
city. Each point of interest in a city provides
different benefits for different categories (e.g.,
culture, shopping). Each tourist has different
preferences for the different categories when
selecting and visiting the points of interests
(e.g., museums, churches). Hence, a multiobjective
decision situation arises. To determine all the
Pareto optimal solutions, two metaheuristic search
techniques are developed and applied. We use the
Pareto ant colony optimization algorithm and extend
the design of the variable neighborhood search
method to the multiobjective case. Both methods are
hybridized with path relinking procedures. The
performances of the two algorithms are tested on
several benchmark instances as well as on real world
instances from different Austrian regions and the
cities of Vienna and Padua. The computational
results show that both implemented methods are well
performing algorithms to solve the multiobjective
orienteering problem.

[1103]

Martin Schlüter, Jose A. Egea, and Julio R. Banga.
Extended ant colony optimization for nonconvex mixed integer
nonlinear programming.
Computers & Operations Research, 36(7):2217–2229, 2009.
[ bib 
DOI ]

[1104]

Oliver Schütze, X. Esquivel, A. Lara, and Carlos A. Coello Coello.
Using the Averaged Hausdorff Distance as a Performance Measure
in Evolutionary Multiobjective Optimization.
IEEE Transactions on Evolutionary Computation, 16(4):504–522,
2012.
[ bib ]

[1105]

Josef Schmee and Gerald J. Hahn.
A Simple Method for Regression Analysis with Censored Data.
Technometrics, 21(4):417–432, 1979.
[ bib 
DOI ]

[1106]

Mark Schillinger, Benjamin Hartmann, Patric Skalecki, Mona Meister, Duy
NguyenTuong, and Oliver Nelles.
Safe active learning and safe Bayesian optimization for tuning
a PIcontroller.
IFACPapersOnLine, 50(1):5967–5972, 2017.
[ bib 
DOI ]

[1107]

Julie R. Schames, Richard H. Henchman, Jay S. Siegel, Christoph A. Sotriffer,
Haihong Ni, and J. Andrew McCammon.
Discovery of a Novel Binding Trench in HIV Integrase.
Journal of Medicinal Chemistry, 47(8):1879–1881, 2004.
[ bib 
DOI ]
Evolutionary optimization of the first clinically approved
antiviral drug for HIV

[1108]

Oliver Schütze, Carlos Hernández, ElGhazali Talbi, JianQiao Sun,
Yousef Naranjani, and FR Xiong.
Archivers for the representation of the set of approximate
solutions for MOPs.
Journal of Heuristics, 25:71–105, 2019.
[ bib 
DOI ]
Keywords: archiving, nearly optimality, epsilondominance, epsilonapproximation, hausdorff convergence

[1109]

Jeffrey C. Schank and Thomas J. Koehnle.
Pseudoreplication is a pseudoproblem.
Journal of Comparative Psychology, 123(4):421–433, 2009.
[ bib ]

[1110]

Oliver Schütze, A. Lara, and Carlos A. Coello Coello.
On the Influence of the Number of Objectives on the Hardness of
a Multiobjective Optimization Problem.
IEEE Transactions on Evolutionary Computation, 15(4):444–455,
2011.
[ bib ]

[1111]

Oliver Schütze, Marco Laumanns, Carlos A. Coello Coello, Michael
Dellnitz, and ElGhazali Talbi.
Convergence of stochastic search algorithms to finite size
Pareto set approximations.
Journal of Global Optimization, 41(4):559–577, 2008.
[ bib ]

[1112]

Oliver Schütze, Marco Laumanns, Emilia Tantar, Carlos A. Coello Coello,
and ElGhazali Talbi.
Computing gap free Pareto front approximations with stochastic
search algorithms.
Evolutionary Computation, 18(1):65–96, 2010.
[ bib ]

[1113]

G. R. Schreiber and Olivier Martin.
Cut Size Statistics of Graph Bisection Heuristics.
SIAM Journal on Optimization, 10(1):231–251, 1999.
[ bib ]

[1114]

Gerhard Schrimpf, Johannes Schneider, Hermann StammWilbrandt, and Gunter
Dueck.
Record Breaking Optimization Results Using the Ruin and Recreate
Principle.
Journal of Computational Physics, 159(2):139–171, 2000.
[ bib ]

[1115]

Marie Schmidt, Anita Schöbel, and Lisa Thom.
Minordering and maxordering scalarization methods for
multiobjective robust optimization.
European Journal of Operational Research, 275(2):446–459,
2019.
[ bib ]

[1116]

Eric Schulz, Maarten Speekenbrink, and Andreas Krause.
A tutorial on Gaussian process regression: Modelling,
exploring, and exploiting functions.
Journal of Mathematical Psychology, 85:1–16, August 2018.
[ bib 
DOI ]

[1117]

Tommaso Schiavinotto and Thomas Stützle.
The Linear Ordering Problem: Instances, Search Space Analysis
and Algorithms.
Journal of Mathematical Modelling and Algorithms,
3(4):367–402, 2004.
[ bib ]

[1118]

Tommaso Schiavinotto and Thomas Stützle.
A Review of Metrics on Permutations for Search Space Analysis.
Computers & Operations Research, 34(10):3143–3153, 2007.
[ bib ]

[1119]

Tom Schrijvers, Guido Tack, Pieter Wuille, Horst Samulowitz, and Peter J.
Stuckey.
Search Combinators.
Constraints, 18(2):269–305, 2013.
[ bib ]

[1120]

Oliver Schütze, Massimiliano Vasile, and Carlos A. Coello Coello.
Computing the Set of EpsilonEfficient Solutions in
Multiobjective Space Mission Design.
Journal of Aerospace Computing, Information, and Communication,
8(3):53–70, 2011.
[ bib 
DOI ]

[1121]

Matthias Schonlau, William J. Welch, and Donald R. Jones.
Global versus Local Search in Constrained Optimization of
Computer Models.
Lecture NotesMonograph Series, 34:11–25, 1998.
[ bib 
DOI ]

[1122]

Pauli Virtanen et al.
SciPy 1.0: Fundamental Algorithms for Scientific Computing in
Python.
Nature Methods, 17:261–272, 2020.
[ bib 
DOI 
epub ]

[1123]

Babooshka Shavazipour, Manuel LópezIbáñez, and Kaisa Miettinen.
Visualizations for Decision Support in Scenariobased
Multiobjective Optimization.
Information Sciences, 578:1–21, 2021.
[ bib 
DOI 
supplementary material ]
We address challenges of decision problems when managers need
to optimize several conflicting objectives simultaneously
under uncertainty. We propose visualization tools to support
the solution of such scenariobased multiobjective
optimization problems. Suitable graphical visualizations are
necessary to support managers in understanding, evaluating,
and comparing the performances of management decisions
according to all objectives in all plausible scenarios. To
date, no appropriate visualization has been suggested. This
paper fills this gap by proposing two visualization methods:
a novel extension of empirical attainment functions for
scenarios and an adapted version of heatmaps. They help a
decisionmaker in gaining insight into realizations of
tradeoffs and comparisons between objective functions in
different scenarios. Some fundamental questions that a
decisionmaker may wish to answer with the help of
visualizations are also identified. Several examples are
utilized to illustrate how the proposed visualizations
support a decisionmaker in evaluating and comparing
solutions to be able to make a robust decision by answering
the questions. Finally, we validate the usefulness of the
proposed visualizations in a realworld problem with a real
decisionmaker. We conclude with guidelines regarding which
of the proposed visualizations are best suited for different
problem classes.

[1124]

Weishi Shao, Dechang Pi, and Zhongshi Shao.
Memetic algorithm with node and edge histogram for noidle flow
shop scheduling problem to minimize the makespan criterion.
Applied Soft Computing, 54:164–182, 2017.
[ bib ]

[1125]

Weishi Shao, Dechang Pi, and Zhongshi Shao.
A hybrid discrete teachinglearning based metaheuristic for
solving noidle flow shop scheduling problem with total tardiness criterion.
Computers & Operations Research, 94:89–105, 2018.
[ bib ]

[1126]

Ke Shang, Tianye Shu, Hisao Ishibuchi, Yang Nan, and Lie Meng Pang.
Benchmarking largescale subset selection in evolutionary
multiobjective optimization.
Information Sciences, 622:755–770, 2023.
[ bib 
DOI ]

[1127]

Babooshka Shavazipour and T. J. Stewart.
Multiobjective optimisation under deep uncertainty.
Operational Research, September 2019.
[ bib 
DOI ]
This paper presents a scenariobased MultiObjective
structure to handle decision problems under deep
uncertainty. Most of the decisions in reallife problems need
to be made in the absence of complete knowledge about the
consequences of the decision and/or are characterised by
uncertainties about the future which is unpredictable. These
uncertainties are almost impossible to reduce by gathering
more information and are not statistical in
nature. Therefore, classical probabilitybased approaches,
such as stochastic programming, do not address these
problems; as they require a correctlydefined complete sample
space, strong assumptions (e.g. normality), or both. The
proposed method extends the concept of twostage stochastic
programming with recourse to address the capability of
dealing with deep uncertainty through the use of scenario
planning rather than statistical expectation. In this
research, scenarios are used as a dimension of preference to
avoid problems relating to the assessment and use of
probabilities under deep uncertainty. Such scenariobased
thinking involved a multiobjective representation of
performance under different future conditions as an
alternative to expectation. To the best of our knowledge,
this is the first attempt of performing a multicriteria
evaluation under deep uncertainty through a structured
optimisation model. The proposed structure replacing
probabilities (in dynamic systems with deep uncertainties) by
aspirations within a goal programming structure. In fact,
this paper also proposes an extension of the goal programming
paradigm to deal with deep uncertainty. Furthermore, we will
explain how this structure can be modelled, implemented, and
solved by Goal Programming using some simple, but not
trivial, examples. Further discussion and comparisons with
some popular existing methods will also provided to highlight
the superiorities of the proposed structure.

[1128]

Babooshka Shavazipour, Jonas Stray, and T. J. Stewart.
Sustainable planning in sugarbioethanol supply chain under deep
uncertainty: A case study of South African sugarcane industry.
Computers & Chemical Engineering, 143:107091, 2020.
[ bib 
DOI ]
In this paper, the strategic planning of sugarbioethanol
supply chains (SCs) under deep uncertainty has been addressed
by applying a twostage scenariobased multiobjective
optimisation methodology. In practice, the depth of
uncertainty is very high, potential outcomes are not
precisely enumerable, and probabilities of outcomes are not
properly definable. To date, no appropriate framework has
been suggested for dealing with deep uncertainty in supply
chain management and energyrelated problems. This study is
the first try to fills this gap. Particularly, the
sustainability of the whole infrastructure of the
sugarbioethanol SCs is analysed in such a way that the final
solutions are sustainable, robust and adaptable for a broad
range of plausible futures. Three objectives are considered
in this problem under six uncertain parameters. A case study
of South African sugarcane industry is utilised to study and
examine the proposed model. The results prove the economic
profitability and sustainability of the project.
Keywords: Supply chain management, Multiobjective optimisation, Deep
uncertainty, Scenario planning, Renewable energy,

[1129]

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and Nando de Freitas.
Taking the human out of the loop: A review of Bayesian
optimization.
Proceedings of the IEEE, 104(1):148–175, 2016.
[ bib ]

[1130]

Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas.
Taking the Human Out of the Loop: A Review of Bayesian
Optimization.
Proceedings of the IEEE, 104(1):148–175, 2016.
[ bib ]

[1131]

Ofer M. Shir and Thomas Bäck.
Niching with derandomized evolution strategies in artificial and
realworld landscapes.
Natural Computing, 8(1):171–196, 2009.
[ bib 
DOI ]

[1132]

David Shilane, Jarno Martikainen, Sandrine Dudoit, and Seppo J. Ovaska.
A general framework for statistical performance comparison of
evolutionary computation algorithms.
Information Sciences, 178(14):2870–2879, 2008.
[ bib 
DOI ]

[1133]

Michael D. Shields and Jiaxin Zhang.
The generalization of Latin hypercube sampling.
Reliability Engineering & System Safety, 148:96–108, 2016.
[ bib ]

[1134]

A. Shmygelska and Holger H. Hoos.
An Ant Colony Optimisation Algorithm for the 2D and 3D
Hydrophobic Polar Protein Folding Problem.
BMC Bioinformatics, 6:30, 2005.
[ bib 
DOI ]

[1135]

Moisés SilvaMuñoz, Alberto Franzin, and Hughes Bersini.
Automatic configuration of the Cassandra database using irace.
PeerJ Computer Science, 7:e634, 2021.
[ bib 
DOI ]

[1136]

Paulo Vitor Silvestrin and Marcus Ritt.
An Iterated Tabu Search for the Multicompartment Vehicle
Routing Problem.
Computers & Operations Research, 81:192–202, 2017.
[ bib ]

[1137]

Marcos Melo Silva, Anand Subramanian, and Luiz Satoru Ochi.
An Iterated Local Search Heuristic for the Split Delivery
Vehicle Routing Problem.
Computers & Operations Research, 53:234–249, 2015.
[ bib ]

[1138]

Olivier Simonin, François Charpillet, and Eric Thierry.
Revisiting wavefront construction with collective agents: an
approach to foraging.
Swarm Intelligence, 9(2):113–138, 2014.
[ bib 
DOI ]
Keywords: irace

[1139]

Kevin Sim, Emma Hart, and Ben Paechter.
A Lifelong Learning Hyperheuristic Method for Bin Packing.
Evolutionary Computation, 23(1):37–67, 2015.
[ bib 
DOI ]

[1140]

Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn.
FalsePositive Psychology: Undisclosed Flexibility in Data
Collection and Analysis Allows Presenting Anything as Significant.
Psychological Science, 2011.
[ bib 
http ]
Proposed the term phacking

[1141]

Herbert A. Simon and Allen Newell.
Heuristic Problem Solving: The Next Advance in Operations
Research.
Operations Research, 6(1):1–10, 1958.
[ bib 
DOI ]

[1142]

Herbert A. Simon.
A Behavioral Model of Rational Choice.
The Quarterly Journal of Economics, 69(1):99–118, 1955.
[ bib 
epub ]

[1143]

Hemant Kumar Singh, Amitay Isaacs, and Tapabrata Ray.
A Pareto Corner Search Evolutionary Algorithm and
Dimensionality Reduction in ManyObjective Optimization Problems.
IEEE Transactions on Evolutionary Computation, 15(4):539–556,
2011.
[ bib 
DOI ]
Manyobjective optimization refers to the optimization
problems containing large number of objectives, typically
more than four. Nondominance is an inadequate strategy for
convergence to the Pareto front for such problems, as almost
all solutions in the population become nondominated,
resulting in loss of convergence pressure. However, for some
problems, it may be possible to generate the Pareto front
using only a few of the objectives, rendering the rest of the
objectives redundant. Such problems may be reducible to a
manageable number of relevant objectives, which can be
optimized using conventional multiobjective evolutionary
algorithms (MOEAs). For dimensionality reduction, most
proposals in the paper rely on analysis of a representative
set of solutions obtained by running a conventional MOEA for
a large number of generations, which is computationally
overbearing. A novel algorithm, Pareto corner search
evolutionary algorithm (PCSEA), is introduced in this paper,
which searches for the corners of the Pareto front instead of
searching for the complete Pareto front. The solutions
obtained using PCSEA are then used for dimensionality
reduction to identify the relevant objectives. The potential
of the proposed approach is demonstrated by studying its
performance on a set of benchmark test problems and two
engineering examples. While the preliminary results obtained
using PCSEA are promising, there are a number of areas that
need further investigation. This paper provides a number of
useful insights into dimensionality reduction and, in
particular, highlights some of the roadblocks that need to be
cleared for future development of algorithms attempting to
use few selected solutions for identifying relevant
objectives

[1144]

Marcos Singer and Michael L. Pinedo.
A Computational Study of Branch and Bound Techniques for
Minimizing the Total Weighted Tardiness in Job Shops.
IIE Transactions, 30(2):109–118, 1998.
[ bib ]

[1145]

Ankur Sinha, Dhish Kumar Saxena, Kalyanmoy Deb, and Ashutosh Tiwari.
Using objective reduction and interactive procedure to handle
manyobjective optimization problems.
Applied Soft Computing, 13(1):415–427, 2013.
[ bib 
DOI ]
A number of practical optimization problems are posed as
manyobjective (more than three objectives) problems. Most of
the existing evolutionary multiobjective optimization
algorithms, which target the entire Paretofront are not
equipped to handle manyobjective problems. Though there have
been copious efforts to overcome the challenges posed by such
problems, there does not exist a generic procedure to
effectively handle them. This paper presents a simplify and
solve framework for handling manyobjective optimization
problems. In that, a given problem is simplified by
identification and elimination of the redundant objectives,
before interactively engaging the decision maker to converge
to the most preferred solution on the Paretooptimal
front. The merit of performing objective reduction before
interacting with the decision maker is two fold. Firstly, the
revelation that certain objectives are redundant,
significantly reduces the complexity of the optimization
problem, implying lower computational cost and higher search
efficiency. Secondly, it is well known that human beings are
not efficient in handling several factors (objectives in the
current context) at a time. Hence, simplifying the problem a
priori addresses the fundamental issue of cognitive overload
for the decision maker, which may help avoid inconsistent
preferences during the different stages of interactive
engagement. The implementation of the proposed framework is
first demonstrated on a threeobjective problem, followed by
its application on two realworld engineering problems.
Keywords: Evolutionary algorithms, Evolutionary multi and
manyobjective optimization, Multicriteria decision making,
Machine learning, Interactive optimization

[1146]

Hemant Kumar Singh, Kalyan Shankar Bhattacharjee, and Tapabrata Ray.
Distancebased subset selection for benchmarking in evolutionary
multi/manyobjective optimization.
IEEE Transactions on Evolutionary Computation, 23(5):904–912,
2019.
[ bib ]

[1147]

Aymen Sioud and Caroline Gagné.
Enhanced migrating birds optimization algorithm for the
permutation flow shop problem with sequence dependent setup times.
European Journal of Operational Research, 264(1):66–73, 2018.
[ bib ]

[1148]

Ben G. Small, Barry W. McColl, Richard Allmendinger, Jürgen Pahle, Gloria
LópezCastejón, Nancy J. Rothwell, Joshua D. Knowles, Pedro Mendes,
David Brough, and Douglas B. Kell.
Efficient discovery of antiinflammatory smallmolecule
combinations using evolutionary computing.
Nature Chemical Biology, 7(12):902–908, 2011.
[ bib ]

[1149]

Kate SmithMiles and Simon Bowly.
Generating New Test Instances by Evolving in Instance Space.
Computers & Operations Research, 63:102–113, 2015.
[ bib ]

[1150]

Kate SmithMiles, Jeffrey Christiansen, and Mario A. Muñoz.
Revisiting where are the hard knapsack problems? via Instance
Space Analysis.
Computers & Operations Research, 128:105184, 2021.
[ bib ]

[1151]

Kate SmithMiles and Leo Lopes.
Measuring instance difficulty for combinatorial optimization
problems.
Computers & Operations Research, 39:875–889, 2012.
[ bib ]

[1152]

Kate SmithMiles.
Crossdisciplinary Perspectives on Metalearning for Algorithm
Selection.
ACM Computing Surveys, 41(1):1–25, 2008.
[ bib ]

[1153]

Krzysztof Socha and Christian Blum.
An ant colony optimization algorithm for continuous
optimization: An application to feedforward neural network training.
Neural Computing & Applications, 16(3):235–247, 2007.
[ bib ]

[1154]

Krzysztof Socha and Marco Dorigo.
Ant Colony Optimization for Continuous Domains.
European Journal of Operational Research, 185(3):1155–1173,
2008.
[ bib 
DOI ]
Proposed ACOR (ACO_{}R)
Keywords: ACOR

[1155]

Christine Solnon.
Ants Can Solve Constraint Satisfaction Problems.
IEEE Transactions on Evolutionary Computation, 6(4):347–357,
2002.
[ bib ]

[1156]

D. Soler, E. Martínez, and J. C. Micó.
A Transformation for the Mixed General Routing Problem with Turn
Penalties.
Journal of the Operational Research Society, 59:540–547, 2008.
[ bib ]

[1157]

M. M. Solomon.
Algorithms for the Vehicle Routing and Scheduling Problems with
Time Windows.
Operations Research, 35:254–265, 1987.
[ bib ]

[1158]

Zhenshou Song, Handing Wang, Cheng He, and Yaochu Jin.
A Krigingassisted twoarchive evolutionary algorithm for
expensive manyobjective optimization.
IEEE Transactions on Evolutionary Computation,
25(6):1013–1027, 2021.
[ bib ]

[1159]

Kenneth Sörensen.
Metaheuristics—the metaphor exposed.
International Transactions in Operational Research,
22(1):3–18, 2015.
[ bib 
DOI ]

[1160]

Kenneth Sörensen, Florian Arnold, and Daniel Palhazi Cuervo.
A critical analysis of the “improved Clarke and Wright savings
algorithm”.
International Transactions in Operational Research,
26(1):54–63, 2017.
[ bib 
DOI ]
Keywords: reproducibility, vehicle routing

[1161]

Jorge A. SoriaAlcaraz, Gabriela Ochoa, Marco A. SoteloFigeroa, and Edmund K.
Burke.
A Methodology for Determining an Effective Subset of Heuristics
in Selection Hyperheuristics.
European Journal of Operational Research, 260:972–983, 2017.
[ bib ]

[1162]

Marcelo De Souza, Marcus Ritt, and Manuel LópezIbáñez.
Capping Methods for the Automatic Configuration of Optimization
Algorithms.
Computers & Operations Research, 139:105615, 2022.
[ bib 
DOI 
supplementary material ]
Automatic configuration techniques are widely and
successfully used to find good parameter settings for
optimization algorithms. Configuration is costly, because it
is necessary to evaluate many configurations on different
instances. For decision problems, when the objective is to
minimize the running time of the algorithm, many
configurators implement capping methods to discard poor
configurations early. Such methods are not directly
applicable to optimization problems, when the objective is to
optimize the cost of the best solution found, given a
predefined running time limit. We propose new capping methods
for the automatic configuration of optimization
algorithms. They use the previous executions to determine a
performance envelope, which is used to evaluate new
executions and cap those that do not satisfy the envelope
conditions. We integrate the capping methods into the irace
configurator and evaluate them on different optimization
scenarios. Our results show that the proposed methods can
save from about 5% to 78% of the configuration effort,
while finding configurations of the same quality. Based on
the computational analysis, we identify two conservative and
two aggressive methods, that save an average of about 20%
and 45% of the configuration effort, respectively. We also
provide evidence that capping can help to better use the
available budget in scenarios with a configuration time
limit.

[1163]

Abdelghani Souilah.
Simulated annealing for manufacturing systems layout design.
European Journal of Operational Research, 82(3):592–614, 1995.
[ bib ]

[1164]

Charles Spearman.
The proof and measurement of association between two things.
The American journal of psychology, 15(1):72–101, 1904.
[ bib ]

[1165]

J. L. Henning.
SPEC CPU2000: measuring CPU performance in the New
Millennium.
Computer, 33(7):28–35, 2000.
[ bib 
DOI ]

[1166]

Daniel A. Spielman and ShangHua Teng.
Smoothed analysis of algorithms: Why the simplex algorithm
usually takes polynomial time.
Journal of the ACM, 51(3):385–463, 2004.
[ bib ]

[1167]

Arno Sprecher, Sönke Hartmann, and Andreas Drexl.
An exact algorithm for project scheduling with multiple modes.
OR Spektrum, 19(3):195–203, 1997.
[ bib 
DOI ]
Keywords: branchandbound, multimode resourceconstrained
project scheduling, project scheduling

[1168]

Arno Sprecher, Rainer Kolisch, and Andreas Drexl.
Semiactive, active, and nondelay schedules for the
resourceconstrained project scheduling problem.
European Journal of Operational Research, 80(1):94–102, 1995.
[ bib 
DOI ]
We consider the resourceconstrained project
scheduling problem (RCPSP). The focus of the paper
is on a formal definition of semiactive, active,
and nondelay schedules. Traditionally these
schedules establish basic concepts within the job
shop scheduling literature. There they are usually
defined in a rather informal way which does not
create any substantial problems. Using these
concepts in the more general RCPSP without giving
a formal definition may cause serious
problems. After providing a formal definition of
semiactive, active, and nondelay schedules for the
RCPSP we outline some of these problems occurring
within the disjunctive arc concept.
Keywords: active schedules, Branchandbound methods,
nondelay schedules, Resourceconstrained project
scheduling, Semiactive schedules

[1169]

N. Srinivas and Kalyanmoy Deb.
Multiobjective Optimization Using Nondominated Sorting in
Genetic Algorithms.
Evolutionary Computation, 2(3):221–248, 1994.
[ bib ]

[1170]

T. J. Stewart.
Robustness of Additive Value Function Methods in MCDM.
Journal of MultiCriteria Decision Analysis, 5(4):301–309,
1996.
[ bib ]
Keywords: machine decisionmaking

[1171]

T. J. Stewart.
Evaluation and refinement of aspirationbased methods in
MCDM.
European Journal of Operational Research, 113(3):643–652,
1999.
[ bib ]
Keywords: machine decisionmaking

[1172]

T. J. Stewart.
Goal programming and cognitive biases in decisionmaking.
Journal of the Operational Research Society, 56(10):1166–1175,
2005.
[ bib 
DOI ]
Keywords: machine decision making

[1173]

T. J. Stewart, Simon French, and Jesus Rios.
Integrating multicriteria decision analysis and scenario
planning: Review and extension.
Omega, 41(4):679–688, 2013.
[ bib 
DOI ]
Keywords: Multicriteria decision analysis

[1174]

Helena Stegherr, Michael Heider, and Jörg Hähner.
Classifying Metaheuristics: Towards a unified multilevel
classification system.
Natural Computing, 2020.
[ bib 
DOI ]

[1175]

Sarah Steiner and Tomasz Radzik.
Computing all efficient solutions of the biobjective minimum
spanning tree problem.
Computers & Operations Research, 35(1):198–211, 2008.
[ bib ]

[1176]

Victoria Stodden.
What scientific idea is ready for retirement? Reproducibility.
Edge, 2014.
[ bib 
http ]
Introduces computational reproducibility, empirical
reproducibility and statistical reproducibility

[1177]

Daniel H. Stolfi and Enrique Alba.
Red Swarm: Reducing travel times in smart cities by using
bioinspired algorithms.
Applied Soft Computing, 24:181–195, 2014.
[ bib 
DOI ]
This article presents an innovative approach to solve one of
the most relevant problems related to smart mobility: the
reduction of vehicles' travel time. Our original approach,
called Red Swarm, suggests a potentially customized route to
each vehicle by using several spots located at traffic lights
in order to avoid traffic jams by using {V2I}
communications. That is quite different from other existing
proposals, as it deals with real maps and actual streets, as
well as several road traffic distributions. We propose an
evolutionary algorithm (later efficiently parallelized) to
optimize our case studies which have been imported from
OpenStreetMap into {SUMO} as they belong to a real city. We
have also developed a Rerouting Algorithm which accesses the
configuration of the Red Swarm and communicates the route
chosen to vehicles, using the spots (via WiFi
link). Moreover, we have developed three competing algorithms
in order to compare their results to those of Red Swarm and
have observed that Red Swarm not only achieved the best
results, but also outperformed the experts' solutions in a
total of 60 scenarios tested, with up to 19% shorter travel
times.
Keywords: Evolutionary algorithm,Road traffic,Smart city,Smart
mobility,Traffic light,WiFi connections

[1178]

Victoria Stodden, Marcia McNutt, David H. Bailey, Ewa Deelman, Yolanda Gil,
Brooks Hanson, Michael A. Heroux, John P. A. Ioannidis, and Michela Taufer.
Enhancing reproducibility for computational methods.
Science, 354(6317):1240–1241, December 2016.
[ bib 
DOI ]

[1179]

Rainer Storn and Kenneth Price.
Differential Evolution – A Simple and Efficient Heuristic for
Global Optimization over Continuous Spaces.
Journal of Global Optimization, 11(4):341–359, 1997.
[ bib 
DOI ]
Proposed differential evolution

[1180]

Victoria Stodden, Jennifer Seiler, and Zhaokun Ma.
An empirical analysis of journal policy effectiveness for
computational reproducibility.
Proceedings of the National Academy of Sciences,
115(11):2584–2589, March 2018.
[ bib 
DOI ]

[1181]

Philip N. Strenski and Scott Kirkpatrick.
Analysis of Finite Length Annealing Schedules.
Algorithmica, 6(16):346–366, 1991.
[ bib ]

[1182]

Patrycja Strycharczuk, Manuel LópezIbáñez, Georgina Brown, and
Adrian Leemann.
General Northern English: Exploring regional variation in the
North of England with machine learning.
Frontiers in Artificial Intelligence, 2020.
[ bib 
DOI ]
In this paper, we present a novel computational approach to the analysis of accent variation. The case study is dialect leveling in the North of England, manifested as reduction of accent variation across the North and emergence of General Northern English (GNE), a panregional standard accent associated with middleclass speakers. We investigated this instance of dialect leveling using random forest classification, with audio data from a crowdsourced corpus of 105 urban, mostly highlyeducated speakers from five northern UK cities: Leeds, Liverpool, Manchester, Newcastle upon Tyne, and Sheffield. We trained random forest models to identify individual northern cities from a sample of other northern accents, based on first two formant measurements of full vowel systems. We tested the models using unseen data. We relied on undersampling, bagging (bootstrap aggregation) and leaveoneout crossvalidation to address some challenges associated with the data set, such as unbalanced data and relatively small sample size. The accuracy of classification provides us with a measure of relative similarity between different pairs of cities, while calculating conditional feature importance allows us to identify which input features (which vowels and which formants) have the largest influence in the prediction. We do find a considerable degree of leveling, especially between Manchester, Leeds and Sheffield, although some differences persist. The features that contribute to these differences most systematically are typically not the ones discussed in previous dialect descriptions. We propose that the most systematic regional features are also not salient, and as such, they serve as sociolinguistic regional indicators. We supplement the random forest results with a more traditional variationist description of bycity vowel systems, and we use both sources of evidence to inform a description of the vowels of General Northern English.
Keywords: vowels, accent features, dialect leveling, Random forest
(bagging), Feature selecion

[1183]

Thomas Stützle.
Iterated Local Search for the Quadratic Assignment Problem.
European Journal of Operational Research, 174(3):1519–1539,
2006.
[ bib ]

[1184]

Thomas Stützle and Marco Dorigo.
A Short Convergence Proof for a Class of ACO Algorithms.
IEEE Transactions on Evolutionary Computation, 6(4):358–365,
2002.
[ bib ]

[1185]

Thomas Stützle and Holger H. Hoos.
MaxMin Ant System.
Future Generation Computer Systems, 16(8):889–914, 2000.
[ bib ]

[1186]

Zhaopin Su, Guofu Zhang, Feng Yue, Dezhi Zhan, Miqing Li, Bin Li, and Xin Yao.
Enhanced Constraint Handling for ReliabilityConstrained
Multiobjective Testing Resource Allocation.
IEEE Transactions on Evolutionary Computation, 25(3):537–551,
2021.
[ bib ]

[1187]

Anand Subramanian and Maria Battarra.
An Iterated Local Search Algorithm for the Travelling Salesman
Problem with Pickups and Deliveries.
Journal of the Operational Research Society, 64(3):402–409,
2013.
[ bib ]

[1188]

Anand Subramanian, Maria Battarra, and Chris N. Potts.
An Iterated Local Search Heuristic for the Single Machine Total
Weighted Tardiness Scheduling Problem with Sequencedependent Setup Times.
International Journal of Production Research, 52(9):2729–2742,
2014.
[ bib ]

[1189]

Yanan Sui, Vincent Zhuang, Joel W. Burdick, and Yisong Yue.
Stagewise Safe Bayesian Optimization with Gaussian
Processes.
Arxiv preprint arXiv:1806.07555, 2018.
Published as [2454].
[ bib 
http ]
Enforcing safety is a key aspect of many problems pertaining
to sequential decision making under uncertainty, which
require the decisions made at every step to be both
informative of the optimal decision and also safe. For
example, we value both efficacy and comfort in medical
therapy, and efficiency and safety in robotic control. We
consider this problem of optimizing an unknown utility
function with absolute feedback or preference feedback
subject to unknown safety constraints. We develop an
efficient safe Bayesian optimization algorithm, StageOpt,
that separates safe region expansion and utility function
maximization into two distinct stages. Compared to existing
approaches which interleave between expansion and
optimization, we show that StageOpt is more efficient and
naturally applicable to a broader class of problems. We
provide theoretical guarantees for both the satisfaction of
safety constraints as well as convergence to the optimal
utility value. We evaluate StageOpt on both a variety of
synthetic experiments, as well as in clinical practice. We
demonstrate that StageOpt is more effective than existing
safe optimization approaches, and is able to safely and
effectively optimize spinal cord stimulation therapy in our
clinical experiments.
Keywords: StageOpt

[1190]

Yanan Sun, Gary G. Yen, and Zhang Yi.
IGD Indicatorbased Evolutionary Algorithm for Manyobjective
Optimization Problems.
IEEE Transactions on Evolutionary Computation, 23(2):173–187,
2019.
[ bib 
DOI ]

[1191]

A. Suppapitnarm, K. A. Seffen, G. T. Parks, and P. J. Clarkson.
A simulated annealing algorithm for multiobjective
optimization.
Engineering Optimization, 33(1):59–85, 2000.
[ bib ]

[1192]

Johan A. K. Suykens and Joos Vandewalle.
Least Squares Support Vector Machine Classifiers.
Neural Processing Letters, 9(3):293–300, 1999.
[ bib 
DOI ]
Keywords: LSSVM

[1193]

Jerry Swan, Steven Adriaensen, Alexander E. I. Brownlee, Kevin Hammond,
Colin G. Johnson, Ahmed Kheiri, Faustyna Krawiec, JuanJulián Merelo,
Leandro L. Minku, Ender Özcan, Gisele Pappa, Pablo
GarcíaSánchez, Kenneth Sörensen, Stefan Voß, Markus Wagner,
and David R. White.
Metaheuristics “In the Large”.
European Journal of Operational Research, 297(2):393–406,
March 2022.
[ bib 
DOI ]

[1194]

Jerry Swan, John R. Woodward, Ender Özcan, Graham Kendall, and Edmund K.
Burke.
Searching the Hyperheuristic Design Space.
Cognitive Computation, 6(1):66–73, March 2014.
[ bib 
DOI ]

[1195]

Harold Szu and Ralph Hartley.
Fast Simulated Annealing.
Physics Letters A, 122(3):157–162, 1987.
[ bib ]

[1196]

Éric D. Taillard.
Some Efficient Heuristic Methods for the Flow Shop Sequencing
Problem.
European Journal of Operational Research, 47(1):65–74, 1990.
[ bib ]

[1197]

Éric D. Taillard.
Robust Taboo Search for the Quadratic Assignment Problem.
Parallel Computing, 17(45):443–455, 1991.
[ bib ]
faster 2exchange delta evaluation in QAP

[1198]

Éric D. Taillard.
Benchmarks for Basic Scheduling Problems.
European Journal of Operational Research, 64(2):278–285, 1993.
[ bib ]

[1199]

Éric D. Taillard.
Comparison of Iterative Searches for the Quadratic Assignment
Problem.
Location Science, 3(2):87–105, 1995.
[ bib ]

[1200]

ElGhazali Talbi.
A Taxonomy of Hybrid Metaheuristics.
Journal of Heuristics, 8(5):541–564, 2002.
[ bib ]

[1201]

Kar Yan Tam.
A Simulated Annealing Algorithm for Allocating Space to
Manufacturing Cells.
International Journal of Production Research, 30(1):63–87,
1992.
[ bib ]

[1202]

Shunji Tanaka and Mituhiko Araki.
An Exact Algorithm for the Singlemachine Total Weighted
Tardiness Problem with Sequencedependent Setup Times.
Computers & Operations Research, 40(1):344–352, 2013.
[ bib ]

[1203]

Ryoji Tanabe and Hisao Ishibuchi.
An easytouse realworld multiobjective optimization problem
suite.
Applied Soft Computing, 89:106078, 2020.
[ bib ]
Proposed the RE benchmark suite

[1204]

Ryoji Tanabe, Hisao Ishibuchi, and Akira Oyama.
Benchmarking Multi and ManyObjective Evolutionary Algorithms
Under Two Optimization Scenarios.
IEEE Access, 5:19597–19619, 2017.
[ bib ]
compared a number of MOEAs using a wide range of numbers of
objectives and stopping criteria, with and without archivers; unbounded archive

[1205]

Lixin Tang and Xianpeng Wang.
Iterated local search algorithm based on very largescale
neighborhood for prizecollecting vehicle routing problem.
International Journal of Advanced Manufacturing Technology,
29(11):1246–1258, 2006.
[ bib ]

[1206]

A. J. Tarquin and J. Dowdy.
Optimal pump operation in water distribution.
Journal of Hydraulic Engineering, ASCE, 115(2):158–169 or
496–501, February 1989.
[ bib ]

[1207]

M. F. Tasgetiren, D. Kizilay, QuanKe Pan, and Ponnuthurai N. Suganthan.
Iterated Greedy Algorithms for the Blocking Flowshop Scheduling
Problem with Makespan Criterion.
Computers & Operations Research, 77:111–126, 2017.
[ bib ]

[1208]

M. Fatih Tasgetiren, YunChia Liang, Mehmet Sevkli, and Gunes Gencyilmaz.
A particle swarm optimization algorithm for makespan and total
flowtime minimization in the permutation flowshop sequencing problem.
European Journal of Operational Research, 177(3):1930 – 1947,
2007.
[ bib 
DOI ]

[1209]

M. Fatih Tasgetiren, QuanKe Pan, Ponnuthurai N. Suganthan, and Ozge
Buyukdagli.
A variable iterated greedy algorithm with differential evolution
for the noidle permutation flowshop scheduling problem.
Computers & Operations Research, 40(7):1729–1743, 2013.
[ bib ]

[1210]

Joc Cing Tay and Nhu Binh Ho.
Evolving dispatching rules using genetic programming for solving
multiobjective flexible jobshop problems.
Computers and Industrial Engineering, 54(3):453 – 473, 2008.
[ bib 
DOI ]

[1211]

Cristina Teixeira, José Covas, Thomas Stützle, and António
GasparCunha.
Engineering an Efficient TwoPhase Local Search for the
CoRotating TwinScrew Configuration Problem.
International Transactions in Operational Research,
18(2):271–291, 2011.
[ bib ]

[1212]

Cristina Teixeira, José Covas, Thomas Stützle, and António
GasparCunha.
MultiObjective Ant Colony Optimization for Solving the
TwinScrew Extrusion Configuration Problem.
Engineering Optimization, 44(3):351–371, 2012.
[ bib ]

[1213]

Cristina Teixeira, José Covas, Thomas Stützle, and António
GasparCunha.
Hybrid Algorithms for the TwinScrew Extrusion Configuration
Problem.
Applied Soft Computing, 23:298–307, 2014.
[ bib ]

[1214]

Fitsum Teklu, Agachai Sumalee, and David Watling.
A Genetic Algorithm Approach for Optimizing Traffic Control
Signals Considering Routing.
ComputerAided Civil and Infrastructure Engineering,
22(1):31–43, January 2007.
[ bib 
DOI ]

[1215]

J. B. Tenenbaum, V. D. Silva, and J. C. Langford.
A global geometric framework for nonlinear dimensionality
reduction.
Science, 290(5500):2319–2323, 2000.
[ bib ]

[1216]

J. Teo and Hussein A. Abbass.
Automatic generation of controllers for embodied legged
organisms: A Pareto evolutionary multiobjective approach.
Evolutionary Computation, 12(3):355–394, 2004.
[ bib 
DOI ]

[1217]

Kei Terayama, Masato Sumita, Ryo Tamura, and Koji Tsuda.
BlackBox Optimization for Automated Discovery.
Accounts of Chemical Research, 54(6):1334–1346, March 2021.
[ bib 
DOI ]
In chemistry and materials science, researchers and engineers
discover, design, and optimize chemical compounds or
materials with their professional knowledge and
techniques. At the highest level of abstraction, this process
is formulated as blackbox optimization. For instance, the
trialanderror process of synthesizing various molecules for
better material properties can be regarded as optimizing a
blackbox function describing the relation between a chemical
formula and its properties. Various blackbox optimization
algorithms have been developed in the machine learning and
statistics communities. Recently, a number of researchers
have reported successful applications of such algorithms to
chemistry. They include the design of photofunctional
molecules and medical drugs, optimization of thermal emission
materials and high Liion conductive solid electrolytes, and
discovery of a new phase in inorganic thin films for solar
cells.There are a wide variety of algorithms available for
blackbox optimization, such as Bayesian optimization,
reinforcement learning, and active learning. Practitioners
need to select an appropriate algorithm or, in some cases,
develop novel algorithms to meet their demands. It is also
necessary to determine how to best combine machine learning
techniques with quantum mechanics and molecular
mechanicsbased simulations, and experiments. In this
Account, we give an overview of recent studies regarding
automated discovery, design, and optimization based on
blackbox optimization. The Account covers the following
algorithms: Bayesian optimization to optimize the chemical or
physical properties, an optimization method using a quantum
annealer, bestarm identification, graybox optimization, and
reinforcement learning. In addition, we introduce active
learning and boundless objectivefree exploration, which may
not fall into the category of blackbox optimization.Data
quality and quantity are key for the success of these
automated discovery techniques. As laboratory automation and
robotics are put forward, automated discovery algorithms
would be able to match human performance at least in some
domains in the near future.

[1218]

Patrick Thibodeau.
Machinebased decisionmaking is coming.
Computer World, November 2011.
Last accessed: 15 January 2014.
[ bib 
http ]

[1219]

Lothar Thiele, Kaisa Miettinen, Pekka J. Korhonen, and Julián Molina.
A PreferenceBased Evolutionary Algorithm for MultiObjective
Optimization.
Evolutionary Computation, 17(3):411–436, 2009.
[ bib 
DOI ]
Abstract In this paper, we discuss the idea of incorporating
preference information into evolutionary multiobjective
optimization and propose a preferencebased evolutionary
approach that can be used as an integral part of an
interactive algorithm. One algorithm is proposed in the
paper. At each iteration, the decision maker is asked to give
preference information in terms of his or her reference point
consisting of desirable aspiration levels for objective
functions. The information is used in an evolutionary
algorithm to generate a new population by combining the
fitness function and an achievement scalarizing function. In
multiobjective optimization, achievement scalarizing
functions are widely used to project a given reference point
into the Pareto optimal set. In our approach, the next
population is thus more concentrated in the area where more
preferred alternatives are assumed to lie and the whole
Pareto optimal set does not have to be generated with equal
accuracy. The approach is demonstrated by numerical
examples.

[1220]

Ye Tian, Ran Cheng, Xingyi Zhang, Fan Cheng, and Yaochu Jin.
An IndicatorBased Multiobjective Evolutionary Algorithm With
Reference Point Adaptation for Better Versatility.
IEEE Transactions on Evolutionary Computation, 22(4):609–622,
2018.
[ bib 
DOI ]
IGDbased archiver

[1221]

TiewOn Ting, M. V. C. Rao, C. K. Loo, and S. S. Ngu.
Solving Unit Commitment Problem Using Hybrid Particle Swarm
Optimization.
Journal of Heuristics, 9(6):507–520, 2003.
[ bib 
DOI ]

[1222]

Santosh Tiwari, Georges Fadel, and Kalyanmoy Deb.
AMGA2: Improving the performance of the archivebased
microgenetic algorithm for multiobjective optimization.
Engineering Optimization, 43(4):377–401, 2011.
[ bib ]

[1223]

V. T'Kindt, Nicolas Monmarché, F. Tercinet, and D. Laügt.
An ant colony optimization algorithm to solve a 2machine
bicriteria flowshop scheduling problem.
European Journal of Operational Research, 142(2):250–257,
2002.
[ bib ]

[1224]

Michal K Tomczyk and Milosz Kadziński.
Decompositionbased interactive evolutionary algorithm for
multiple objective optimization.
IEEE Transactions on Evolutionary Computation, 24(2):320–334,
2019.
[ bib 
DOI ]
We propose a decompositionbased interactive evolutionary
algorithm (EA) for multiple objective optimization. During an
evolutionary search, a decision maker (DM) is asked to
compare pairwise solutions from the current population. Using
the Monte Carlo simulation, the proposed algorithm generates
from a uniform distribution a set of instances of the
preference model compatible with such an indirect preference
information. These instances are incorporated as the search
directions with the aim of systematically converging a
population toward the DMs most preferred region of the Pareto
front. The experimental comparison proves that the proposed
decompositionbased method outperforms the stateoftheart
interactive counterparts of the dominancebased EAs. We also
show that the quality of constructed solutions is highly
affected by the form of the incorporated preference model.
Keywords: interactive multiobjective; decisionmaking

[1225]

Michal K Tomczyk and Milosz Kadziński.
EMOSOR: Evolutionary multiple objective optimization guided by
interactive stochastic ordinal regression.
Computers & Operations Research, 108:134–154, 2019.
[ bib 
DOI ]
We propose a family of algorithms, called EMOSOR, combining
Evolutionary Multiple Objective Optimization with Stochastic
Ordinal Regression. The proposed methods ask the Decision
Maker (DM) to holistically compare, at regular intervals, a
pair of solutions, and use the Monte Carlo simulation to
construct a set of preference model instances compatible with
such indirect and incomplete information. The specific
variants of EMOSOR are distinguished by the following three
aspects. Firstly, they make use of two different preference
models, i.e., either an additive value function or a
Chebyshev function. Secondly, they aggregate the
acceptability indices derived from the stochastic analysis in
various ways, and use thus constructed indicators or
relations to sort the solutions obtained in each
generation. Thirdly, they incorporate different active
learning strategies for selecting pairs of solutions to be
critically judged by the DM. The extensive computational
experiments performed on a set of benchmark optimization
problems reveal that EMOSOR is able to bias an evolutionary
search towards a part of the Pareto front being the most
relevant to the DM, outperforming in this regard the
stateoftheart interactive evolutionary hybrids. Moreover,
we demonstrate that the performance of EMOSOR improves in
case the forms of a preference model used by the method and
the DMâ€™s value system align. Furthermore, we discuss how
vastly incorporation of different indicators based on the
stochastic acceptability indices influences the quality of
both the best constructed solution and an entire
population. Finally, we demonstrate that our novel
questioning strategies allow to reduce a number of
interactions with the DM until a highquality solution is
constructed or, alternatively, to discover a better solution
after the same number of interactions.
Keywords: Multiple objective optimization, Interactive evolutionary
hybrids, Stochastic ordinal regression, Preference
disaggregation, Pairwise comparisons, Active learning

[1226]

Michal K Tomczyk and Milosz Kadziński.
Decompositionbased coevolutionary algorithm for interactive
multiple objective optimization.
Information Sciences, 549:178–199, 2021.
[ bib 
DOI ]
We propose a novel coevolutionary algorithm for interactive
multiple objective optimization, named CIEMO/D. It aims at
finding a region in the Pareto front that is highly relevant
to the Decision Maker (DM). For this reason, CIEMO/D asks the
DM, at regular intervals, to compare pairs of solutions from
the current population and uses such preference information
to bias the evolutionary search. Unlike the existing
interactive evolutionary algorithms dealing with just a
single population, CIEMO/D coevolves a pool of
subpopulations in a steadystate decompositionbased
evolutionary framework. The evolution of each subpopulation
is driven by the use of a different preference model. In this
way, the algorithm explores various regions in the objective
space, thus increasing the chances of finding DMâ€™s most
preferred solution. To improve the pace of the evolutionary
search, CIEMO/D allows for the migration of solutions between
different subpopulations. It also dynamically alters the
subpopulationsâ€™ size based on compatibility between the
incorporated preference models and the decision examples
supplied by the DM. The extensive experimental evaluation
reveals that CIEMO/D can successfully adjust to different
DMâ€™s decision policies. We also compare CIEMO/D with selected
stateoftheart interactive evolutionary hybrids that make
use of the DMâ€™s pairwise comparisons, demonstrating its high
competitiveness.
Keywords: Evolutionary multiple objective optimization, Coevolution,
Decomposition, Indirect preference information, Preference
learning

[1227]

C. E. Torres, L. F. Rossi, J. Keffer, K. Li, and C.C. Shen.
Modeling, analysis and simulation of antbased network routing
protocols.
Swarm Intelligence, 4(3):221–244, 2010.
[ bib ]

[1228]

Heike Trautmann and Jörn Mehnen.
Preferencebased Pareto optimization in certain and noisy
environments.
Engineering Optimization, 41(1):23–38, January 2009.
[ bib ]

[1229]

Vito Trianni and Manuel LópezIbáñez.
Advantages of TaskSpecific MultiObjective Optimisation in
Evolutionary Robotics.
PLoS One, 10(8):e0136406, 2015.
[ bib 
DOI ]
The application of multiobjective optimisation to
evolutionary robotics is receiving increasing attention. A
survey of the literature reveals the different possibilities
it offers to improve the automatic design of efficient and
adaptive robotic systems, and points to the successful
demonstrations available for both taskspecific and
taskagnostic approaches (i.e., with or without reference to
the specific design problem to be tackled). However, the
advantages of multiobjective approaches over
singleobjective ones have not been clearly spelled out and
experimentally demonstrated. This paper fills this gap for
taskspecific approaches: starting from wellknown results in
multiobjective optimisation, we discuss how to tackle
commonly recognised problems in evolutionary robotics. In
particular, we show that multiobjective optimisation (i)
allows evolving a more varied set of behaviours by exploring
multiple tradeoffs of the objectives to optimise, (ii)
supports the evolution of the desired behaviour through the
introduction of objectives as proxies, (iii) avoids the
premature convergence to local optima possibly introduced by
multicomponent fitness functions, and (iv) solves the
bootstrap problem exploiting ancillary objectives to guide
evolution in the early phases. We present an experimental
demonstration of these benefits in three different case
studies: maze navigation in a single robot domain, flocking
in a swarm robotics context, and a strictly collaborative
task in collective robotics.

[1230]

Vito Trianni and S. Nolfi.
Engineering the evolution of selforganizing behaviors in swarm
robotics: A case study.
Artificial Life, 17(3):183–202, 2011.
[ bib ]

[1231]

Anupam Trivedi, Dipti Srinivasan, Krishnendu Sanyal, and Abhiroop Ghosh.
A survey of multiobjective evolutionary algorithms based on
decomposition.
IEEE Transactions on Evolutionary Computation, 21(3):440–462,
2016.
[ bib ]

[1232]

L.Y. Tseng and Y.T. Lin.
A hybrid genetic local search algorithm for the permutation
flowshop scheduling problem.
European Journal of Operational Research, 198(1):84–92, 2009.
[ bib ]

[1233]

S. Tsutsui.
Ant Colony Optimization with Cunning Ants.
Transactions of the Japanese Society for Artificial
Intelligence, 22:29–36, 2007.
[ bib 
DOI ]
Keywords: ant colony optimization, traveling salesman problem, cunning
ant, donor ant, local search

[1234]

Alexis Tugilimana, Ashley P. Thrall, and Rajan Filomeno Coelho.
Conceptual Design of Modular Bridges Including Layout
Optimization and Component Reusability.
Journal of Bridge Engineering, 22(11):04017094, 2017.
[ bib 
DOI ]
Keywords: scenariobased

[1235]

Renata TurkeÅ¡, Kenneth Sörensen, and Lars Magnus Hvattum.
Metaanalysis of metaheuristics: Quantifying the effect of
adaptiveness in adaptive large neighborhood search.
European Journal of Operational Research, 292(2):423–42, 2021.
[ bib 
DOI ]
Keywords: Metaheuristics, Metaanalysis, Adaptive large neighborhood
search

[1236]

Tea Tušar and Bogdan Filipič.
Visualizing Exact and Approximated 3D Empirical Attainment
Functions.
Mathematical Problems in Engineering, 2014, 2014.
Article ID 569346, 18 pages.
[ bib 
DOI ]

[1237]

Tea Tušar and Bogdan Filipič.
Visualization of Pareto front approximations in evolutionary
multiobjective optimization: A critical review and the prosection method.
IEEE Transactions on Evolutionary Computation, 19(2):225–245,
2015.
[ bib 
DOI ]

[1238]

D. Tuyttens, Jacques Teghem, Philippe Fortemps, and K. Van Nieuwenhuyze.
Performance of the MOSA Method for the Bicriteria Assignment
Problem.
Journal of Heuristics, 6:295–310, 2000.
[ bib ]

[1239]

Amos Tversky and Daniel Kahneman.
Judgment under uncertainty: Heuristics and biases.
Science, 185(4157):1124–1131, 1974.
[ bib ]

[1240]

Amos Tversky and Daniel Kahneman.
Loss aversion in riskless choice: a referencedependent model.
The Quarterly Journal of Economics, 106(4):1039–1061, 1991.
[ bib ]

[1241]

Amos Tversky.
Choice by elimination.
Journal of Mathematical Psychology, 9(4):341–367, 1972.
[ bib ]

[1242]

Colin Twomey, Thomas Stützle, Marco Dorigo, Max Manfrin, and Mauro
Birattari.
An Analysis of Communication Policies for Homogeneous
Multicolony ACO Algorithms.
Information Sciences, 180(12):2390–2404, 2010.
[ bib 
DOI ]

[1243]

E. Ulungu and Jacques Teghem.
The two phases method: An efficient procedure to solve
biobjective combinatorial optimization problems.
Foundations of Computing and Decision Sciences, 20(2):149–165,
1995.
[ bib ]

[1244]

E. Ulungu, Jacques Teghem, P. H. Fortemps, and D. Tuyttens.
MOSA method: a tool for solving multiobjective combinatorial
optimization problems.
Journal of MultiCriteria Decision Analysis, 8(4):221–236,
1999.
[ bib ]

[1245]

Thijs Urlings, Rubén Ruiz, and F. SivrikayaŞerifoğlu.
Genetic Algorithms for Complex Hybrid Flexible Flow Line
Problems.
International Journal of Metaheuristics, 1(1):30–54, 2010.
[ bib ]

[1246]

Thijs Urlings, Rubén Ruiz, and Thomas Stützle.
Shifting Representation Search for Hybrid Flexible Flowline
Problems.
European Journal of Operational Research, 207(2):1086–1095,
2010.
[ bib 
DOI ]

[1247]

Rob J. M. Vaessens, Emile H. L. Aarts, and Jan Karel Lenstra.
A Local Search Template.
Computers & Operations Research, 25(11):969–979, 1998.
[ bib 
DOI ]

[1248]

Claudio Lucio do Val Lopes, Flávio Vinícius Cruzeiro Martins,
Elizabeth F. Wanner, and Kalyanmoy Deb.
Analyzing dominance move (MIPDoM) indicator for multiand
manyobjective optimization.
IEEE Transactions on Evolutionary Computation, 2021.
[ bib ]

[1249]

Eva Vallada and Rubén Ruiz.
Genetic algorithms with path relinking for the minimum tardiness
permutation flowshop problem.
Omega, 38(1–2):57–67, 2010.
[ bib 
DOI ]

[1250]

Eva Vallada, Rubén Ruiz, and Jose M. Framiñán.
New hard benchmark for flowshop scheduling problems minimising
makespan.
European Journal of Operational Research, 240(3):666–677,
2015.
[ bib 
DOI ]

[1251]

Eva Vallada, Rubén Ruiz, and Gerardo Minella.
Minimising total tardiness in the mmachine flowshop problem: A
review and evaluation of heuristics and metaheuristics.
Computers & Operations Research, 35(4):1350–1373, 2008.
[ bib ]

[1252]

Pieter Vansteenwegen and Manuel Mateo.
An Iterated Local Search Algorithm for the Singlevehicle Cyclic
Inventory Routing Problem.
European Journal of Operational Research, 237(3):802–813,
2014.
[ bib ]

[1253]

Pieter Vansteenwegen, Wouter Souffriau, Greet Vanden Berghe, and Dirk Van
Oudheusden.
Iterated Local Search for the Team Orienteering Problem with
Time Tindows.
Computers & Operations Research, 36(12):3281–3290, 2009.
[ bib ]

[1254]

Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, and Luis Torgo.
OpenML: Networked Science in Machine Learning.
ACM SIGKDD Explorations Newsletter, 15(2):49–60, June
2014.
[ bib 
DOI ]

[1255]

A. Vargha and H. D. Delaney.
A critique and improvement of the CL common language effect
size statistics of McGraw and Wong.
Journal of Educational and Behavioral Statistics,
25(2):101–132, 2000.
[ bib ]
Keywords: effect size test, A12 test

[1256]

T. K. Varadharajan and C. Rajendran.
A multiobjective simulatedannealing algorithm for scheduling
in flowshops to minimize the makespan and total flowtime of jobs.
European Journal of Operational Research, 167(3):772–795,
2005.
[ bib ]

[1257]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones,
Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention Is All You Need.
Arxiv preprint arXiv:1706.03762, 2017.
[ bib 
http ]
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoderdecoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 EnglishtoGerman translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 EnglishtoFrench translation task, our model establishes a new singlemodel stateoftheart BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

[1258]

A. Vasan and Slobodan P. Simonovic.
Optimization of Water Distribution Network Design Using
Differential Evolution.
Journal of Water Resources Planning and Management, ASCE,
136(2):279–287, 2010.
[ bib ]

[1259]

Sergei Vassilvitskii and Mihalis Yannakakis.
Efficiently computing succinct tradeoff curves.
Theoretical Computer Science, 348(23):334–356, 2005.
[ bib ]

[1260]

J. A. VázquezRodríguez and Gabriela Ochoa.
On the Automatic Discovery of Variants of the NEH Procedure
for Flow Shop Scheduling Using Genetic Programming.
Journal of the Operational Research Society, 62(2):381–396,
2010.
[ bib ]

[1261]

Daniel Vaz, Luís Paquete, Carlos M. Fonseca, Kathrin Klamroth, and Michael
Stiglmayr.
Representation of the nondominated set in biobjective discrete
optimization.
Computers & Operations Research, 63:172–186, 2015.
[ bib 
DOI ]

[1262]

David A. Van Veldhuizen and Gary B. Lamont.
Multiobjective Evolutionary Algorithms: Analyzing the
Stateoftheart.
Evolutionary Computation, 8(2):125–147, 2000.
[ bib 
DOI ]

[1263]

Amit Verma and Mark Lewis.
Penalty and partitioning techniques to improve performance of
QUBO solvers.
Discrete Optimization, p. 100594, 2020.
[ bib 
DOI ]
Keywords: Quadratic Unconstrained Binary Optimization, Nonlinear
optimization, PseudoBoolean optimization, Equality
constraint, Inequality constraint

[1264]

Sébastien Verel, Arnaud Liefooghe, Laetitia Jourdan, and Clarisse Dhaenens.
On the Structure of Multiobjective Combinatorial Search Space:
MNKlandscapes with Correlated Objectives.
European Journal of Operational Research, 227(2):331–342,
2013.
[ bib 
DOI ]

[1265]

Paolo Viappiani, Boi Faltings, and Pearl Pu.
Preferencebased Search using ExampleCritiquing with
Suggestions.
Journal of Artificial Intelligence Research, 27:465–503, 2006.
[ bib ]

[1266]

Paolo Viappiani, Pearl Pu, and Boi Faltings.
Preferencebased Search with Adaptive Recommendations.
AI Communications, 21(2):155–175, 2008.
[ bib ]

[1267]

Thibaut Vidal, Teodor Gabriel Crainic, Michel Gendreau, and Christian Prins.
Heuristics for Multiattribute Vehicle Routing Problems: A
Survey and Synthesis.
European Journal of Operational Research, 231(1):1–21, 2013.
[ bib ]

[1268]

Thibaut Vidal, Teodor Gabriel Crainic, Michel Gendreau, and Christian Prins.
A Unified Solution Framework for Multiattribute Vehicle Routing
Problems.
European Journal of Operational Research, 234(3):658–673,
2014.
[ bib ]

[1269]

Matheus Guedes Vilas Boas, Haroldo Gambini Santos, Luiz Henrique de Campos
Merschmann, and Greet Vanden Berghe.
Optimal decision trees for the algorithm selection problem:
integer programming based approaches.
International Transactions in Operational Research,
28(5):2759–2781, 2021.
[ bib 
DOI ]

[1270]

Christos Voudouris and Edward P. K. Tsang.
Guided Local Search and its Application to the Travelling
Salesman Problem.
European Journal of Operational Research, 113(2):469–499,
1999.
[ bib ]

[1271]

Jyrki Wallenius.
Comparative Evaluation of Some Interactive Approaches to
Multicriterion Optimization.
Management Science, 21(12):1387–1396, 1975.
[ bib ]

[1272]

C. Walshaw and M. Cross.
Mesh Partitioning: A Multilevel Balancing and Refinement
Algorithm.
SIAM Journal on Scientific Computing, 22(1):63–80, 2000.
[ bib 
DOI ]

[1273]

David J. Walker, Richard M. Everson, and Jonathan E. Fieldsend.
Visualizing mutually nondominating solution sets in
manyobjective optimization.
IEEE Transactions on Evolutionary Computation, 17(2):165–184,
2012.
[ bib ]

[1274]

Chengen Wang, Chengbin Chu, and JeanMarie Proth.
Heuristic Approaches for n/m/F/ΣCi Scheduling
Problems.
European Journal of Operational Research, 96(3):636–644, 1997.
[ bib 
DOI ]

[1275]

Handing Wang, Licheng Jiao, and Xin Yao.
TwoArch2: An improved twoarchive algorithm for manyobjective
optimization.
IEEE Transactions on Evolutionary Computation, 19(4):524–541,
2015.
[ bib ]

[1276]

Yang Wang, Zhipeng Lü, Fred Glover, and JinKao Hao.
Probabilistic GRASPTabu Search algorithms for the UBQP
problem.
Computers & Operations Research, 40(12):3100–3107, 2013.
[ bib 
DOI ]

[1277]

Yang Wang, Zhipeng Lü, Fred Glover, and JinKao Hao.
Backbone Guided Tabu Search for Solving the UBQP Problem.
Journal of Heuristics, 19(4):679–695, 2013.
[ bib 
DOI ]

[1278]

Rui Wang, Robin C. Purshouse, and Peter J. Fleming.
PreferenceInspired Coevolutionary Algorithms for ManyObjective
Optimization.
IEEE Transactions on Evolutionary Computation, 17(4):474–494,
2013.
[ bib ]

[1279]

Rui Wang, Jian Xiong, Minfan He, Liang Gao, and Ling Wang.
Multiobjective optimal design of hybrid renewable energy system
under multiple scenarios.
Renewable Energy, 151:226–237, 2020.
[ bib 
DOI ]

[1280]

Yang Wang, Zhipeng Lü, Fred Glover, and JinKao Hao.
Path relinking for unconstrained binary quadratic programming.
European Journal of Operational Research, 223(3):595–604,
2012.
[ bib 
DOI ]

[1281]

JeanPaul Watson, L. Barbulescu, Darrell Whitley, and Adele E. Howe.
Contrasting Structured and Random Permutation FlowShop
Scheduling Problems: Search Space Topology and Algorithm Performance.
INFORMS Journal on Computing, 14(2):98–123, 2002.
[ bib ]

[1282]

JeanPaul Watson, J. C. Beck, A. E. Howe, and Darrell Whitley.
Problem Difficulty for Tabu Search in JobShop Scheduling.
Artificial Intelligence, 143(2):189–217, 2003.
[ bib ]

[1283]

JeanPaul Watson, Adele E Howe, and Darrell Whitley.
Deconstructing Nowicki and Smutnicki's iTSAB tabu search
algorithm for the jobshop scheduling problem.
Computers & Operations Research, 33(9):2623–2644, 2006.
[ bib ]

[1284]

Abigail A. Watson and Joseph R. Kasprzyk.
Incorporating deeply uncertain factors into the many objective
search process.
Environmental Modelling & Software, 89:159–171, 2017.
[ bib ]
Keywords: scenariobased

[1285]

Edward J. Wegman.
Hyperdimensional data analysis using parallel coordinates.
Journal of the American Statistical Association,
85(411):664–675, 1990.
[ bib ]

[1286]

Edward D. Weinberger.
Local properties of Kauffman's Nk model: A tunably rugged
energy landscape.
Physical Review A, 44(10):6399, 1991.
[ bib ]

[1287]

Bernard L. Welch.
The significance of the difference between two means when the
population variances are unequal.
Biometrika, 29(3/4):350–362, 1938.
[ bib ]

[1288]

Simon Wessing and Manuel LópezIbáñez.
Latin Hypercube Designs with Branching and Nested Factors for
Initialization of Automatic Algorithm Configuration.
Evolutionary Computation, 27(1):129–145, 2018.
[ bib 
DOI ]

[1289]

Dennis Weyland.
A Rigorous Analysis of the Harmony Search Algorithm: How the
Research Community can be misled by a “novel” Methodology.
International Journal of Applied Metaheuristic Computing,
12(2):50–60, 2010.
[ bib ]

[1290]

Dennis Weyland.
A critical analysis of the harmony search algorithm: How not to
solve Sudoku.
Operations Research Perspectives, 2:97–105, 2015.
[ bib ]

[1291]

D. R. White, A. Arcuri, and J. A. Clark.
Evolutionary Improvement of Programs.
IEEE Transactions on Evolutionary Computation, 15(4):515–538,
2011.
[ bib ]

[1292]

L. While, L. Bradstreet, and L. Barone.
A Fast Way of Calculating Exact Hypervolumes.
IEEE Transactions on Evolutionary Computation, 16(1):86–95,
2012.
[ bib ]

[1293]

Darrell Whitley, Soraya Rana, John Dzubera, and Keith E. Mathias.
Evaluating Evolutionary Algorithms.
Artificial Intelligence, 85:245–296, 1996.
[ bib ]

[1294]

R. J. Williams.
Simple Statistical GradientFollowing Algorithms for
Connectionist Reinforcement Learning.
Machine Learning, 8(3):229–256, 1992.
[ bib ]

[1295]

P. Winkler.
Random Orders.
Order, 1:317–331, 1985.
[ bib ]
Showed that fraction of Paretooptimal increases with number
of objectives

[1296]

Carsten Witt.
Analysis of an Iterated Local Search Algorithm for Vertex Cover
in Sparse Random Graphs.
Theoretical Computer Science, 425:117–125, 2012.
[ bib ]

[1297]

D. H. Wolpert and W. G. Macready.
No Free Lunch Theorems for Optimization.
IEEE Transactions on Evolutionary Computation, 1(1):67–82,
1997.
[ bib 
DOI ]

[1298]

Matthew J. Woodruff, Patrick M. Reed, and Timothy W. Simpson.
Many objective visual analytics: rethinking the design of
complex engineered systems.
Structural and Multidisciplinary Optimization, 48(1):201–219,
2013.
[ bib 
DOI ]

[1299]

David L. Woodruff, Ulrike Ritzinger, and Johan Oppen.
Research Note: The Point of Diminishing Returns in Heuristic
Search.
International Journal of Metaheuristics, 1(3):222–231, 2011.
[ bib 
DOI ]
Keywords: anytime

[1300]

H. S. Woo and D. S. Yim.
A Heuristic Algorithm for Mean Flowtime Objective in Flowshop
Scheduling.
Computers & Operations Research, 25(3):175–182, 1998.
[ bib ]

[1301]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang
Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al.
Google's neural machine translation system: Bridging the gap
between human and machine translation.
Arxiv preprint arXiv:1609.08144 [cs.CL], 2016.
[ bib 
http ]

[1302]

Xindong Wu, Xingquan Zhu, GongQing Wu, and Wei Ding.
Data mining with big data.
IEEE Transactions on Knowledge and Data Engineering,
26(1):97–107, 2014.
[ bib ]

[1303]

B. Xin, L. Chen, J. Chen, Hisao Ishibuchi, K. Hirota, and B. Liu.
Interactive Multiobjective Optimization: A Review of the
StateoftheArt.
IEEE Access, 6:41256–41279, 2018.
[ bib 
DOI ]
Interactive multiobjective optimization (IMO) aims at finding
the most preferred solution of a decision maker with the
guidance of his/her preferences which are provided
progressively. During the process, the decision maker can
adjust his/her preferences and explore only interested
regions of the search space. In recent decades, IMO has
gradually become a common interest of two distinct
communities, namely, the multiple criteria decision making
(MCDM) and the evolutionary multiobjective optimization
(EMO). The IMO methods developed by the MCDM community
usually use the mathematical programming methodology to
search for a single preferred Pareto optimal solution, while
those which are rooted in EMO often employ evolutionary
algorithms to generate a representative set of solutions in
the decision maker's preferred region. This paper aims to
give a review of IMO research from both MCDM and EMO
perspectives. Taking into account four classification
criteria including the interaction pattern, preference
information, preference model, and search engine (i.e.,
optimization algorithm), a taxonomy is established to
identify important IMO factors and differentiate various IMO
methods. According to the taxonomy, stateoftheart IMO
methods are categorized and reviewed and the design ideas
behind them are summarized. A collection of important issues,
e.g., the burdens, cognitive biases and preference
inconsistency of decision makers, and the performance
measures and metrics for evaluating IMO methods, are
highlighted and discussed. Several promising directions
worthy of future research are also presented.
Keywords: Decision making, Evolutionary computation, Pareto
optimization, Evolutionary multiobjective optimization,
interactive multiobjective optimization, multiple criteria
decision making, preference information, preference models

[1304]

Jiefeng Xu, Steve Y. Chiu, and Fred Glover.
Finetuning a tabu search algorithm with statistical tests.
International Transactions in Operational Research,
5(3):233–244, 1998.
[ bib 
DOI ]

[1305]

Lin Xu, Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
SATzilla: Portfoliobased Algorithm Selection for SAT.
Journal of Artificial Intelligence Research, 32:565–606, June
2008.
[ bib 
DOI 
epub ]

[1306]

Hongyun Xu, Zhipeng Lü, and T. C. E. Cheng.
Iterated Local Search for Singlemachine Scheduling with
Sequencedependent Setup Times to Minimize Total Weighted Tardiness.
Journal of Scheduling, 17(3):271–287, 2014.
[ bib ]

[1307]

Mutsunori Yagiura, M. Kishida, and Toshihide Ibaraki.
A 3Flip Neighborhood Local Search for the Set Covering
Problem.
European Journal of Operational Research, 172(2):472–499,
2006.
[ bib ]

[1308]

Yuki Yamada.
How to Crack Preregistration: Toward Transparent and Open
Science.
Frontiers in Psychology, 9, September 2018.
[ bib 
DOI ]
Keywords: HARKing; PARKing

[1309]

Kaifeng Yang, Michael T. M. Emmerich, André H. Deutz, and Thomas
Bäck.
MultiObjective Bayesian Global Optimization using Expected
Hypervolume Improvement Gradient.
Swarm and Evolutionary Computation, 44:945–956, February 2019.
[ bib 
DOI ]
Keywords: Bayesian Optimisation with preferences

[1310]

Y. Yang, S. Kreipl, and M. L. Pinedo.
Heuristics for Minimizing Total Weighted Tardiness in Flexible
Flow Shops.
Journal of Scheduling, 3(2):89–108, 2000.
[ bib ]

[1311]

S. Yang, Miqing Li, X. Liu, and J. Zheng.
A GridBased Evolutionary Algorithm for ManyObjective
Optimization.
IEEE Transactions on Evolutionary Computation, 17(5):721–736,
2013.
[ bib 
DOI ]
epsilongrid

[1312]

Vincent F. Yu and ShihWei Lin.
Iterated Greedy Heuristic for the Timedependent
Prizecollecting Arc Routing Problem.
Computers and Industrial Engineering, 90:54–66, 2015.
[ bib ]

[1313]

G. Yu, R. S. Powell, and M. J. H. Sterling.
Optimized Pump Scheduling in Water Distribution Systems.
Journal of Optimization Theory and Applications,
83(3):463–488, 1994.
[ bib ]

[1314]

Zhi Yuan, Marco A. Montes de Oca, Thomas Stützle, and Mauro Birattari.
Continuous Optimization Algorithms for Tuning Real and Integer
Algorithm Parameters of Swarm Intelligence Algorithms.
Swarm Intelligence, 6(1):49–75, 2012.
[ bib ]

[1315]

Q. Zeng and Z. Yang.
Integrating Simulation and Optimization to Schedule Loading
Operations in Container Terminals.
Computers & Operations Research, 36(6):1935–1944, 2009.
[ bib 
DOI ]

[1316]

Tiantian Zhang, Michael Georgiopoulos, and Georgios C. Anagnostopoulos.
MultiObjective Model Selection via Racing.
IEEE Transactions on Cybernetics, 46(8):1863–1876, 2016.
[ bib ]

[1317]

Qingfu Zhang and Hui Li.
MOEA/D: A Multiobjective Evolutionary Algorithm Based on
Decomposition.
IEEE Transactions on Evolutionary Computation, 11(6):712–731,
2007.
[ bib 
DOI ]
Introduces penaltybased boundary intersection (PBI)
function

[1318]

Jingqiao Zhang and Arthur C. Sanderson.
JADE: Adaptive differential evolution with optional external
archive.
IEEE Transactions on Evolutionary Computation, 13(5):945–958,
2009.
[ bib 
DOI ]

[1319]

H. Zhao and Sudha Ram.
Constrained cascade generalization of decision trees.
IEEE Transactions on Knowledge and Data Engineering,
16(6):727–739, 2004.
[ bib 
DOI ]

[1320]

Lu Zhen and DaoFang Chang.
A biobjective model for robust berth allocation scheduling.
Computers and Industrial Engineering, 63(1):262–273, 2012.
[ bib ]

[1321]

A. Zhou, Qingfu Zhang, and Yaochu Jin.
Approximating the set of Paretooptimal solutions in both the
decision and objective spaces by an estimation of distribution algorithm.
IEEE Transactions on Evolutionary Computation,
13(5):1167–1189, 2009.
[ bib 
DOI ]
Keywords: multimodal, IGDX

[1322]

Shlomo Zilberstein.
Using Anytime Algorithms in Intelligent Systems.
AI Magazine, 17(3):73–83, 1996.
[ bib 
DOI 
epub ]
Anytime algorithms give intelligent systems the capability to trade deliberation time for quality of results. This capability is essential for successful operation in domains such as signal interpretation, realtime diagnosis and repair, and mobile robot control. What characterizes these domains is that it is not feasible (computationally) or desirable (economically) to compute the optimal answer. This article surveys the main control problems that arise when a system is composed of several anytime algorithms. These problems relate to optimal management of uncertainty and precision. After a brief introduction to anytime computation, I outline a wide range of existing solutions to the metalevel control problem and describe current work that is aimed at increasing the applicability of anytime computation.
Keywords: performance profiles

[1323]

Stanley Zionts and Jyrki Wallenius.
An interactive multiple objective linear programming method for
a class of underlying nonlinear utility functions.
Management Science, 29(5):519–529, 1983.
[ bib ]

[1324]

Eckart Zitzler and Lothar Thiele.
Multiobjective Evolutionary Algorithms: A Comparative Case
Study and the Strength Pareto Approach.
IEEE Transactions on Evolutionary Computation, 3(4):257–271,
1999.
[ bib 
DOI ]
Proposed SPEA,
http://www.tik.ee.ethz.ch/sop/publicationListFiles/zt1999a.pdf

[1325]

Eckart Zitzler, Lothar Thiele, and Johannes Bader.
On SetBased Multiobjective Optimization.
IEEE Transactions on Evolutionary Computation, 14(1):58–79,
2010.
[ bib 
DOI ]
Proposed SPAM and explores combination of quality indicators
Keywords: Performance assessment; Preference articulation; refinement;
Set Partitioning; Setpreference

[1326]

Eckart Zitzler, Lothar Thiele, and Kalyanmoy Deb.
Comparison of Multiobjective Evolutionary Algorithms: Empirical
Results.
Evolutionary Computation, 8(2):173–195, 2000.
[ bib 
DOI ]
Keywords: ZDT benchmark

[1327]

Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M. Fonseca, and Viviane
Grunert da Fonseca.
Performance Assessment of Multiobjective Optimizers: an Analysis
and Review.
IEEE Transactions on Evolutionary Computation, 7(2):117–132,
2003.
[ bib 
DOI ]
Proposed the combination of quality indicators; proposed epsilonindicator

[1328]

M. Zlochin, Mauro Birattari, N. Meuleau, and Marco Dorigo.
ModelBased Search for Combinatorial Optimization: A Critical
Survey.
Annals of Operations Research, 131(1–4):373–395, 2004.
[ bib ]

[1329]

Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas, and
Michel Lang.
mlrMBO: A Modular Framework for ModelBased Optimization of
Expensive BlackBox Functions.
Arxiv preprint arXiv:1703.03373 [stat.ML], 2017.
[ bib 
http ]

[1330]

Oscar Cordón, Francisco Herrera, and Thomas Stützle.
Special Issue on Ant Colony Optimization: Models and
Applications.
Mathware & Soft Computing, 9(3):137–268, 2002.
[ bib ]

[1331]

G. McCormick and R. S. Powell.
Optimal Pump Scheduling in Water Supply Systems with Maximum
Demand Charges.
Journal of Water Resources Planning and Management, ASCE,
129(5):372–379, September / October 2003.
[ bib ]

[1332]

Gang Quan, Garrison W. Greenwood, Donglin Liu, and Sharon Hu.
Searching for multiobjective preventive maintenance schedules:
Combining preferences with evolutionary algorithms.
European Journal of Operational Research, 177(3):1969–1984,
2007.
[ bib 
DOI ]
Heavy industry maintenance facilities at aircraft service
centers or railroad yards must contend with scheduling
preventive maintenance tasks to ensure critical equipment
remains available. The workforce that performs these tasks
are often highpaid, which means the task scheduling should
minimize worker idle time. Idle time can always be minimized
by reducing the workforce. However, all preventive
maintenance tasks should be completed as quickly as possible
to make equipment available. This means the completion time
should be also minimized. Unfortunately, a small workforce
cannot complete many maintenance tasks per hour. Hence, there
is a tradeoff: should the workforce be small to reduce idle
time or should it be large so more maintenance can be
performed each hour? A cost effective schedule should strike
some balance between a minimum schedule and a minimum size
workforce. This paper uses evolutionary algorithms to solve
this multiobjective problem. However, rather than conducting
a conventional dominancebased Pareto search, we introduce a
form of utility theory to find Pareto optimal solutions. The
advantage of this method is the user can target specific
subsets of the Pareto front by merely ranking a small set of
initial solutions. A large example problem is used to
demonstrate our method.
Keywords: Evolutionary computations, Scheduling, Utility theory,
Preventive maintenance, Multiobjective optimization,
rankingbased, interactive

[1333]

Marvin N. Wright and Andreas Ziegler.
ranger: A Fast Implementation of Random Forests for
High Dimensional Data in C++ and R.
Arxiv preprint arXiv:1508.04409 [stat.ML], 2015.
[ bib 
http ]

[1334]

Marvin N. Wright and Andreas Ziegler.
ranger: A Fast Implementation of Random Forests for
High Dimensional Data in C++ and R.
Journal of Statistical Software, 77(1):1–17, 2017.
[ bib 
DOI ]

[1335]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel,
M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos,
D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.
Scikitlearn: Machine learning in Python.
Journal of Machine Learning Research, 12:2825–2830, 2011.
[ bib ]

[1336]

Jakobus E. van Zyl, Dragan A. Savic, and Godfrey A. Walters.
Operational Optimization of Water Distribution Systems using a
Hybrid Genetic Algorithm.
Journal of Water Resources Planning and Management, ASCE,
130(2):160–170, March 2004.
[ bib ]

[1337]

AAAI.
35th AAAI Conference on Artificial Intelligence: Reproducibility
Checklist.
https://aaai.org/Conferences/AAAI21/reproducibilitychecklist/, 2021.
Last accessed: June 6th, 2021.
[ bib ]

[1338]

ACM.
Artifact Review and Badging Version 1.1.
https://www.acm.org/publications/policies/artifactreviewandbadgingcurrent,
August 2020.
[ bib ]

[1339]

Emile H. L. Aarts, Jan H. M. Korst, and Wil Michiels.
Simulated Annealing.
In E. K. Burke and G. Kendall, editors, Search Methodologies,
pp. 187–210. Springer, Boston, MA, 2005.
[ bib 
DOI ]

[1340]

Hussein A. Abbass.
The selfadaptive Pareto differential evolution algorithm.
In Proceedings of the 2002 Congress on Evolutionary Computation
(CEC'02), pp. 831–836, Piscataway, NJ, 2002. IEEE Press.
[ bib ]

[1341]

Ricardo Henrique Remes de Lima and Aurora Trinidad Ramirez Pozo.
A study on autoconfiguration of MultiObjective Particle Swarm
Optimization Algorithm.
In Proceedings of the 2017 Congress on Evolutionary Computation
(CEC 2017), pp. 718–725, Piscataway, NJ, 2017. IEEE Press.
[ bib 
DOI ]

[1342]

Hussein A. Abbass, Ruhul Sarker, and Charles Newton.
PDE: a Paretofrontier differential evolution approach for
multiobjective optimization problems.
In Proceedings of the 2001 Congress on Evolutionary Computation
(CEC'01), pp. 971–978, Piscataway, NJ, 2001. IEEE Press.
[ bib ]

[1343]

F. Ben Abdelaziz, S. Krichen, and J. Chaouachi.
A hybrid heuristic for multiobjective knapsack problems.
In M. G. C. Resende and J. Pinho de Souza, editors, Proceedings
of MIC 1997, the 2nd Metaheuristics International Conference, pp. 205–212,
1997.
[ bib 
DOI ]

[1344]

A. Acan.
An external memory implementation in ant colony optimization.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pp. 73–84. Springer, Heidelberg, 2004.
[ bib ]
Keywords: memorybased ACO

[1345]

A. Acan.
An external partial permutations memory for ant colony
optimization.
In G. R. Raidl and J. Gottlieb, editors, Proceedings of EvoCOP
2005 – 5th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 3448 of Lecture Notes in Computer Science, pp.
1–11. Springer, Heidelberg, 2005.
[ bib ]
Keywords: memorybased ACO

[1346]

Hernán E. Aguirre, Saúl Zapotecas, Arnaud Liefooghe, Sébastien
Verel, and Kiyoshi Tanaka.
Approaches for ManyObjective Optimization: Analysis and
Comparison on MNKLandscapes.
In S. Bonnevay et al., editors, Artificial Evolution: 12th
International Conference, Evolution Artificielle, EA, 2015, volume 9554 of
Lecture Notes in Computer Science, pp. 14–28. Springer, Cham,
Switzerland, 2016.
[ bib 
DOI ]

[1347]

Weiwei Cheng, Jens Hühn, and Eyke Hüllermeier.
Decision Tree and InstanceBased Learning for Label Ranking.
In A. P. Danyluk, L. Bottou, and M. L. Littman, editors,
Proceedings of the 26th International Conference on Machine Learning, ICML
2009, pp. 161–168, New York, NY, 2009. ACM Press.
[ bib 
DOI ]

[1348]

Hernán E. Aguirre and Kiyoshi Tanaka.
ManyObjective Optimization by Space Partitioning and Adaptive
εRanking on MNKLandscapes.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pp. 407–422.
Springer, Heidelberg, 2009.
[ bib ]

[1349]

Hernán E. Aguirre.
Advances on Manyobjective Evolutionary Optimization.
In C. Blum and E. Alba, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO Companion 2013, pp. 641–666,
New York, NY, 2013. ACM Press.
[ bib ]
Keywords: manyobjective evolutionary optimization

[1350]

A. Aho, J. Hopcroft, and J. Ullman.
Data structures and algorithms.
AddisonWesley, Reading, MA, 1983.
[ bib ]

[1351]

R. K. Ahuja, T. Magnanti, and J. B. Orlin.
Network Flows: Theory, Algorithms and Applications.
PrenticeHall, 1993.
[ bib ]

[1352]

Uwe Aickelin, Edmund K. Burke, and Jingpeng Li.
Improved Squeaky Wheel Optimisation for Driver Scheduling.
In T. P. Runarsson, H.G. Beyer, E. K. Burke, J.J. Merelo,
D. Whitley, and X. Yao, editors, Parallel Problem Solving from Nature –
PPSN IX, volume 4193 of Lecture Notes in Computer Science, pp.
182–191. Springer, Heidelberg, 2006.
[ bib ]

[1353]

Hassene Aissi and Bernard Roy.
Robustness in Multicriteria Decision Aiding.
In M. Ehrgott, J. R. Figueira, and S. Greco, editors, Trends in
Multiple Criteria Decision Analysis, volume 142 of International Series
in Operations Research & Management Science, chapter 4, pp. 87–121.
Springer, US, 2010.
[ bib ]

[1354]

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama.
Optuna: A Nextgeneration Hyperparameter Optimization
Framework.
In Teredesai et al., editors, 25th ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining, pp. 2623–2631. ACM
Press, New York, NY, July 2019.
[ bib 
DOI ]

[1355]

S. M. Aktürk, Alper Atamtürk, and S. Gürel.
A Strong Conic Quadratic Reformulation for MachineJob
Assignment with Controllable Processing Times.
Research Report BCOL.07.01, University of CaliforniaBerkeley, 2007.
[ bib ]

[1356]

I. Alaya, Christine Solnon, and Khaled Ghédira.
Ant Colony Optimization for MultiObjective Optimization
Problems.
In 19th IEEE International Conference on Tools with Artificial
Intelligence (ICTAI 2007), volume 1, pp. 450–457. IEEE Computer Society
Press, Los Alamitos, CA, 2007.
[ bib ]

[1357]

I. Alaya, Christine Solnon, and Khaled Ghédira.
Ant algorithm for the multidimensional knapsack problem.
In B. Filipič and J. Šilc, editors, International
Conference on Bioinspired Optimization Methods and their Applications (BIOMA
2004), pp. 63–72, 2004.
[ bib 
http ]

[1358]

Enrique Alba and Francisco Chicano.
ACOhg: dealing with huge graphs.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pp. 10–17. ACM Press, New
York, NY, 2007.
[ bib 
DOI ]

[1359]

Mohamad Alissa, Kevin Sim, and Emma Hart.
Algorithm Selection Using Deep Learning without Feature
Extraction.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pp. 198–206, New York, NY, 2019. ACM Press.
[ bib 
DOI 
epub ]

[1360]

Richard Allmendinger and Joshua D. Knowles.
Evolutionary Optimization on Problems Subject to Changes of
Variables.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of
Lecture Notes in Computer Science, pp. 151–160. Springer, Heidelberg,
2010.
[ bib ]
Motivated by an experimental problem involving the
identification of effective drug combinations drawn from a
nonstatic drug library, this paper examines evolutionary
algorithm strategies for dealing with changes of
variables. We consider four standard techniques from dynamic
optimization, and propose one new technique. The results show
that only little additional diversity needs to be introduced
into the population when changing a small number of
variables, while changing many variables or optimizing a
rugged landscape requires often a restart of the optimization
process

[1361]

Richard Allmendinger and Joshua D. Knowles.
Evolutionary Search in Lethal Environments.
In International Conference on Evolutionary Computation Theory
and Applications, pp. 63–72. SciTePress, 2011.
[ bib 
DOI 
epub ]

[1362]

Richard Allmendinger and Joshua D. Knowles.
Policy Learning in ResourceConstrained Optimization.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp.
1971–1979. ACM Press, New York, NY, 2011.
[ bib 
DOI ]
We consider an optimization scenario in which resources are
required in the evaluation process of candidate
solutions. The challenge we are focussing on is that certain
resources have to be committed to for some period of time
whenever they are used by an optimizer. This has the effect
that certain solutions may be temporarily nonevaluable
during the optimization. Previous analysis revealed that
evolutionary algorithms (EAs) can be effective against this
resourcing issue when augmented with static strategies for
dealing with nonevaluable solutions, such as repairing,
waiting, or penalty methods. Moreover, it is possible to
select a suitable strategy for resourceconstrained problems
offline if the resourcing issue is known in advance. In this
paper we demonstrate that an EA that uses a reinforcement
learning (RL) agent, here Sarsa(λ), to learn
offline when to switch between static strategies, can be more
effective than any of the static strategies themselves. We
also show that learning the same task as the RL agent but
online using an adaptive strategy selection method, here
DMAB, is not as effective; nevertheless, online learning is
an alternative to static strategies.

[1363]

Joseph Allen, Ahmed Moussa, and Xudong Liu.
HumanintheLoop Learning of Qualitative Preference Models.
In R. Barták and K. W. Brawner, editors, Proceedings of
the ThirtySecond International Florida Artificial Intelligence Research
Society Conference, pp. 108–111. AAAI Press, 2019.
[ bib 
DOI ]

[1364]

Richard Allmendinger.
Tuning Evolutionary Search for ClosedLoop Optimization.
PhD thesis, The University of Manchester, UK, January 2012.
[ bib ]

[1365]

A. Alsheddy and E. Tsang.
Guided Pareto local search and its application to the 0/1
multiobjective knapsack problems.
In M. Caserta and S. Voß, editors, Proceedings of MIC 2009,
the 8th Metaheuristics International Conference, Hamburg, Germany, 2010.
University of Hamburg.
[ bib ]

[1366]

Sanae Amani, Mahnoosh Alizadeh, and Christos Thrampoulidis.
Linear Stochastic Bandits Under Safety Constraints.
In H. M. Wallach, H. Larochelle, A. Beygelzimer,
F. d'AlchéBuc, E. B. Fox, and R. Garnett, editors, Advances in
Neural Information Processing Systems (NeurIPS 32), pp. 9256–9266, 2019.
[ bib 
epub ]

[1367]

Klaus Andersen, René Victor Valqui Vidal, and Villy Bæk Iversen.
Design of a Teleprocessing Communication Network Using Simulated
Annealing.
In R. V. V. Vidal, editor, Applied Simulated Annealing, pp.
201–215. Springer, 1993.
[ bib ]

[1368]

J. H. Andersen and R. S. Powell.
The Use of Continuous Decision Variables in an Optimising Fixed
Speed Pump Scheduling Algorithm.
In R. S. Powell and K. S. Hindi, editors, Computing and Control
for the Water Industry, pp. 119–128. Research Studies Press Ltd., 1999.
[ bib ]

[1369]

D. Anghinolfi, A. Boccalatte, M. Paolucci, and C. Vecchiola.
Performance Evaluation of an Adaptive Ant Colony Optimization
Applied to Single Machine Scheduling.
In X. Li et al., editors, Simulated Evolution and Learning, 7th
International Conference, SEAL 2008, volume 5361 of Lecture Notes in
Computer Science, pp. 411–420. Springer, Heidelberg, 2008.
[ bib ]

[1370]

Daniel Angus.
PopulationBased Ant Colony Optimisation for Multiobjective
Function Optimisation.
In M. Randall, H. A. Abbass, and J. Wiles, editors, Progress in
Artificial Life (ACAL), volume 4828 of Lecture Notes in Computer
Science, pp. 232–244. Springer, Heidelberg, 2007.
[ bib 
DOI ]

[1371]

J. Ansel, S. Kamil, K. Veeramachaneni, J. RaganKelley, J. Bosboom, UnaMay
O'Reilly, and S. Amarasinghe.
OpenTuner: An extensible framework for program autotuning.
In Proceedings of the 23rd International Conference on Parallel
Architectures and Compilation, pp. 303–315, New York, NY, 2014. ACM Press.
[ bib 
DOI ]

[1372]

Carlos Ansótegui, Yuri Malitsky, Horst Samulowitz, Meinolf Sellmann, and
Kevin Tierney.
ModelBased Genetic Algorithms for Algorithm Configuration.
In Q. Yang and M. Wooldridge, editors, Proceedings of the 24th
International Joint Conference on Artificial Intelligence (IJCAI15), pp.
733–739. IJCAI/AAAI Press, Menlo Park, CA, 2015.
[ bib 
epub ]
Keywords: GGA++

[1373]

Carlos Ansótegui, Yuri Malitsky, and Meinolf Sellmann.
MaxSAT by Improved InstanceSpecific Algorithm Configuration.
In D. Stracuzzi et al., editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pp. 2594–2600. AAAI Press, 2014.
[ bib ]

[1374]

Carlos Ansótegui, Meinolf Sellmann, and Kevin Tierney.
A GenderBased Genetic Algorithm for the Automatic Configuration
of Algorithms.
In I. P. Gent, editor, Principles and Practice of Constraint
Programming, CP 2009, volume 5732 of Lecture Notes in Computer
Science, pp. 142–157. Springer, Heidelberg, 2009.
[ bib 
DOI ]
Keywords: GGA

[1375]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Finding Cuts in the TSP.
Technical Report 95–05, DIMACS Center, Rutgers University,
Piscataway, NJ, USA, March 1995.
[ bib ]

[1376]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Finding Tours in the TSP.
Technical Report 99885, Forschungsinstitut für Diskrete
Mathematik, University of Bonn, Germany, 1999.
[ bib ]

[1377]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
The Traveling Salesman Problem: A Computational Study.
Princeton University Press, Princeton, NJ, 2006.
[ bib ]

[1378]

Jay April, Fred Glover, James P. Kelly, and Manuel Laguna.
Simulationbased optimization: Practical introduction to
simulation optimization.
In S. E. Chick, P. J. Sanchez, D. M. Ferrin, and D. J. Morrice,
editors, Proceedings of the 35th Winter Simulation Conference: Driving
Innovation, volume 1, pp. 71–78, New York, NY, December 2003. ACM Press.
[ bib 
DOI ]

[1379]

Sanjeev Arora and Boaz Barak.
Computational complexity: a modern approach.
Cambridge University Press, 2009.
[ bib ]

[1380]

Etor Arza, Josu Ceberio, Aritz Pérez, and Ekhine Irurozki.
Approaching the quadratic assignment problem with kernels of
mallows models under the hamming distance.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO Companion 2019. ACM Press, New York, NY, 2019.
[ bib 
DOI 
epub ]
Keywords: QAP, EDA, Mallows

[1381]

Y. Asahiro, K. Iwama, and E. Miyano.
Random Generation of Test Instances with Controlled Attributes.
In D. S. Johnson and M. A. Trick, editors, Cliques, Coloring,
and Satisfiability: Second DIMACS Implementation Challenge, volume 26 of
DIMACS Series on Discrete Mathematics and Theoretical Computer
Science, pp. 377–393. American Mathematical Society, Providence, RI, 1996.
[ bib ]

[1382]

N. Ascheuer.
Hamiltonian Path Problems in the Online Optimization of
Flexible Manufacturing Systems.
PhD thesis, Technische Universität Berlin, Berlin, Germany, 1995.
[ bib ]

[1383]

R. Atkinson, Jakobus E. van Zyl, Godfrey A. Walters, and Dragan A. Savic.
Genetic algorithm optimisation of levelcontrolled pumping
station operation.
In Water network modelling for optimal design and management,
pp. 79–90. Centre for Water Systems, Exeter, UK, 2000.
[ bib ]

[1384]

Charles Audet, CongKien Dang, and Dominique Orban.
Algorithmic Parameter Optimization of the DFO Method with the
OPAL Framework.
In K. Naono, K. Teranishi, J. Cavazos, and R. Suda, editors,
Software Automatic Tuning: From Concepts to StateoftheArt Results, pp.
255–274. Springer, 2010.
[ bib ]

[1385]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Articulating User Preferences in ManyObjective Problems by
Sampling the Weighted Hypervolume.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pp. 555–562. ACM Press,
New York, NY, 2009.
[ bib ]

[1386]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Investigating and Exploiting the Bias of the Weighted
Hypervolume to Articulate User Preferences.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pp. 563–570. ACM Press,
New York, NY, 2009.
[ bib ]

[1387]

Anne Auger, Johannes Bader, Dimo Brockhoff, and Eckart Zitzler.
Theory of the hypervolume indicator: optimal μdistributions
and the choice of the reference point.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2009, pp. 87–102. ACM Press,
New York, NY, 2009.
[ bib ]

[1388]

Anne Auger, Dimo Brockhoff, Manuel LópezIbáñez, Kaisa Miettinen,
Boris Naujoks, and Günther Rudolph.
Which questions should be asked to find the most appropriate
method for decision making and problem solving? (Working Group
“Algorithm Design Methods”).
In S. Greco, J. D. Knowles, K. Miettinen, and E. Zitzler, editors,
Learning in Multiobjective Optimization (Dagstuhl Seminar 12041),
volume 2(1) of Dagstuhl Reports, pp. 92–93. Schloss
Dagstuhl–LeibnizZentrum für Informatik, Germany, 2012.
[ bib 
DOI ]

[1389]

A. Auger and B. Doerr, editors.
Theory of Randomized Search Heuristics: Foundations and Recent
Developments, volume 1 of Series on Theoretical Computer Science.
World Scientific Publishing Co., Singapore, 2011.
[ bib ]

[1390]

Anne Auger and Nikolaus Hansen.
A restart CMA evolution strategy with increasing population
size.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pp. 1769–1776. IEEE Press, Piscataway, NJ, September 2005.
[ bib 
DOI ]
Keywords: IPOPCMAES

[1391]

Anne Auger and Nikolaus Hansen.
Performance evaluation of an advanced local search evolutionary
algorithm.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pp. 1777–1784. IEEE Press, Piscataway, NJ, September 2005.
[ bib ]
Keywords: LRCMAES

[1392]

Andreea Avramescu, Richard Allmendinger, and Manuel LópezIbáñez.
A MultiObjective MultiType Facility Location Problem for the
Delivery of Personalised Medicine.
In P. Castillo and J. L. Jiménez Laredo, editors,
Applications of Evolutionary Computation, volume 12694 of Lecture Notes
in Computer Science, pp. 388–403. Springer, Cham, Switzerland, 2021.
[ bib 
DOI 
supplementary material ]
Advances in personalised medicine targeting specific
subpopulations and individuals pose a challenge to the
traditional pharmaceutical industry. With a higher level of
personalisation, an already critical supply chain is facing
additional demands added by the very sensitive nature of its
products. Nevertheless, studies concerned with the efficient
development and delivery of these products are scarce. Thus,
this paper presents the case of personalised medicine and the
challenges imposed by its mass delivery. We propose a
multiobjective mathematical model for the
locationallocation problem with two interdependent facility
types in the case of personalised medicine products. We show
its practical application through a cell and gene therapy
case study. A multiobjective genetic algorithm with a novel
population initialisation procedure is used as solution
method.
Keywords: Personalised medicine, Biopharmaceuticals Supply chain,
Facility locationallocation, Evolutionary multiobjective
optimisation

[1393]

Doǧan Aydın, Gürcan Yavuz, Serdar Özyön, Celal Yasar, and
Thomas Stützle.
Artificial Bee Colony Framework to Nonconvex Economic Dispatch
Problem with Valve Point Effects: A Case Study.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO Companion 2017, pp. 1311–1318,
New York, NY, 2017. ACM Press.
[ bib ]

[1394]

Mayowa Ayodele, Richard Allmendinger, Manuel LópezIbáñez, and
Matthieu Parizy.
MultiObjective QUBO Solver: BiObjective Quadratic Assignment
Problem.
In J. E. Fieldsend and M. Wagner, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2022, pp. 467–475.
ACM Press, New York, NY, 2022.
[ bib 
DOI ]
Quantum and quantuminspired optimisation algorithms are
designed to solve problems represented in binary, quadratic
and unconstrained form. Combinatorial optimisation problems
are therefore often formulated as Quadratic Unconstrained
Binary Optimisation Problems (QUBO) to solve them with these
algorithms. Moreover, these QUBO solvers are often
implemented using specialised hardware to achieve enormous
speedups, e.g. Fujitsu's Digital Annealer (DA) and DWave's
Quantum Annealer. However, these are singleobjective
solvers, while many realworld problems feature multiple
conflicting objectives. Thus, a common practice when using
these QUBO solvers is to scalarise such multiobjective
problems into a sequence of singleobjective problems. Due to
design tradeoffs of these solvers, formulating each
scalarisation may require more time than finding a local
optimum. We present the first attempt to extend the algorithm
supporting a commercial QUBO solver as a multiobjective
solver that is not based on scalarisation. The proposed
multiobjective DA algorithm is validated on the biobjective
Quadratic Assignment Problem. We observe that algorithm
performance significantly depends on the archiving strategy
adopted, and that combining DA with nonscalarisation methods
to optimise multiple objectives outperforms the current
scalarised version of the DA in terms of final solution
quality.
Keywords: digital annealer, multiobjective, biobjective QAP, QUBO

[1395]

Mayowa Ayodele.
Penalty Weights in QUBO Formulations: Permutation Problems.
In L. Pérez Cáceres and S. Verel, editors, Proceedings
of EvoCOP 2022 – 22nd European Conference on Evolutionary Computation in
Combinatorial Optimization, Lecture Notes in Computer Science, pp.
159–174. Springer, Cham, Switzerland, 2022.
[ bib ]

[1396]

Amine AzizAlaoui, Carola Doerr, and Johann Dréo.
Towards Large Scale Automated Algorithm Design by Integrating
Modular Benchmarking Frameworks.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO Companion 2021, pp.
1365–1374, New York, NY, 2021. ACM Press.
[ bib 
DOI ]

[1397]

Ilya Loshchilov and T. Glasmachers.
Black Box Optimization Competition, 2017.
[ bib 
http ]

[1398]

Anne Auger, Dimo Brockhoff, Nikolaus Hansen, Dejan Tusar, Tea Tušar, and
Tobias Wagner.
GECCO Workshop on RealParameter BlackBox Optimization
Benchmarking (BBOB 2016): Focus on multiobjective problems.
https://numbbo.github.io/workshops/BBOB2016/, 2016.
[ bib ]

[1399]

Eckart Zitzler, Marco Laumanns, and S. Bleuler.
A tutorial on evolutionary multiobjective optimization.
In X. Gandibleux, M. Sevaux, K. Sörensen, and V. T'Kindt,
editors, Metaheuristics for Multiobjective Optimisation, volume 535 of
Lecture Notes in Economics and Mathematical Systems, pp. 3–37.
Springer, Berlin, Germany, 2004.
[ bib ]

[1400]

S. Bleuler, Marco Laumanns, Lothar Thiele, and Eckart Zitzler.
PISA – A Platform and Programming Language Independent
Interface for Search Algorithms.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pp. 494–508. Springer,
Heidelberg, 2003.
[ bib ]

[1401]

Domagoj Babić.
Spear theorem prover.
https://www.domagojbabic.com/index.php/ResearchProjects/Spear,
2008.
[ bib ]

[1402]

Domagoj Babić and Alan J. Hu.
Structural Abstraction of Software Verification Conditions.
In Computer Aided Verification: 19th International Conference,
CAV 2007, pp. 366–378, 2007.
[ bib ]
Spearswv instances,
http://www.cs.ubc.ca/labs/beta/Projects/ParamILS/benchmark_instances/SpearSWV/SWVscrambledfirst302.tar.gz,
http://www.cs.ubc.ca/labs/beta/Projects/ParamILS/benchmark_instances/SpearSWV/SWVscrambledlast302.tar.gz

[1403]

Domagoj Babić and Frank Hutter.
Spear Theorem Prover.
In SAT'08: Proceedings of the SAT 2008 Race, 2008.
[ bib 
epub 
supplementary material ]
Unreviewed paper

[1404]

Thomas Bäck, David B. Fogel, and Zbigniew Michalewicz.
Handbook of evolutionary computation.
IOP Publishing, 1997.
[ bib ]

[1405]

Achim Bachem, Barthel Steckemetz, and Michael Wottawa.
An efficient parallel clusterheuristic for large Traveling
Salesman Problems.
Technical Report 94150, University of Koln, Germany, 1994.
[ bib ]
Keywords: Genetic Edge Recombination (ERX)

[1406]

Thomas Bäck.
Evolutionary algorithms in theory and practice: evolution
strategies, evolutionary programming, genetic algorithms.
Oxford University Press, 1996.
[ bib ]

[1407]

Prasanna Balaprakash, Mauro Birattari, Thomas Stützle, and Marco Dorigo.
Incremental local search in ant colony optimization: Why it
fails for the quadratic assignment problem.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 5th International Workshop, ANTS 2006, volume 4150 of
Lecture Notes in Computer Science, pp. 156–166. Springer, Heidelberg,
2006.
[ bib ]

[1408]

Prasanna Balaprakash, Mauro Birattari, and Thomas Stützle.
Improvement Strategies for the FRace Algorithm: Sampling
Design and Iterative Refinement.
In T. BartzBeielstein, M. J. Blesa, C. Blum, B. Naujoks, A. Roli,
G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics, volume 4771
of Lecture Notes in Computer Science, pp. 108–122. Springer,
Heidelberg, 2007.
[ bib 
DOI ]
Keywords: Iterated Race

[1409]

Egon Balas and Andrew Ho.
Set Covering Algorithms Using Cutting Planes, Heuristics, and
Subgradient Optimization: A Computational Study.
In M. W. Padberg, editor, Combinatorial optimization, volume 12
of Mathematical Programming Studies, pp. 37–60. Springer,
Berlin/Heidelberg, 1980.
[ bib 
DOI ]

[1410]

P. Baptiste and L. K. Hguny.
A branch and bound algorithm for the
F/no_idle/C_{}max.
In Proceedings of the international conference on industrial
engineering and production management, IEPM'97, pp. 429–438, Lyon, 1997.
[ bib ]

[1411]

Thomas BartzBeielstein.
Experimental Research in Evolutionary Computation: The New
Experimentalism.
Springer, Berlin, Germany, 2006.
[ bib ]
Keywords: SPO

[1412]

Thomas BartzBeielstein.
How to Create Generalizable Results.
In J. Kacprzyk and W. Pedrycz, editors, Springer Handbook of
Computational Intelligence, pp. 1127–1142. Springer, Berlin/
Heidelberg, 2015.
[ bib ]
Keywords: Mixedeffects models, randomeffects model, problem instance
generation

[1413]

Thomas BartzBeielstein, Oliver Flasch, Patrick Koch, and Wolfgang Konen.
SPOT: A Toolbox for Interactive and Automatic Tuning in the
R Environment.
In Proceedings 20. Workshop Computational Intelligence, pp.
264–273, Karlsruhe, 2010. KIT Scientific Publishing.
[ bib ]

[1414]

Thomas BartzBeielstein, C. Lasarczyk, and Mike Preuss.
Sequential Parameter Optimization.
In Proceedings of the 2005 Congress on Evolutionary Computation
(CEC 2005), pp. 773–780, Piscataway, NJ, September 2005. IEEE Press.
[ bib ]

[1415]

Thomas BartzBeielstein, C. Lasarczyk, and Mike Preuss.
The Sequential Parameter Optimization Toolbox.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pp. 337–360. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Keywords: SPOT

[1416]

Thomas BartzBeielstein and Sandor Markon.
Tuning search algorithms for realworld applications: A
regression tree based approach.
In Proceedings of the 2004 Congress on Evolutionary Computation
(CEC 2004), pp. 1111–1118, Piscataway, NJ, September 2004. IEEE Press.
[ bib ]

[1417]

Elias Bareinboim and Judea Pearl.
Transportability of causal effects: Completeness results.
In J. Hoffmann and B. Selman, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pp. 698,704. AAAI Press, 2012.
[ bib ]

[1418]

Thomas BartzBeielstein and Mike Preuss.
Considerations of budget allocation for sequential parameter
optimization (SPO).
In L. Paquete, M. Chiarandini, and D. Basso, editors, Empirical
Methods for the Analysis of Algorithms, Workshop EMAA 2006, Proceedings,
pp. 35–40, Reykjavik, Iceland, 2006.
[ bib ]

[1419]

Thomas BartzBeielstein and Mike Preuss.
Experimental Analysis of Optimization Algorithms: Tuning and
Beyond.
In Y. Borenstein and A. Moraglio, editors, Theory and Principled
Methods for the Design of Metaheuristics, Natural Computing Series, pp.
205–245. Springer, Berlin/Heidelberg, 2014.
[ bib 
DOI ]

[1420]

Benjamín Barán and Matilde Schaerer.
A multiobjective ant colony system for vehicle routing problem
with time windows.
In Proceedings of the Twentyfirst IASTED International
Conference on Applied Informatics, pp. 97–102, Insbruck, Austria, 2003.
[ bib ]

[1421]

Matthieu Basseur, Adrien Goëffon, Arnaud Liefooghe, and Sébastien
Verel.
On Setbased Local Search for Multiobjective Combinatorial
Optimization.
In C. Blum and E. Alba, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2013, pp. 471–478. ACM Press,
New York, NY, 2013.
[ bib 
DOI ]

[1422]

Roberto Battiti, M. Brunato, and Franco Mascia.
Reactive Search and Intelligent Optimization, volume 45 of
Operations Research/Computer Science Interfaces.
Springer, New York, NY, 2008.
[ bib 
DOI ]

[1423]

Roberto Battiti and Paolo Campigotto.
Reactive search optimization: Learning while optimizing. An
experiment in interactive multiobjective optimization.
In M. Caserta and S. Voß, editors, Proceedings of MIC 2009,
the 8th Metaheuristics International Conference, Hamburg, Germany, 2010.
University of Hamburg.
[ bib ]

[1424]

Michele Battistutta, Andrea Schaerf, and Tommaso Urli.
Featurebased tuning of singlestage simulated annealing for
examination timetabling.
In E. Özcan, E. K. Burke, and B. McCollum, editors, PATAT
2014: Proceedings of the 10th International Conference of the Practice and
Theory of Automated Timetabling, pp. 53–61. PATAT, 2014.
[ bib ]
Keywords: Frace

[1425]

E. B. Baum.
Iterated Descent: A Better Algorithm for Local Search in
Combinatorial Optimization Problems.
Manuscript, 1986.
[ bib ]

[1426]

E. B. Baum.
Towards Practical “Neural” Computation for Combinatorial
Optimization Problems.
In Neural Networks for Computing, AIP Conference Proceedings,
pp. 53–64, 1986.
[ bib ]

[1427]

A. Baykasoglu, T. Dereli, and I. Sabuncu.
A multiple objective ant colony optimization approach to
assembly line balancing problems.
In 35th International Conference on Computers and Industrial
Engineering (CIE35), pp. 263–268, Istanbul, Turkey, 2005.
[ bib ]

[1428]

Brian Beachkofski and Ramana Grandhi.
Improved Distributed Hypercube Sampling.
In Proceedings of the 43rd AIAA/ASME/ASCE/AHS/ASC Structures,
Structural Dynamics, and Materials Conference. AIAA paper 20021274,
American Institute of Aeronautics and Astronautics, 2002.
[ bib ]

[1429]

John E. Beasley.
Heuristic algorithms for the unconstrained binary quadratic
programming problem.
Technical report, The Management School, Imperial College, London,
England, 1998.
[ bib 
epub ]

[1430]

S. Becker, J. Gottlieb, and Thomas Stützle.
Applications of Racing Algorithms: An Industrial Perspective.
In E.G. Talbi, P. Liardet, P. Collet, E. Lutton, and M. Schoenauer,
editors, Artificial Evolution, volume 3871 of Lecture Notes in
Computer Science, pp. 271–283, Heidelberg, 2005. Springer.
[ bib ]

[1431]

David D. Bedworth and James E. Bailey.
Integrated Production Control Systems: Management, Analysis,
Design, volume 2.
John Wiley & Sons, New York, NY, 1982.
[ bib ]

[1432]

Andreas Beham, Michael Affenzeller, and Stefan Wagner.
Instancebased algorithm selection on quadratic assignment
problem landscapes.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO Companion 2017, pp. 1471–1478,
New York, NY, 2017. ACM Press.
[ bib ]

[1433]

Valerie Belton, Jürgen Branke, Petri Eskelinen, Salvatore Greco, Julián
Molina, Francisco Ruiz, and Roman Slowiński.
Interactive Multiobjective Optimization from a Learning
Perspective.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science, pp. 405–433.
Springer, Heidelberg, 2008.
[ bib 
DOI ]

[1434]

Nacim Belkhir, Johann Dréo, Pierre Savéant, and Marc Schoenauer.
Feature Based Algorithm Configuration: A Case Study with
Differential Evolution.
In J. Handl, E. Hart, P. R. Lewis, M. LópezIbáñez,
G. Ochoa, and B. Paechter, editors, Parallel Problem Solving from Nature
– PPSN XIV, volume 9921 of Lecture Notes in Computer Science,
pp. 156–166, Heidelberg, 2016. Springer.
[ bib 
DOI ]

[1435]

Nacim Belkhir, Johann Dréo, Pierre Savéant, and Marc Schoenauer.
Per Instance Algorithm Configuration of CMAES with Limited
Budget.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pp. 681–688. ACM Press,
New York, NY, 2017.
[ bib ]

[1436]

Jon Louis Bentley.
Experiments on Traveling Salesman Heuristics.
In D. S. Johnson, editor, Proceedings of the First Annual
ACMSIAM Symposium on Discrete Algorithms, pp. 91–99. Society for
Industrial and Applied Mathematics, Philadelphia, PA, USA, 1990.
[ bib ]

[1437]

Nawal Benabbou, Cassandre Leroy, and Thibaut Lust.
An Interactive RegretBased Genetic Algorithm for Solving
MultiObjective Combinatorial Optimization Problems.
In Proceedings of the AAAI Conference on Artificial
Intelligence, pp. 2335–2342. AAAI Press, 2020.
[ bib 
DOI ]
Keywords: interactive, multiobjective, decisionmakers

[1438]

Alexander Javier Benavides and Marcus Ritt.
Iterated Local Search Heuristics for Minimizing Total Completion
Time in Permutation and Nonpermutation Flow Shops.
In R. I. Brafman, C. Domshlak, P. Haslum, and S. Zilberstein,
editors, Proceedings of the TwentyFifth International Conference on
Automated Planning and Scheduling, ICAPS 2015, pp. 34–41. AAAI Press,
Menlo Park, CA, 2015.
[ bib ]
Keywords: irace

[1439]

Stefano Benedettini, Andrea Roli, and Christian Blum.
A Randomized Iterated Greedy Algorithm for the Founder Sequence
Reconstruction Problem.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pp. 37–51. Springer, Heidelberg, 2010.
[ bib 
DOI ]

[1440]

Stefano Benedettini, Andrea Roli, and Luca Di Gaspero.
Twolevel ACO for Haplotype Inference under Pure Parsimony.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 6th International Conference, ANTS 2008, volume 5217 of
Lecture Notes in Computer Science, pp. 179–190. Springer, Heidelberg,
2008.
[ bib ]

[1441]

D. Bertsekas.
Dynamic Programming and Optimal Control.
Athena Scientific, Belmont, MA, 1995.
[ bib ]

[1442]

D. Bertsekas.
Network Optimization: Continuous and Discrete Models.
Athena Scientific, Belmont, MA, 1998.
[ bib ]

[1443]

James S. Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl.
Algorithms for HyperParameter Optimization.
In J. ShaweTaylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and
K. Q. Weinberger, editors, Advances in Neural Information Processing
Systems (NIPS 24), pp. 2546–2554. Curran Associates, Red Hook, NY, 2011.
[ bib 
http ]

[1444]

Hughes Bersini, Marco Dorigo, S. Langerman, G. Seront, and L. M. Gambardella.
Results of the First International Contest on Evolutionary
Optimisation.
In T. Bäck, T. Fukuda, and Z. Michalewicz, editors,
Proceedings of the 1996 IEEE International Conference on Evolutionary
Computation (ICEC'96), pp. 611–615, Piscataway, NJ, 1996. IEEE Press.
[ bib ]

[1445]

Felix Berkenkamp, Angela P. Schoellig, and Andreas Krause.
Safe controller optimization for quadrotors with Gaussian
processes.
In 2016 IEEE International Conference on Robotics and Automation
(ICRA), pp. 491–496. IEEE, 2016.
[ bib 
DOI ]

[1446]

James S. Bergstra, Daniel Yasmin, and David Cox.
Making a science of model search: Hyperparameter optimization in
hundreds of dimensions for vision architectures.
In S. Dasgupta and D. McAllester, editors, Proceedings of the
30th International Conference on Machine Learning, ICML 2013, volume 28,
pp. 115–123, 2013.
[ bib 
http ]

[1447]

Matthijs L. den Besten, Thomas Stützle, and Marco Dorigo.
Ant Colony Optimization for the Total Weighted Tardiness
Problem.
In M. Schoenauer et al., editors, Parallel Problem Solving from
Nature – PPSN VI, volume 1917 of Lecture Notes in Computer
Science, pp. 611–620, Heidelberg, 2000. Springer.
[ bib ]

[1448]

Matthijs L. den Besten, Thomas Stützle, and Marco Dorigo.
Design of Iterated Local Search Algorithms: An Example
Application to the Single Machine Total Weighted Tardiness Problem.
In E. J. W. Boers et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2001, volume 2037 of Lecture
Notes in Computer Science, pp. 441–452. Springer, Heidelberg, 2001.
[ bib ]

[1449]

Nicola Beume and Günther Rudolph.
Faster SMetric Calculation by Considering Dominated
Hypervolume as Klee's Measure Problem.
In B. Kovalerchuk, editor, Proceedings of the Second IASTED
Conference on Computational Intelligence, pp. 231–236. ACTA Press,
Anaheim, 2006.
[ bib ]

[1450]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Generation of MultiObjective ACO Algorithms for the
Biobjective Knapsack.
In M. Dorigo et al., editors, Swarm Intelligence, 8th
International Conference, ANTS 2012, volume 7461 of Lecture Notes in
Computer Science, pp. 37–48. Springer, Heidelberg, 2012.
[ bib 
DOI 
supplementary material ]

[1451]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Generation of MOACO Algorithms for the Biobjective
Bidimensional Knapsack Problem: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012008/, 2012.
[ bib ]

[1452]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An Analysis of Local Search for the Biobjective Bidimensional
Knapsack: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2012016/, 2013.
[ bib ]

[1453]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Deconstructing MultiObjective Evolutionary Algorithms: An
Iterative Analysis on the Permutation Flowshop: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2013010/, 2013.
[ bib ]

[1454]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An Analysis of Local Search for the Biobjective Bidimensional
Knapsack Problem.
In M. Middendorf and C. Blum, editors, Proceedings of EvoCOP
2013 – 13th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7832 of Lecture Notes in Computer Science, pp.
85–96. Springer, Heidelberg, 2013.
[ bib 
DOI ]

[1455]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic ComponentWise Design of MultiObjective
Evolutionary Algorithms.
Technical Report TR/IRIDIA/2014012, IRIDIA, Université Libre de
Bruxelles, Belgium, August 2014.
[ bib ]

[1456]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Deconstructing MultiObjective Evolutionary Algorithms: An
Iterative Analysis on the Permutation Flowshop.
In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, and J. L.
Walteros, editors, Learning and Intelligent Optimization, 8th
International Conference, LION 8, volume 8426 of Lecture Notes in
Computer Science, pp. 57–172. Springer, Heidelberg, 2014.
[ bib 
DOI 
supplementary material ]

[1457]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Design of Evolutionary Algorithms for MultiObjective
Combinatorial Optimization.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, Parallel Problem Solving from Nature – PPSN XIII, volume
8672 of Lecture Notes in Computer Science, pp. 508–517. Springer,
Heidelberg, 2014.
[ bib 
DOI ]

[1458]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Design of Evolutionary Algorithms for MultiObjective
Combinatorial Optimization.
http://iridia.ulb.ac.be/supp/IridiaSupp2014007/, 2014.
[ bib ]

[1459]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic ComponentWise Design of MultiObjective Evolutionary
Algorithms.
https://github.com/iridiaulb/automoeatevc2016, 2015.
[ bib ]

[1460]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
To DE or Not to DE? Multiobjective Differential Evolution
Revisited from a ComponentWise Perspective: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2015001/, 2015.
[ bib ]

[1461]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
To DE or Not to DE? Multiobjective Differential Evolution
Revisited from a ComponentWise Perspective.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pp. 48–63.
Springer, Heidelberg, 2015.
[ bib 
DOI ]

[1462]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Comparing DecompositionBased and Automatically
ComponentWise Designed MultiObjective Evolutionary Algorithms.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pp. 396–410,
Heidelberg, 2015. Springer.
[ bib 
DOI ]

[1463]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
http://iridia.ulb.ac.be/supp/IridiaSupp2015007/, 2017.
[ bib ]

[1464]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
A LargeScale Experimental Evaluation of HighPerforming Multi
and ManyObjective Evolutionary Algorithms.
Technical Report TR/IRIDIA/2017005, IRIDIA, Université Libre de
Bruxelles, Belgium, February 2017.
[ bib ]

[1465]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
An Empirical Assessment of the Properties of Inverted
Generational Distance Indicators on Multi and Manyobjective Optimization.
In H. Trautmann, G. Rudolph, K. Klamroth, O. Schütze, M. M.
Wiecek, Y. Jin, and C. Grimme, editors, Evolutionary Multicriterion
Optimization, EMO 2017, volume 10173 of Lecture Notes in Computer
Science, pp. 31–45. Springer International Publishing, Cham, Switzerland,
2017.
[ bib 
DOI ]

[1466]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatically Designing StateoftheArt Multi and
ManyObjective Evolutionary Algorithms: Supplementary material.
https://github.com/iridiaulb/automoeaecj2020, 2019.
[ bib ]

[1467]

Hao Wang, Chaoli Sun, Yaochu Jin, Shufen Qin, and Haibo Yu.
A Multiindicator based Selection Strategy for Evolutionary
Manyobjective Optimization.
In Proceedings of the 2019 Congress on Evolutionary Computation
(CEC 2019), pp. 2042–2049, Piscataway, NJ, 2019. IEEE Press.
[ bib ]
unbounded archive

[1468]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Archiver Effects on the Performance of Stateoftheart Multi
and Manyobjective Evolutionary Algorithms.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019. ACM Press, New York, NY, 2019.
[ bib 
DOI 
epub 
supplementary material ]

[1469]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Archiver Effects on the Performance of Stateoftheart Multi
and Manyobjective Evolutionary Algorithms: Supplementary material.
http://iridia.ulb.ac.be/supp/IridiaSupp2019004/, 2019.
[ bib ]

[1470]

Leonardo C. T. Bezerra, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Configuration of Multiobjective Optimizers and
Multiobjective Configuration.
In T. BartzBeielstein, B. Filipič, P. Korošec, and E.G.
Talbi, editors, HighPerformance SimulationBased Optimization, pp.
69–92. Springer International Publishing, Cham, Switzerland, 2020.
[ bib 
DOI ]
Heuristic optimizers are an important tool in academia and industry, and their performanceoptimizing configuration requires a significant amount of expertise. As the proper configuration of algorithms is a crucial aspect in the engineering of heuristic algorithms, a significant research effort has been dedicated over the last years towards moving this step to the computer and, thus, make it automatic. These research efforts go way beyond tuning only numerical parameters of already fully defined algorithms, but exploit automatic configuration as a means for automatic algorithm design. In this chapter, we review two main aspects where the research on automatic configuration and multiobjective optimization intersect. The first is the automatic configuration of multiobjective optimizers, where we discuss means and specific approaches. In addition, we detail a case study that shows how these approaches can be used to design new, highperforming multiobjective evolutionary algorithms. The second aspect is the research on multiobjective configuration, that is, the possibility of using multiple performance metrics for the evaluation of algorithm configurations. We highlight some few examples in this direction.

[1471]

Leonardo C. T. Bezerra.
A componentwise approach to multiobjective evolutionary
algorithms: from flexible frameworks to automatic design.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2016.
[ bib ]
Supervised by Thomas Stützle and Manuel LópezIbáñez

[1472]

Leonora Bianchi, L. M. Gambardella, and Marco Dorigo.
An Ant Colony Optimization Approach to the Probabilistic
Traveling Salesman Problem.
In J.J. Merelo et al., editors, Parallel Problem Solving from
Nature – PPSN VII, volume 2439 of Lecture Notes in Computer
Science, pp. 883–892, Heidelberg, 2002. Springer.
[ bib ]

[1473]

Armin Biere.
Yet another Local Search Solver and Lingeling and Friends
Entering the SAT Competition 2014.
In A. Belov, D. Diepold, M. Heule, and M. Järvisalo, editors,
Proceedings of SAT Competition 2014: Solver and Benchmark Descriptions,
volume B20142 of Science Series of Publications B, pp. 39–40.
University of Helsinki, 2014.
[ bib ]

[1474]

André Biedenkapp, H. Furkan Bozkurt, Theresa Eimer, Frank Hutter, and
Marius Thomas Lindauer.
Dynamic Algorithm Configuration: Foundation of a New
MetaAlgorithmic Framework.
In G. D. Giacomo, A. Catala, B. Dilkina, M. Milano, S. Barro,
A. BugarÃn, and J. Lang, editors, Proceedings of the 24th European
Conference on Artificial Intelligence (ECAI), volume 325 of Frontiers
in Artificial Intelligence and Applications, pp. 427–434. IOS Press, 2020.
[ bib 
epub ]

[1475]

André Biedenkapp, Marius Thomas Lindauer, Katharina Eggensperger, Frank
Hutter, Chris Fawcett, and Holger H. Hoos.
Efficient Parameter Importance Analysis via Ablation with
Surrogates.
In S. P. Singh and S. Markovitch, editors, Proceedings of the
AAAI Conference on Artificial Intelligence. AAAI Press, February 2017.
[ bib 
DOI ]

[1476]

André Biedenkapp, Joshua Marben, Marius Thomas Lindauer, and Frank Hutter.
Cave: Configuration assessment, visualization and evaluation.
In R. Battiti, M. Brunato, I. Kotsireas, and P. M. Pardalos, editors,
Learning and Intelligent Optimization, 12th International Conference,
LION 12, volume 11353 of Lecture Notes in Computer Science, pp.
115–130, Cham, Switzerland, 2018. Springer.
[ bib ]

[1477]

George Bilchev and Ian C. Parmee.
The Ant Colony Metaphor for Searching Continuous Design Spaces.
In T. C. Fogarty, editor, Evolutionary Computing, AISB
Workshop, volume 993 of Lecture Notes in Computer Science, pp.
25–39. Springer, Berlin, Germany, 1995.
[ bib 
DOI ]

[1478]

Mauro Birattari, Prasanna Balaprakash, and Marco Dorigo.
The ACO/FRACE algorithm for combinatorial optimization under
uncertainty.
In K. F. Doerner, M. Gendreau, P. Greistorfer, W. J. Gutjahr, R. F.
Hartl, and M. Reimann, editors, Metaheuristics – Progress in Complex
Systems Optimization, volume 39 of Operations Research/Computer Science
Interfaces Series, pp. 189–203. Springer, New York, NY, 2006.
[ bib ]

[1479]

Mauro Birattari, Marco Chiarandini, Marco Saerens, and Thomas Stützle.
Learning Graphical Models for Algorithm Configuration.
In T. Berthold, A. M. Gleixner, S. Heinz, and T. Koch, editors,
Integration of AI and OR Techniques in Contraint Programming for
Combinatorial Optimization Problems, Lecture Notes in Computer Science.
Springer, Heidelberg, 2011.
[ bib ]

[1480]

Mauro Birattari, Gianni A. Di Caro, and Marco Dorigo.
Toward the formal foundation of Ant Programming.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pp. 188–201. Springer, Heidelberg, 2002.
[ bib ]

[1481]

Steven Bird, Ewan Klein, and Edward Loper.
Natural language processing with Python: analyzing text with
the natural language toolkit.
O'Reilly Media, Inc., 2009.
[ bib ]

[1482]

Mauro Birattari, Thomas Stützle, Luís Paquete, and Klaus Varrentrapp.
A Racing Algorithm for Configuring Metaheuristics.
In W. B. Langdon et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2002, pp. 11–18. Morgan
Kaufmann Publishers, San Francisco, CA, 2002.
[ bib 
epub ]
Keywords: Frace

[1483]

Mauro Birattari, Zhi Yuan, Prasanna Balaprakash, and Thomas Stützle.
FRace and Iterated FRace: An Overview.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pp. 311–336. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Keywords: Frace, iterated Frace, irace, tuning

[1484]

Mauro Birattari, Zhi Yuan, Prasanna Balaprakash, and Thomas Stützle.
Parameter Adaptation in Ant Colony Optimization.
In M. Caserta and S. Voß, editors, Proceedings of MIC 2009,
the 8th Metaheuristics International Conference, Hamburg, Germany, 2010.
University of Hamburg.
[ bib ]

[1485]

Mauro Birattari.
Tuning Metaheuristics: A Machine Learning Perspective, volume
197 of Studies in Computational Intelligence.
Springer, Berlin/Heidelberg, 2009.
[ bib 
DOI ]
Based on the PhD thesis [1486]

[1486]

Mauro Birattari.
The Problem of Tuning Metaheuristics as Seen from a Machine
Learning Perspective.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2004.
[ bib ]
Supervised by Marco Dorigo

[1487]

Francesco Biscani, Dario Izzo, and Chit Hong Yam.
A Global Optimisation Toolbox for Massively Parallel Engineering
Optimisation.
In Astrodynamics Tools and Techniques (ICATT 2010), 4th
International Conference on, 2010.
[ bib 
http ]
Keywords: PaGMO

[1488]

Bernd Bischl, Olaf Mersmann, Heike Trautmann, and Mike Preuss.
Algorithm Selection Based on Exploratory Landscape Analysis and
Costsensitive Learning.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pp. 313–320. ACM
Press, New York, NY, 2012.
[ bib ]
Keywords: continuous optimization, landscape analysis, algorithm selection

[1489]

Christopher M. Bishop.
Pattern recognition and machine learning.
Springer, 2006.
[ bib ]

[1490]

Erdem Bıyık, Jonathan Margoliash, Shahrouz Ryan Alimo, and Dorsa
Sadigh.
Efficient and Safe Exploration in Deterministic Markov
Decision Processes with Unknown Transition Models.
In 2019 American Control Conference (ACC), pp. 1792–1799.
IEEE, 2019.
[ bib 
DOI ]

[1491]

María J. Blesa and Christian Blum.
Ant Colony Optimization for the Maximum EdgeDisjoint Paths
Problem.
In G. R. Raidl et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2004, volume 3005 of Lecture
Notes in Computer Science, pp. 160–169. Springer, Heidelberg, 2004.
[ bib ]

[1492]

Aymeric Blot, Holger H. Hoos, Laetitia Jourdan, MarieEléonore
KessaciMarmion, and Heike Trautmann.
MOParamILS: A Multiobjective Automatic Algorithm
Configuration Framework.
In P. Festa, M. Sellmann, and J. Vanschoren, editors, Learning
and Intelligent Optimization, 10th International Conference, LION 10, volume
10079 of Lecture Notes in Computer Science, pp. 32–47. Springer,
Cham, Switzerland, 2016.
[ bib ]

[1493]

Aymeric Blot, Laetitia Jourdan, and MarieEléonore KessaciMarmion.
Automatic design of multiobjective local search algorithms:
case study on a biobjective permutation flowshop scheduling problem.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pp. 227–234. ACM Press,
New York, NY, 2017.
[ bib 
DOI ]

[1494]

Aymeric Blot, Manuel LópezIbáñez, MarieEléonore
KessaciMarmion, and Laetitia Jourdan.
New Initialisation Techniques for MultiObjective Local Search:
Application to the Biobjective Permutation Flowshop.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature – PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pp.
323–334. Springer, Cham, Switzerland, 2018.
[ bib 
DOI ]

[1495]

Aymeric Blot, Alexis Pernet, Laetitia Jourdan, MarieEléonore
KessaciMarmion, and Holger H. Hoos.
Automatically Configuring Multiobjective Local Search Using
Multiobjective Optimisation.
In H. Trautmann, G. Rudolph, K. Klamroth, O. Schütze, M. M.
Wiecek, Y. Jin, and C. Grimme, editors, Evolutionary Multicriterion
Optimization, EMO 2017, volume 10173 of Lecture Notes in Computer
Science, pp. 61–76. Springer International Publishing, Cham, Switzerland,
2017.
[ bib ]

[1496]

Christian Blum, J. Bautista, and J. Pereira.
BeamACO applied to assembly line balancing.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 5th International Workshop, ANTS 2006, volume 4150 of
Lecture Notes in Computer Science, pp. 96–107. Springer, Heidelberg, 2006.
[ bib 
DOI ]

[1497]

Christian Blum, María J. Blesa, and Manuel LópezIbáñez.
Beam Search for the Longest Common Subsequence Problem.
Technical Report LSI0829, Department LSI, Universitat
Politècnica de Catalunya, 2008.
Published in Computers & Operations
Research [137].
[ bib ]

[1498]

Christian Blum, Carlos Cotta, Antonio J. Fernández, and J. E. Gallardo.
A probabilistic beam search algorithm for the shortest common
supersequence problem.
In C. Cotta et al., editors, Proceedings of EvoCOP 2007 –
Seventh European Conference on Evolutionary Computation in Combinatorial
Optimisation, volume 4446 of Lecture Notes in Computer Science, pp.
36–47. Springer, Berlin, Germany, 2007.
[ bib ]

[1499]

Christian Blum and Manuel LópezIbáñez.
Ant Colony Optimization.
In The Industrial Electronics Handbook: Intelligent Systems.
CRC Press, 2nd edition, 2011.
[ bib 
http ]

[1500]

Christian Blum and M. Mastrolilli.
Using Branch & Bound Concepts in ConstructionBased
Metaheuristics: Exploiting the Dual Problem Knowledge.
In T. BartzBeielstein, M. J. Blesa, C. Blum, B. Naujoks, A. Roli,
G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics, volume 4771
of Lecture Notes in Computer Science, pp. 123–139. Springer,
Heidelberg, 2007.
[ bib ]

[1501]

C. Blum and D. Merkle, editors.
Swarm Intelligence–Introduction and Applications.
Natural Computing Series. Springer Verlag, Berlin, Germany, 2008.
[ bib ]

[1502]

Christian Blum and Günther R. Raidl.
Hybrid Metaheuristics—Powerful Tools for Optimization.
Artificial Intelligence: Foundations, Theory, and Algorithms.
Springer, Berlin, Germany, 2016.
[ bib ]

[1503]

Christian Blum and Andrea Roli.
Hybrid metaheuristics: an introduction.
In C. Blum, M. J. Blesa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics: An emergent approach for optimization, volume 114 of
Studies in Computational Intelligence, pp. 1–30. Springer, Berlin,
Germany, 2008.
[ bib ]

[1504]

Christian Blum and M. Yábar Vallès.
Multilevel ant colony optimization for DNA sequencing by
hybridization.
In F. Almeida et al., editors, Hybrid Metaheuristics, volume
4030 of Lecture Notes in Computer Science, pp. 94–109. Springer,
Heidelberg, 2006.
[ bib 
DOI ]

[1505]

K. D. Boese.
Models for Iterative Global Optimization.
PhD thesis, University of California, Computer Science Department,
Los Angeles, CA, 1996.
[ bib ]

[1506]

Béla Bollobás.
Random Graphs.
Cambridge University Press, New York, NY, 2nd edition, 2001.
[ bib ]

[1507]

Grady Booch, James E. Rumbaugh, and Ivar Jacobson.
The Unified Modeling Language User Guide.
AddisonWesley, 2nd edition, 2005.
[ bib ]

[1508]

P. C. Borges and Michael Pilegaard Hansen.
A basis for future successes in multiobjective combinatorial
optimization.
Technical Report IMMREP19988, Institute of Mathematical Modelling,
Technical University of Denmark, Lyngby, Denmark, 1998.
[ bib ]

[1509]

Allan Borodin and Ran ElYaniv.
Online computation and competitive analysis.
Cambridge University Press, New York, NY, 1998.
[ bib ]

[1510]

Michael Borenstein, Larry V. Hedges, Julian P. T. Higgins, and Hannah R.
Rothstein.
Introduction to MetaAnalysis.
Wiley, 2009.
[ bib ]

[1511]

Bernhard E. Boser, Isabelle Guyon, and Vladimir Vapnik.
A Training Algorithm for Optimal Margin Classifiers.
In D. Haussler, editor, COLT'92, pp. 144–152. ACM Press,
1992.
[ bib 
DOI ]
Proposed SVM

[1512]

Jakob Bossek, Pascal Kerschke, Aneta Neumann, Markus Wagner, Frank Neumann, and
Heike Trautmann.
Evolving Diverse TSP Instances by Means of Novel and Creative
Mutation Operators.
In T. Friedrich, C. Doerr, and D. V. Arnold, editors,
Proceedings of the 15th ACM/SIGEVO Conference on Foundations of Genetic
Algorithms, pp. 58–71. ACM, 2019.
[ bib ]

[1513]

Paul F. Boulos, Chun Hou Orr, Werner de Schaetzen, J. G. Chatila, Michael
Moore, Paul Hsiung, and Devan Thomas.
Optimal pump operation of water distribution systems using
genetic algorithms.
In AWWA Distribution System Symp., Denver, USA, 2001. American
Water Works Association.
[ bib ]

[1514]

V. Bowman and Jr. Joseph.
On the Relationship of the Tchebycheff Norm and the Efficient
Frontier of MultipleCriteria Objectives.
In H. Thiriez and S. Zionts, editors, Multiple Criteria Decision
Making, volume 130 of Lecture Notes in Economics and Mathematical
Systems, pp. 76–86. Springer, Berlin/Heidelberg, 1976.
[ bib 
DOI ]

[1515]

George E. P. Box and Norman R. Draper.
Response surfaces, mixtures, and ridge analyses.
John Wiley & Sons, 2007.
[ bib ]

[1516]

G. E. P. Box, W. G. Hunter, and J. S. Hunter.
Statistics for experimenters: an introduction to design, data
analysis, and model building.
John Wiley & Sons, New York, NY, 1978.
[ bib ]

[1517]

A. Brandt.
Multilevel Computations: Review and Recent Developments.
In S. F. McCormick, editor, Multigrid Methods: Theory,
Applications, and Supercomputing, Proceedings of the 3rd Copper Mountain
Conference on Multigrid Methods, volume 110 of Lecture Notes in Pure
and Applied Mathematics, pp. 35–62. Marcel Dekker, New York, NY, 1988.
[ bib ]

[1518]

L. Bradstreet, L. Barone, L. While, S. Huband, and P. Hingston.
Use of the WFG Toolkit and PISA for Comparison of MOEAs.
In IEEE Symposium on Computational Intelligence in
Multicriteria DecisionMaking, IEEE MCDM, pp. 382–389, 2007.
[ bib ]

[1519]

Cristóbal BarbaGonzález, Vesa Ojalehto, José GarcíaNieto,
Antonio J. Nebro, Kaisa Miettinen, and José F. AldanaMontes.
Artificial Decision Maker Driven by PSO: An Approach for
Testing Reference Point Based Interactive Methods.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature – PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pp.
274–285. Springer, Cham, Switzerland, 2018.
[ bib 
DOI ]
Keywords: machine decisionmaker

[1520]

Jürgen Branke, Salvatore Corrente, Salvatore Greco, Milosz
Kadziński, Manuel LópezIbáñez, Vincent Mousseau, Mauro
Munerato, and Roman Slowiński.
BehaviorRealistic Artificial DecisionMakers to Test
PreferenceBased Multiobjective Optimization Method (Working Group
“Machine DecisionMaking”).
In S. Greco, K. Klamroth, J. D. Knowles, and G. Rudolph, editors,
Understanding Complexity in Multiobjective Optimization (Dagstuhl
Seminar 15031), volume 5(1) of Dagstuhl Reports, pp. 110–116.
Schloss Dagstuhl–LeibnizZentrum für Informatik, Germany, 2015.
[ bib 
DOI ]
Keywords: multiple criteria decision making, evolutionary
multiobjective optimization

[1521]

Yesnier Bravo, Javier Ferrer, Gabriel J. Luque, and Enrique Alba.
Smart Mobility by Optimizing the Traffic Lights: A New Tool for
Traffic Control Centers.
In E. Alba, F. Chicano, and G. J. Luque, editors, Smart Cities
(SmartCT 2016), Lecture Notes in Computer Science, pp. 147–156. Springer,
Cham, Switzerland, 2016.
[ bib 
DOI ]
Urban traffic planning is a fertile area of Smart Cities to
improve efficiency, environmental care, and safety, since the
traffic jams and congestion are one of the biggest sources of
pollution and noise. Traffic lights play an important role in
solving these problems since they control the flow of the
vehicular network at the city. However, the increasing number
of vehicles makes necessary to go from a local control at one
single intersection to a holistic approach considering a
large urban area, only possible using advanced computational
resources and techniques. Here we propose HITUL, a system
that supports the decisions of the traffic control managers
in a large urban area. HITUL takes the real traffic
conditions and compute optimal traffic lights plans using
bioinspired techniques and microsimulations. We compare our
system against plans provided by experts. Our solutions not
only enable continuous traffic flows but reduce the
pollution. A case study of Málaga city allows us to
validate the approach and show its benefits for other cities
as well.
Keywords: Multiobjective optimization, Smart mobility, Traffic lights
planning

[1522]

JeanPierre Brans and Bertrand Mareschal.
PROMETHEEGAIA. Une méthode d'aide à la décision
en présence de critères multiples.
Editions Ellipses, Paris, FR, 2002.
[ bib ]

[1523]

JeanPierre Brans and Bertrand Mareschal.
PROMETHEE Methods.
In J. R. Figueira, S. Greco, and M. Ehrgott, editors, Multiple
Criteria Decision Analysis, State of the Art Surveys, chapter 5, pp.
163–195. Springer, 2005.
[ bib ]

[1524]

Jürgen Branke, C. Schmidt, and H. Schmeck.
Efficient fitness estimation in noisy environments.
In E. D. Goodman, editor, Proceedings of the 3rd Annual
Conference on Genetic and Evolutionary Computation, GECCO 2001, pp.
243–250. Morgan Kaufmann Publishers, San Francisco, CA, 2001.
[ bib ]

[1525]

Jürgen Branke, Salvatore Corrente, Salvatore Greco, Roman Slowiński,
and P. Zielniewicz.
Using Choquet integral as preference model in interactive
evolutionary multiobjective optimization.
Technical report, WBS, University of Warwick, 2014.
[ bib ]

[1526]

Jürgen Branke and Jawad Elomari.
Simultaneous tuning of metaheuristic parameters for various
computing budgets.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 263–264.
ACM Press, New York, NY, 2011.
[ bib 
DOI ]
Keywords: metaoptimization, offline parameter optimization

[1527]

Jürgen Branke and Jawad Elomari.
Racing with a Fixed Budget and a SelfAdaptive Significance
Level.
In P. M. Pardalos and G. Nicosia, editors, Learning and
Intelligent Optimization, 7th International Conference, LION 7, volume 7997
of Lecture Notes in Computer Science. Springer, Heidelberg, 2013.
[ bib ]

[1528]

Leo Breiman, Jerome Friedman, Charles J. Stone, and Richard A. Olshen.
Classification and regression trees.
CRC Press, 1984.
[ bib ]

[1529]

Mátyás Brendel and Marc Schoenauer.
LearnandOptimize: A Parameter Tuning Framework for
Evolutionary AI Planning.
In J.K. Hao, P. Legrand, P. Collet, N. Monmarché, E. Lutton, and
M. Schoenauer, editors, Artificial Evolution: 10th International
Conference, Evolution Artificielle, EA, 2011, volume 7401 of Lecture
Notes in Computer Science, pp. 145–155. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1530]

Mátyás Brendel and Marc Schoenauer.
Instancebased Parameter Tuning for Evolutionary AI Planning.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO Companion 2011, pp.
591–598, New York, NY, 2011. ACM Press.
[ bib 
DOI ]

[1531]

Karl Bringmann and Tobias Friedrich.
Approximating the Least Hypervolume Contributor: NPHard in
General, But Fast in Practice.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pp. 6–20.
Springer, Heidelberg, 2009.
[ bib ]
Extended version published in [172]

[1532]

Karl Bringmann and Tobias Friedrich.
The Maximum Hypervolume Set Yields Nearoptimal Approximation.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pp. 511–518. ACM
Press, New York, NY, 2010.
[ bib ]
Proved that hypervolume approximates the additive
εindicator, converging quickly as N increases,
that is, sets that maximize hypervolume are near optimal on
additive ε too, with the gap diminishing as quickly
as O(1/N).

[1533]

Karl Bringmann and Tobias Friedrich.
Tight bounds for the approximation ratio of the hypervolume
indicator.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of Lecture
Notes in Computer Science, pp. 607–616, Heidelberg, 2010. Springer.
[ bib ]

[1534]

Karl Bringmann and Tobias Friedrich.
Convergence of HypervolumeBased Archiving Algorithms I:
Effectiveness.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 745–752.
ACM Press, New York, NY, 2011.
[ bib 
DOI ]
Extended version published as [174]

[1535]

Karl Bringmann and Tobias Friedrich.
Convergence of HypervolumeBased Archiving Algorithms II:
Competitiveness.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pp. 457–464. ACM
Press, New York, NY, 2012.
[ bib 
DOI ]
Extended version published as [174]

[1536]

Karl Bringmann, Tobias Friedrich, and Patrick Klitzke.
Generic postprocessing via subset selection for hypervolume and
epsilonindicator.
In T. BartzBeielstein, J. Branke, B. Filipič, and J. Smith,
editors, Parallel Problem Solving from Nature – PPSN XIII, volume
8672 of Lecture Notes in Computer Science, pp. 518–527. Springer,
Heidelberg, 2014.
[ bib ]

[1537]

Karl Bringmann, Tobias Friedrich, and Patrick Klitzke.
Twodimensional subset selection for hypervolume and
epsilonindicator.
In C. Igel and D. V. Arnold, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2014. ACM Press, New York,
NY, 2014.
[ bib 
DOI ]

[1538]

Andre Britto and Aurora Pozo.
Using archiving methods to control convergence and diversity for
manyobjective problems in particle swarm optimization.
In Proceedings of the 2012 Congress on Evolutionary Computation
(CEC 2012), pp. 1–8, Piscataway, NJ, 2012. IEEE Press.
[ bib 
DOI ]

[1539]

Karl Bringmann and Tobias Friedrich.
Don't be greedy when calculating hypervolume contributions.
In I. I. Garibay, T. Jansen, R. P. Wiegand, and A. S. Wu, editors,
Proceedings of the Tenth ACM SIGEVO Workshop on Foundations of Genetic
Algorithms (FOGA), pp. 103–112. ACM, 2009.
[ bib ]
Extended version published in [173]

[1540]

Karl Bringmann, Tobias Friedrich, Frank Neumann, and Markus Wagner.
Approximationguided Evolutionary Multiobjective Optimization.
In T. Walsh, editor, Proceedings of the 22nd International Joint
Conference on Artificial Intelligence (IJCAI11), pp. 1198–1203.
IJCAI/AAAI Press, Menlo Park, CA, 2011.
[ bib ]

[1541]

Dimo Brockhoff.
A Bug in the Multiobjective Optimizer IBEA: Salutary Lessons
for Code Release and a Performance ReAssessment.
In A. GasparCunha, C. H. Antunes, and C. A. Coello Coello,
editors, Evolutionary Multicriterion Optimization, EMO 2015 Part I,
volume 9018 of Lecture Notes in Computer Science, pp. 187–201.
Springer, Heidelberg, 2015.
[ bib 
DOI ]

[1542]

Dimo Brockhoff, Roberto Calandra, Manuel LópezIbáñez, Frank
Neumann, and Selvakumar Ulaganathan.
Metamodeling for (interactive) multiobjective optimization
(WG5).
In K. Klamroth, J. D. Knowles, G. Rudolph, and M. M. Wiecek, editors,
Personalized Multiobjective Optimization: An Analytics Perspective
(Dagstuhl Seminar 18031), volume 8(1) of Dagstuhl Reports, pp.
85–94. Schloss Dagstuhl–LeibnizZentrum für Informatik, Germany, 2018.
[ bib 
DOI ]
Keywords: multiple criteria decision making, evolutionary
multiobjective optimization

[1543]

Dimo Brockhoff, Tobias Friedrich, N. Hebbinghaus, C. Klein, Frank Neumann, and
Eckart Zitzler.
Do Additional Objectives Make a Problem Harder?
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pp. 765–772. ACM Press,
New York, NY, 2007.
[ bib 
DOI ]

[1544]

Dimo Brockhoff, Manuel LópezIbáñez, Boris Naujoks, and Günther
Rudolph.
Runtime Analysis of Simple Interactive Evolutionary Biobjective
Optimization Algorithms.
In C. A. Coello Coello et al., editors, Parallel Problem
Solving from Nature – PPSN XII, Part I, volume 7491 of Lecture
Notes in Computer Science, pp. 123–132. Springer, Heidelberg, 2012.
[ bib 
DOI ]
Development and deployment of interactive evolutionary
multiobjective optimization algorithms (EMOAs) have recently
gained broad interest. In this study, first steps towards a
theory of interactive EMOAs are made by deriving bounds on
the expected number of function evaluations and queries to a
decision maker. We analyze randomized local search and the
(1+1)EA on the biobjective problems LOTZ and COCZ under the
scenario that the decision maker interacts with these
algorithms by providing a subjective preference whenever
solutions are incomparable. It is assumed that this decision
is based on the decision maker's internal utility
function. We show that the performance of the interactive
EMOAs may dramatically worsen if the utility function is
nonlinear instead of linear.

[1545]

Dimo Brockhoff, Dhish Kumar Saxena, Kalyanmoy Deb, and Eckart Zitzler.
On Handling a Large Number of Objectives A Posteriori and During
Optimization.
In J. D. Knowles, D. Corne, K. Deb, and D. R. Chair, editors,
Multiobjective Problem Solving from Nature, Natural Computing Series, pp.
377–403. Springer, Berlin/Heidelberg, 2008.
[ bib 
DOI ]
Dimensionality reduction methods are used routinely in
statistics, pattern recognition, data mining, and machine
learning to cope with highdimensional spaces. Also in the
case of highdimensional multiobjective optimization
problems, a reduction of the objective space can be
beneficial both for search and decision making. New questions
arise in this context, e.g., how to select a subset of
objectives while preserving most of the problem structure. In
this chapter, two different approaches to the task of
objective reduction are developed, one based on assessing
explicit conflicts, the other based on principal component
analysis (PCA). Although both methods use different
principles and preserve different properties of the
underlying optimization problems, they can be effectively
utilized either in an a posteriori scenario or during
search. Here, we demonstrate the usability of the
conflictbased approach in a decisionmaking scenario after
the search and show how the principalcomponentbased
approach can be integrated into an evolutionary
multicriterion optimization (EMO) procedure.

[1546]

Dimo Brockhoff and Tea Tušar.
Benchmarking algorithms from the platypus framework on the
biobjective bbobbiobj testbed.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO Companion 2019, pp. 1905–1911. ACM Press, New York, NY,
2019.
[ bib 
DOI 
epub ]
Keywords: unbounded archive

[1547]

Dimo Brockhoff, T. Wagner, and Heike Trautmann.
On the properties of the R2 indicator.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pp. 465–472. ACM
Press, New York, NY, 2012.
[ bib ]
Proof that R2 is weakly Pareto compliant.

[1548]

Dimo Brockhoff and Eckart Zitzler.
Are All Objectives Necessary? On Dimensionality Reduction in
Evolutionary Multiobjective Optimization.
In T. P. Runarsson, H.G. Beyer, E. K. Burke, J.J. Merelo,
D. Whitley, and X. Yao, editors, Parallel Problem Solving from Nature –
PPSN IX, volume 4193 of Lecture Notes in Computer Science, pp.
533–542, Heidelberg, 2006. Springer.
[ bib ]
Most of the available multiobjective evolutionary algorithms
(MOEA) for approximating the Pareto set have been designed
for and tested on low dimensional problems (≤3
objectives). However, it is known that problems with a high
number of objectives cause additional difficulties in terms
of the quality of the Pareto set approximation and running
time. Furthermore, the decision making process becomes the
harder the more objectives are involved. In this context, the
question arises whether all objectives are necessary to
preserve the problem characteristics. One may also ask under
which conditions such an objective reduction is feasible, and
how a minimum set of objectives can be computed. In this
paper, we propose a general mathematical framework, suited to
answer these three questions, and corresponding algorithms,
exact and heuristic ones. The heuristic variants are geared
towards direct integration into the evolutionary search
process. Moreover, extensive experiments for four wellknown
test problems show that substantial dimensionality reductions
are possible on the basis of the proposed methodology.

[1549]

Dimo Brockhoff and Eckart Zitzler.
Dimensionality Reduction in Multiobjective Optimization: The
Minimum Objective Subset Problem.
In K.H. Waldmann and U. M. Stocker, editors, Operations
Research Proceedings 2006, pp. 423–429. Springer, Berlin/Heidelberg,
2007.
[ bib 
DOI ]
The number of objectives in a multiobjective optimization
problem strongly influences both the performance of
generating methods and the decision making process in
general. On the one hand, with more objectives, more
incomparable solutions can arise, the number of which affects
the generating method's performance. On the other hand, the
more objectives are involved the more complex is the choice
of an appropriate solution for a (human) decision maker. In
this context, the question arises whether all objectives are
actually necessary and whether some of the objectives may be
omitted; this question in turn is closely linked to the
fundamental issue of conflicting and nonconflicting
optimization criteria. Besides a general definition of
conflicts between objective sets, we here introduce the
NPhard problem of computing a minimum subset of objectives
without losing information (MOSS). Furthermore, we present
for MOSS both an approximation algorithm with optimum
approximation ratio and an exact algorithm which works well
for small input instances. We conclude with experimental
results for a random problem and the multiobjective
0/1knapsack problem
Keywords: objective reduction

[1550]

Dimo Brockhoff and Eckart Zitzler.
Improving hypervolumebased multiobjective evolutionary
algorithms by using objective reduction methods.
In Proceedings of the 2007 Congress on Evolutionary Computation
(CEC 2007), pp. 2086–2093, Piscataway, NJ, 2007. IEEE Press.
[ bib 
DOI ]
Keywords: objective reduction

[1551]

Artur Brum and Marcus Ritt.
Automatic Design of Heuristics for Minimizing the Makespan in
Permutation Flow Shops.
In Proceedings of the 2018 Congress on Evolutionary Computation
(CEC 2018), pp. 1–8, Piscataway, NJ, 2018. IEEE Press.
[ bib 
DOI ]

[1552]

Artur Brum and Marcus Ritt.
Automatic Algorithm Configuration for the Permutation Flow Shop
Scheduling Problem Minimizing Total Completion Time.
In A. Liefooghe and M. LópezIbáñez, editors,
Proceedings of EvoCOP 2018 – 18th European Conference on Evolutionary
Computation in Combinatorial Optimization, volume 10782 of Lecture
Notes in Computer Science, pp. 85–100, Heidelberg, 2018. Springer.
[ bib 
DOI ]

[1553]

T. N. Bui and J. R. Rizzo, Jr.
Finding Maximum Cliques with Distributed Ants.
In K. Deb et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2004, Part I, volume 3102 of
Lecture Notes in Computer Science, pp. 24–35. Springer, Heidelberg, 2004.
[ bib ]

[1554]

Edmund K. Burke and Yuri Bykov.
The Late Acceptance HillClimbing Heuristic.
Technical Report CSM192, University of Stirling, 2012.
[ bib ]

[1555]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, and John R. Woodward.
Automatic Heuristic Generation with Genetic Programming:
Evolving a Jackofalltrades or a Master of One.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pp. 1559–1565, New York,
NY, 2007. ACM Press.
[ bib 
DOI ]

[1556]

Edmund K. Burke, Matthew R. Hyde, Graham Kendall, Gabriela Ochoa, Ender
Özcan, and John R. Woodward.
A Classification of HyperHeuristic Approaches: Revisited.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 272 of International Series in Operations
Research & Management Science, chapter 14, pp. 453–477. Springer, 2019.
[ bib 
DOI ]

[1557]

Rainer E. Burkard, Eranda Çela, Panos M. Pardalos, and L. S. Pitsoulis.
The quadratic assignment problem.
In P. M. Pardalos and D.Z. Du, editors, Handbook of
Combinatorial Optimization, volume 2, pp. 241–338. Kluwer Academic
Publishers, 1998.
[ bib ]

[1558]

Maxim Buzdalov.
Towards better estimation of statistical significance when
comparing evolutionary algorithms.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO Companion 2019, pp. 1782–1788. ACM Press, New York, NY,
2019.
[ bib 
DOI 
epub ]

[1559]

Nicola Beume, Carlos M. Fonseca, Manuel LópezIbáñez, Luís
Paquete, and Jan Vahrenhold.
On the Complexity of Computing the Hypervolume Indicator.
Technical Report CI235/07, University of Dortmund, December 2007.
Published in IEEE Transactions on Evolutionary
Computation [116].
[ bib ]

[1560]

COnfiguration and SElection of ALgorithms.
http://www.coseal.net, 2017.
[ bib ]

[1561]

IBM.
ILOG CPLEX Optimizer.
http://www.ibm.com/software/integration/optimization/cplexoptimizer/,
2017.
[ bib ]

[1562]

Christian Leonardo CamachoVillalón, Marco Dorigo, and Thomas Stützle.
Why the Intelligent Water Drops Cannot Be Considered as a Novel
Algorithm.
In M. Dorigo, M. Birattari, A. L. Christensen, A. Reina, and
V. Trianni, editors, Swarm Intelligence, 11th International Conference,
ANTS 2018, volume 11172 of Lecture Notes in Computer Science, pp.
302–314, Heidelberg, 2018. Springer.
[ bib ]

[1563]

Paolo Campigotto and Andrea Passerini.
Adapting to a realistic decision maker: experiments towards a
reactive multiobjective optimizer.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pp. 338–341. Springer, Heidelberg,
2010.
[ bib 
DOI ]

[1564]

Christian Leonardo CamachoVillalón, Thomas Stützle, and Marco Dorigo.
Grey Wolf, Firefly and Bat Algorithms: Three Widespread
Algorithms that Do Not Contain Any Novelty.
In M. Dorigo, T. Stützle, M. J. Blesa, C. Blum, H. Hamann, and
M. K. Heinrich, editors, Swarm Intelligence, 12th International
Conference, ANTS 2020, volume 12421 of Lecture Notes in Computer
Science, pp. 121–133. Springer, Heidelberg, 2020.
[ bib ]

[1565]

Felipe Campelo, Áthila R. Trindade, and Manuel LópezIbáñez.
Pseudoreplication in Racing Methods for Tuning Metaheuristics.
In preparation, 2017.
[ bib ]

[1566]

E. CantúPaz.
Efficient and Accurate Parallel Genetic Algorithms.
Kluwer Academic Publishers, Boston, MA, 2000.
[ bib ]

[1567]

P. Cardoso, M. Jesus, and A. Marquez.
MONACO: multiobjective network optimisation based on an
ACO.
In Proc. X Encuentros de Geometría Computacional, Seville,
Spain, 2003.
[ bib ]

[1568]

Alex Guimarães Cardoso de Sá, Walter José G. S. Pinto, Luiz
Otávio Vilas Boas Oliveira, and Gisele Pappa.
RECIPE: A GrammarBased Framework for Automatically Evolving
Classification Pipelines.
In J. McDermott, M. Castelli, L. Sekanina, E. Haasdijk, and
P. GarcíaSánchez, editors, Proceedings of the 20th European
Conference on Genetic Programming, EuroGP 2017, volume 10196 of Lecture
Notes in Computer Science, pp. 246–261. Springer, Heidelberg, 2017.
[ bib 
DOI ]

[1569]

Ioannis Caragiannis, Ariel D. Procaccia, and Nisarg Shah.
When Do Noisy Votes Reveal the Truth?
In M. J. Kearns, R. P. McAfee, and É. Tardos, editors,
Proceedings of the Fourteenth ACM Conference on Electronic Commerce, pp.
143–160, New York, NY, 2013. ACM Press.
[ bib 
DOI ]
A wellstudied approach to the design of voting rules views
them as maximum likelihood estimators; given votes that are
seen as noisy estimates of a true ranking of the
alternatives, the rule must reconstruct the most likely true
ranking. We argue that this is too stringent a requirement,
and instead ask: How many votes does a voting rule need to
reconstruct the true ranking? We define the family of
pairwisemajority consistent rules, and show that for all
rules in this family the number of samples required from the
Mallows noise model is logarithmic in the number of
alternatives, and that no rule can do asymptotically better
(while some rules like plurality do much worse). Taking a
more normative point of view, we consider voting rules that
surely return the true ranking as the number of samples tends
to infinity (we call this property accuracy in the limit);
this allows us to move to a higher level of abstraction. We
study families of noise models that are parametrized by
distance functions, and find voting rules that are accurate
in the limit for all noise models in such general
families. We characterize the distance functions that induce
noise models for which pairwisemajority consistent rules are
accurate in the limit, and provide a similar result for
another novel family of positiondominance consistent
rules. These characterizations capture three wellknown
distance functions.
Keywords: computer social choice, mallows model, sample complexity

[1570]

Josu Ceberio, Alexander Mendiburu, and José A. Lozano.
Kernels of Mallows Models for Solving Permutationbased
Problems.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pp. 505–512. ACM Press, New York, NY, 2015.
[ bib ]

[1571]

Eranda Çela.
The Quadratic Assignment Problem: Theory and Algorithms.
Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998.
[ bib ]

[1572]

Amadeo Cesta, Angelo Oddi, and Stephen F. Smith.
Iterative Flattening: A Scalable Method for Solving
MultiCapacity Scheduling Problems.
In H. A. Kautz and B. W. Porter, editors, Proceedings of AAAI
2000 – Seventeenth National Conference on Artificial Intelligence, pp.
742–747. AAAI Press/MIT Press, Menlo Park, CA, 2000.
[ bib ]

[1573]

S. T. H. Chang.
Optimizing the Real Time Operation of a Pumping Station at a
Water Filtration Plant using Genetic Algorithms.
Honors thesis, Department of Civil and Environmental Engineering, The
University of Adelaide, 1999.
[ bib ]

[1574]

Donald V. Chase and Lindell E. Ormsbee.
Optimal pump operation of water distribution systems with
multiple storage tanks.
In Proceedings of American Water Works Association Computer
Specialty Conference, pp. 205–214, Denver, USA, 1989. AWWA.
[ bib ]

[1575]

Donald V. Chase and Lindell E. Ormsbee.
An alternate formulation of time as a decision variable to
facilitate realtime operation of water supply systems.
In Proceedings of the 18th Annual Conference of Water Resources
Planning and Management, pp. 923–927, New York, NY, 1991. ASCE.
[ bib ]

[1576]

Fei Chen, Yang Gao, Zhaoqian Chen, and Shifu Chen.
SCGA: Controlling genetic algorithms with Sarsa(0).
In Computational Intelligence for Modelling, Control and
Automation, 2005 and International Conference on Intelligent Agents, Web
Technologies and Internet Commerce, International Conference on, volume 1,
pp. 1177–1183. IEEE, 2005.
[ bib 
DOI ]

[1577]

Clément Chevalier, David Ginsbourger, Julien Bect, and Ilya Molchanov.
Estimating and Quantifying Uncertainties on Level Sets Using the
Vorob'ev Expectation and Deviation with Gaussian Process Models.
In D. Ucinski, A. C. Atkinson, and M. Patan, editors, mODa
10–Advances in ModelOriented Design and Analysis, pp. 35–43. Springer
International Publishing, Heidelberg, 2013.
[ bib 
DOI ]
Several methods based on Kriging have recently been proposed
for calculating a probability of failure involving
costlytoevaluate functions. A closely related problem is to
estimate the set of inputs leading to a response exceeding a
given threshold. Now, estimating such a level set—and not
solely its volume—and quantifying uncertainties on it are
not straightforward. Here we use notions from random set
theory to obtain an estimate of the level set, together with
a quantification of estimation uncertainty. We give explicit
formulae in the Gaussian process setup and provide a
consistency result. We then illustrate how spacefilling
versus adaptive design strategies may sequentially reduce
level set estimation uncertainty.

[1578]

Weiyu Chen, Hisao Ishibuchi, and Ke Shang.
ClusteringBased Subset Selection in Evolutionary Multiobjective
Optimization.
In 2021 IEEE International Conference on Systems, Man, and
Cybernetics, pp. 468–475. IEEE, 2021.
[ bib ]

[1579]

Peter C. Cheeseman, Bob Kanefsky, and William M. Taylor.
Where the Really Hard Problems Are.
In J. Mylopoulos and R. Reiter, editors, Proceedings of the 12th
International Joint Conference on Artificial Intelligence (IJCAI91), pp.
331–340. Morgan Kaufmann Publishers, 1995.
[ bib ]

[1580]

Lu Chen, Bin Xin, Jie Chen, and Juan Li.
A virtualdecisionmaker library considering personalities and
dynamically changing preference structures for interactive multiobjective
optimization.
In Proceedings of the 2017 Congress on Evolutionary Computation
(CEC 2017), pp. 636–641, Piscataway, NJ, 2017. IEEE Press.
[ bib 
DOI ]
Keywords: machine DM, interactive EMOA

[1581]

L. Chen, X. H. Xu, and Y. X. Chen.
An adaptive ant colony clustering algorithm.
In I. Cloete, K.P. Wong, and M. Berthold, editors, Proceedings
of the International Conference on Machine Learning and Cybernetics, pp.
1387–1392. IEEE Press, 2004.
[ bib ]

[1582]

Weiyu Chen, Hisao Ishibuchi, and Ke Shang.
Modified Distancebased Subset Selection for Evolutionary
Multiobjective Optimization Algorithms.
In Proceedings of the 2020 Congress on Evolutionary Computation
(CEC 2020), pp. 1–8, Piscataway, NJ, 2020. IEEE Press.
[ bib ]
Keywords: IGD+

[1583]

Marco Chiarandini and Yuri Goegebeur.
Mixed Models for the Analysis of Optimization Algorithms.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pp. 225–264. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Preliminary version available as Tech. Rep. MF200907001 at the The Danish Mathematical Society

[1584]

Marco Chiarandini.
Stochastic Local Search Methods for Highly Constrained
Combinatorial Optimisation Problems.
PhD thesis, FB Informatik, TU Darmstadt, Germany, 2005.
[ bib ]

[1585]

TsungChe Chiang.
nsga3cpp: A C++ implementation of NSGAIII.
http://web.ntnu.edu.tw/~tcchiang/publications/nsga3cpp/nsga3cpp.htm,
2014.
[ bib ]

[1586]

Matthias Christen, Olaf Schenk, and Helmar Burkhart.
PATUS: A Code Generation and Autotuning Framework for Parallel
Iterative Stencil Computations on Modern Microarchitectures.
In F. Mueller, editor, Proceedings of the 2011 IEEE
International Parallel & Distributed Processing Symposium, IPDPS '11, pp.
676–687. IEEE Computer Society, 2011.
[ bib 
DOI ]

[1587]

Jan Christiaens and Greet Vanden Berghe.
Slack Induction by String Removals for Vehicle Routing
Problems.
Technical Report 7052018, Department of Computing Science, KU
Leuven, Gent, Belgium, 2018.
[ bib ]

[1588]

Nicos Christofides.
Worstcase analysis of a new heuristic for the travelling
salesman problem.
Technical Report 388, Graduate School of Industrial Administration,
CarnegieMellon University, Pittsburgh, PA, 1976.
[ bib ]

[1589]

Tinkle Chugh and Manuel LópezIbáñez.
Maximising Hypervolume and Minimising εIndicators
using Bayesian Optimisation over Sets.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO Companion 2021, pp.
1326–1334. ACM Press, New York, NY, 2021.
[ bib 
DOI 
supplementary material ]
Keywords: multiobjective, surrogate models, epsilon, hypervolume

[1590]

S. Chusanapiputt, D. Nualhong, S. Jantarang, and S. Phoomvuthisarn.
Selective selfadaptive approach to ant system for solving unit
commitment problem.
In M. Cattolico et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2006, pp. 1729–1736. ACM Press,
New York, NY, 2006.
[ bib ]

[1591]

Tinkle Chugh.
Handling expensive multiobjective optimization problems with
evolutionary algorithms.
PhD thesis, University of Jyväskylä, 2017.
[ bib ]

[1592]

Tinkle Chugh.
Scalarizing Functions in Bayesian Multiobjective Optimization.
In Proceedings of the 2020 Congress on Evolutionary Computation
(CEC 2020), pp. 1–8, Piscataway, NJ, 2020. IEEE Press.
[ bib 
DOI ]

[1593]

Christian Cintrano, Javier Ferrer, Manuel LópezIbáñez, and Enrique
Alba.
Hybridization of Racing Methods with Evolutionary Operators for
Simulation Optimization of Traffic Lights Programs.
In C. Zarges and S. Verel, editors, Proceedings of EvoCOP 2021
– 21th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 12692 of Lecture Notes in Computer Science, pp.
17–33. Springer, Cham, Switzerland, 2021.
[ bib 
DOI ]
In many realworld optimization problems, like the traffic
light scheduling problem tackled here, the evaluation of
candidate solutions requires the simulation of a process
under various scenarios. Thus, good solutions should not only
achieve good objective function values, but they must be
robust (low variance) across all different scenarios.
Previous work has revealed the effectiveness of IRACE for
this task. However, the operators used by IRACE to generate
new solutions were designed for configuring algorithmic
parameters, that have various data types (categorical,
numerical, etc.). Meanwhile, evolutionary algorithms have
powerful operators for numerical optimization, which could
help to sample new solutions from the best ones found in the
search. Therefore, in this work, we propose a hybridization
of the elitist iterated racing mechanism of IRACE with
evolutionary operators from differential evo lution and
genetic algorithms. We consider a realistic case study
derived from the traffic network of Malaga (Spain) with 275
traffic lights that should be scheduled optimally. After a
meticulous study, we discovered that the hybrid algorithm
comprising IRACE plus differential evolution offers
statistically better results than conventional algorithms and
also improves travel times and reduces pollution.
Extended version published in Evolutionary Computation journal [246].
Keywords: Hybrid algorithms, Evolutionary algorithms, Simulation
optimization, Uncertainty, Traffic light planning

[1594]

Jill Cirasella, David S. Johnson, Lyle A. McGeoch, and Weixiong Zhang.
The Asymmetric Traveling Salesman Problem: Algorithms, Instance
Generators, and Tests.
In A. L. Buchsbaum and J. Snoeyink, editors, Algorithm
Engineering and Experimentation, Third International Workshop, ALENEX 2001,
Washington, DC, USA, January 56, 2001, Revised Papers, volume 2153 of
Lecture Notes in Computer Science, pp. 32–59, Berlin, Germany, 2001.
Springer.
[ bib 
DOI ]

[1595]

Jon Claerbout and Martin Karrenbach.
Electronic documents give reproducible research a new meaning.
In SEG Technical Program Expanded Abstracts 1992, pp.
601–604. Society of Exploration Geophysicists, 1992.
[ bib 
DOI ]
Proposed a reproducibility taxonomy, defined reproducibility
and taxonomy

[1596]

Maurice Clerc and J. Kennedy.
Standard PSO 2011.
Particle Swarm Central, 2011.
[ bib 
http ]

[1597]

Maurice Clerc.
Standard Particle Swarm Optimisation.
hal00764996, September 2012.
[ bib 
http ]
Since 2006, three successive standard PSO versions have been
put on line on the Particle Swarm Central
(http://particleswarm.info), namely SPSO 2006, 2007,
and 2011. The basic principles of all three versions can be
informally described the same way, and in general, this
statement holds for almost all PSO variants. However, the
exact formulae are slightly different, because they took
advantage of latest theoretical analysis available at the
time they were designed.
Keywords: particle swarm optimisation

[1598]

Carlos A. Coello Coello.
Multiobjective Evolutionary Algorithms in RealWorld
Applications: Some Recent Results and Current Challenges.
In Advances in Evolutionary and Deterministic Methods for
Design, Optimization and Control in Engineering and Sciences, pp. 3–18.
Springer, 2015.
[ bib 
DOI ]

[1599]

Carlos A. Coello Coello, Gary B. Lamont, and David A. Van Veldhuizen.
Evolutionary Algorithms for Solving MultiObjective Problems.
Springer, New York, NY, 2nd edition, 2007.
[ bib 
DOI ]

[1600]

Carlos A. Coello Coello and Margarita ReyesSierra.
A Study of the Parallelization of a Coevolutionary
Multiobjective Evolutionary Algorithm.
In R. Monroy, G. ArroyoFigueroa, L. E. Sucar, and H. Sossa, editors,
Proceedings of MICAI, volume 2972 of Lecture Notes in Artificial
Intelligence, pp. 688–697. Springer, Heidelberg, 2004.
[ bib ]
Introduces Inverted Generational Distance (IGD)
Keywords: IGD

[1601]

Carlos A. Coello Coello.
Handling Preferences in Evolutionary Multiobjective
Optimization: A Survey.
In Proceedings of the 2000 Congress on Evolutionary Computation
(CEC'00), pp. 30–37. IEEE Press, Piscataway, NJ, July 2000.
[ bib ]

[1602]

Carlos A. Coello Coello.
Recent Results and Open Problems in Evolutionary Multiobjective
Optimization.
In C. MartínVide, R. Neruda, and M. A. VegaRodríguez,
editors, Theory and Practice of Natural Computing  6th International
Conference, TPNC 2017, volume 10687 of Lecture Notes in Computer
Science, pp. 3–21. Springer International Publishing, Cham, Switzerland,
2017.
[ bib ]

[1603]

Paul R. Cohen.
Empirical Methods for Artificial Intelligence.
MIT Press, Cambridge, MA, 1995.
[ bib ]

[1604]

G. Cohen.
Optimal Control of Water Supply Networks.
In S. G. Tzafestas, editor, Optimization and Control of Dynamic
Operational Research Models, volume 4, chapter 8, pp. 251–276.
NorthHolland Publishing Company, Amsterdam, 1982.
[ bib ]

[1605]

Alberto Colorni, Marco Dorigo, and Vittorio Maniezzo.
Distributed Optimization by Ant Colonies.
In F. J. Varela and P. Bourgine, editors, Proceedings of the
First European Conference on Artificial Life, pp. 134–142. MIT Press,
Cambridge, MA, 1992.
[ bib ]

[1606]

Sonia Colas, Nicolas Monmarché, Pierre Gaucher, and Mohamed Slimane.
Artificial Ants for the Optimization of Virtual Keyboard
Arrangement for Disabled People.
In N. Monmarché, E.G. Talbi, P. Collet, M. Schoenauer, and
E. Lutton, editors, Artificial Evolution, volume 4926 of Lecture
Notes in Computer Science, pp. 87–99. Springer, Heidelberg, 2008.
[ bib 
DOI ]

[1607]

Andrew R. Conn, Katya Scheinberg, and Luis N. Vicente.
Introduction to DerivativeFree Optimization.
MPS–SIAM Series on Optimization. Society for Industrial and Applied
Mathematics, Philadelphia, PA, USA, 2009.
[ bib ]

[1608]

David Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook.
Concorde TSP Solver.
http://www.math.uwaterloo.ca/tsp/concorde.html, 2014.
Version visited last on 15 April 2014.
[ bib ]

[1609]

W. J. Conover.
Practical Nonparametric Statistics.
John Wiley & Sons, New York, NY, 3rd edition, 1999.
[ bib ]

[1610]

Stephen A. Cook.
The Complexity of Theoremproving Procedures.
In Proceedings of the Third Annual ACM Symposium on Theory of
Computing, STOC '71, pp. 151–158. ACM, 1971.
[ bib 
DOI ]

[1611]

William J. Cook.
In Pursuit of the Traveling Salesman.
Princeton University Press, Princeton, NJ, 2012.
[ bib ]

[1612]

William J. Cook.
Computing in Combinatorial Optimization.
In B. Steffen and G. Woeginger, editors, Computing and Software
Science: State of the Art and Perspectives, volume 10000 of Lecture
Notes in Computer Science, pp. 27–47. Springer, Cham, Switzerland, 2019.
[ bib 
DOI ]

[1613]

David Corne, Nick R. Jerram, Joshua D. Knowles, and Martin J. Oates.
PESAII: RegionBased Selection in Evolutionary Multiobjective
Optimization.
In E. D. Goodman, editor, Proceedings of the 3rd Annual
Conference on Genetic and Evolutionary Computation, GECCO 2001, pp.
283–290. Morgan Kaufmann Publishers, San Francisco, CA, 2001.
[ bib 
DOI ]

[1614]

David Corne and Joshua D. Knowles.
Some Multiobjective Optimizers are Better than Others.
In Proceedings of the 2003 Congress on Evolutionary Computation
(CEC'03), pp. 2506–2512. IEEE Press, Piscataway, NJ, December 2003.
[ bib ]

[1615]

David Corne and Joshua D. Knowles.
No free lunch and free leftovers theorems for multiobjective
optimisation problems.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pp. 327–341. Springer,
Heidelberg, 2003.
[ bib 
DOI ]

[1616]

David Corne, Joshua D. Knowles, and M. J. Oates.
The Pareto EnvelopeBased Selection Algorithm for
Multiobjective Optimization.
In M. Schoenauer et al., editors, Parallel Problem Solving from
Nature – PPSN VI, volume 1917 of Lecture Notes in Computer
Science, pp. 839–848. Springer, Heidelberg, 2000.
[ bib ]

[1617]

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein.
Introduction to algorithms.
MIT Press, Cambridge, MA, 2009.
[ bib ]

[1618]

David Corne and Alan Reynolds.
Evaluating optimization algorithms: bounds on the performance of
optimizers on unseen problems.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 707–710,
New York, NY, 2011. ACM Press.
[ bib 
DOI 
supplementary material ]

[1619]

Oscar Cordón, I. Fernández de Viana, Francisco Herrera, and L. Moreno.
A New ACO Model Integrating Evolutionary Computation Concepts:
The BestWorst Ant System.
In M. Dorigo et al., editors, Abstract proceedings of ANTS 2000
– From Ant Colonies to Artificial Ants: Second International Workshop on Ant
Algorithms, pp. 22–29. IRIDIA, Université Libre de Bruxelles, Belgium,
September, 7–9 2000.
[ bib ]

[1620]

Peter I. Cowling, Graham Kendall, and Eric Soubeiga.
A Hyperheuristic Approach to Scheduling a Sales Summit.
In E. K. Burke and W. Erben, editors, PATAT 2000: Proceedings of
the 3rd International Conference of the Practice and Theory of Automated
Timetabling, volume 2079 of Lecture Notes in Computer Science, pp.
176–190. Springer, 2000.
[ bib 
DOI ]
First mention of the term hyperheuristic.

[1621]

M. J. Crawley.
The R Book.
Wiley, 2nd edition, 2012.
[ bib ]

[1622]

W. B. Crowston, F. Glover, G. L. Thompson, and J. D. Trawick.
Probabilistic and Parametric Learning Combinations of Local Job
Shop Scheduling Rules.
ONR Research Memorandum No. 117, GSIA, CarnegieMellon University,
Pittsburgh, PA, USA, 1963.
[ bib ]

[1623]

Joseph C. Culberson.
Iterated Greedy Graph Coloring and the Difficulty Landscape.
Technical Report 9207, Department of Computing Science, The
University of Alberta, Edmonton, Alberta, Canada, 1992.
[ bib ]

[1624]

Joseph C. Culberson, A. Beacham, and D. Papp.
Hiding our Colors.
In Proceedings of the CP'95 Workshop on Studying and Solving
Really Hard Problems, pp. 31–42, Cassis, France, September 1995.
[ bib ]

[1625]

Joseph C. Culberson and F. Luo.
Exploring the kcolorable Landscape with Iterated Greedy.
In D. S. Johnson and M. A. Trick, editors, Cliques, Coloring,
and Satisfiability: Second DIMACS Implementation Challenge, volume 26 of
DIMACS Series on Discrete Mathematics and Theoretical Computer
Science, pp. 245–284. American Mathematical Society, Providence, RI, 1996.
[ bib ]

[1626]

Jeff Cumming.
Understanding the New Statistics – Effect Sizes, Confidence
Intervals, and Metaanalysis.
Taylor & Francis, 2012.
[ bib ]

[1627]

Nguyen Dang Thi Thanh and Patrick De Causmaecker.
Motivations for the Development of a Multiobjective Algorithm
Configurator.
In B. Vitoriano, E. Pinson, and F. Valente, editors, ICORES
2014  Proceedings of the 3rd International Conference on Operations Research
and Enterprise Systems, pp. 328–333. SciTePress, 2014.
[ bib ]

[1628]

Nguyen Dang Thi Thanh and Patrick De Causmaecker.
Characterization of Neighborhood Behaviours in a
Multineighborhood Local Search Algorithm.
In P. Festa, M. Sellmann, and J. Vanschoren, editors, Learning
and Intelligent Optimization, 10th International Conference, LION 10, volume
10079 of Lecture Notes in Computer Science, pp. 234–239. Springer,
Cham, Switzerland, 2016.
[ bib ]

[1629]

Nguyen Dang and Carola Doerr.
Hyperparameter tuning for the (1 + (λ,
λ)) GA.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pp. 889–897. ACM Press, New York, NY, 2019.
[ bib 
DOI 
epub ]
Keywords: irace; theory

[1630]

Nguyen Dang Thi Thanh, Leslie Pérez Cáceres, Patrick De
Causmaecker, and Thomas Stützle.
Configuring irace Using Surrogate Configuration
Benchmarks.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pp. 243–250. ACM Press,
New York, NY, 2017.
[ bib 
DOI ]
Keywords: irace

[1631]

Augusto Lopez Dantas and Aurora Trinidad Ramirez Pozo.
A MetaLearning Algorithm Selection Approach for the Quadratic
Assignment Problem.
In Proceedings of the 2018 Congress on Evolutionary Computation
(CEC 2018), pp. 1–8, Piscataway, NJ, 2018. IEEE Press.
[ bib ]

[1632]

Graeme C. Dandy and Matthew S. Gibbs.
Optimizing System Operations and Water Quality.
In P. Bizier and P. DeBarry, editors, Proceedings of World Water
and Environmental Resources Congress. ASCE, Philadelphia, USA, 2003.
on CDROM.
[ bib 
DOI ]

[1633]

Nguyen Dang Thi Thanh.
Data analytics for algorithm design.
PhD thesis, KU Leuven, Belgium, 2018.
[ bib ]
Supervised by Patrick De Causmaecker

[1634]

Fabio Daolio, Sébastien Verel, Gabriela Ochoa, and Marco Tomassini.
Local Optima Networks and the Performance of Iterated Local
Search.
In T. Soule and J. H. Moore, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2012, pp. 369–376. ACM
Press, New York, NY, 2012.
[ bib ]

[1635]

Samuel Daulton, Maximilian Balandat, and Eytan Bakshy.
Differentiable Expected Hypervolume Improvement for Parallel
MultiObjective Bayesian Optimization.
In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin,
editors, Advances in Neural Information Processing Systems (NeurIPS
33), pp. 9851–9864, 2020.
[ bib 
epub ]

[1636]

Werner de Schaetzen, Dragan A. Savic, and Godfrey A. Walters.
A genetic algorithm approach to pump scheduling in water
supply.
In V. Babovic and L. C. Larsen, editors, Hydroinformatics '98,
pp. 897–899, Rotterdam, Balkema, 1998.
[ bib ]

[1637]

Thomas Dean and Mark S. Boddy.
An Analysis of TimeDependent Planning.
In H. E. Shrobe, T. M. Mitchell, and R. G. Smith, editors,
Proceedings of the 7th National Conference on Artificial Intelligence,
AAAI88, pp. 49–54. AAAI Press/MIT Press, Menlo Park, CA, 1988.
[ bib 
http ]
Keywords: anytime, performance profiles

[1638]

Angela Dean and Daniel Voss.
Design and Analysis of Experiments.
Springer, London, UK, 1999.
[ bib 
DOI ]

[1639]

Kalyanmoy Deb.
Introduction to evolutionary multiobjective optimization.
In J. Branke, K. Deb, K. Miettinen, and R. Slowiński, editors,
Multiobjective Optimization: Interactive and Evolutionary Approaches,
volume 5252 of Lecture Notes in Computer Science, pp. 59–96.
Springer, Heidelberg, 2008.
[ bib 
DOI ]
In its current state, evolutionary multiobjective
optimization (EMO) is an established field of research and
application with more than 150 PhD theses, more than ten
dedicated texts and edited books, commercial softwares and
numerous freely downloadable codes, a biannual conference
series running successfully since 2001, special sessions and
workshops held at all major evolutionary computing
conferences, and fulltime researchers from universities and
industries from all around the globe. In this chapter, we
provide a brief introduction to EMO principles, illustrate
some EMO algorithms with simulated results, and outline the
current research and application potential of EMO. For
solving multiobjective optimization problems, EMO procedures
attempt to find a set of welldistributed Paretooptimal
points, so that an idea of the extent and shape of the
Paretooptimal front can be obtained. Although this task was
the early motivation of EMO research, EMO principles are now
being found to be useful in various other problem solving
tasks, enabling one to treat problems naturally as they
are. One of the major current research thrusts is to combine
EMO procedures with other multiple criterion decision making
(MCDM) tools so as to develop hybrid and interactive
multiobjective optimization algorithms for finding a set of
tradeoff optimal solutions and then choose a preferred
solution for implementation. This chapter provides the
background of EMO principles and their potential to launch
such collaborative studies with MCDM researchers in the
coming years.

[1640]

Kalyanmoy Deb.
Multiobjective optimization.
In E. K. Burke and G. Kendall, editors, Search Methodologies,
pp. 273–316. Springer, Boston, MA, 2005.
[ bib 
DOI ]

[1641]

Kalyanmoy Deb.
MultiObjective Optimization Using Evolutionary Algorithms.
Wiley, Chichester, UK, 2001.
[ bib ]

[1642]

Kalyanmoy Deb and S. Agrawal.
A NichedPenalty Approach for Constraint Handling in Genetic
Algorithms.
In A. Dobnikar, N. C. Steele, D. W. Pearson, and R. F. Albrecht,
editors, Artificial Neural Nets and Genetic Algorithms (ICANNGA99),
pp. 235–243. Springer Verlag, 1999.
[ bib 
DOI ]
Keywords: polynomial mutation

[1643]

Kalyanmoy Deb, S. Agarwal, A. Pratap, and T. Meyarivan.
A fast elitist nondominated sorting genetic algorithm for
multiobjective optimization: NSGAII.
In M. Schoenauer et al., editors, Parallel Problem Solving from
Nature – PPSN VI, volume 1917 of Lecture Notes in Computer
Science, pp. 849–858. Springer, Heidelberg, 2000.
[ bib ]

[1644]

Kalyanmoy Deb and Sachin Jain.
MultiSpeed Gearbox Design Using MultiObjective Evolutionary
Algorithms.
Technical Report 2002001, KanGAL, February 2002.
[ bib ]

[1645]

Kalyanmoy Deb and Christie Myburgh.
Breaking the billionvariable barrier in realworld optimization
using a customized evolutionary algorithm.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2016, pp. 653–660. ACM Press, New York, NY, 2016.
[ bib ]

[1646]

Kalyanmoy Deb and Ankur Sinha.
Solving Bilevel MultiObjective Optimization Problems Using
Evolutionary Algorithms.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pp. 110–124.
Springer, Heidelberg, 2009.
[ bib ]

[1647]

Kalyanmoy Deb and J. Sundar.
Reference point based multiobjective optimization using
evolutionary algorithms.
In M. Cattolico et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2006, pp. 635–642. ACM Press,
New York, NY, 2006.
[ bib 
DOI ]
Proposed RNSGAII

[1648]

Kalyanmoy Deb, Rahul Tewari, Mayur Dixit, and Joydeep Dutta.
Finding tradeoff solutions close to KKT points using
evolutionary multiobjective optimization.
In Proceedings of the 2007 Congress on Evolutionary Computation
(CEC 2007), pp. 2109–2116. IEEE Press, Piscataway, NJ, 2007.
[ bib ]

[1649]

Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler.
Scalable Test Problems for Evolutionary MultiObjective
Optimization.
Technical Report 112, Computer Engineering and Networks Laboratory
(TIK), Swiss Federal Institute of Technology (ETH), Zürich, Switzerland,
2001.
Do not cite this TR! It is incorrect and it is superseeded
by [1650].
[ bib ]
Keywords: DTLZ benchmark

[1650]

Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler.
Scalable Test Problems for Evolutionary Multiobjective
Optimization.
In A. Abraham, L. Jain, and R. Goldberg, editors, Evolutionary
Multiobjective Optimization, Advanced Information and Knowledge Processing,
pp. 105–145. Springer, London, UK, January 2005.
[ bib 
DOI ]
Keywords: DTLZ benchmark

[1651]

William A. Dees, Jr. and Patrick G. Karger.
Automated Ripup and Reroute Techniques.
In DAC'82, Proceedings of the 19th Design Automation Workshop,
pp. 432–439. IEEE Press, 1982.
[ bib ]

[1652]

Matthijs L. den Besten.
Simple Metaheuristics for Scheduling.
PhD thesis, FB Informatik, TU Darmstadt, Germany, 2004.
[ bib 
http ]

[1653]

Roman Denysiuk, Lino Costa, and Isabel Espírito Santo.
Manyobjective optimization using differential evolution with
variablewise mutation restriction.
In C. Blum and E. Alba, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2013, pp. 591–598. ACM Press,
New York, NY, 2013.
[ bib ]

[1654]

Jia Deng, Wei Dong, Richard Socher, LiJia Li, Kai Li, and Li FeiFei.
Imagenet: A largescale hierarchical image database.
In Computer Vision and Pattern Recognition, 2009. CVPR 2009.
IEEE Conference on, pp. 248–255. IEEE, 2009.
[ bib ]

[1655]

Marcelo De Souza and Marcus Ritt.
An Automatically Designed Recombination Heuristic for the
TestAssignment Problem.
In Proceedings of the 2018 Congress on Evolutionary Computation
(CEC 2018), pp. 1–8, Piscataway, NJ, 2018. IEEE Press.
[ bib 
DOI ]

[1656]

Marcelo De Souza and Marcus Ritt.
Automatic GrammarBased Design of Heuristic Algorithms for
Unconstrained Binary Quadratic Programming.
In A. Liefooghe and M. LópezIbáñez, editors,
Proceedings of EvoCOP 2018 – 18th European Conference on Evolutionary
Computation in Combinatorial Optimization, volume 10782 of Lecture
Notes in Computer Science, pp. 67–84. Springer, Heidelberg, 2018.
[ bib 
DOI ]

[1657]

Marcelo De Souza and Marcus Ritt.
Hybrid Heuristic for Unconstrained Binary Quadratic Programming
– Source Code of HHBQP.
https://github.com/souzamarcelo/hhbqp, 2018.
[ bib ]

[1658]

Marcelo De Souza, Marcus Ritt, Manuel LópezIbáñez, and Leslie
Pérez Cáceres.
ACVIZ: A Tool for the Visual Analysis of the
Configuration of Algorithms with irace – Source Code.
https://github.com/souzamarcelo/acviz, 2020.
[ bib ]

[1659]

Marcelo De Souza, Marcus Ritt, Manuel LópezIbáñez, and Leslie
Pérez Cáceres.
ACVIZ: Algorithm Configuration
Visualizations for irace: Supplementary material.
http://doi.org/10.5281/zenodo.4714582, September 2020.
[ bib ]

[1660]

Sophie Dewez.
On the toll setting problem.
PhD thesis, Faculté de Sciences, Université Libre de
Bruxelles, 2014.
[ bib ]
Supervised by Dr. Martine Labbé

[1661]

Ilias Diakonikolas and Mihalis Yannakakis.
Succinct approximate convex Pareto curves.
In Proceedings of the nineteenth annual ACMSIAM symposium on
Discrete algorithms, pp. 74–83. Society for Industrial and Applied
Mathematics, 2008.
[ bib ]

[1662]

Diego Díaz, Pablo Valledor, Paula Areces, Jorge Rodil, and Montserrat
Suárez.
An ACO Algorithm to Solve an Extended Cutting Stock Problem
for Scrap Minimization in a Bar Mill.
In M. Dorigo et al., editors, Swarm Intelligence, 9th
International Conference, ANTS 2014, volume 8667 of Lecture Notes in
Computer Science, pp. 13–24. Springer, Heidelberg, 2014.
[ bib ]

[1663]

Luca Di Gaspero, Marco Chiarandini, and Andrea Schaerf.
A Study on the ShortTerm Prohibition Mechanisms in Tabu
Search.
In G. Brewka, S. Coradeschi, A. Perini, and P. Traverso, editors,
Proceedings of the 17th European Conference on Artificial Intelligence,
ECAI 2006, Riva del Garda, Italy, August29  September 1, 2006, pp.
83–87. IOS Press, 2006.
[ bib ]

[1664]

Luca Di Gaspero, Andrea Rendl, and Tommaso Urli.
ConstraintBased Approaches for Balancing Bike Sharing Systems.
In C. Schulte, editor, Principles and Practice of Constraint
Programming, volume 8124 of Lecture Notes in Computer Science, pp.
758–773. Springer, Heidelberg, 2013.
[ bib 
DOI ]
Keywords: Frace

[1665]

Luca Di Gaspero, Andrea Rendl, and Tommaso Urli.
A Hybrid ACO+CP for Balancing Bicycle Sharing Systems.
In M. J. Blesa, C. Blum, P. Festa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics, volume 7919 of Lecture Notes in Computer
Science, pp. 198–212. Springer, Heidelberg, 2013.
[ bib 
DOI ]
Keywords: Frace

[1666]

Daniel Doblas, Antonio J. Nebro, Manuel LópezIbáñez, José
GarcíaNieto, and Carlos A. Coello Coello.
Automatic Design of Multiobjective Particle Swarm Optimizers.
In M. Dorigo, H. Hamann, M. LópezIbáñez,
J. GarcíaNieto, A. Engelbrecht, C. Pinciroli, V. Strobel, and C. L.
CamachoVillalón, editors, Swarm Intelligence, 13th International
Conference, ANTS 2022, volume 13491 of Lecture Notes in Computer
Science, pp. 28–40. Springer, Cham, Switzerland, 2022.
[ bib 
DOI ]

[1667]

Pedro Domingos and Geoff Hulten.
Mining highspeed data streams.
In R. Ramakrishnan, S. J. Stolfo, R. J. Bayardo, and I. Parsa,
editors, The 6th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, KDD 2000, pp. 71–80. ACM Press, New York, NY,
2000.
[ bib 
epub ]

[1668]

Marco Dorigo and Gianni A. Di Caro.
The Ant Colony Optimization MetaHeuristic.
In D. Corne, M. Dorigo, and F. Glover, editors, New Ideas in
Optimization, pp. 11–32. McGraw Hill, London, UK, 1999.
[ bib ]

[1669]

Marco Dorigo and L. M. Gambardella.
Ant Colony System.
Technical Report IRIDIA/9605, IRIDIA, Université Libre de
Bruxelles, Belgium, 1996.
[ bib ]

[1670]

Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni.
The Ant System: An autocatalytic optimizing process.
Technical Report 91016 Revised, Dipartimento di Elettronica,
Politecnico di Milano, Italy, 1991.
[ bib ]

[1671]

Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni.
Positive Feedback as a Search Strategy.
Technical Report 91016, Dipartimento di Elettronica, Politecnico di
Milano, Italy, 1991.
[ bib ]

[1672]

Marco Dorigo, Marco A. Montes de Oca, Sabrina Oliveira, and Thomas
Stützle.
Ant Colony Optimization.
In J. J. Cochran, editor, Wiley Encyclopedia of Operations
Research and Management Science, volume 1, pp. 114–125. John Wiley &
Sons, 2011.
[ bib 
DOI ]

[1673]

Marco Dorigo and Thomas Stützle.
The Ant Colony Optimization Metaheuristic: Algorithms,
Applications and Advances.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pp. 251–285. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib ]

[1674]

Marco Dorigo and Thomas Stützle.
Ant Colony Optimization.
MIT Press, Cambridge, MA, 2004.
[ bib ]

[1675]

Marco Dorigo.
Optimization, Learning and Natural Algorithms.
PhD thesis, Dipartimento di Elettronica, Politecnico di Milano,
Italy, 1992.
In Italian.
[ bib ]

[1676]

Johann Dréo.
Using performance fronts for parameter setting of stochastic
metaheuristics.
In F. Rothlauf, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO Companion 2009, pp. 2197–2200.
ACM Press, New York, NY, 2009.
[ bib 
DOI ]

[1677]

Johann Dréo, Carola Doerr, and Yann Semet.
Coupling the design of benchmark with algorithm in
landscapeaware solver design.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO Companion 2019, pp. 1419–1420. ACM Press, New York, NY,
2019.
[ bib 
DOI 
epub ]

[1678]

Johann Dréo, Arnaud Liefooghe, Sébastien Verel, Marc Schoenauer,
JuanJulián Merelo, Alexandre Quemy, Benjamin Bouvier, and Jan Gmys.
Paradiseo: from a modular framework for evolutionary computation
to the automated design of metaheuristics.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO Companion 2021. ACM
Press, New York, NY, 2021.
[ bib 
DOI ]

[1679]

Johann Dréo and P. Siarry.
A New Ant Colony Algorithm Using the Heterarchical Concept Aimed
at Optimization of Multiminima Continuous Functions.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pp. 216–221. Springer, Heidelberg, 2002.
[ bib ]

[1680]

Stefan Droste, Thomas Jansen, and Ingo Wegener.
A new framework for the valuation of algorithms for
blackboxoptimization.
In K. A. De Jong, R. Poli, and J. E. Rowe, editors, Proceedings
of the Seventh Workshop on Foundations of Genetic Algorithms (FOGA), pp.
253–270. Morgan Kaufmann Publishers, 2002.
[ bib ]

[1681]

Hisao Ishibuchi, Lie Meng Pang, and Ke Shang.
A new framework of evolutionary multiobjective algorithms with
an unbounded external archive.
In G. D. Giacomo, A. Catala, B. Dilkina, M. Milano, S. Barro,
A. BugarÃn, and J. Lang, editors, Proceedings of the 24th European
Conference on Artificial Intelligence (ECAI), volume 325 of Frontiers
in Artificial Intelligence and Applications. IOS Press, 2020.
[ bib ]

[1682]

Chris Drummond.
Replicability is not Reproducibility: Nor is it Good Science.
In Proceedings of the Evaluation Methods for Machine Learning
Workshop at the 26th ICML, Montreal, Canada, 2009.
[ bib 
http ]

[1683]

Mădălina M. Drugan and Dirk Thierens.
PathGuided Mutation for Stochastic Pareto Local Search
Algorithms.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of Lecture
Notes in Computer Science, pp. 485–495. Springer, Heidelberg, 2010.
[ bib ]

[1684]

Jérémie DuboisLacoste.
Weight Setting Strategies for TwoPhase Local Search: A Study on
Biobjective Permutation Flowshop Scheduling.
Technical Report TR/IRIDIA/2009024, IRIDIA, Université Libre de
Bruxelles, Belgium, 2009.
[ bib ]

[1685]

Jérémie DuboisLacoste, Holger H. Hoos, and Thomas Stützle.
On the Empirical Scaling Behaviour of Stateoftheart Local
Search Algorithms for the Euclidean TSP.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pp. 377–384, New York, NY, 2015. ACM Press.
[ bib 
DOI ]

[1686]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Effective Hybrid Stochastic Local Search Algorithms for
Biobjective Permutation Flowshop Scheduling.
In M. J. Blesa, C. Blum, L. Di Gaspero, A. Roli, M. Sampels, and
A. Schaerf, editors, Hybrid Metaheuristics, volume 5818 of Lecture
Notes in Computer Science, pp. 100–114. Springer, Heidelberg, 2009.
[ bib 
DOI ]

[1687]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary material: Improving the Anytime Behavior of
TwoPhase Local Search.
http://iridia.ulb.ac.be/supp/IridiaSupp2010012, 2010.
[ bib ]

[1688]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Supplementary material: A Hybrid TP+PLS Algorithm for
Biobjective Flowshop Scheduling Problems.
http://iridia.ulb.ac.be/supp/IridiaSupp2010001, 2010.
[ bib ]

[1689]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Adaptive “Anytime” TwoPhase Local Search.
In C. Blum and R. Battiti, editors, Learning and Intelligent
Optimization, 4th International Conference, LION 4, volume 6073 of
Lecture Notes in Computer Science, pp. 52–67. Springer, Heidelberg, 2010.
[ bib 
DOI ]

[1690]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Automatic Configuration of Stateoftheart MultiObjective
Optimizers Using the TP+PLS Framework.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp.
2019–2026. ACM Press, New York, NY, 2011.
[ bib 
DOI ]

[1691]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Pareto Local Search Algorithms for Anytime Biobjective
Optimization.
In J.K. Hao and M. Middendorf, editors, Proceedings of EvoCOP
2012 – 12th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 7245 of Lecture Notes in Computer Science, pp.
206–217. Springer, Heidelberg, 2012.
[ bib 
DOI ]

[1692]

Jérémie DuboisLacoste, Manuel LópezIbáñez, and Thomas
Stützle.
Combining Two Search Paradigms for Multiobjective Optimization:
TwoPhase and Pareto Local Search.
In E.G. Talbi, editor, Hybrid Metaheuristics, volume 434 of
Studies in Computational Intelligence, pp. 97–117. Springer Verlag,
2013.
[ bib 
DOI 
http ]

[1693]

Jérémie DuboisLacoste, Federico Pagnozzi, and Thomas Stützle.
Supplementary material: An iterated greedy algorithm with
optimization of partial solutions for the permutation flowshop problem.
http://iridia.ulb.ac.be/supp/IridiaSupp2013006, 2017.
[ bib ]

[1694]

Jérémie DuboisLacoste and Thomas Stützle.
Tuning of a Stigmergybased Traffic Light Controller as a
Dynamic Optimization Problem.
In Proceedings of the 2017 Congress on Evolutionary Computation
(CEC 2017), pp. 1–8. IEEE Press, Piscataway, NJ, 2017.
[ bib ]

[1695]

Jérémie DuboisLacoste.
A study of Pareto and TwoPhase Local Search Algorithms for
Biobjective Permutation Flowshop Scheduling.
Master's thesis, IRIDIA, Université Libre de Bruxelles, Belgium,
2009.
[ bib ]

[1696]

Jérémie DuboisLacoste.
Effective Stochastic Local Search Algorithms For BiObjective
Permutation Flowshop Scheduling.
Rapport d'avancement de recherches présenté pour la formation
doctorale en sciences de l'ingénieur, IRIDIA, Université Libre de
Bruxelles, Belgium, 2010.
[ bib ]

[1697]

Jérémie DuboisLacoste.
Anytime Local Search for MultiObjective Combinatorial
Optimization: Design, Analysis and Automatic Configuration.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2014.
[ bib ]
Supervised by Thomas Stützle and Manuel LópezIbáñez

[1698]

Gunter Dueck, Martin Maehler, Johannes Schneider, Gerhard Schrimpf, and Hermann
StammWilbrandt.
Optimization with Ruin Recreate.
US Patent 6,418,398 B1, July 2002.
Filed on October 1, 1999 and granted on July 9, 2002; Assignee is IBM
Corporation, Armonk, NY (US).
[ bib ]

[1699]

Irina Dumitrescu and Thomas Stützle.
Combinations of Local Search and Exact Algorithms.
In G. R. Raidl and J. Gottlieb, editors, Proceedings of EvoCOP
2003 – 3rd European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 2611 of Lecture Notes in Computer Science, pp.
211–223. Springer, Heidelberg, 2003.
[ bib 
DOI ]

[1700]

Irina Dumitrescu and Thomas Stützle.
Usage of Exact Algorithms to Enhance Stochastic Local Search
Algorithms.
In V. Maniezzo, T. Stützle, and S. Voß, editors,
Matheuristics—Hybridizing Metaheuristics and Mathematical Programming,
volume 10 of Annals of Information Systems, pp. 103–134. Springer,
New York, NY, 2009.
[ bib ]

[1701]

Juan J. Durillo, José GarcíaNieto, Antonio J. Nebro, Carlos A. Coello
Coello, Francisco Luna, and Enrique Alba.
MultiObjective Particle Swarm Optimizers: An Experimental
Comparison.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pp. 495–509.
Springer, Heidelberg, 2009.
[ bib ]
Particle Swarm Optimization (PSO) has received increasing
attention in the optimization research community since its
first appearance in the mid1990s. Regarding multiobjective
optimization, a considerable number of algorithms based on
MultiObjective Particle Swarm Optimizers (MOPSOs) can be
found in the specialized literature. Unfortunately, no
experimental comparisons have been made in order to clarify
which MOPSO version shows the best performance. In this
paper, we use a benchmark composed of three wellknown
problem families (ZDT, DTLZ, and WFG) with the aim of
analyzing the search capabilities of six representative
stateoftheart MOPSOs, namely, NSPSO, SigmaMOPSO, OMOPSO,
AMOPSO, MOPSOpd, and CLMOPSO. We additionally propose a new
MOPSO algorithm, called SMPSO, characterized by including a
velocity constraint mechanism, obtaining promising results
where the rest perform inadequately.

[1702]

Juan J. Durillo, Antonio J. Nebro, Francisco Luna, and Enrique Alba.
On the Effect of the SteadyState Selection Scheme in
MultiObjective Genetic Algorithms.
In M. Ehrgott, C. M. Fonseca, X. Gandibleux, J.K. Hao, and
M. Sevaux, editors, Evolutionary Multicriterion Optimization, EMO
2009, volume 5467 of Lecture Notes in Computer Science, pp. 183–197.
Springer, Heidelberg, 2009.
[ bib ]

[1703]

Cynthia Dwork, Ravi Kumar, Moni Naor, and D. Sivakumar.
Rank aggregation methods for the Web.
In V. Y. Shen, N. Saito, M. R. Lyu, and M. E. Zurko, editors,
Proceedings of the Tenth International World Wide Web Conference, WWW 10,
pp. 613–622. ACM Press, New York, NY, 2001.
[ bib 
DOI ]
Keywords: Kemeny ranking,multiword queries,rank aggregation,ranking
functions,spam

[1704]

L. A. Rossman.
EPANET 2 Users Manual.
U.S. Environmental Protection Agency, Cincinnati, USA, 2000.
[ bib ]

[1705]

L. A. Rossman.
EPANET User's Guide.
Risk Reduction Engineering Laboratory, Office of Research and
Development, U.S. Environmental Protection Agency, Cincinnati, USA, 1994.
[ bib ]

[1706]

L. A. Rossman.
The EPANET Programmer's Toolkit for Analysis of Water
Distribution Systems.
In Proceedings of the Annual Water Resources Planning and
Management Conference, Reston, USA, 1999. ASCE.
[ bib ]

[1707]

Russell C. Eberhart and J. Kennedy.
A New Optimizer Using Particle Swarm Theory.
In Proceedings of the Sixth International Symposium on Micro
Machine and Human Science, pp. 39–43, 1995.
[ bib ]

[1708]

Katharina Eggensperger, Frank Hutter, Holger H. Hoos, and Kevin LeytonBrown.
Efficient Benchmarking of Hyperparameter Optimizers via
Surrogates.
In B. Bonet and S. Koenig, editors, Proceedings of the AAAI
Conference on Artificial Intelligence, pp. 1114–1120. AAAI Press, 2015.
[ bib 
DOI ]

[1709]

Werner Ehm.
Reproducibility from the perspective of metaanalysis.
In H. Atmanspacher and S. Maasen, editors, Reproducibility –
Principles, problems, practices and prospects, pp. 141–168. Wiley, 2016.
[ bib ]

[1710]

Matthias Ehrgott and Xavier Gandibleux.
Hybrid Metaheuristics for Multiobjective Combinatorial
Optimization.
In C. Blum, M. J. Blesa, A. Roli, and M. Sampels, editors,
Hybrid Metaheuristics: An emergent approach for optimization, volume 114 of
Studies in Computational Intelligence, pp. 221–259. Springer, Berlin,
Germany, 2008.
[ bib 
DOI ]
Many realworld optimization problems can be
modelled as combinatorial optimization
problems. Often, these problems are characterized by
their large size and the presence of multiple,
conflicting objectives. Despite progress in solving
multiobjective combinatorial optimization problems
exactly, the large size often means that heuristics
are required for their solution in acceptable time.
Since the middle of the nineties the trend is
towards heuristics that “pick and choose” elements
from several of the established metaheuristic
schemes. Such hybrid approximation techniques may
even combine exact and heuristic approaches. In this
chapter we give an overview over approximation
methods in multiobjective combinatorial
optimization. We briefly summarize “classical”
metaheuristics and focus on recent approaches, where
metaheuristics are hybridized and/or combined with
exact methods.

[1711]

Matthias Ehrgott.
Multicriteria Optimization, volume 491 of Lecture Notes in
Economics and Mathematical Systems.
Springer, Berlin, Germany, 2000.
[ bib ]

[1712]

Matthias Ehrgott.
Multicriteria Optimization.
Springer, Berlin, Germany, 2nd edition, 2005.
[ bib 
DOI ]

[1713]

Agoston E. Eiben, Mark Horvath, Wojtek Kowalczyk, and Martijn C. Schut.
Reinforcement learning for online control of evolutionary
algorithms.
In International Workshop on Engineering SelfOrganising
Applications, pp. 151–160. Springer, 2006.
[ bib ]

[1714]

Agoston E. Eiben and M. Jelasity.
A critical note on experimental research methodology in EC.
In Proceedings of the 2002 Congress on Evolutionary Computation
(CEC'02), pp. 582–587, Piscataway, NJ, 2002. IEEE Press.
[ bib 
DOI ]
Discusses reproducibility, generalizability and separation
between training (for tuning and experimentation) and testing
instances (for comparisons).

[1715]

Agoston E. Eiben, Zbigniew Michalewicz, Marc Schoenauer, and James E. Smith.
Parameter Control in Evolutionary Algorithms.
In F. Lobo, C. F. Lima, and Z. Michalewicz, editors, Parameter
Setting in Evolutionary Algorithms, pp. 19–46. Springer, Berlin, Germany,
2007.
[ bib ]

[1716]

Agoston E. Eiben and James E. Smith.
Introduction to Evolutionary Computing.
Springer, 2003.
[ bib ]

[1717]

Agoston E. Eiben and James E. Smith.
Introduction to Evolutionary Computing.
Natural Computing Series. Springer, 2nd edition, 2007.
[ bib ]

[1718]

Mohammed ElAbd.
Oppositionbased Artificial Bee Colony Algorithm.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 109–116.
ACM Press, New York, NY, 2011.
[ bib ]

[1719]

Michael T. M. Emmerich, André H. Deutz, and J. W. Klinkenberg.
Hypervolumebased expected improvement: Monotonicity properties
and exact computation.
In Proceedings of the 2011 Congress on Evolutionary Computation
(CEC 2011), pp. 2147–2154, Piscataway, NJ, 2011. IEEE Press.
[ bib 
DOI ]
Proposed Expected Hypervolume Improvement (EHVI)

[1720]

Michael T. M. Emmerich and Carlos M. Fonseca.
Computing Hypervolume Contributions in Low Dimensions:
Asymptotically Optimal Algorithm and Complexity Results.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pp. 121–135. Springer, Heidelberg, 2011.
[ bib 
DOI ]
Given a finite set Y ⊂R^{d} of n mutually
nondominated vectors in d ≥2 dimensions, the
hypervolume contribution of a point y ∈Y is the
difference between the hypervolume indicator of Y
and the hypervolume indicator of Y ∖ {y}. In
multiobjective metaheuristics, hypervolume
contributions are computed in several selection and
boundedsize archiving procedures. This paper
presents new results on the (time) complexity of
computing all hypervolume contributions. It is
proved that for d = 2 and 3 the problem has time
complexity Θ(n logn), and, for d > 3,
the time complexity is bounded below by Ω(n
logn). Moreover, complexity bounds are derived for
computing a single hypervolume contribution. A
dimension sweep algorithm with time complexity
O (n logn) and space
complexity O(n) is
proposed for computing all hypervolume contributions
in three dimensions. It improves the complexity of
the best known algorithm for d = 3 by a factor of
√(n) . Theoretical results
are complemented by performance tests on randomly
generated testproblems.

[1721]

Stefan Eppe, Yves De Smet, and Thomas Stützle.
A biobjective optimization model to eliciting decision maker's
preferences for the PROMETHEE II method.
In R. I. Brafman, F. Roberts, and A. Tsoukiàs, editors,
Algorithmic Decision Theory, Third International Conference, ADT 2011,
volume 6992 of Lecture Notes in Artificial Intelligence, pp. 56–66.
Springer, Heidelberg, 2011.
[ bib ]

[1722]

Stefan Eppe, Manuel LópezIbáñez, Thomas Stützle, and Yves De
Smet.
An Experimental Study of Preference Model Integration into
MultiObjective Optimization Heuristics.
In Proceedings of the 2011 Congress on Evolutionary Computation
(CEC 2011), pp. 2751–2758. IEEE Press, Piscataway, NJ, 2011.
[ bib 
DOI ]

[1723]

David Eriksson, Michael Pearce, Jacob Gardner, Ryan D. Turner, and Matthias
Poloczek.
Scalable Global Optimization via Local Bayesian Optimization.
In H. M. Wallach, H. Larochelle, A. Beygelzimer,
F. d'AlchéBuc, E. B. Fox, and R. Garnett, editors, Advances in
Neural Information Processing Systems (NeurIPS 32), pp. 5496–5507, 2019.
[ bib 
epub ]
Arxiv preprint arXiv: https://arxiv.org/abs/1910.01739

[1724]

Emre Ertin, Anthony N. Dean, Mathew L. Moore, and Kevin L. Priddy.
Dynamic Optimization for Optimal Control of Water Distribution
Systems.
In K. L. Priddy, P. E. Keller, and P. J. Angeline, editors,
Applications and Science of Computational Intelligence IV, Proceedings of
SPIE, volume 4390, pp. 142–149, March 2001.
[ bib ]

[1725]

V. Esat and M. Hall.
Water resources system optimization using genetic algorithms.
In A. Verwey, A. Minns, V. Babovic, and C. Maksimović, editors,
Hydroinformatics'94, pp. 225–231, Balkema, Rotterdam, The
Netherlands, 1994.
[ bib ]

[1726]

Larry J. Eshelman and J. David Schaffer.
RealCoded Genetic Algorithms and IntervalSchemata.
In D. Whitley, editor, Foundations of Genetic Algorithms
(FOGA), pp. 187–202. Morgan Kaufmann Publishers, 1993.
[ bib ]

[1727]

Larry J. Eshelman, A. Caruana, and J. David Schaffer.
Biases in the Crossover Landscape.
In J. D. Schaffer, editor, Proceedings of the Third
International Conference on Genetic Algorithms (ICGA'89), pp. 86–91.
Morgan Kaufmann Publishers, San Mateo, CA, 1989.
[ bib ]

[1728]

Richard M. Everson, Jonathan E. Fieldsend, and Sameer Singh.
Full Elite Sets for Multiobjective Optimisation.
In Adaptive Computing in Design and Manufacture V, pp.
343–354. Springer, London, UK, 2002.
[ bib 
DOI ]
Extended version published as [419]

[1729]

C. J. Eyckelhof and M. Snoek.
Ant Systems for a Dynamic TSP: Ants Caught in a Traffic
Jam.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pp. 88–99. Springer, Heidelberg, 2002.
[ bib ]

[1730]

Stefan Falkner, Marius Thomas Lindauer, and Frank Hutter.
SpySMAC: Automated configuration and performance analysis of
SAT solvers.
In M. Heule and S. Weaver, editors, Theory and Applications of
Satisfiability Testing – SAT 2015, volume 9340 of Lecture Notes in
Computer Science, pp. 215–222. Springer, Cham, Switzerland, 2015.
[ bib 
DOI ]

[1731]

Jesús Guillermo FalcónCardona, Saúl
ZapotecasMartínez, and Abel GarcíaNájera.
Pareto compliance from a practical point of view.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 395–402.
ACM Press, New York, NY, 2021.
[ bib 
DOI ]

[1732]

M. Farina and P. Amato.
On the Optimal Solution Definition for Manycriteria
Optimization Problems.
In Proceedings of the NAFIPSFLINT International
Conference'2002, pp. 233–238, Piscataway, New Jersey, June 2002. IEEE
Service Center.
[ bib 
DOI ]
First to mention exponential number of nondominated solutions
with respect to many objectives?

[1733]

D. Favaretto, E. Moretti, and Paola Pellegrini.
On the explorative behavior of MaxMin Ant System.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2009, volume 5752 of Lecture Notes
in Computer Science, pp. 115–119. Springer, Heidelberg, 2009.
[ bib ]

[1734]

Chris Fawcett, Malte Helmert, Holger H. Hoos, Erez Karpas, Gabriele Röger,
and Jendrik Seipp.
FDAutotune: DomainSpecific Configuration using
FastDownward.
In E. Karpas, S. Jiménez Celorrio, and S. Kambhampati, editors,
Proceedings of ICAPSPAL11, 2011.
[ bib ]

[1735]

Chris Fawcett and Holger H. Hoos.
Analysing Differences between Algorithm Configurations through
Ablation.
In Proceedings of MIC 2013, the 10th Metaheuristics
International Conference, pp. 123–132, 2013.
[ bib 
epub ]

[1736]

Silvino Fernández, Segundo Álvarez, Diego Díaz, Miguel Iglesias,
and Borja Ena.
Scheduling a Galvanizing Line by Ant Colony Optimization.
In M. Dorigo et al., editors, Swarm Intelligence, 9th
International Conference, ANTS 2014, volume 8667 of Lecture Notes in
Computer Science, pp. 146–157. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1737]

Silvino Fernández, Segundo Álvarez, Eneko Malatsetxebarria, Pablo
Valledor, and Diego Díaz.
Performance Comparison of Ant Colony Algorithms for the
Scheduling of Steel Production Lines.
In J. L. Jiménez Laredo, S. Silva, and A. I.
EsparciaAlcázar, editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO Companion 2015. ACM Press, New
York, NY, 2015.
[ bib 
DOI ]
Keywords: irace

[1738]

José C. Ferreira, Carlos M. Fonseca, and António GasparCunha.
Methodology to select solutions from the Paretooptimal set: a
comparative study.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pp. 789–796. ACM Press,
New York, NY, 2007.
[ bib ]

[1739]

F. J. Ferri, P. Pudil, M. Hatef, and J. Kittler.
Comparative study of techniques for largescale feature
selection.
In E. S. Gelsema and L. S. Kanal, editors, Pattern Recognition
in Practice IV, volume 16 of Machine Intelligence and Pattern
Recognition, pp. 403–413. NorthHolland, 1994.
[ bib 
DOI ]
The combinatorial search problem arising in feature selection
in high dimensional spaces is considered. Recently developed
techniques based on the classical sequential methods and the
(l, r) search called Floating search algorithms are compared
against the Genetic approach to feature subset search. Both
approaches have been designed with the view to give a good
compromise between efficiency and effectiveness for large
problems. The purpose of this paper is to investigate the
applicability of these techniques to high dimensional
problems of feature selection. The aim is to establish
whether the properties inferred for these techniques from
medium scale experiments involving up to a few tens of
dimensions extend to dimensionalities of one order of
magnitude higher. Further, relative merits of these
techniques visavis such high dimensional problems are
explored and the possibility of exploiting the best aspects
of these methods to create a composite feature selection
procedure with superior properties is considered.
Describes sequential forward / backward selection

[1740]

Silvino Fernández, Pablo Valledor, Diego Díaz, Eneko Malatsetxebarria,
and Miguel Iglesias.
Criticality of Response Time in the usage of Metaheuristics in
Industry.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
Companion 2016, pp. 937–940. ACM Press, New York, NY, 2016.
[ bib ]

[1741]

Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel
Blum, and Frank Hutter.
Efficient and robust automated machine learning.
In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett,
editors, Advances in Neural Information Processing Systems (NIPS 28),
pp. 2962–2970, 2015.
[ bib 
epub 
http ]

[1742]

Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg,
Manuel Blum, and Frank Hutter.
Autosklearn: Efficient and Robust Automated Machine Learning.
In F. Hutter, L. Kotthoff, and J. Vanschoren, editors, Automated
Machine Learning, pp. 113–134. Springer, 2019.
[ bib 
DOI 
epub ]
The success of machine learning in a broad range of
applications has led to an evergrowing demand for machine
learning systems that can be used off the shelf by
nonexperts. To be effective in practice, such systems need
to automatically choose a good algorithm and feature
preprocessing steps for a new dataset at hand, and also set
their respective hyperparameters. Recent work has started to
tackle this automated machine learning (AutoML) problem with
the help of efficient Bayesian optimization methods. Building
on this, we introduce a robust new AutoML system based on the
Python machine learning package scikitlearn (using 15
classifiers, 14 feature preprocessing methods, and 4 data
preprocessing methods, giving rise to a structured hypothesis
space with 110 hyperparameters). This system, which we dub
Autosklearn, improves on existing AutoML methods by
automatically taking into account past performance on similar
datasets, and by constructing ensembles from the models
evaluated during the optimization. Our system won six out of
ten phases of the first ChaLearn AutoML challenge, and our
comprehensive analysis on over 100 diverse datasets shows
that it substantially outperforms the previous state of the
art in AutoML. We also demonstrate the performance gains due
to each of our contributions and derive insights into the
effectiveness of the individual components of Autosklearn.

[1743]

Álvaro Fialho, Raymond Ros, Marc Schoenauer, and Michèle Sebag.
Comparisonbased adaptive strategy selection with bandits in
differential evolution.
In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors,
Parallel Problem Solving from Nature, PPSN XI, volume 6238 of Lecture
Notes in Computer Science, pp. 194–203. Springer, Heidelberg, 2010.
[ bib ]

[1744]

Álvaro Fialho, Marc Schoenauer, and Michèle Sebag.
FitnessAUC bandit adaptive strategy selection vs. the
probability matching one within differential evolution: an empirical
comparison on the BBOB2010 noiseless testbed.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO Companion 2010, pp.
1535–1542. ACM Press, New York, NY, 2010.
[ bib ]

[1745]

Álvaro Fialho, Marc Schoenauer, and Michèle Sebag.
Toward comparisonbased adaptive operator selection.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pp. 767–774. ACM
Press, New York, NY, 2010.
[ bib ]
Proposed FAUC and FSR

[1746]

Álvaro Fialho.
Adaptive operator selection for optimization.
PhD thesis, Université Paris SudParis XI, 2010.
[ bib ]

[1747]

Jonathan E. Fieldsend.
University staff teaching allocation: formulating and optimising
a manyobjective problem.
In P. A. N. Bosman, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2017, pp. 1097–1104. ACM Press,
New York, NY, 2017.
[ bib 
DOI ]
Example of NSGAIII deteriorating.

[1748]

Jonathan E. Fieldsend and Richard M. Everson.
Visualising highdimensional Pareto relationships in
twodimensional scatterplots.
In R. C. Purshouse, P. J. Fleming, C. M. Fonseca, S. Greco, and
J. Shaw, editors, Evolutionary Multicriterion Optimization, EMO 2013,
volume 7811 of Lecture Notes in Computer Science, pp. 558–572.
Springer, Heidelberg, 2013.
[ bib 
DOI ]

[1749]

Jonathan E. Fieldsend.
Data structures for nondominated sets: implementations and
empirical assessment of two decades of advances.
In C. A. Coello Coello, editor, Proceedings of the 2020
Genetic and Evolutionary Computation Conference, pp. 489–497. ACM Press,
New York, NY, 2020.
[ bib 
DOI 
epub ]
unbounded archives

[1750]

Andreas Fink and Stefan Voß.
HotFrame: A Heuristic Optimization Framework.
In S. Voß and D. L. Woodruff, editors, Optimization Software
Class Libraries, pp. 81–154. Kluwer Academic Publishers, Boston, MA, 2002.
[ bib ]

[1751]

Benjamin Fisset, Clarisse Dhaenens, and Laetitia Jourdan.
MOMine_{}clust: A Framework for MultiObjective
Clustering.
In C. Dhaenens, L. Jourdan, and M.E. Marmion, editors, Learning
and Intelligent Optimization, 9th International Conference, LION 9, volume
8994 of Lecture Notes in Computer Science, pp. 293–305. Springer,
Heidelberg, 2015.
[ bib ]
Keywords: irace

[1752]

Peter J. Fleming, Robin C. Purshouse, and Robert J. Lygoe.
Manyobjective optimization: An engineering design perspective.
In C. A. Coello Coello, A. Hernández Aguirre, and E. Zitzler,
editors, Evolutionary Multicriterion Optimization, EMO 2005, volume
3410 of Lecture Notes in Computer Science, pp. 14–32. Springer,
Heidelberg, 2005.
[ bib ]

[1753]

Robin C. Purshouse, Cezar Jalbă, and Peter J. Fleming.
PreferenceDriven CoEvolutionary Algorithms Show Promise for
ManyObjective optimisation.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pp. 136–150. Springer, Heidelberg, 2011.
[ bib ]

[1754]

M. Fleischer.
The Measure of Pareto Optima. Applications to Multiobjective
Metaheuristics.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pp. 519–533. Springer,
Heidelberg, 2003.
[ bib ]

[1755]

R. Fletcher.
Practical methods of optimization.
John Wiley & Sons, New York, NY, 1987.
[ bib ]
BFGS

[1756]

Dario Floreano and Francesco Mondada.
Automatic creation of an autonomous agent: Genetic evolution of
a neural network driven robot.
In D. Cliff, P. Husbands, J.A. Meyer, and S. Wilson, editors,
Proceedings of the third international conference on Simulation of adaptive
behavior: From Animals to Animats 3, pp. 421–430. MIT Press, Cambridge,
MA, 1994.
[ bib ]
LISCONF1994003

[1757]

Filippo Focacci, François Laburthe, and Andrea Lodi.
Local Search and Constraint Programming.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pp. 369–403. Kluwer Academic Publishers, Norwell, MA,
2002.
[ bib ]

[1758]

David B. Fogel, Alvin J. Owens, and Michael J. Walsh.
Artificial Intelligence Through Simulated Evolution.
John Wiley & Sons, 1966.
[ bib ]

[1759]

David B. Fogel.
Evolutionary Computation. Toward a New Philosophy of Machine
Intelligence.
IEEE Press, 1995.
[ bib ]

[1760]

Carlos M. Fonseca and Peter J. Fleming.
Genetic Algorithms for Multiobjective Optimization: Formulation,
Discussion and Generalization.
In S. Forrest, editor, Proceedings of the Fifth International
Conference on Genetic Algorithms (ICGA'93), pp. 416–423. Morgan Kaufmann
Publishers, 1993.
[ bib 
epub ]
Proposes MOGA and PMOGA

[1761]

Carlos M. Fonseca and Peter J. Fleming.
On the Performance Assessment and Comparison of Stochastic
Multiobjective Optimizers.
In H.M. Voigt et al., editors, Parallel Problem Solving from
Nature – PPSN IV, volume 1141 of Lecture Notes in Computer
Science, pp. 584–593. Springer, Heidelberg, 1996.
[ bib ]

[1762]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
The Relationship between the Covered Fraction, Completeness and
Hypervolume Indicators.
In J.K. Hao, P. Legrand, P. Collet, N. Monmarché, E. Lutton, and
M. Schoenauer, editors, Artificial Evolution: 10th International
Conference, Evolution Artificielle, EA, 2011, volume 7401 of Lecture
Notes in Computer Science, pp. 25–36. Springer, Heidelberg, 2012.
[ bib ]

[1763]

Carlos M. Fonseca, Viviane Grunert da Fonseca, and Luís Paquete.
Exploring the Performance of Stochastic Multiobjective
Optimisers with the SecondOrder Attainment Function.
In C. A. Coello Coello, A. Hernández Aguirre, and E. Zitzler,
editors, Evolutionary Multicriterion Optimization, EMO 2005, volume
3410 of Lecture Notes in Computer Science, pp. 250–264. Springer,
Heidelberg, 2005.
[ bib 
DOI ]
The attainment function has been proposed as a
measure of the statistical performance of stochastic
multiobjective optimisers which encompasses both the
quality of individual nondominated solutions in
objective space and their spread along the tradeoff
surface. It has also been related to results from
random closedset theory, and cast as a meanlike,
firstorder moment measure of the outcomes of
multiobjective optimisers. In this work, the use of
more informative, secondorder moment measures for
the evaluation and comparison of multiobjective
optimiser performance is explored experimentally,
with emphasis on the interpretability of the
results.

[1764]

Carlos M. Fonseca, Andreia P. Guerreiro, Manuel LópezIbáñez, and
Luís Paquete.
On the Computation of the Empirical Attainment Function.
In R. H. C. Takahashi et al., editors, Evolutionary
Multicriterion Optimization, EMO 2011, volume 6576 of Lecture Notes in
Computer Science, pp. 106–120. Springer, Heidelberg, 2011.
[ bib 
DOI ]
The attainment function provides a description of the
location of the distribution of a random nondominated point
set. This function can be estimated from experimental data
via its empirical counterpart, the empirical attainment
function (EAF). However, computation of the EAF in more than
two dimensions is a nontrivial task. In this article, the
problem of computing the empirical attainment function is
formalised, and upper and lower bounds on the corresponding
number of output points are presented. In addition, efficient
algorithms for the two and threedimensional cases are
proposed, and their time complexities are related to lower
bounds derived for each case.

[1765]

Carlos M. Fonseca, Luís Paquete, and Manuel LópezIbáñez.
An improved dimension sweep algorithm for the
hypervolume indicator.
In Proceedings of the 2006 Congress on Evolutionary Computation
(CEC 2006), pp. 1157–1163. IEEE Press, Piscataway, NJ, July 2006.
[ bib 
DOI ]
This paper presents a recursive, dimensionsweep
algorithm for computing the hypervolume indicator of
the quality of a set of n nondominated points in
d>2 dimensions. It improves upon the existing HSO
(Hypervolume by Slicing Objectives) algorithm by
pruning the recursion tree to avoid repeated
dominance checks and the recalculation of partial
hypervolumes. Additionally, it incorporates a recent
result for the threedimensional special case. The
proposed algorithm achieves O(n^{d2} log n) time
and linear space complexity in the worstcase, but
experimental results show that the pruning
techniques used may reduce the time complexity
exponent even further.

[1766]

Jorge Ramón Fonseca Cacho and Kazem Taghva.
The State of Reproducible Research in Computer Science.
In S. Latifi, editor, 17th International Conference on
Information TechnologyNew Generations (ITNG 2020), Advances in
Intelligent Systems and Computing, pp. 519–524. Springer International
Publishing, 2020.
[ bib 
DOI ]
Reproducible research is the cornerstone of cumulative
science and yet is one of the most serious crisis that we
face today in all fields. This paper aims to describe the
ongoing reproducible research crisis along with
counterarguments of whether it really is a crisis, suggest
solutions to problems limiting reproducible research along
with the tools to implement such solutions by covering the
latest publications involving reproducible research.
Keywords: Docker, Improving transparency, OCR, Open science,
Replicability, Reproducibility

[1767]

Manuel Förster, Bettina Bickel, Bernd Hardung, and Gabriella Kókai.
Selfadaptive ant colony optimisation applied to function
allocation in vehicle networks.
In D. Thierens et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2007, pp. 1991–1998. ACM Press,
New York, NY, 2007.
[ bib ]

[1768]

Michael Foster, Matthew Hughes, George O'Brien, Pietro S. Oliveto, James Pyle,
Dirk Sudholt, and James Williams.
Do sophisticated evolutionary algorithms perform better than
simple ones?
In C. A. Coello Coello, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2020, pp. 184–192, New York,
NY, 2020. ACM Press.
[ bib 
DOI 
epub ]

[1769]

Robert Fourer, David M. Gay, and Brian W. Kernighan.
AMPL: A Modeling Language for Mathematical Programming.
Duxbury, 2nd edition, 2002.
[ bib ]

[1770]

Bennett L. Fox.
Uniting probabilistic methods for optimization.
In Proceedings of the 24th conference on Winter simulation,
pp. 500–505. ACM, 1992.
[ bib ]

[1771]

Bennett L. Fox.
Simulated annealing: folklore, facts, and directions.
In Monte Carlo and QuasiMonte Carlo Methods in Scientific
Computing, pp. 17–48. Springer, 1995.
[ bib ]

[1772]

Alberto Franzin.
Empirical Analysis of Stochastic Local Search Behaviour:
Connecting Structure, Components and Landscape.
PhD thesis, IRIDIA, École polytechnique, Université Libre de
Bruxelles, Belgium, 2021.
[ bib ]

[1773]

C. B. Fraser.
Subsequences and Supersequences of Strings.
PhD thesis, University of Glasgow, 1995.
[ bib ]

[1774]

Alberto Franzin, Raphaël Gyory, JeanCharles Nadé, Guillaume Aubert,
Georges Klenkle, and Hughes Bersini.
Philéas: Anomaly Detection for IoT Monitoring.
In L. Cao, W. Kosters, and J. Lijffijt, editors, Proceedings of
the 32nd Benelux Conference on Artificial Intelligence, BNAIC 2020, Leiden,
The Netherlands, 1920 November 2020, pp. 56–70, 2020.
[ bib 
http ]

[1775]

Jose M. Framiñán, Rainer Leisten, and Rubén Ruiz.
Manufacturing Scheduling Systems: An Integrated View on Models,
Methods, and Tools.
Springer, New York, NY, 2014.
[ bib ]

[1776]

Alberto Franzin and Thomas Stützle.
Exploration of Metaheuristics through Automatic Algorithm
Configuration Techniques and Algorithmic Frameworks.
In T. Friedrich, F. Neumann, and A. M. Sutton, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
Companion 2016, pp. 1341–1347. ACM Press, New York, NY, 2016.
[ bib ]

[1777]

Alberto Franzin and Thomas Stützle.
Comparison of Acceptance Criteria in Randomized Local Searches.
In E. Lutton, P. Legrand, P. Parrend, N. Monmarché, and
M. Schoenauer, editors, EA 2017: Artificial Evolution, volume 10764 of
Lecture Notes in Computer Science, pp. 16–29, Heidelberg, 2017.
Springer.
[ bib ]

[1778]

Alberto Franzin and Thomas Stützle.
Revisiting Simulated Annealing: a ComponentBased Analysis:
Supplementaty Material.
http://iridia.ulb.ac.be/supp/IridiaSupp2018001, 2018.
[ bib ]

[1779]

Alberto Franzin and Thomas Stützle.
Towards transferring algorithm configurations across problems.
In M. Vlastelica, J. Song, A. Ferber, B. Amos, G. Martius,
B. Dilkina, and Y. Yue, editors, Learning Meets Combinatorial Algorithms
Workshop at NeurIPS 2020, LMCA 2020, Vancouver, Canada, December 12, 2020,
pp. 1–6, 2020.
[ bib ]

[1780]

Alberto Franzin and Thomas Stützle.
A causal framework for understanding optimisation algorithms.
In F. Heintz, M. Milano, and B. O'Sullivan, editors, Trustworthy
AI – Integrating Learning, Optimization and Reasoning. TAILOR 2020, volume
12641 of Lecture Notes in Computer Science, pp. 140–145. Springer,
Cham, Switzerland, 2021.
[ bib ]

[1781]

Alberto Franzin and Thomas Stützle.
A Landscapebased Analysis of Fixed Temperature and Simulated
Annealing: Supplementaty Material.
http://iridia.ulb.ac.be/supp/IridiaSupp2021002, 2021.
[ bib ]

[1782]

A. R. R. Freitas, Peter J. Fleming, and Frederico G. Guimarães.
A Nonparametric HarmonyBased Objective Reduction Method for
ManyObjective Optimization.
In 2013 IEEE International Conference on Systems, Man, and
Cybernetics, pp. 651–656. IEEE Press, 2013.
[ bib 
DOI ]

[1783]

B. Freisleben and P. Merz.
A Genetic Local Search Algorithm for Solving Symmetric and
Asymmetric Traveling Salesman Problems.
In T. Bäck, T. Fukuda, and Z. Michalewicz, editors,
Proceedings of the 1996 IEEE International Conference on Evolutionary
Computation (ICEC'96), pp. 616–621, Piscataway, NJ, 1996. IEEE Press.
[ bib ]

[1784]

Tobias Friedrich, Andreas Göbel, Francesco Quinzan, and Markus Wagner.
HeavyTailed Mutation Operators in SingleObjective
Combinatorial Optimization.
In A. Auger, C. M. Fonseca, N. Lourenço, P. Machado, L. Paquete,
and D. Whitley, editors, Parallel Problem Solving from Nature – PPSN
XV, volume 11101 of Lecture Notes in Computer Science, pp.
134–145. Springer, Cham, Switzerland, 2018.
[ bib ]
A core feature of evolutionary algorithms is their mutation
operator. Recently, much attention has been devoted to the
study of mutation operators with dynamic and nonuniform
mutation rates. Following up on this line of work, we propose
a new mutation operator and analyze its performance on the
(1+1) Evolutionary Algorithm (EA). Our analyses show that
this mutation operator competes with preexisting ones, when
used by the (1+1)EA on classes of problems for which
results on the other mutation operators are available. We
present a “jump” function for which the performance of the
(1+1)EA using any static uniform mutation and any restart
strategy can be worse than the performance of the (1+1)EA
using our mutation operator with no restarts. We show that
the (1+1)EA using our mutation operator finds a
(1/3)approximation ratio on any nonnegative submodular
function in polynomial time. This performance matches that of
combinatorial local search algorithms specifically designed
to solve this problem.

[1785]

Tobias Friedrich, Timo Kötzing, Martin S. Krejca, and Andrew M. Sutton.
Robustness of Ant Colony Optimization to Noise.
In S. Silva and A. I. EsparciaAlcázar, editors,
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO
2015, pp. 17–24. ACM Press, New York, NY, 2015.
[ bib 
DOI ]
Keywords: ant colony optimization, noisy fitness, run time analysis,
theory

[1786]

Tobias Friedrich, Timo Kötzing, and Markus Wagner.
A Generic BetandRun Strategy for Speeding Up Stochastic Local
Search.
In S. P. Singh and S. Markovitch, editors, Proceedings of the
AAAI Conference on Artificial Intelligence, pp. 801–807. AAAI Press,
February 2017.
[ bib ]

[1787]

Tobias Friedrich, Francesco Quinzan, and Markus Wagner.
Escaping Large Deceptive Basins of Attraction with Heavytailed
Mutation Operators.
In H. E. Aguirre and K. Takadama, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2018, pp. 293–300.
ACM Press, New York, NY, 2018.
[ bib 
DOI ]
Keywords: combinatorial optimization, heavytailed mutation,
singleobjective optimization, experimentsmotivated theory,
irace

[1788]

Michael Friendly.
Statistical graphics for multivariate data.
In SAS Conference Proceedings: SAS Users Group International 16
(SUGI 16), 1991.
[ bib ]
February 1720, 1991, New Orleans, Louisiana, 297 papers

[1789]

D. Fudenberg and J. Tirole.
Game Theory.
MIT Press, Cambridge, MA, 1983.
[ bib ]

[1790]

Noriyuki Fujimoto and Kouki Nanai.
Solving QUBO with GPU parallel MOPSO.
In F. Chicano and K. Krawiec, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO Companion 2021, pp.
1788–1794. ACM Press, New York, NY, 2021.
[ bib ]

[1791]

Alex S. Fukunaga.
Evolving Local Search Heuristics for SAT Using Genetic
Programming.
In K. Deb et al., editors, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2004, Part II, volume 3103 of
Lecture Notes in Computer Science, pp. 483–494. Springer, Heidelberg,
2004.
[ bib ]
Satisfiability testing (SAT) is a very active area
of research today, with numerous realworld
applications. We describe CLASS2.0, a genetic
programming system for semiautomatically designing
SAT local search heuristics. An empirical
comparison shows that that the heuristics generated
by our GP system outperform the state of the art
humandesigned local search algorithms, as well as
previously proposed evolutionary approaches, with
respect to both runtime as well as search efficiency
(number of variable flips to solve a problem).

[1792]

Nancy E. Furlong, Eugene A. Lovelace, and Kristin L. Lovelace.
Research Methods and Statistics: An Integrated Approach.
Harcourt College Publishers, 2000.
[ bib ]

[1793]

D. Gaertner and K. Clark.
On Optimal Parameters for Ant Colony Optimization Algorithms.
In H. R. Arabnia and R. Joshua, editors, Proceedings of the 2005
International Conference on Artificial Intelligence, ICAI 2005, pp. 83–89.
CSREA Press, 2005.
[ bib ]

[1794]

Matteo Gagliolo and Catherine Legrand.
Algorithm Survival Analysis.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pp. 161–184. Springer, Berlin, Germany, 2010.
[ bib 
DOI ]
Algorithm selection is typically based on models of
algorithm performance,learned during a separate
offline training sequence, which can be
prohibitively expensive. In recent work, we adopted
an online approach, in which models of the runtime
distributions of the available algorithms are
iteratively updated and used to guide the allocation
of computational resources, while solving a sequence
of problem instances. The models are estimated using
survival analysis techniques, which allow us to
reduce computation time, censoring the runtimes of
the slower algorithms. Here, we review the
statistical aspects of our online selection method,
discussing the bias induced in the runtime
distributions (RTD) models by the competition of
different algorithms on the same problem instances.

[1795]

L. M. Gambardella and Marco Dorigo.
AntQ: A Reinforcement Learning Approach to the Traveling
Salesman Problem.
In A. Prieditis and S. Russell, editors, Proceedings of the
Twelfth International Conference on Machine Learning (ML95), pp. 252–260.
Morgan Kaufmann Publishers, Palo Alto, CA, 1995.
[ bib ]

[1796]

L. M. Gambardella and Marco Dorigo.
Solving Symmetric and Asymmetric TSPs by Ant Colonies.
In T. Bäck, T. Fukuda, and Z. Michalewicz, editors,
Proceedings of the 1996 IEEE International Conference on Evolutionary
Computation (ICEC'96), pp. 622–627, Piscataway, NJ, 1996. IEEE Press.
[ bib ]

[1797]

L. M. Gambardella, Éric D. Taillard, and G. Agazzi.
MACSVRPTW: A Multiple Ant Colony System for Vehicle Routing
Problems with Time Windows.
In D. Corne, M. Dorigo, and F. Glover, editors, New Ideas in
Optimization, pp. 63–76. McGraw Hill, London, UK, 1999.
[ bib ]

[1798]

Xavier Gandibleux, X. Delorme, and V. T'Kindt.
An Ant Colony Optimisation Algorithm for the Set Packing
Problem.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pp. 49–60. Springer, Heidelberg, 2004.
[ bib ]

[1799]

Xavier Gandibleux, N. Mezdaoui, and A. Fréville.
A tabu search procedure to solve multiobjective combinatorial
optimization problem.
In R. Caballero, F. Ruiz, and R. Steuer, editors, Advances in
Multiple Objective and Goal Programming, volume 455 of Lecture Notes in
Economics and Mathematical Systems, pp. 291–300, Heidelberg, 1997.
Springer.
[ bib ]

[1800]

Xavier Gandibleux, H. Morita, and N. Katoh.
Use of a genetic heritage for solving the assignment problem
with two objectives.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pp. 43–57. Springer,
Heidelberg, 2003.
[ bib ]

[1801]

Huiru Gao, Haifeng Nie, and Ke Li.
Visualisation of Pareto Front Approximation: A Short Survey
and Empirical Comparisons.
In Proceedings of the 2019 Congress on Evolutionary Computation
(CEC 2019), pp. 1750–1757, Piscataway, NJ, 2019. IEEE Press.
[ bib 
DOI ]

[1802]

Deon Garrett and Dipankar Dasgupta.
Multiobjective landscape analysis and the generalized assignment
problem.
In V. Maniezzo, R. Battiti, and J.P. Watson, editors, Learning
and Intelligent Optimization, Second International Conference, LION 2,
volume 5313 of Lecture Notes in Computer Science, pp. 110–124.
Springer, Heidelberg, 2008.
[ bib ]

[1803]

M. R. Garey and David S. Johnson.
Computers and Intractability: A Guide to the Theory of
NPCompleteness.
Freeman & Co, San Francisco, CA, 1979.
[ bib ]

[1804]

José GarcíaNieto, Esteban LópezCamacho, María Jesús
Godoy García, Antonio J. Nebro, Juan J. Durillo, and José F.
AldanaMontes.
A study of archiving strategies in multiobjective PSO for
molecular docking.
In M. Dorigo, M. Birattari, X. Li, M. LópezIbáñez,
K. Ohkura, C. Pinciroli, and T. Stützle, editors, Swarm
Intelligence, 10th International Conference, ANTS 2016, volume 9882 of
Lecture Notes in Computer Science, pp. 40–52. Springer, Heidelberg, 2016.
[ bib 
DOI ]

[1805]

Beatriz A. Garro, Humberto Sossa, and Roberto A. Vazquez.
Evolving ant colony system for optimizing path planning in
mobile robots.
In Electronics, Robotics and Automotive Mechanics Conference,
pp. 444–449, Los Alamitos, CA, 2007. IEEE Computer Society.
[ bib 
DOI ]

[1806]

Luca Di Gaspero and Andrea Schaerf.
Easysyn++: A tool for automatic synthesis of stochastic local
search algorithms.
In T. Stützle, M. Birattari, and H. H. Hoos, editors,
Engineering Stochastic Local Search Algorithms. Designing, Implementing and
Analyzing Effective Heuristics. SLS 2007, volume 4638 of Lecture Notes
in Computer Science, pp. 177–181. Springer, Heidelberg, 2007.
[ bib ]

[1807]

Martin Gebser, Roland Kaminski, Benjamin Kaufmann, Torsten Schaub,
Marius Thomas Schneider, and Stefan Ziller.
A portfolio solver for answer set programming: Preliminary
report.
In P. Calabar and T. C. Son, editors, Logic Programming and
Nonmonotonic Reasoning, volume 8148 of Lecture Notes in Artificial
Intelligence, pp. 352–357. Springer, Heidelberg, 2013.
[ bib ]

[1808]

Peter Geibel.
Reinforcement Learning for MDPs with Constraints.
In J. Fürnkranz, T. Scheffer, and M. Spiliopoulou, editors,
Machine Learning: ECML 2006, volume 4212 of Lecture Notes in Computer
Science, pp. 646–653, 2006.
[ bib 
DOI ]
In this article, I will consider Markov Decision Processes
with two criteria, each defined as the expected value of an
infinite horizon cumulative return. The second criterion is
either itself subject to an inequality constraint, or there
is maximum allowable probability that the single returns
violate the constraint. I describe and discuss three new
reinforcement learning approaches for solving such control
problems.
Keywords: Safe RL

[1809]

Ian P. Gent, Stuart A. Grant, Ewen MacIntyre, Patrick Prosser, Paul Shaw,
Barbara M. Smith, and Toby Walsh.
How Not To Do It.
Technical Report 97.27, School of Computer Studies, University of
Leeds, May 1997.
[ bib ]
We give some dos and don'ts for those analysing algorithms
experimentally. We illustrate these with many examples from
our own research on the study of algorithms for NPcomplete
problems such as satisfiability and constraint
satisfaction. Where we have not followed these maxims, we
have suffered as a result.

[1810]

Ian P. Gent, Holger H. Hoos, P. Prosser, and T. Walsh.
Morphing: Combining Structure and Randomness.
In Proceedings of the Sixteenth National Conference on
Artificial Intelligence, pp. 654–660, 1999.
[ bib ]

[1811]

Michel Gendreau and JeanYves Potvin.
Tabu Search.
In M. Gendreau and J.Y. Potvin, editors, Handbook of
Metaheuristics, volume 146 of International Series in Operations
Research & Management Science, pp. 41–59. Springer, New York, NY, 2nd
edition, 2010.
[ bib ]

[1812]

Daniel Geschwender, Frank Hutter, Lars Kotthoff, Yuri Malitsky, Holger H. Hoos,
and Kevin LeytonBrown.
Algorithm Configuration in the Cloud: A Feasibility Study.
In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, and J. L.
Walteros, editors, Learning and Intelligent Optimization, 8th
International Conference, LION 8, volume 8426 of Lecture Notes in
Computer Science, pp. 41–46. Springer, Heidelberg, 2014.
[ bib 
DOI ]

[1813]

Matthew S. Gibbs, Graeme C. Dandy, Holger R. Maier, and John B. Nixon.
Calibrating genetic algorithms for water distribution system
optimisation.
In 7th Annual Symposium on Water Distribution Systems Analysis.
ASCE, May 2005.
[ bib ]

[1814]

T. Glasmachers.
A fast incremental BSP tree archive for nondominated points.
In H. Trautmann, G. Rudolph, K. Klamroth, O. Schütze, M. M.
Wiecek, Y. Jin, and C. Grimme, editors, Evolutionary Multicriterion
Optimization, EMO 2017, volume 10173 of Lecture Notes in Computer
Science, pp. 252–266. Springer International Publishing, Cham,
Switzerland, 2017.
[ bib ]
Keywords: archiving

[1815]

Fred Glover.
A Template for Scatter Search and Path Relinking.
In J.K. Hao, E. Lutton, E. M. A. Ronald, M. Schoenauer, and
D. Snyers, editors, Artificial Evolution, volume 1363 of Lecture
Notes in Computer Science, pp. 1–51. Springer, Heidelberg, 1998.
[ bib 
DOI ]

[1816]

Xavier Glorot and Yoshua Bengio.
Understanding the difficulty of training deep feedforward neural
networks.
In Proceedings of the Thirteenth International Conference on
Artificial Intelligence and Statistics, pp. 249–256, 2010.
[ bib ]

[1817]

Fred Glover and Gary A. Kochenberger.
Critical Even Tabu Search for Multidimensional Knapsack
Problems.
In I. H. Osman and J. P. Kelly, editors, Metaheuristics: Theory
& Applications, pp. 407–427. Kluwer Academic Publishers, Norwell, MA,
1996.
[ bib ]

[1818]

Fred Glover and Manuel Laguna.
Tabu Search.
Kluwer Academic Publishers, Boston, MA, USA, 1997.
[ bib ]

[1819]

Fred Glover, Manuel Laguna, and Rafael Martí.
Scatter Search and Path Relinking: Advances and Applications.
In F. Glover and G. A. Kochenberger, editors, Handbook of
Metaheuristics, pp. 1–35. Kluwer Academic Publishers, Norwell, MA, 2002.
[ bib ]

[1820]

Elizabeth Ferreira Gouvêa Goldbarg, Givanaldo R. Souza, and Marco Cesar
Goldbarg.
Particle Swarm for the Traveling Salesman Problem.
In J. Gottlieb and G. R. Raidl, editors, Proceedings of EvoCOP
2006 – 6th European Conference on Evolutionary Computation in Combinatorial
Optimization, volume 3906 of Lecture Notes in Computer Science, pp.
99–110. Springer, Heidelberg, 2006.
[ bib ]

[1821]

David E. Goldberg.
Genetic Algorithms in Search, Optimization and Machine
Learning.
AddisonWesley, Boston, MA, 1989.
[ bib ]

[1822]

Fred E. Goldman and Larry W. Mays.
The Application of Simulated Annealing to the Optimal Operation
of Water Systems.
In Proceedings of 26th Annual Water Resources Planning and
Management Conference, Tempe, USA, June 2000. ASCE.
[ bib ]

[1823]

Ralph E. Gomory.
An algorithm for integer solutions to linear programs.
In R. Graves and P. Wolfe, editors, Recent Advances in
Mathematical Programming, pp. 260–302. McGraw Hill, New York, NY, 1963.
[ bib ]

[1824]

Wenyin Gong, Álvaro Fialho, and Zhihua Cai.
Adaptive strategy selection in differential evolution.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO 2010, pp. 409–416. ACM
Press, New York, NY, 2010.
[ bib 
DOI ]

[1825]

M. GorgesSchleuter.
Asparagos96 and the Travelling Salesman Problem.
In T. Bäck, Z. Michalewicz, and X. Yao, editors, Proceedings
of the 1997 IEEE International Conference on Evolutionary Computation
(ICEC'97), pp. 171–174, Piscataway, NJ, 1997. IEEE Press.
[ bib ]

[1826]

J. Gottlieb, M. Puchta, and Christine Solnon.
A Study of Greedy, Local Search, and Ant Colony Optimization
Approaches for Car Sequencing Problems.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2003, volume 2611 of Lecture
Notes in Computer Science, pp. 246–257, Heidelberg, 2003. Springer.
[ bib ]

[1827]

Alex Graves, Abdelrahman Mohamed, and Geoffrey Hinton.
Speech recognition with deep recurrent neural networks.
In Acoustics, speech and signal processing (icassp), 2013 ieee
international conference on, pp. 6645–6649. IEEE, 2013.
[ bib ]

[1828]

Garrison W. Greenwood, Xiaobo Hu, and Joseph G. D'Ambrosio.
Fitness functions for multiple objective optimization problems:
Combining preferences with Pareto rankings.
In R. K. Belew and M. D. Vose, editors, Foundations of Genetic
Algorithms (FOGA), pp. 437–455. Morgan Kaufmann Publishers, 1996.
[ bib ]

[1829]

Salvatore Greco, Benedetto Matarazzo, and Roman Slowiński.
Interactive evolutionary multiobjective optimization using
dominancebased rough set approach.
In H. Ishibuchi et al., editors, Proceedings of the 2010
Congress on Evolutionary Computation (CEC 2010), pp. 1–8. IEEE Press,
Piscataway, NJ, 2010.
[ bib ]

[1830]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
A characterization of the outcomes of stochastic multiobjective
optimizers through a reduction of the hitting function test sets.
Technical report, CSI, Universidade do Algarve, 2004.
[ bib ]
Keywords: highorder EAF

[1831]

Viviane Grunert da Fonseca and Carlos M. Fonseca.
The AttainmentFunction Approach to Stochastic Multiobjective
Optimizer Assessment and Comparison.
In T. BartzBeielstein, M. Chiarandini, L. Paquete, and M. Preuss,
editors, Experimental Methods for the Analysis of Optimization
Algorithms, pp. 103–130. Springer, Berlin, Germany, 2010.
[ bib ]

[1832]

Viviane Grunert da Fonseca, Carlos M. Fonseca, and Andreia O. Hall.
Inferential Performance Assessment of Stochastic Optimisers and
the Attainment Function.
In E. Zitzler, K. Deb, L. Thiele, C. A. Coello Coello, and
D. Corne, editors, Evolutionary Multicriterion Optimization, EMO 2001,
volume 1993 of Lecture Notes in Computer Science, pp. 213–225.
Springer, Heidelberg, 2001.
[ bib 
DOI ]
The performance of stochastic optimisers can be assessed
experimentally on given problems by performing multiple
optimisation runs, and analysing the results. Since an
optimiser may be viewed as an estimator for the (Pareto)
minimum of a (vector) function, stochastic optimiser
performance is discussed in the light of the criteria
applicable to more usual statistical
estimators. Multiobjective optimisers are shown to deviate
considerably from standard point estimators, and to require
special statistical methodology. The attainment function is
formulated, and related results from random closedset theory
are presented, which cast the attainment function as a
meanlike measure for the outcomes of multiobjective
optimisers. Finally, a covariancemeasure is defined, which
should bring additional insight into the stochastic behaviour
of multiobjective optimisers. Computational issues and
directions for further work are discussed at the end of the
paper.
Proposed looking at anytime behavior as a multiobjective
problem
Keywords: EAF

[1833]

C. Guéret, Nicolas Monmarché, and M. Slimane.
Ants Can Play Music.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pp. 310–317. Springer, Heidelberg,
2004.
[ bib ]

[1834]

M. Guntsch and Jürgen Branke.
New Ideas for Applying Ant Colony Optimization to the
Probabilistic TSP.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2003, volume 2611 of Lecture
Notes in Computer Science, pp. 165–175, Heidelberg, 2003. Springer.
[ bib ]

[1835]

M. Guntsch and Martin Middendorf.
Pheromone Modification Strategies for Ant Algorithms Applied to
Dynamic TSP.
In E. J. W. Boers et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2001, volume 2037 of Lecture
Notes in Computer Science, pp. 213–222. Springer, Heidelberg, 2001.
[ bib ]

[1836]

M. Guntsch and Martin Middendorf.
A Population Based Approach for ACO.
In S. Cagnoni et al., editors, Applications of Evolutionary
Computing, Proceedings of EvoWorkshops 2002, volume 2279 of Lecture
Notes in Computer Science, pp. 71–80, Heidelberg, 2002. Springer.
[ bib ]

[1837]

M. Guntsch and Martin Middendorf.
Solving MultiObjective Permutation Problems with Population
Based ACO.
In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele,
editors, Evolutionary Multicriterion Optimization, EMO 2003, volume
2632 of Lecture Notes in Computer Science, pp. 464–478. Springer,
Heidelberg, 2003.
[ bib ]

[1838]

M. Guntsch and Martin Middendorf.
Applying Population Based ACO to Dynamic Optimization
Problems.
In M. Dorigo et al., editors, Ant Algorithms, Third
International Workshop, ANTS 2002, volume 2463 of Lecture Notes in
Computer Science, pp. 111–122. Springer, Heidelberg, 2002.
[ bib ]

[1839]

Gurobi.
Gurobi Optimizer.
http://www.gurobi.com/products/gurobioptimizer, 2017.
[ bib ]

[1840]

D. Gusfield.
Algorithms on Strings, Trees, and Sequences.
In Computer Science and Computational Biology. Cambridge
University Press, 1997.
[ bib ]

[1841]

Walter J. Gutjahr.
SACO: An AntBased Approach to Combinatorial Optimization
Under Uncertainty.
In M. Dorigo et al., editors, Ant Colony Optimization and Swarm
Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of
Lecture Notes in Computer Science, pp. 238–249. Springer, Heidelberg,
2004.
[ bib ]

[1842]

Walter J. Gutjahr.
A converging ACO algorithm for stochastic combinatorial
optimization.
In A. Albrecht and K. Steinhöfel, editors, Stochastic
Algorithms: Foundations and Applications, volume 2827 of Lecture Notes
in Computer Science, pp. 10–25. Springer Verlag, 2003.
[ bib 
DOI ]

[1843]

Evert Haasdijk, Arif Attaul Qayyum, and Agoston E. Eiben.
Racing to improve online, onboard evolutionary robotics.
In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 187–194.
ACM Press, New York, NY, 2011.
[ bib ]

[1844]

S. Häckel, M. Fischer, D. Zechel, and T. Teich.
A multiobjective ant colony approach for Paretooptimization
using dynamic programming.
In C. Ryan, editor, Proceedings of the Genetic and Evolutionary
Computation Conference, GECCO 2008, pp. 33–40. ACM Press, New York, NY,
2008.
[ bib ]

[1845]

David Hadka, Patrick M. Reed, and T. W. Simpson.
Diagnostic assessment of the Borg MOEA for manyobjective
product family design problems.
In Proceedings of the 2012 Congress on Evolutionary Computation
(CEC 2012), pp. 1–10, Piscataway, NJ, 2012. IEEE Press.
[ bib ]

[1846]

Apache Software Foundation.
Hadoop, 2008.
[ bib 
http ]

[1847]

Thomas M. Walski, Donald V. Chase, Dragan A. Savic, Walter Grayman, Stephen
Beckwith, and Edmundo Koelle.
Advanced Water Distribution Modeling and Management.
Haestad Methods, Inc., Haestad Press, 1st edition, 2003.
[ bib ]

[1848]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
On the impact of the cutoff time on the performance of algorithm
configurators.
In M. LópezIbáñez, A. Auger, and T. Stützle,
editors, Proceedings of the Genetic and Evolutionary Computation
Conference, GECCO 2019, pp. 907–915. ACM Press, New York, NY, 2019.
[ bib 
DOI 
epub ]
Keywords: theory, automatic configuration, capping

[1849]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
Fast Perturbative Algorithm Configurators.
In T. Bäck, M. Preuss, A. Deutz, H. Wang, C. Doerr, M. T. M.
Emmerich, and H. Trautmann, editors, Parallel Problem Solving from
Nature – PPSN XVI, volume 12269 of Lecture Notes in Computer
Science, pp. 19–32. Springer, Cham, Switzerland, 2020.
[ bib 
DOI ]

[1850]

George T. Hall, Pietro S. Oliveto, and Dirk Sudholt.
Analysis of the performance of algorithm configurators for
search heuristics with global mutation operators.
In C. A. Coello Coello, editor, Proceedings of the Genetic and
Evolutionary Computation Conference, GECCO 2020, pp. 823–831. ACM Press,
New York, NY, 2020.
[ bib 
DOI 
epub ]

[1851]

Greg Hamerly and Charles Elkan.
Learning the k in kmeans.
In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in
Neural Information Processing Systems (NIPS 16). MIT Press, 2003.
[ bib 
epub ]

[1852]

Hayfa Hammami and Thomas Stützle.
A Computational Study of Neighborhood Operators for JobShop
Scheduling Problems with Regular Objectives.
In B. Hu and M. LópezIbáñez, editors, Proceedings
of EvoCOP 2017 – 17th European Conference on Evolutionary Computation in
Combinatorial Optimization, volume 10197 of Lecture Notes in Computer
Science, pp. 1–17. Springer, Heidelberg, 2017.
[ bib 
DOI ]

[1853]

Michael Pilegaard Hansen.
Tabu search for multiobjective optimization: MOTS.
In J. Climaco, editor, Proceedings of the 13th International
Conference on Multiple Criteria Decision Making (MCDM'97), pp. 574–586.
Springer Verlag, 1997.
[ bib ]

[1854]

Nikolaus Hansen, Anne Auger, S. Finck, and R. Ros.
RealParameter BlackBox Optimization Benchmarking 2009:
Experimental setup.
Technical Report RR6828, INRIA, France, 2009.
[ bib 
supplementary material ]

[1855]

Nikolaus Hansen, Anne Auger, Raymond Ros, Steffen Finck, and Petr
Pošík.
Comparing Results of 31 Algorithms from the BlackBox
Optimization Benchmarking BBOB2009.
In M. Pelikan and J. Branke, editors, Proceedings of the Genetic
and Evolutionary Computation Conference, GECCO Companion 2010, pp.
1689–1696. ACM Press, New York, NY, 2010.
[ bib 
DOI ]
This paper presents results of the BBOB2009 benchmarking of
31 search algorithms on 24 noiseless functions in a blackbox
optimization scenario in continuous domain. The runtime of
the algorithms, measured in number of function evaluations,
is investigated and a connection between a single convergence
graph and the runtime distribution is uncovered. Performance
is investigated for different dimensions up to 40D, for
different target precision values, and in different subgroups
of functions. Searching in larger dimension and multimodal
functions appears to be more difficult. The choice of the
best algorithm also depends remarkably on the available
budget of function evaluations.
Keywords: benchmarking, blackbox optimization

[1856]

Nikolaus Hansen, Steffen Finck, Raymond Ros, and Anne Auger.
RealParameter BlackBox Optimization Benchmarking 2009:
Noiseless Functions Definitions.
Technical Report RR6829, INRIA, France, 2009.
Updated February 2010.
[ bib 
epub ]
http://coco.gforge.inria.fr/bbob2012downloads

[1857]

Michael Pilegaard Hansen and Andrzej Jaszkiewicz.
Evaluating the quality of approximations to the nondominated
set.
Technical Report IMMREP19987, Institute of Mathematical Modelling,
Technical University of Denmark, Lyngby, Denmark, 1998.
[ bib ]
Proposed R2 indicator

[1858]

Julia Handl and Joshua D. Knowles.
Modes of Pr