Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf ·...

211
Evolutionary Algorithms and Dynamic Optimization Problems Von der Fakult¨ at Informatik, Elektrotechnik und Informationstechnik der Universit¨ at Stuttgart zur Erlangung der W ¨ urde eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigte Abhandlung Vorgelegt von Karsten Weicker aus Ludwigsburg Hauptberichter: Prof. Dr. V. Claus Mitberichter: Prof. Dr. H. Schmeck Prof. Dr. K. De Jong Tag der m¨ undlichen Pr ¨ ufung: 24.02.2003

Transcript of Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf ·...

Page 1: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

Evolutionary Algorithms andDynamic Optimization Problems

Von der Fakultat Informatik, Elektrotechnik undInformationstechnik der Universitat Stuttgartzur Erlangung der Wurde eines Doktors der

Naturwissenschaften (Dr. rer. nat.) genehmigteAbhandlung

Vorgelegt von

Karsten Weickeraus Ludwigsburg

Hauptberichter: Prof. Dr. V. ClausMitberichter: Prof. Dr. H. Schmeck

Prof. Dr. K. De Jong

Tag der mundlichen Prufung: 24.02.2003

Page 2: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

Institut fur Formale Methoden der Informatik derUniversitat Stuttgart

2003

2

Page 3: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

3

Page 4: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

Summary

This thesis examines evolutionary algorithms, a universal optimization method,applied to dynamic problems, i.e. the problems are changing during optimization.The thesis is motivated by a lack of foundations for the field and the incomparabilityof most publications that are of an empirical nature.

To establish a basis for the comparison of evolutionary algorithms applied to differ-ent time-dependent problems, a mathematical framework for the description of themajority of dynamic problems is introduced in the first part of the thesis. Withinthe framework, the dynamics of a problem are defined exactly for the changesbetween the discrete time steps. At one time step, the overall fitness function isdefined as the maximum of several static component functions at each point ofthe search space. Each component functions may be modified between the timesteps by stretching it with respect to the optimum, by rescaling the fitness, and bycoordinate transformations to relocate the optimum in the search space. The prop-erties of the modifications can be described mathematically within the framework.This leads to a classification of the considered dynamic problems. As a result ex-aminations on distinct problems can be integrated in an overall picture using theirsimilarities concerning the dynamics. On the one hand, this is used to create amapping between problem classes and special techniques used in dynamic envi-ronments. This mapping is based on an analysis of the literature in the field. It isa first step toward the identification of design patterns in evolutionary computingto support the development of evolutionary algorithms for new dynamic problems.On the other hand, the problem classes of the framework are used as basis for anexamination of performance measures within dynamic environments.

The second part of the thesis analyzes one specific technique, namely local vari-ation, applied to one problem class, namely tracking problems or drifting land-scapes, in detail. For this purpose, the optimization process of a (1, λ)-strategyapplied to a simple two-dimensional problem is modeled using a Markov chain.This enables the exact computation of the probability to be within a certain dis-tance to the optimum at any time step of the optimization process. By variationof the strength of the dynamics, the step width parameter of the mutation, cer-tain paradigms concerning the mutation operator, and the population size, findingsconcerning the optimal calibration of the optimization methods are derived. Thisleads to ten qualitative design rules for the application of local variation to track-ing problems. In particular statements are made concerning the choice of the stepwidth parameter, directed mutations, and mutations penalizing small steps. Goodsettings of the offspring population size are deduced by correlating fitness evalua-tions and the strength of the dynamics. Moreover, external memorizing techniques

4

Page 5: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

and self-adaptation mechanisms are considered. In order to exhaust the frontiersof the technique, an extreme case of dynamic problems is analyzed that is hard forcurrent self-adaptation techniques. This problem may serve as benchmark for thedevelopment of self-adaptation mechanisms tailored to dynamic problems. The de-sign rules are validated rudimentary on a small set of test functions using evolutionstrategies as optimization algorithm. Two new techniques to cope with the newbenchmark are proposed.

5

Page 6: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

Zusammenfassung

Diese Arbeit beschaftigt sich mit dem universell einsetzbaren Optimierungsver-fahren der evolutionaren Algorithmen angewandt auf so genannte dynamische Prob-leme, d.h. sich wahrend der Optimierung verandernde Probleme. Die Dissertationist im Wesentlichen durch einen Mangel an Grundlagen zu dieser Thematik sowievielen meist unvergleichbaren empirischen Arbeiten motiviert.

Um eine Vergleichsgrundlage fur evolutionare Algorithmen auf unterschiedlichenzeitabhangigen Problemen zu schaffen, wird im ersten Teil der Arbeit eine mathe-matische Beschreibung fur eine große Klasse von dynamischen Problemen eingefuhrt.Diese erlaubt die exakte Definition der Dynamik anhand von Veranderungen zwis-chen den diskreten Zeitschritten. Dabei wird von mehreren statischen Basisfunk-tionen ausgegangen, die uber Maximumsbildung zu einer Gesamtfunktion zusam-mengefuhrt werden. Die einzelnen Basisfunktionen konnen mittels Streckung,Fitnessskalierung und Koordinatentransformation modifiziert werden. Die Eigen-schaften dieser drei Modifikationsmoglichkeiten lassen sich wiederum mathema-tisch beschreiben und fuhren zu einer Klassifikation der dynamischen Probleme.Damit konnen empirische Untersuchungen, die unterschiedliche Probleme benutzen,bezuglich der Ahnlichkeit der zugrunde liegenden Dynamik eingeordnet werden.Auf der einen Seite dient dies der Erstellung einer Zuordnung von speziellen Tech-niken fur dynamische Probleme zu den Problemklassen. Eine erste solche Zuord-nung wird auf der Basis einer Analyse der Literatur zum Thema erstellt. Dies ist einerster Schritt zur Identifikation von Entwurfsmustern zur Unterstutzung der Losungzukunftiger dynamischer Probleme. Auf der anderen Seite werden die Problemk-lassen auch als Basis fur eine Untersuchung von Leistungsmaßen auf dynamischenProblemen herangezogen.

Im zweiten Teil der Arbeit wird eine spezielle Technik, die lokale Veranderung,auf einer Problemklasse, den so genannten Tracking-Problemen oder driftendenLandschaften, im Detail analysiert. Hierfur wird der Optimierungsprozess eineseinfachen zweidimensionalen Problems durch eine (1, λ)-Strategie mittels einerMarkov-Kette modelliert. Dies ermoglicht exakte Berechnungen der Wahrschein-lichkeiten, sich zu einem beliebigen Zeitschritt in einer bestimmten Entfernungzum Optimum zu befinden. Durch Variation der Starke der Dynamik, des Schrit-tweitenparameters der Mutation, bestimmter grundlegender Eigenschaften der Mu-tation sowie der Populationsgroße lassen sich Ergebnisse zur Einstellung des Op-timierungsverfahrens ableiten. Dies fuhrt zu zehn qualitativen Entwurfsregeln zurAnwendung von lokaler Variation auf Tracking-Probleme. Insbesondere werdendabei Aussagen zur Wahl des Schrittweitenparameters sowie zum Einfluss einergerichteten Mutation oder einer Mutation, die kleine Schrittweiten benachteiligt,

6

Page 7: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

gemacht. Sinnvolle Populationsgroßen lassen sich ableiten, indem die Anzahl derFitnessevaluationen pro Generation mit der Starke der Dynamik korreliert wird.Ebenso werden im Rahmen des zweiten Teils der Dissertation Techniken mit ex-ternem Erinnerungsvermogen sowie Selbstanpassungstechniken betrachtet. Umdie Grenzen der Technik auszuloten, wird ein Extremfall eines dynamischen Prob-lems betrachtet, der als zukunftiger Benchmark fur die Entwicklung von Selbstan-passungsmechanismen in dynamischen Problemen dienen kann. Die Entwurfs-regeln werden ansatzweise auf einer kleinen Menge an Testfunktionen fur die Evo-lutionsstrategie validiert. Zwei neue Techniken zur Bewaltigung des neuen Bench-marks werden dabei prasentiert.

7

Page 8: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,
Page 9: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

Contents

1 Introduction 13

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.2 Organization of this Thesis . . . . . . . . . . . . . . . . . . . . . 14

1.3 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Background and Related Work 17

2.1 Optimization problems . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.1 Static problems . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.2 Dynamic problems . . . . . . . . . . . . . . . . . . . . . 19

2.2 Evolutionary algorithms . . . . . . . . . . . . . . . . . . . . . . 22

2.2.1 Basic algorithm . . . . . . . . . . . . . . . . . . . . . . . 22

2.2.2 Paradigms . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.3 Dynamic Optimization . . . . . . . . . . . . . . . . . . . . . . . 31

3 Contribution and Methodology 37

3.1 Limitations of Previous Work . . . . . . . . . . . . . . . . . . . . 37

9

Page 10: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CONTENTS

3.2 Focus and Contribution of this Thesis . . . . . . . . . . . . . . . 38

3.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4 A Classification of Dynamic Problems 41

4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4.2 Existing Classifications . . . . . . . . . . . . . . . . . . . . . . . 43

4.3 Dynamic Problem Framework . . . . . . . . . . . . . . . . . . . 44

4.4 Problem Properties . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.4.1 Coordinate Transformations . . . . . . . . . . . . . . . . 51

4.4.2 Fitness Rescalings . . . . . . . . . . . . . . . . . . . . . 56

4.4.3 Stretching Factors . . . . . . . . . . . . . . . . . . . . . 61

4.4.4 Frequency of Changes . . . . . . . . . . . . . . . . . . . 62

4.4.5 Resulting Classification . . . . . . . . . . . . . . . . . . 63

4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5 Measuring Performance in Dynamic Environments 67

5.1 Goals of Dynamic Optimization . . . . . . . . . . . . . . . . . . 68

5.1.1 Optimization accuracy . . . . . . . . . . . . . . . . . . . 69

5.1.2 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

5.1.3 Reactivity . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.1.4 Technical aspects of adaptation . . . . . . . . . . . . . . 73

5.2 Performance Measures . . . . . . . . . . . . . . . . . . . . . . . 73

5.2.1 Measures for optimization accuracy . . . . . . . . . . . . 75

5.2.2 Measures for stability . . . . . . . . . . . . . . . . . . . . 78

5.2.3 Measures for reactivity . . . . . . . . . . . . . . . . . . . 79

5.2.4 Comparing algorithms . . . . . . . . . . . . . . . . . . . 79

5.3 Examination of Performance Measures . . . . . . . . . . . . . . . 80

5.3.1 Considered problems . . . . . . . . . . . . . . . . . . . . 80

5.3.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . 81

5.3.3 Statistical examination of the measures . . . . . . . . . . 81

10

Page 11: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CONTENTS

5.3.4 Discussion of the Results . . . . . . . . . . . . . . . . . . 84

5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6 Techniques for Dynamic Environments 87

6.1 Restarting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.2 Local variation . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.3 Memorizing previous solutions . . . . . . . . . . . . . . . . . . . 89

6.3.1 Explicit memory . . . . . . . . . . . . . . . . . . . . . . 90

6.3.2 Implicit memory . . . . . . . . . . . . . . . . . . . . . . 90

6.4 Preserving diversity . . . . . . . . . . . . . . . . . . . . . . . . . 91

6.4.1 Diversity increasing techniques . . . . . . . . . . . . . . 91

6.4.2 Niching techniques . . . . . . . . . . . . . . . . . . . . . 92

6.4.3 Restricted mating . . . . . . . . . . . . . . . . . . . . . . 93

6.5 Adaptive and self-adaptive techniques . . . . . . . . . . . . . . . 93

6.6 Algorithms with overlapping generations . . . . . . . . . . . . . . 94

6.7 Non-local encoding . . . . . . . . . . . . . . . . . . . . . . . . . 95

6.8 Learning of the underlying dynamics . . . . . . . . . . . . . . . . 98

6.9 Resulting Problem-Techniques Mapping . . . . . . . . . . . . . . 98

6.10 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

7 Analysis of Local Operators for Tracking 101

7.1 Theoretical framework . . . . . . . . . . . . . . . . . . . . . . . 103

7.1.1 Exact Markov chain model . . . . . . . . . . . . . . . . . 106

7.1.2 Worst-case Markov chain model . . . . . . . . . . . . . . 111

7.2 Feasible tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 124

7.3 Optimal parameter settings . . . . . . . . . . . . . . . . . . . . . 128

7.4 Non-zero-mean mutation . . . . . . . . . . . . . . . . . . . . . . 132

7.5 Proposition of bigger steps . . . . . . . . . . . . . . . . . . . . . 137

7.6 Dependence on the population size . . . . . . . . . . . . . . . . . 146

7.7 Memorizing techniques . . . . . . . . . . . . . . . . . . . . . . . 152

11

Page 12: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CONTENTS

7.8 Issues of adaptation and self-adaptation . . . . . . . . . . . . . . 156

7.8.1 Evaluation of the presented operators . . . . . . . . . . . 156

7.8.2 Limits of self-adaptation: uncentered tracking . . . . . . . 158

7.8.3 Alternative adaptation mechanisms . . . . . . . . . . . . 160

7.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

8 Four Case Studies Concerning the Design Rules 165

8.1 Adapting and self-adapting local operators . . . . . . . . . . . . . 166

8.1.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . 166

8.1.2 Limitations of local operators . . . . . . . . . . . . . . . 172

8.1.3 Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . 178

8.1.4 Self-adaptation . . . . . . . . . . . . . . . . . . . . . . . 178

8.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 180

8.2 Proposition of bigger steps . . . . . . . . . . . . . . . . . . . . . 181

8.3 Self-adaptation for the moving corridor . . . . . . . . . . . . . . 184

8.4 Building a model of the dynamics . . . . . . . . . . . . . . . . . 189

9 Conclusions and Future Work 195

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

12

Page 13: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 1

Introduction

1.1 Motivation

In nature, biological evolution is an adaptation process in an always changing envi-ronment. Changes occur from external reasons like climate changes or geophysicalcatastrophes as well as from evolution itself since all species, animals and plants,are coevolving and thus determining their common environment. Evolution of onespecies comes along with a change in the environment of all other species in itshabitat. Although natural evolution is always aiming at optimality, it will neverconverge in an equilibrium and reach optimality. The reasons are the changing en-vironment and a certain inertia of evolution. As a consequence natural evolution isalso not reversible.

Since the 1950s natural evolution has been modeled by engineers and scientists inorder to solve their problems (Reed, Toombs, & Barricelli, 1967; Fogel, Owens, &Walsh, 1965; Bremermann, 1962; Friedman, 1956; Friedberg, Dunham, & North,1959; Friedberg, 1958; Fraser, 1957; Box, 1957). The emerging evolutionary al-gorithms (EAs) have been developed in four different streams, genetic algorithms(GA, Holland, 1975, 1992; Goldberg, 1989), evolutionary programming (EP, Fo-gel, Owens, & Walsh, 1966; Fogel, 1995), evolution strategies (ES, Rechenberg,

13

Page 14: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

1. INTRODUCTION

1973, 1994; Schwefel, 1977, 1995), and genetic programming (GP, Koza, 1992a).In the case of GAs the initial idea was to control processes by adaptive reproductiveplans (cf. Section 3.5 in Holland, 1992; De Jong, 1993) which comes very close tothe idea of natural evolution. However, most research restricted its focus on opti-mization in static environments. As a consequence, the standard algorithms are notdesigned for non-stationary problems and only applicable with certain difficulty.

Lately an increase in the popularity of dynamic optimization can be observed re-flected in the number of research papers presented at conferences and workshops.Even real-world applications of evolutionary algorithms on dynamic problems (e.g.Vavak, Jukes, & Fogarty, 1997; Rana-Stevens, Lubin, & Montana, 2000) are aris-ing, paving the way for a huge number of potential applications.

Numerous special techniques have been proposed to tackle dynamic problems moreeffectively. However, there are many diverse non-stationary test problems with dif-ferent characteristics such that comparisons among results or generalizing conclu-sions are almost impossible. Also, we notice a remarkable absence of theoreticalfoundations for the classification of problems, for the comparison of different al-gorithms, and for the development and explanation of techniques. Nevertheless,the author of this thesis believes that such a foundation is necessary to drive theemerging field of evolutionary dynamic optimization to fruition.

1.2 Organization of this Thesis

Chapter 2 provides a short introduction to stationary and dynamic optimizationproblems, evolutionary algorithms in general, and an overview of the issues andthe major results in dynamic optimization.

Chapter 3 shows how this thesis fits into this picture. It discusses the most urgentopen problems and the contribution of the thesis.

A more general and formal discussion of dynamic function optimization problemsfollows in Chapter 4. Here, properties of these problems are defined within a math-ematical framework which enables a profound problem classification.

In Chapter 5, different aspects concerning the goals of optimization in dynamicenvironments are discussed. Various performance measures are reviewed and newmeasures are proposed. They are examined on four problem classes of the intro-duced framework.

Chapter 6 reviews the major publications in dynamic optimization and categorizesthe empirical investigations concerning the problems as well as the techniques used.

14

Page 15: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

1.3. ACKNOWLEDGEMENTS

The remainder of the thesis focuses on a special class of problems, namely driftingproblems. The performance of local operators on these problems is investigated indetail in a formal, theoretical investigation in Chapter 7.

The findings of Chapter 7, a set of design rules, are applied to a few scenarios inChapter 8.

Eventually, Chapter 9 concludes the work with a summary and discussion.

1.3 Acknowledgements

First of all, I would like to thank my advisor, Prof. Dr. Volker Claus, who gaveme the opportunity to work in his group and to develop this thesis. It was always aprivilege to experience the freedom of following my own ideas in research as wellas in teaching. He also made the integration of both research and family possible.

Very special thanks go to the co-referees Prof. Dr. Hartmut Schmeck and Prof.Kenneth A. De Jong Ph.D. for reviewing my thesis and for many helpful com-ments. I am very grateful for the discussions with Prof. De Jong, Dr. ChristopherRonnewinkel, and Dr. Jurgen Branke. They all contributed valuable impulses tomy work. This is even more true for the perpetual discussions with my wife Dr.Nicole Weicker. Also I appreciate the support of all members of our group “FormalConcepts” at the University of Stuttgart.

Moreover, I want to thank Prof. Eliot Moss Ph.D. at the University of Mas-sachusetts and Prof. Dr. Andreas Zell at the University of Tubingen. Both hadan impact on my way of doing research and it was a pleasure to work with them.

I am grateful to the staff of Schloss Dagstuhl who enabled me to stay one weekat Dagstuhl to have unimpaired attention to promote my research. Large parts ofChapter 7 are due to this research stay.

Finally, I want to thank my family—Nicole for her support and advice and thechildren for bearing their busy Dad. I also owe thanks to my parents who havesupported me such a long time and gave me the opportunity to become what I am.And last, I am grateful to my brother Norbert whose “technical support” was oftena big help when I was focusing on my research.

15

Page 16: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,
Page 17: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 2

Background and Related Work

This chapter summarizes the relevant definitions and results that are necessary tounderstand and to rate the topic of this thesis. Section 2.1 focuses on optimizationproblems, Section 2.2 is concerned with evolutionary algorithms, and Section 2.3gives the background information on optimization in non-stationary environments.

2.1 Optimization problems

This section gives a formal definition of optimization problems and, then, advancesto the special class of dynamic or non-stationary problems.

2.1.1 Static problems

Optimization problems occur in all areas of industry, research, and management.To illustrate the term “optimization” within these areas, a few typical short scenar-ios are presented. A first example is an improved utilization of existing resources:in job shop scheduling the through-put of a factory or a set of machines is increased,or the task of flight crew assignment reduces the number of idle flights of airplane

17

Page 18: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

crew members. A second example is the design of technical objects, e.g. the eco-nomical construction of spatial structures like bridges or cranes, the improvementof nozzles, or the optimization of hardware circuits. Another optimization task isthe mere calibration of parameters for existing processes—a technical example isthe optimization of electronic control units for combustion engines. A last exam-ple is the search for biochemical lead structures in the context of pharmaceuticaldrug design, which differs from the previous scenarios in the fact that any feasiblestructure must be found contrary to the improvement of existing solutions. Each ofthose problems has different characteristics which must be taken into considerationby a suitable optimization algorithm.

The following definition formalizes an optimization problem by introducing thepossible candidate solutions as the search space Ω and a quality value for eachcandidate solution enabling an assessment of the candidate solution’s quality and acomparison of different candidate solutions.

Definition 2.1 (Optimization problem) An optimization problem is defined by theclosed search space Ω, a quality function f : Ω → R, and a comparison relation“ ” ⊂ <,>.The task is to determine the set of global optima X ⊆ Ω defined as

X = x ∈ Ω | ∀x′∈Ωf(x) f(x′) . ♦

Note, that often the identification of one solution x ∈ X suffices for a successfuloptimization. Moreover, in real-world applications, any improvement of the qualityover the best candidate solution known so far is usually already a success. As aconsequence, the detection of the exact global optimum should usually be replacedby an approximation, i.e. the task is to find a candidate solution x∗ with f(x∗) asclose as possible to f(x) (x ∈ X ).

A very simple example to illustrate the definition is the sphere function.

Example 2.1 (Sphere function) The search space is defined as

Ω = [−5.12, 5.12]× · · · × [−5.12, 5.12] ⊂ Rk.

The quality function is f(x1, . . . , xk) =∑k

i=1 x2i .

It is a minimization problem, i.e. “” ≡ “<”. The set of global optima followsimmediately as X = ~0. This problem is unimodal which means that there existsonly one local and global optimum. The problem is also separable since the qualityfunction can be expressed as a sum of terms where each term depends on only oneobject variable xi. ♦

18

Page 19: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.1. OPTIMIZATION PROBLEMS

Note, that this general definition of optimization problems includes problems ofvarying difficulty. Toy problems like Example 2.1 are usually used as benchmarkproblems to compare various general optimization methods or to analyze EA the-oretically. In the context of evolutionary algorithms rather uninteresting problemsare those where deterministic, polynomial time algorithms exist, e.g. minimumspanning trees, shortest paths, or maximum flow in graphs. More relevant forreal-world applications are the difficult problems, i.e. those problems which areNP-hard. The complexity class NP contains all problems which may be solved bya non-deterministic Turing machine in polynomial time. A problem X is calledNP-hard iff any problem Y in the class NP may be reduced, i.e. rephrased, to an in-stance of X using a polynomial time algorithm and the problem Y may be solvedby solving X . If X is furthermore an instance of NP the problem is called NP-complete. A popular NP-complete problem is the traveling salesperson problem(Papadimitriou, 1977).

2.1.2 Dynamic problems

Static problems have been the focus of evolutionary algorithm research for almost20 years, and their solution proved to be of considerable difficulty. However, theintroduction of time dependency by Goldberg and Smith (1987) adds a new andvery distinct degree of difficulty. Before these issues are discussed in more detail,the non-stationary version of an optimization problem is defined formally.

Definition 2.2 (Dynamic optimization problem) A dynamic optimization prob-lem is defined by the search space Ω, a set of quality functions f (t) : Ω → R(t ∈ N0), and a comparison relation “ ” ∈ <,>.The goal is to determine the set of all global optima X (t) ⊆ Ω (t ∈ N0) defined as

X (t) =x ∈ Ω | ∀x′∈Ωf

(t)(x) f (t)(x′). ♦

In dynamic optimization, a complete solution of the problem at each time step isusually unrealistic or infeasible. As a consequence, the search for exact globaloptima must be replaced again by the search for acceptable approximations.

Note, for the sake of simplicity, in all examinations of this thesis only problemswith one global optimum are considered.

Example 2.2 (Dynamic sphere function) Again, the search space is defined as

Ω = [−5.12, 5.12]× · · · × [−5.12, 5.12] ⊂ Rk.

19

Page 20: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

The sequence of quality functions is given by

f (t)(x1, . . . , xk) =k∑i=1

(xi − y(t)i )2.

with y(t)i = 5sin(ρt) for t ∈ N0, a scaling factor ρ ∈ R, and 1 ≤ i ≤ k. It

is a minimization problem, i.e. “” ≡ “<”. The set of global optima followsas X (t) = (5sin(ρt), . . . , 5sin(ρt). Depending on the factor ρ, changes to theproblem from t to t+ 1 are slight or significant. This leads to a high variance in thedegree of difficulty of the problem. ♦

The time dimension has a completely different impact on the characteristics of aproblem than any other dimension of the search space. As long as changes in thelandscape occur only occasionally the problem can be optimized as it were staticin the periods between those changes. However, as soon as the changes occur moreoften or even continuously, there is only a very restricted time span available todeliver a new approximation for the problem and, as a consequence, only a lim-ited number of quality evaluations must suffice. This is a completely different taskcompared to static optimization problems. In fact, incorporating dynamics can turna very simple unimodal problem like the sphere in Example 2.1 into such a com-plex task that standard algorithms cannot cope with the problem. This underlinesthe need to develop a foundation for the application of evolutionary algorithms todynamic environments.

In the literature, the dynamics inherent in the problem are often referred to as ex-ogenous dynamics in contrast to endogenous dynamics stemming from the dynam-ics in the evolutionary optimizer itself. Examples for the latter are changes due tocoevolutionary methods or effects in small finite populations.

Two real-world examples for dynamic problems are the classical tasks of time se-ries prediction (in the context of evolutionary computation tackled by Fogel et al.,1966; Angeline, Fogel, & Fogel, 1996; Angeline & Fogel, 1997; Angeline, 1998;Neubauer, 1997) and control problems like the stabilization of a pole-cart system(e.g. solved with means of evolutionary computation by Odetayo & McGregor,1989). Another application of increasing importance is dynamic scheduling, e.g.in a job shop scheduling problem, where a number of jobs has to be assigned to asetup of machines guaranteeing a high through-put and reacting flexibly on newlyarriving jobs. Examples tackled by evolutionary computation can be found in thepublications of Biegel and Davern (1990); Bierwirth, Kopfer, Mattfeld, and Rixen(1995); Bierwirth and Mattfeld (1999); Branke and Mattfeld (2000); Fang, Ross,and Corne (1993); Hart and Ross (1998); Lin, Goodman, and Punch III (1997);

20

Page 21: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.1. OPTIMIZATION PROBLEMS

Mattfeld and Bierwirth (1999); Cartwright and Tuson (1994) and Rixen, Bierwirth,and Kopfer (1995). Other real world problems that have been tackled successfullyby evolutionary computation are combustion balancing in multiple burner boiler(Vavak et al., 1997), load balancing (Munetomo, Takai, & Sato, 1996), digital sig-nal processing (Neubauer, 1996), reduction of air traffic congestion (Oussedik, De-lahaye, & Schoenauer, 1999), speech recognition (Spalanzani & Kabre, 1998), andparameter identification (Fogarty, Vavak, & Cheng, 1995). However, those realworld problems are often difficult to examine and to analyze. Very often they re-quire also rather special optimization techniques and are, therefore, inadequate toget insights into the usage of a general problem solver to tackle dynamic problems.

As a consequence, artificial non-stationary problems are usually the basis for theexamination of general dynamic optimization techniques. The most simple way tocreate an artificial dynamic problem is to take a stationary problem function andto move it like in Example 2.2. Frequently, very simple unimodal problems areused because the focus of the difficulty is one the introduced dynamics. In theliterature there are examples for a linear movement (see Vavak, Fogarty, & Jukes,1996a, 1996b; Salomon & Eggenberger, 1997; Ryan & Collins, 1998), movementaccording to a sine function (see Cobb, 1990; Dasgupta, 1995), or along a cyclictrace in the search space (see Angeline, 1997; Back, 1998). Often, there are alsocertain stationary periods involved.

Instead of a continuous movement, random relocation of the stationary problemis considered by Angeline (1997) and Back (1997, 1998). Collard, Escazut, andGaspar (1996, 1997) relocated a multi-dimensional Gaussian function according toan enumerating function.

Where shifting and relocating introduces a special kind of dynamics which is ap-plied similarly to each point of the stationary problem, more complex dynamicscan be introduced by the moving hills technique. Here several unimodal prob-lems (hills) are placed in the search space which can individually change theirheight (e.g. Cedeno & Vemuri, 1997; Liles & De Jong, 1999; Trojanowski &Michalewicz, 1999b), the position of the maximum peak or all peaks (e.g. Grefen-stette, 1992; Cobb & Grefenstette, 1993; Vavak et al., 1997; Vavak, Jukes, &Fogarty, 1998; Sarma & De Jong, 1999; Smith & Vavak, 1999), or the shape ofthe peaks. Morrison and De Jong (1999), Grefenstette (1999), and Branke (1999c)proposed different moving hills test problem generators comprising most describedchanges. These or similar problem generators have been used by Kirley and Green(2000), Morrison and De Jong (2000), Saleem and Reynolds (2000), and Ursem(2000).

21

Page 22: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

Besides the described problems in a real-valued search space, there is also a wideset of artificial dynamic problems using binary search spaces. One popular ap-proach are the dynamic match functions (or pattern tracking) where a binary stringis given and the fitness of each individual is measured as the number of commonbits with the bit string. The most simple version of this kind of problem is the time-varying counting ones function in which alternating the number of ones and zerosare maximized (cf. Back, 1997, 1999). In a more general version of the problemall g generations d bits in a target string are changed (cf. Vavak & Fogarty, 1996;Collard et al., 1997; Escazut & Collard, 1997; Gaspar & Collard, 1997, 1999a,1999b; Stanhope & Daida, 1998, 1999). In a very early version of the problem,Pettit and Swigger (1983) changed each bit according to an individual stochastictransition table.

Similarly to relocation according to an enumerating function in real-valued searchspaces, Collard et al. (1996, 1997) applied the same scheme to a two-bit needle-in-a-haystack function.

Probably the most popular binary dynamic test function is the time-varying knap-sack problem where the size of the knapsack or the weight of the items changesover time. As an extreme, the dynamic problem can alternate between completelydifferent knapsack instances. For various dynamic knapsack problems refer to thework of Dasgupta and McGregor (1992), Goldberg and Smith (1987), Hadad andEick (1997), Lewis, Hart, and Ritchie (1998), Mori, Kita, and Nishikawa (1996,1998), Ng and Wong (1995), Ryan (1996, 1997), Smith and Goldberg (1992), andSmith and Vavak (1999).

2.2 Evolutionary algorithms

This section outlines a general generic evolutionary algorithm and, very briefly,presents the four major paradigms, namely genetic algorithms, evolution strategies,evolutionary programming, and genetic programming.

2.2.1 Basic algorithm

Figure 2.1 shows the general evolutionary cycle which is the generic basis for allevolutionary algorithms (EA).

The fundamental idea of evolutionary computing is the mimicry of natural evolu-tion: an initial multi-set of candidate solutions undergoes a process of simulatedevolution. That means that candidate solutions are able to reproduce themselves

22

Page 23: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.2. EVOLUTIONARY ALGORITHMS

yesno

Outputresult

selection

mutationevaluation

environmentalselection

terminationcriterion

parental

recombination

initialization

evaluation

Figure 2.1 Schematic description of the generation loop in evolutionary algo-rithms.

and are subject to an additional selection pressure. Following the biological termi-nology, a candidate solution is referred to as individual and a multi-set (or a tuple)of individuals is called a population. Usually populations in EAs are of a fixed sizecontrary to the varying population sizes in nature, where changes in the populationsize are one means of direct response to changing environmental conditions.

In the evolutionary cycle in Figure 2.1 parental and environmental selection, re-combination, and mutation are clearly biologically inspired components where theinitialization, the direct evaluation of individuals, and the termination criteria areadditional elements, necessary for the use of evolution as an optimization method.The components are discussed in more detail in the next paragraphs. One passthrough the evolutionary cycle is called a generation.

In the initialization a first population of individuals is created. Usually, those indi-viduals are chosen at random – under certain circumstances also concrete individ-uals, e.g. known good candidate solutions, are included in the initial population.

Since individuals in simulated evolution do not live in a real environment strugglingfor survival, the quality function of the optimization problem replaces the interac-tion of individuals with the environment: it establishes a means of comparison ofindividuals to guide the evolutionary search process. In the context of evolutionaryoptimization we refer to the quality of an individual as fitness.

23

Page 24: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

In order to create new individuals it is necessary to select parents from the currentpopulation. This selection and the assignment of a number of offspring to eachparent is one of two possible positions in the evolutionary cycle where selectivepressure may take place. No selective pressure can be obtained in the parentalselection by a uniform random selection of parents.

Between the selected parents a recombination takes place and at least one offspringis created by the combination of the genetic material of the parents where the term“combination” should be understood in a wider sense. Offspring inherit certaintraits of their parents—however often also completely new traits may be created.Note, that there are evolutionary algorithms that do not use recombination.

Also in analogy to natural evolution an error rate in the process of reproductionis considered within the mutation operators that are applied when generating off-spring individuals. Usually only rather small changes should be added to individu-als since evolutionary progress relies on the inheritance of parental traits.

Those recombined and mutated individuals are evaluated using the quality func-tion to determine their fitness. The fitness of the individuals is the basis for theenvironmental selection where for each individual a decision is met whether it willsurvive and be a potential parent in the next iteration. There are two extreme casesof environmental selection, namely the steady state EA, where just one offspring iscreated and replaces an individual in the parental population, and the generationalEA, where the whole parental population is replaced by new individuals. In be-tween those extremes there exist many environmental selection strategies, e.g. byselecting the best individuals from the union of parents and offspring or by replac-ing more than one individual in the parental population. The latter case is usuallydetermined by a degree of overlap between the generations and a replacement strat-egy.

Contrary, to natural evolution at the end of each cycle the termination criteria testswhether the goal of the optimization has been met already. Usually, an additionalmaximal number of generations is given such that the EA always halts.

The processing scheme of the general evolutionary algorithm is shown in Algo-rithm 2.1 in pseudo code. Parameters are the population size as well as the numberof offspring that have to be created each generation. Moreover a genotypic searchspace G must be determined together with a decoding function dec : G → Ωthat determines to which phenotypic candidate solution a genotype is mapped. Theinteraction between genotype and phenotype is shown in Figure 2.2. Ideally sucha mapping from genotype to phenotype is bijective. But due to the optimizationproblem or restrictions caused by the chosen evolutionary algorithm, there is oftenredundancy in the encoding, i.e. several genotypes are mapped to one phenotype,

24

Page 25: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.2. EVOLUTIONARY ALGORITHMS

Algorithm 2.1 General evolutionary algorithm EA

1: INPUTS: certain parameter µ, λ, . . . , quality function f : Ω→ R2: PARAMETERS: population size µ, number of offspring λ, genotype G, de-

coding function dec3: t← 04: P (t)← create a population of size µ5: evaluate individuals in P (t) using dec and f6: while termination criteria not fulfilled do7: E ← select parents for λ offspring from P (t)8: P ′ ← create offspring by recombination of individuals in E9: P ′′ ← mutate individuals in P ′

10: evaluate individuals in P ′′ using dec and f11: t← t+ 112: P (t)← select µ individuals from P ′′ (and P (t− 1))13: end while14: OUTPUT: best individual in P (t)

or the mapping is not surjective, i.e. there are phenotypes which are not encodedby any genotype. Whenever the decoding function is not important, it is omittedand the induced quality function is used instead.

phenotypicsearch space Ω

quality functionIR+

f

inducedquality function

F = f decG ⊆ G1 × . . .×Glgenotype

phenotype

decodingdec

Figure 2.2 Genotypic coding of the search space.

For each application of the evolutionary algorithm displayed in Algorithm 2.1, theoperators must be chosen in accordance with the respective problem. The operatorsrecombination, mutation, and parental and environmental selection must conformto the following definition.

Definition 2.3 (Operators) Given a gentoypic search space G, a mutation operatoris defined by the function

M ξ : G → G

25

Page 26: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

where ξ ∈ Ξ represents a random number (or several random numbers).

Analogously, the recombination operator is defined for r ≥ 2 parents and s ≥ 1offspring by the function (r, s ∈ N)

Rξ : Gr → Gs.

A selection operator on a population P = 〈A1, . . . , Ar〉 with Ai ∈ G selectings ∈ N individuals is defined by the function

Sξ,dec,f : Gr → Gs

where for the resulting individuals Sξ,dec,f (P ) = 〈B1, . . . , Bs〉 it holds that Bi ∈set(P ) for 1 ≤ i ≤ s and set(P ) = Ai ∈ G | P = 〈A1, . . . , Ar〉, 1 ≤ i ≤ s.The function S can depend on a random number ξ ∈ Ξ, the genotype–phenotype–mapping dec, and the quality function f . ♦

2.2.2 Paradigms

In this subsection, we review those standard algorithms (and paradigms) that arerelevant in the remainder of this work, namely genetic algorithms, evolution strate-gies, and evolutionary programming. The fourth paradigm, genetic programming,is described superficially at the end of this section.

Genetic algorithms (GAs) have their roots in the early work of Holland (1969,1973), resulting in his book on adaptive systems (Holland, 1975) where GAs arecalled reproductive plans. The use of genetic algorithms as optimization tools wasestablished by his students (e.g. De Jong, 1975). Today’s popularity of GAs isprimarily due to the book by Goldberg (1989).

The canonical GA is outlined in Algorithm 2.2. The historical form is character-ized by a binary genotypic search space G = Bl and a coding function mappingthis space to the phenotypic search space, e.g. Ω = Rn. For the encoding ofeach phenotypic dimension usually a standard binary encoding or a Gray code isused (Caruana & Schaffer, 1988). The operators to modify the individuals are a re-combinative crossover operator which exchanges certain bits of two parents and abit-flipping mutation where each bit in an individual is flipped with a certain proba-bility. In GAs there is a high emphasis on the recombination which is applied witha rather high probability. The mutation has the role of a background operator witha very low bit-flipping probability. The selective pressure is created by parentalselection only, using fitness proportional selection, i.e. selection is probabilistic

26

Page 27: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.2. EVOLUTIONARY ALGORITHMS

Algorithm 2.2 Genetic algorithm GA

1: INPUTS: quality function f2: PARAMETERS: population size µ, decoding function dec : Bl → Ω,

mutation rate pm, crossover probability px3: t← 04: P (t)← create initial population of size µ5: evaluate individuals in P (t) (using dec and f )6: while termination criteria not fulfilled do7: P ′ ← select µ parents from P (t) with a probability proportional to the indi-

viduals quality8: P ′′ ← recombine the parents in P ′ using the crossover operator (each appli-

cation results in two new individuals; crossover takes place with probabilitypx—the individuals are copied otherwise)

9: P ′′′ ← apply the mutation to each individual in P ′′ (each bit is flipped withprobability pm)

10: evaluate population P ′′′ (using dec and f )11: t← t+ 112: P (t)← P ′′′

13: end while14: OUTPUT: best individual in P (t)

where better individuals get a higher probability assigned. As a result, better indi-viduals are expected to have more offspring than worse individuals have. Since thenumber of created individuals matches the population size there is no environmen-tal selective pressure.

Three prominent crossover operators are the 1–point crossover, where one crossoverpoint in the individuals is chosen and the left part of the individuals is exchanged,the 2–point crossover, where two crossover points are chosen and the section be-tween those points is exchanged, and the uniform crossover, where each bit is cho-sen individually from one of both parents.

In the course of time, many modifications concerning the selection mechanism havebeen proposed. There are on the one hand certain scaling techniques which are ableto control the selective pressure more effectively. On the other hand there are alsovery distinct selection operators like the tournament selection where an individualis selected as the best of k uniformly chosen individuals.

In the 1980s, there have been also first GAs that use a genotypic search space dif-ferent from the binary space. Namely, real-valued search spaces and permutationsfor combinatorial optimization are used. For these algorithms the development of

27

Page 28: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

special operators was necessary. However, all those modifications are out of thescope of this work.

Evolution strategies (ES) have been developed by Rechenberg (1964) togetherwith Schwefel and Bienert for the design of a nozzle as well as a crooked pipe.Their first experiments have been executed manually.

Within evolution strategies the genotypic search space is always real-valued G = Rl

and usually equivalent to the phenotypic search space. The main operator in ES isthe mutation which generates modifications local to the genotypic search space byadding a Gaussian random number to each dimension of G. The standard deviationof the probability density function used in the mutation is an essential parameterfor success or failure of an optimization. The next paragraph is devoted to thecontrol of this parameter. If the standard deviation is identical for all search spacedimensions, the mutation is called isotropic. Originally there was no recombinationwhich was later introduced as secondary operator. Selective pressure is only gener-ated by an environmental selection – the parents are chosen with uniform probabil-ity. In a population of size µ, the environmental selective pressure is generated byincreasing the population size with newly created individuals and reducing it againby selecting only the µ best individuals. If the individuals are chosen from λ > µoffspring only, it is a comma–strategy; if the original population is expanded by λnew individuals, it is a plus–strategy.

The first attempt to control the mutation’s standard deviation σ has been the 1/5–success rule by Rechenberg (1973) where σ was diminished if the success rate isless than 1/5 and it is increased if the success rate, i.e. the percentage of offspringbetter than their parent, is greater than 1/5.

σ′ ←

ασ, iff ps > 1/51ασ, iff ps < 1/5

σ, iff ps = 1/5

Usually, α ≈ 1.224 is chosen. Where this rule works sufficiently in unimodalsearch spaces, it gets easily trapped in multimodal search spaces. A much morerobust adaptation mechanism was introduced by Schwefel (1975) with the conceptof self-adaptation. Here, each individual is extended by one (or more) additionalparameter(s) which contain the standard deviation(s) used for the creation of thisindividual. In case of the isotropic mutation, one strategy parameter is needed.This strategy parameter is first mutated and then used to create the next individual.The mutation is carried out for an individual A = 〈A1, . . . , An, σ〉 according to

28

Page 29: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.2. EVOLUTIONARY ALGORITHMS

following rules.

σ′ ← σ exp

(1√lN (0, 1)

)(2.1)

A′i ← Ai +N (0, σ′)

The rule for the strategy parameter is called the lognormal update rule. The pri-mary idea is that high quality individuals are due to good parameter values whichleads to the dominance of well-adapted standard deviations for each individual atits position in the search space. The resulting self-adaptive evolution strategy isdisplayed in Algorithm 2.3.

Algorithm 2.3 Self-adaptive evolution strategy ES

1: INPUTS: quality function f2: PARAMETERS: population size µ, number of offspring λ, recombination

rate pr3: t← 04: P (t)← create initial population of size µ5: evaluate P (t) using f6: while termination criteria not fulfilled do7: P ′ ← 〈〉8: for i ∈ 1, . . . , λ do9: A← select parent uniformly from P (t)

10: if U([0, 1]) < pr then11: B ← select mate uniformly from P (t)12: A← recombine A and B13: end if14: σ′ ← apply mutation on strategy parameter of A15: A′ = 〈A′1, . . . , A′n〉 ← apply mutation on 〈A1, . . . , An〉 using σ′

16: evaluate A′ using f17: P ′ ← P ′ 〈A′〉18: end for19: t← t+ 120: P (t)← select the µ best individuals from P ′ (or from P ′ P (t− 1))21: end while22: OUTPUT: best individual in P (t)

In the above adaptation scheme the same standard deviation is applied to all di-mensions of the search space. This can be changed by individual strategy pa-rameters for each search space dimension, i.e. the individual has the form A =

29

Page 30: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

〈A1, . . . , An, σ1, . . . , σn〉. The strategy parameters are adapted using the followingrule.

σ′i ← σi exp

(√

2lu+1√2√lN (0, 1)

)A′i ← Ai +N (0, σ′i)

where u ∼ N (0, 1) is chosen once for the adaptation of all strategy variables σi(1 ≤ i ≤ n). We refer to this mutation as (simple) non-isotropic mutation. The fac-tors√

2l and 1√2√l

are recommended by Schwefel (1977) as appropriate heuristic

settings.

If in addition the orientation in the search space should be adapted, 12n(n−1) strat-

egy variables are necessary to encode the direction in a n-dimensional search space.One of those techniques is the covariance matrix adaptation (cma) by Hansen andOstermeier (1996, 2001). It is a derandomized approach—that means that first arandom change is added to the object variables, then according to the successs ofthis change the underlying covariance matrix that encloses all strategy variables ischanged. So the mutation does not rely on good random changes of the strategyvariables.

Evolutionary programming (EP) was developed by Fogel et al. (1965, 1966)who evolved finite automata to predict time series. In the late 80s and beginning90s, Fogel (1992a, 1992b) extended the evolutionary programming paradigm toreal-valued search spaces as described in the next paragraph.

There is no selective pressure in the parental selection—each individual in the pop-ulation creates exactly one offspring using mutation. No recombination is used.The mutation is quite similar to the mutation in the evolution strategies. Even a self-adaptation mechanism was developed independently. The mutation and the usageof the strategy parameter σ differs for an individual A = 〈A1, . . . , An, σ1, . . . , σn〉and is described in the following equations.

σ′i ← maxσi +N (0,√sσi), ε (2.2)

A′i ← Ai +N (0, σi)

where parameter s controls the strength of adaptation and ε > 0 is a minimal re-quired standard deviation. As environmental selection each new individual andeach parent are compared to k randomly chosen individuals in the population. Thenumber of wins is assigned to each individual as score. The µ best scoring indi-viduals survive for the next generation. An outline of evolutionary programming isgiven in Algorithm 2.4.

30

Page 31: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.3. DYNAMIC OPTIMIZATION

Algorithm 2.4 Evolutionary programming EP

1: INPUTS: quality function f2: PARAMETERS: tournament size k, population size µ3: t← 04: P (t)← create population of size µ5: evaluate P (t) using f6: while termination criteria not fulfilled do7: P ′ ← mutate each individual in P (t − 1) according to the mutation given

above8: evaluate P ′ using f9: P ′′ ← P ′ P (t− 1)

10: for each individual A in P ′′ do11: Q← choose k individuals randomly from P ′′

12: determine score of A as the number of individuals B ∈ Q where f(A) f(B)

13: end for14: t← t+ 115: P (t)← µ individuals in P ′′ with best score16: end while17: OUTPUT: best individual in P (t)

Genetic programming (GP) was introduced by Koza (1989, 1992a, 1992b). Ini-tially the search space consisted of parse trees, e.g. containing Lisp S-expressions.Later, also genetic programming on graphs and other structures of varying size hasbeen developed. The basic algorithm is quite similar to the algorithm of genetic al-gorithms although the population size is usually very large compared to GAs. In theinitial tree representation, the mutation has been implemented as the replacementof subtrees by random trees, the recombination as exchange of subtrees amongindividuals, and the parental selection as tournament selection.

2.3 Dynamic Optimization

Optimization or adaptation in non-stationary environments is a fairly new researcharea from the perspective of a computer scientist or engineer. However, from abiologist’s viewpoint, adaptation in a changing environment is a common theme:coevolutionary interactions are examined since the 1960s where various speciesare responding to one another modifying their environments reciprocally. Thereis a rich literature on models and examinations (e.g. Futuyma & Slatkin, 1983).

31

Page 32: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

However, as Slatkin (1983) points out, those findings are always restricted by thesimplifications of the models. Also, these results are not transferable to the opti-mization of non-stationary problems in computer science or engineering since thereciprocal redefinition of the environment does not match the character of the dy-namic problems considered there. In dynamic optimization the dynamics are usu-ally exogenous and not affected by the optimization process itself. In the followingparagraphs a quick overview on the state of the art is given.

Related biological results: In recent years there are some theoretical publica-tions (e.g. Wilke, 1999) on exogenous dynamics in biological systems, using theEigen quasispecies model (Eigen, 1971). Primary results of this approach are theexamination of environmentally guided drifts where an adaptive walker can bedragged to a global minimum in a maximization task (Wilke, 1998, 1999). Also os-cillation frequencies are examined (Wilke, Ronnewinkel, & Martinetz, 1999, 2001;Wilke & Ronnewinkel, 2001). These results are related to dynamic optimization,although the relevant evolutionary algorithms are restricted to rather simple vari-ants of standard genetic algorithms.

Approaches to dynamic problems: When tackling problems with exogenousdynamics, there are basically two different evolutionary approaches. First, an evo-lutionary algorithm may be used to evolve a strategy, program, or automaton fortackling the problem. In this case, usually each created individual must be testedfor a certain amount of time in the dynamic environment to determine its fitnessvalue. Nevertheless, although the individual solves a dynamic task, the evolution-ary algorithm faces a more or less static problem since the dynamic task is identicalfor each evaluation. This approach, displayed in Figure 2.3, is also referred to asoffline optimization. It is used in many evolutionary programming (EP) and geneticprogramming (GP) applications, e.g. for time series prediction (Fogel et al., 1966;Angeline et al., 1996; Angeline & Fogel, 1997; Angeline, 1998) or the stabilizationof the pole-cart system (using a GA, Odetayo & McGregor, 1989; Thierens & Ver-cauteren, 1991). The task is the evolution of a strategy that is able to deal with therespective dynamic problem. In general, this is a difficult task where the solutionquality depends on the complexity and regularity of the dynamics and the internalrepresentation of the strategy.

Where offline optimization is completely distinct from the biological examinationsin the previous paragraph, the second approach is the exact analogue to the ex-ogenous biological considerations. Here, the dynamic problem is changing in-dependently while the evolutionary algorithm evolves solutions. This approach,

32

Page 33: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.3. DYNAMIC OPTIMIZATION

EvolutionaryAlgorithm

DynamicProblem Ind.

Tackles

Figure 2.3 Static evolution of a program to solve the dynamic problem.

displayed in Figure 2.4, can be used to tackle more unpredictable problems thanthe first approach. The second approach, also referred to as online optimization, isthe subject of this thesis. The following paragraphs summarize the state of the artconcerning online optimization.

DynamicProblem Ind.

EvolutionaryAlgorithmsolution

is

Figure 2.4 Dynamic evolution of solutions to the problem.

Theoretical results: There are only few theoretical results available and most ofthe results are closely related to the biological examinations. For example, Rowe(1999, 2001) also uses the Eigen quasispecies model. He computes the cyclic at-tractors in periodic (or oscillating) fitness functions. As a result he finds that thetheory conforms to experimental findings for high and modest mutation rates—only modest mutation enables a successful adaptation. For low mutation rates,however, the theory cannot be confirmed since no adaptation could be observed inexperiments.

Another theoretical result concerns the fixed point concentrations of a needle-in-a-haystack problem examined by Ronnewinkel, Wilke, and Martinetz (2000). Thiswork is an extension of the biological investigations of Wilke (1999). It enablesthe derivation of an optimal mutation rate for this simple problem. The work isextended for problems with bigger basins of attraction by Droste (2002).

Stanhope and Daida (1999) analyzed the dynamics of a genetic hillclimber. Theyobserve in their theoretical results as well as in empirical validations that smallperturbations have a huge effect on the optimizers performance. However, for afixed dynamic problem, an optimally chosen mutation rate affects the performance

33

Page 34: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

only little. The authors conclude that self-adaptation techniques might be of littleuse in dynamic problems—however their results are restricted to population size 1.These findings are also related to the results of Angeline (1997) who also concludedfrom experiments that self-adaptation is not suited for non-stationary problems. Asa response, Back (1998) showed that self-adaptation is useful for a small degree ofdynamics.

Evolution strategies with isotropic mutations have been the focus of Arnold andBeyer (2002) who analyze random drifting problems.

Not theoretical in a strict sense but still a fundamental conception is the compari-son of evolutionary algorithms to filters in signal processing by Hirst (1997). Heshowed that they act similar to low pass analogue filters in the continuous domainand band pass non-recursive digital filters in the discrete domain using a genera-tional evolutionary algorithm.

Comparison to learning techniques: In the context of dynamic optimization,evolutionary algorithms have been compared systematically to other learning tech-niques seldom. The early work by Pettit and Swigger (1983) showed in a simpleset-up that cognitive approaches lead to better results than a genetic approach.

The work by Littman and Ackley (1991) did not compare evolutionary computationto a learning algorithm but rather showed that adding a learning component canimprove the response time of the evolutionary algorithm considerably in a dynamic,artificial life–related scenario.

Modeling the dynamic environment: When tackling dynamic problems a nearbyidea is to use well-examined models of the environment, as they are used in manydynamic systems solutions. Since there are manifold applications within that cate-gory, only a partial overview is given here.

For control tasks there exists broad experience with fuzzy controllers. Also, thereis a symbiotic development of fuzzy and evolutionary computing during the pastyears (for an overview see Bonissone, 1997; Karr, 1997; Bersini, 1998). Evolu-tionary algorithms for modifying fuzzy controllers within a dynamic environmentare proposed by Karr (1991, 1999).

Another approach to model the environment is the usage of stochastic learning au-tomata (see Narendra & Thathachar, 1989). In the work of Munetomo et al. (1996)such a model is used: the genetic algorithm generates new actions in the automaton,actions are selected based on their fitness and applied to the environment, and thefitness is recomputed using a linear learning scheme based on the feedback of the

34

Page 35: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2.3. DYNAMIC OPTIMIZATION

environment. A related approach was presented by Papadimitriou and Pomportsis(2000) where a real valued mutation like in ES was used.

Only partially related is the work where evolutionary algorithms are used to supportmethods in control theory. For example, Fadali, Zhang, and Louis (1999) appliedgenetic algorithms to robust stability analysis in dynamic discrete-time systems.

However, all those approaches that use a model of the environment are restricted intheir application by the limits of the model. Especially the deterministic descriptionas linear or non-linear system is often not possible.

Technique oriented research: In the case of online optimization, standard algo-rithms reveal many deficiencies restricting their applicability. Generally speaking,the algorithms loose their adaptability: they are not able to track an optimum anymore or to react to changes like the advent of a new global optimum. One mainreason for this effect is the loss of diversity in the population. Convergence is an in-trinsic characteristic of evolutionary algorithms due to a big enough selective pres-sure. As a consequence, most of the research in the area of dynamic optimizationfocuses on the development of new mechanisms and techniques to avoid conver-gence, to keep a certain level of adaptability, and to create more powerful tailoredevolutionary algorithms. The work of Goldberg and Smith (1987) and Cobb (1990)are two early examples of this kind of research. An overview of the techniques iscontained in Chapter 6. For a more complete list refer to the doctoral thesis ofBranke (2002).

Application oriented research: Another research area with increasing impor-tance is the application of the techniques to real-world problems. Here, successfulapplications are flowshop scheduling (Cartwright & Tuson, 1994), rescheduling(Bierwirth & Mattfeld, 1999), air crew scheduling (Rana-Stevens et al., 2000),speech recognition (Spalanzani & Kabre, 1998), or scheduling of aircraft landings(Beasley, Krishnamoorthy, Sharaiha, & Abramson, 1995). However, as Mattfeldand Bierwirth (1999) argue in their workshop contribution, many of those real-world applications are different from the usually considered benchmark functionssince they replace the exogenous dynamics, an external goal modification, by inter-nal model changes where the representation of candidate solutions changes due tonewly arriving jobs in case of a scheduling problem. As a consequence, techniquesoriented research is only partially relevant to real-world applications.

Systematical examinations: As we have seen above, the theoretical results arevery sparse and consider usually only very simple algorithms and problems. There-

35

Page 36: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

2. BACKGROUND AND RELATED WORK

fore, the question arises how many systematical examinations of parameters maybe found in the literature.

Vavak and Fogarty (1996) and Smith and Vavak (1999) have examined the choicebetween generational and steady state algorithms and, in the latter article, the in-fluence of the replacement strategy in the steady state algorithm.

A systematical experimental investigation of good parameters for the dynamicmatching function (a dynamic version of the onemax problem) was done by Stanhopeand Daida (1998). Also, the doctoral thesis of Branke (2002) contains an extensiveempirical study of parameter choices for two different problems.

Only partially relevant for general dynamic optimization are the examinations ofHartley (1999), who considered the fitness computation in dynamic classificationtasks, and of Coker and Winter (1997), who investigated the optimal number ofparents in an artificial life simulation.

In the face of necessary empirical analyses concerning the usefulness of algorithmsand good parameter setups, the need for dynamic test function generators was rec-ognized. Recently three different generators have been proposed (Grefenstette,1999; Branke, 1999c; Morrison & De Jong, 1999).

Classification: In consideration of the variety of dynamic problems and algo-rithms, the definition of problem clusters and technique clusters is necessary. Be-sides very coarse approaches (e.g. De Jong, 2000), first more profound classifica-tion were proposed by Trojanowski and Michalewicz (1999a) and Branke (2002).

36

Page 37: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 3

Contribution and Methodology

This chapter summarizes the open problems in the field of dynamic optimization(Section 3.1) and clarifies which topics this thesis focuses on (Section 3.2). Fur-thermore, the methodology used to examine these topics is described in Section 3.3.

3.1 Limitations of Previous Work

The discussion in Section 2.3 gives an overview on the current state of dynamicoptimization. The shortcomings and open problems in the field are obvious. Thefollowing list gives an incomplete overview on those topics.

1. Besides very few theoretical examinations of simple algorithms and prob-lems, there is no theoretical or fundamental framework for both theoreticaland empirical work. Such a framework is necessary to enable a systematicresearch and integration of results. Most existing research is technique ori-ented leading to isolated empirical findings that are often not comparable dueto very distinct non-stationary problems.

2. The restriction of theoretical investigations on simple problems, which is truefor almost any analysis of evolutionary computing, is especially problematic

37

Page 38: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

3. CONTRIBUTION AND METHODOLOGY

in dynamic optimization since the characteristics of the dynamics may havean even stronger impact on the difficulty of the problem. As a consequencetheory for more complex problems is needed.

3. Also most theoretical approaches are limited to the analysis of simplifiedgenetic algorithms. All the developed special techniques to tackle dynamicproblems are again not considered in the current theory.

4. There is no fine-grained classification of dynamic problems available. Mostexisting classifications are very coarse and not based on exact mathematicalcriteria.

5. Besides a small number of responsive arguments, there is no fundamentalinvestigation available concerning the choice of performance measures. Boththe discussion what is actually expected from an optimization in a dynamicenvironment as well as the discussion how the attainment of these goals ismeasured are usually omitted in most publications.

6. The missing fundamental framework and the missing classification of prob-lems prevents a mapping of existing techniques to problem classes. Such amapping could serve as decisive factor when designing an evolutionary algo-rithm for a specific dynamic problem.

7. Even for a restricted dynamic problem class there are no design rules avail-able.

Apparently this list can be continued easily—especially if more specific topics con-cerning the techniques and problems are considered. However action seems to bemore imperative for the listed rather general open problems.

3.2 Focus and Contribution of this Thesis

This thesis addresses a few of the topics listed above. Chapter 4 is concerned witha fundamental classification of dynamic problems (Topic 4). This classification isalso intended to serve as a first step toward a formal foundation of dynamic evolu-tionary optimization (Topic 1). It is the first approach to base such a classificationon mathematically defined, exact properties. How it can serve for an integrationof empirical investigations is demonstrated in Chapters 5 and 6. In Chapter 5 thegoals of an optimization in a dynamic environment are clarified and respective

38

Page 39: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

3.3. METHODOLOGY

performance measures are examined for four problem classes within the frame-work (Topic 5). Chapter 6 reviews the primary literature on techniques used indynamic optimization and derives a first tentative mapping between dynamic op-timization techniques and problem characteristics (Topic 6). Thus, the ensembleof Chapters 4–6 is the first attempt to interrelate problem properties, optimizationtechniques, and performance measures on a broad formal basis for dynamic opti-mization.

Where the first half of the thesis is concerned with the complete range of dynamicproblems, it focuses on drifting problems in the second half. In Chapter 7 a rig-orous analysis of local variation operators in drifting landscapes is executed. Itresults in several design rules to guide practitioners when applying a local variationevolutionary algorithm to a drifting problem (Topic 7). These design rules are usedin four small case studies in Chapter 8. Besides the practical guidelines, Chap-ter 7 provides a detailed insight into the processing of local mutation operators indrifting landscapes. The understanding and control of those problems is increasedpersistently.

Altogether this thesis extends both the formal basis for integrative research in thearea of non-stationary optimization and the understanding and usage of local vari-ation to dynamic problems (especially drifting landscapes).

3.3 Methodology

In underpinning and deriving the results of this thesis a very strict methodology hasbeen used for both empirical and theoretical examinations.

In empirical studies, all conclusions for a setup consisting of algorithm, parameters,and problem are based on at least 50 experiments using independent initial randomseeds for the random number generator. Usually averaged results are presented inthe paper. As soon as a comparison of different techniques or parameter settingsis aimed at, a statistical test concerning the confidence in the superiority of eitherchoice is used. Throughout the whole paper Student’s t-test is applied for thispurpose. For each generation of the experiments, the hypothesis test is applied tothe data of that generation. As a consequence the t-test creates a curve of statisticalconfidence per generation. In Chapter 5, Spearman’s rank order correlation andthe mean square error are used additionally to evaluate time series of performancevalues. Details are described in Section 5.3.3.

All algorithms and problems used in the thesis are implemented in C++. Theresulting library is called sea (Stuttgart Evolutionary Algorithms) and provides

39

Page 40: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

3. CONTRIBUTION AND METHODOLOGY

an easily usable and expandable command-line program. Uniform random num-bers are created using the random number generator of Park and Miller (1988)with Bays-Durham shuffle and added safeguards. Random numbers From a Gaus-sian probability density function are generated with the Box-Muller transformation(Box & Muller, 1958). The random bits needed by genetic algorithms are pro-duced by a separate, very fast random number generator (Knuth, 1981, p.28). Allrandom number generators and statistical methods are used in the implementationsof Press, Teukolsky, Vetterling, and Flannery (1992).

The theoretical considerations in Chapter 7 use Markov chains to model the searchdynamics exactly as well as in a worst case scenario. The exact model is used forcomputations of the probabilities to be at certain points in the search space. Thesecomputations are carried out using the GNU multiple precision arithmetic library(Granlund, 1996) to rule out any numerical effects in the computations.

40

Page 41: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 4

A Classification of DynamicProblems

This chapter introduces a new mathematical framework for classifying non-stationaryproblems. After a short motivation in Section 4.1, the existing classifications arereviewed in Section 4.2. Then in Section 4.3 a formal framework for defining ordescribing dynamic problems is given. Section 4.4 extracts certain problem prop-erties from the formal framework and uses them to define a very general classifi-cation scheme for dynamic problems. A short discussion in Section 4.5 concludesthe chapter.

4.1 Motivation

Research in dynamic optimization is characterized by many different non-stationaryproblems and applications. As a consequence, a huge fraction of today’s researchis driven by certain applications or specific exemplary problems. But still there isno common basis for classifying or comparing different problems. However sucha classification is essential if we want to build a general foundation for the designof non-stationary optimizers on the available fragmentary results. In particular the

41

Page 42: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

classification is useful within two stages of the design process.

First, knowledge on the characteristics of a completely new problem might helpwhen designing a respective algorithm. If we are able to identify the problem classthe new problem belongs to, knowledge concerning the techniques to tackle theseproblems may be transfered to the new problem. Since introducing dynamics toproblems leads to a wide range of manifold and complex behavior, it is desirableto have at least certain clues which technique is suited to tackle a given problem.The necessity of this mapping between problem characteristics and algorithmictechniques concerning “good” performance is also a direct consequence of the “NoFree Lunch” theorems by Wolpert and Macready (1995, 1997).

Second, the assessment of the performance of an algorithm depends highly on theproperties of the problem. The desired adaptive process of an evolutionary algo-rithm in a dynamic environment can be qualified closely by a clarification of therequirements. This desired behavior may be influenced by the problem character-istics. An even bigger influence of the problem properties concerns the selection ofthe performance measure to assess the stated goal. Here knowledge on the assess-ment of certain problem classes may help to measure the performance of a newlydesigned algorithm for a new instance of this class and to evaluate and to improvethe algorithm.

CharacteristicProperties

EvolutionaryAlgorithm

DynamicProblem

influences

selectdesign

applyto

assess

identify

Goal/DesiredBehaviorPerformance

Measure

Figure 4.1 Schematic overview of the different aspects involved in this chapter.

Figure 4.1 shows the different stages where knowledge on problem characteristicswithin a classification may be helpful when tackling a new non-stationary prob-lem: design of evolutionary algorithms, desired behavior, and the respective perfor-mance measures. The selection of performance measures is discussed in Chapter 5together with a minor discussion of the desired behavior. Simple design issues arethe topic of Chapter 6.

42

Page 43: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.2. EXISTING CLASSIFICATIONS

Since many applications and benchmark problems are rather incomparable, such aclassification is difficult if not infeasible on a general level. Therefore, this chapterrestricts the task to a mathematical framework of dynamic function optimizationproblems that covers most of the possible problem characteristics.

4.2 Existing Classifications

We can distinguish two different approaches to classify dynamic problems: directdescription of classes and classification by parametrization.

With respect to the former approach, the direct description of classes, there are onlyvery few problem classifications in the literature and most of these classificationsare rather coarse-grained. In spite of different terminology, the following classesare broadly used.

• In alternating (or cyclic) problems, only a few different landscapes occurand alternate with each other—usually with certain static periods betweenchanges (e.g. Collard et al., 1997; Liles & De Jong, 1999; De Jong, 2000).

• In problems with changing morphology, the fitness landscape changes ac-cording to certain topological rules. In most cases those landscapes are char-acterized by rather severe changes (e.g. Collard et al., 1997; Liles & De Jong,1999; De Jong, 2000).

• In drifting landscapes, there is at least one static topology that drifts throughthe landscape; often several, superimposed topologies are used (e.g. Liles &De Jong, 1999; De Jong, 2000).

• In abrupt and discontinuous problems, only very few and unpredictable changesoccur (e.g. De Jong, 2000).

Another class, dynamical encoding problems, is proposed by Collard et al. (1997)as “problems where the interpretation of alleles changes” which is a rather gener-ally applicable description although this term is used only in the context of one spe-cific function. Another classification is defined by Trojanowski and Michalewicz(1999a)—they inspect problems on their randomness and predictability. For prob-lems with non-random and predictable changes, a cyclic behavior is considered inaddition. Furthermore, they are interested in dynamic changes of problem con-straints. However, as already mentioned above, all those classifications are coarse.The different classes are not properly delimited and there is a vast number of classes

43

Page 44: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

in-between. In order to make meaningful statements for certain problem properties,a more detailed classification is necessary.

Where exact properties are missing in the existing classifications so far, they can befound in the second approach, classification by parametrization, as parameters inproblem generators and in the description of problems. Therefore, we can considerthe following parameters as properties of rather fine-grained classifications.

• The frequency of change indicates how often the changes occur or how manygenerations remain static between the changes (Branke, 1999b). The sameproperty is also called transition period (Gaspar & Collard, 1999a), periodduration (Collard et al., 1997), or punctuation (Grefenstette, 1999).

• The severity of change denotes how far e.g. the optimum is moving (Branke,1999b). This property is also called transition range (Gaspar & Collard,1999a) or maximal drift (Grefenstette, 1999). An alternative version of theseverity is the average shape alteration which considers all points in thesearch space (Collard et al., 1997).

• The rule of meta dynamics denotes an underlying regularity concerning thechanges (e.g. Morrison & De Jong, 1999, use the logistics function to spec-ify the meta rule). In a more vague way this property is also called the pre-dictability of change (Branke, 1999b).

• The cycle length considers a periodic behavior and denotes the length of oneperiod (Branke, 1999b).

These properties can be used for a very exact description of various problem classes,e.g. in the case of one moving hill. But there are many scenarios and problems forwhich this classification is not general enough.

The classification presented in the following sections uses a combination of bothapproaches. Within a mathematical framework defined in Section 4.3, a set of basiccharacteristics similar to the fine-grained parameters is defined in an exact manner(cf. Section 4.4). Those basic properties are used to build up problem classessimilar to the coarse-grained classes. The advantage of this method is that for anyproblem covered by the framework an exact mapping to a problem class is possible.

4.3 Dynamic Problem Framework

This section introduces a new approach to classification: a general mathematicalframework to describe and characterize dynamic fitness functions. In order to es-

44

Page 45: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.3. DYNAMIC PROBLEM FRAMEWORK

tablish a well-defined basis for comparison and classification the non-stationaryfitness functions are defined in a formal way.

Similar to the moving hills problems, each dynamic function consists of severalstatic functions where for each static function a rule of dynamics is given describinghow the contribution of this component to the complete problem changes over time.Note that the term “time” is always associated with an equidistant discretizationthroughout the whole paper which is in particular due to the discrete nature ofevolutionary algorithms. The dynamics rule is defined by a sequence of coordinatetransformations, stretching factors for the coordinates, and fitness rescalings.

Definition 4.1 (Dynamic fitness function for maximization) Let Ω be the searchspace with distance metric d : Ω × Ω → R. A dynamic fitness function F ≡(F (t)

)t∈N with F (t) : Ω→ R for t ∈ N is defined by n ∈ N components consisting

of a static fitness function fi : Ω → R (1 ≤ i ≤ n) with optimum at 0 ∈ Ω,fi(0) = 1, and a dynamics rule with

• coordinate transformations(

c(t)i

)t∈N

with c(t)i : Ω → Ω preserving any dis-

tance in the search space, i.e.

d(c(t)i (ω1), c

(t)i (ω2)) = d(ω1, ω2) for all ω1, ω2 ∈ Ω,

,

• stretching factors(

s(t)i

)t∈N

with s(t)i ∈ R+, and

• fitness rescalings(

r(t)i

)t∈N

with r(t)i ∈ R+.

The accumulated dynamics are defined as

• C(t1,t2)i = c

(t2)i . . . c

(t1+1)i for t2 > t1 ≥ 0, C

(0,0)i = id , C

(t)i = C

(0,t)i for

t ≥ 0,

• S(t1,t2)i =

∏t2t=t1+1 s

(t)i for t2 > t1 ≥ 0, S

(0,0)i = 1, S

(t)i = S

(0,t)i , and

• R(t1,t2)i =

∏t2t=t1+1 r

(t)i for t2 > t1 ≥ 0, R

(0,0)i = 1, R

(t)i = R

(0,t)i .

For ω ∈ Ω, the resulting dynamic fitness function is defined as

F (t)(ω) = max

R(t)1 f1(C

(t)1 (S

(t)1

−→0ω)), . . . ,R(t)

n fn(C (t)n (S (t)

n

−→0ω))

45

Page 46: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Moreover, it is required that the following condition holds

∃1≤j≤n ∀1≤k≤n,j 6=k R(t)j > R

(t)k . (4.1)

In generation t = 0, (4.1) holds for j = 1. ♦

Note, that the coordinate transformations are linear translations, rotations, andcombinations of translation and rotation. All those transformations are preservingany distance in the search space

Also note, that condition (4.1) assures that there is at each time step only one globaloptimum, i.e. only one of the component fitness functions attains the optimal fit-ness value. This condition is of a technical nature to simplify the definition of theproperties in the succeeding section.

Before a few examples are presented illustrating how problems from recent pub-lications may be studied within the given framework, the difference between theframework and a problem generator are elucidated in this paragraph. On the onehand, a problem generator produces many different test problems according to cer-tain given parameters. However, the variance concerning various problem proper-ties may be still very high. As a consequence the problem classes defined by theparameters are often very broad with rather fuzzy and imprecise boundaries. Onthe other hand, the given framework defines exactly how the search space changesat any point for all time steps. There is no random variation possible within theframework. As a consequence the properties can be defined formally in the nextsection.

An illustration of constructing one component of the dynamic fitness function iscontained in Figure 4.2

Example 4.1 (Moving Peaks Problem) One example are the moving peaks prob-lems (e.g. Branke, 1999c; Morrison & De Jong, 1999) which can be realized by onecomponent function for each peak. Then, the movement of the peaks is defined bythe coordinate transformations, the peak heights may be changed by fitness rescal-ing, and the width or the slope of a cone may be changed by the stretching factor.The coordinate transformations, the rescaling, and the stretching factor apply usu-ally only very small changes to each hill. In case of the problem generator proposedby Morrison and De Jong (1999) it is possible to choose the randomness of the dif-ferent transformations using a logistic function where small values produce exactlyone deterministic value but for higher values the function bifurcates resulting invarious values with equal probability. That means, by using small values, a linearconstant change can be achieved where higher values lead to a more random behav-ior of the hills. However, each random instance must be described separately within

46

Page 47: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.3. DYNAMIC PROBLEM FRAMEWORK

0C

0.25

0.5

1.0

0.75

0

1.0

0.25

0.5

0.75

v

Rv

0

vS v

0

0

coordinatetransformation

integrate several component functions

fitness rescaling

stretching

Figure 4.2 Transformation of one cone at one time step in an exemplary movinghills problem.

the framework. An exemplary landscape with two hills is defined for Ω = R2 usingthe stationary functions

f1(x, y) = f2(x, y) =

1−

√x2 + y2, if

√x2 + y2 < 1

0, otherwise,

coordinate transformations that provide a linear movement into one direction

c(0)1 :

(xy

)7→(x− 5y − 1

)47

Page 48: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Figure 4.3 The left column shows the movement of one static hill in the fitnesslandscape. The right column shows two hills moving and changingtheir heights additionally.

c(0)2 :

(xy

)7→(x+ 1y + 1

)c

(t)1 :

(xy

)7→(x+ 0.2y + 0.1

)for 1 ≤ t

c(t)2 :

(xy

)7→(x− 0.1y − 0.1

)for 1 ≤ t,

no stretching factors, i.e. the slope of the hills does not change,

s(t)1 = s

(t)2 = 1 for 0 ≤ t, and

opposite fitness rescalings for both hills

r(0)1 = 1 r

(0)2 = 0.1

r(t)1 = 0.9 for 1 ≤ t

r(t)2 = 1.2 for 1 ≤ t.

Figure 4.3 shows schematically how such a landscape is changing. Due to thefitness rescalings the global optimum is jumping from one hill to the other hill. ♦

Example 4.2 (Hills with changing height) This fitness function, introduced by Trojanowskiand Michalewicz (1999b), divides the search space into different segments which

48

Page 49: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.3. DYNAMIC PROBLEM FRAMEWORK

0 1 32

4

8

12

5

13

9

6

10

14

7

11

15

1

2

3

45

6

7

8

1 2

3

4

56

7

8

Figure 4.4 Hills with changing height: the maximum height appears at the markedhills in the order indicated.

hold each a peak where the height of the peaks is changing according to a schedule.In particular, a two-dimensional search space [0, 1)× [0, 1) may be divided into 16cells of equal size. Each cell 0 ≤ k ≤ 15 is defined as Il × Im with k = 4l + m,0 ≤ l,m ≤ 3 and Ij = [lbj, ubj) where

lb0 = 0, lb1 = 0.25, lb2 = 0.5, lb3 = 0.75,ub0 = 0.25, ub1 = 0.5, ub2 = 0.75, ub3 = 1.

Then, for each cell k the fitness function is defined as

f(t)k (x1, x2) =

Maxheight(t)k (ubl − x1)(x1 − lbl)(ubm − x2)(x2 − lbm),

iff lbl ≤ x1 < ubl and lbm ≤ x2 < ubm0, otherwise

.

The hills in the component functions with index k ∈ 0, 3, 6, 9, 10, 12, 13, 15 havea constant height Maxheight

(t)k = 0.5. The other hills form a cycle

cyc(0) = 1, cyc(1) = 2, cyc(2) = 7, cyc(3) = 11,

cyc(4) = 14, cyc(5) = 13, cyc(6) = 8, cyc(7) = 4.

All those hills get the maximal height Maxheight(t)k = 1.0 assigned. Using the

fitness rescalings one of those hills becomes the maximum hill and this role cyclesthrough all eight hills. There are no coordinate or stretching dynamics necessary to

49

Page 50: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

model this problem

c(t)i = 1 (identity)

s(t)i = 1.

The following values define the rescaling of the 8 cycling hills

α0 = 1

αi = 0.5 for 1 ≤ i ≤ 7.

In the first generation, α0 will be associated with hill cyc(0), α1 with cyc(1), andso on. Like in the work of Trojanowski and Michalewicz (1999b), the maximumhill jumps each 5 generations. This is modeled by the fitness rescalings as followsfor 0 ≤ i ≤ 7

r(t)cyc(i) =

αi, if t = 0α(i+ t

5)mod8

α(7+i+ k5 )mod8

, if ∃a∈N t = 5a

1, otherwise

All remaining hills k ∈ 0, 3, 6, 9, 10, 12, 13, 15 are associated with the fitnessrescaling factor r(t)

k = 1. ♦

Example 4.3 (Rotating fitness functions) Another possible dynamic fitness func-tion is the rotation of a static fitness function around a center as introduced byWeicker and Weicker (1999, 2000). This is easily reproducible in this frameworkby defining only one component function with the static fitness function f1 and arotating coordinate transformation

c(t)1 :

(xy

)7→(cos(2π

τ)x− sin(2π

τ)y

sin(2πτ

)x+ cos(2πτ

)y

)for 0 ≤ t,

where τ is the number of time steps necessary for one full rotation. The rotationcenter is ~0 ∈ R2 = Ω.

Stretching factors and fitness rescalings are not necessary and therefore

r(t)1 = 1

s(t)1 = 1. ♦

Figure 4.5 shows how the fitness function changes.

50

Page 51: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

Figure 4.5 Schematic description of the rotating fitness function.

4.4 Problem Properties

The mathematical formulation of dynamic fitness functions in Definition 4.1 en-ables the definition of several problem properties influencing the hardness of a dy-namic problem. The following definitions formalize a few basic problem propertiesinherent in the dynamics of the problem. All properties are defined with respect toa set of time steps T ∈ N (T 6= ∅). For each definition the properties are illustratedusing the examples above. For simplicity, we assume from now on that Ω = Rn.

4.4.1 Coordinate Transformations

The first four definitions concern the coordinate transformations. Since Exam-ple 4.2 makes no use of coordinate transformations those properties are fulfilledfor this example for trivial reasons.

Definition 4.2 (Predictability of coordinate transformations) LetF be a dynamicfitness function consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤ n). Then, the coordinate transformations of Fj (1 ≤ j ≤ n) are pre-dictable with respect to time steps T ⊆ N (T 6= ∅) and error ε ∈ R+

0 iff

predictC T,ε(Fj) ≡ ∀t∈T maxω∈Ω

d(c

(t)j (ω), c

(t+1)j (ω)

)≤ ε. (4.2)

51

Page 52: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Lemma 4.1 Supposed Ω = Rn, a dynamic problem consisting of just one compo-nent function F1 with predictable dynamics for T = t and ε = 0, and the rule ofdynamics being a rotation or a linear translation—no combination of both—, thenit follows from Equation 4.2 that the movement of any point can be predicted attime t + 1 with probability 1 using the preceding movement of n uniformly chosenrandom points ωi (1 ≤ i ≤ n) at time t. ♦

Proof: It follows immediately from Equation 4.2 that the coordinate transformationis the same at times t and t + 1 (since ε = 0). Then, the two cases of coordinatetransformations must be distinguished. First, the case of the linear translation canbe recognized by equal translations for all points ω1, . . . , ωn ∈ Ω:

c(t)1 (ω1)− ω1 = . . . = c

(t)1 (ωn)− ωn. (4.3)

Then the prediction for ω is

c(t+1)1 (ω) = ω + c

(t)1 (ω1)− ω1.

Second, if Equation 4.3 does not hold, the coordinate transformations must be a ro-tation. From one point in the search space ωk and its translation c(t)

1 (ωk)−ωk it canbe deduced that the center of the rotation is in the hyperplane planek determinedby the following conditions:

1. c(t)1 (ωk)− ωk is a normal vector on planek and

2. ωk + 12(c

(t)1 (ωk)− ωk) ∈ planek .

Note, that for any pair 1 ≤ k, k′ ≤ n the probability is 0 that there exists an s ∈ R−→

center ωk = s−→

center ωk′ .

Therefore, the center of the rotation can be computed with probability 1 as

center =⋂

1≤k≤n

planek

The degree of one rotation results as

α =180

π

|c(t)1 (ω1)− ω1||ω1 − center |

.

q.e.d.

52

Page 53: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

Note, if the rule of dynamics is a combination of translation and rotation, there isno simple derivation mechanism to detect and compute the rule.

In the case of Example 4.1 with linear movement, each component is predictableand, as a consequence, this holds for the complete function too. Example 4.3 ispredictable too.

Definition 4.3 (Severity of coordinate transformations) Let F be a dynamic fit-ness function consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤ n). Then, the coordinate transformations of Fj (1 ≤ j ≤ n) have theseverity at t ∈ N and ω ∈ Ω

severityω,t(Fj) = d(c(t)j (ω), ω).

The maximal severity with respect to time steps T ⊆ N (T 6= ∅) is defined as

severityC T (Fj) = maxt∈T, ω∈Ω

severityω,t(Fj). (4.4)♦

Example 4.1 has a constant step size and, therefore, constant severity. Example 4.3has no constant severity since the changes close to the optimum are considerablysmaller than at the border of the search space.

Definition 4.4 (Repetitive coordinate dynamics) LetF be a dynamic fitness func-tion consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤

n). Then, the coordinate transformations of Fj (1 ≤ j ≤ n) are repetitive withrespect to time steps T ⊆ N (|T | ≥ 2) and error ε ∈ R+

0 iff

repetitiveC T,ε(Fj) ≡ ∀t1,t2∈T,t1<t2 ∃ε′∈[1−ε,1+ε] ∀ω∈Ω d(C(t1,t2)j (ω), ω) ≤ ε ♦

The set of time steps T are the moments where the landscape has an almost identicalshape.

Example 4.1 is not repetitive for all T ⊂ N. This follows directly from the lineartranslation and a non-zero severity in each time step. However, Example 4.3 isrepetitive e.g. for

T = 1, 1 + τ, 1 + 2τ, 1 + 3τ, . . ..

Definition 4.5 (Coordinate homogeneity) Let F be a dynamic fitness functionconsisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤ n).

Then, the coordinate transformations of Fj and Fk (1 ≤ j, k ≤ n) are homogeneouswith respect to time steps T ⊆ N (T 6= ∅) iff

homoC T (Fj, Fk) ≡ ∀t∈T c(t)j = c

(t)k . ♦

53

Page 54: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Example 4.3 is homogeneous for trivial reasons. Example 4.1 is not homogenoussince different coordinate transformations are applied to each component function.

Given the various properties of coordinate transformations, dynamic problems canbe classified using the properties concerning the movement of the optimum of onecomponent function in the search space. Therefore, the homogeneity is not con-sidered for this classification. Note, that these classes can be adjusted for differentpurposes by choosing time steps T 6= ∅ and error rates ε, ε′ appropriately.

Static coordinates: Position and orientation of the component function(s) is unaf-fected over the course of time:

∀1≤j≤n severityC T (Fj) = 0 (4.5)

for T = N. The other properties are fulfilled trivially if the coordinates arestationary.

Drifting landscapes: The component functions may change their position or ori-entation slightly between the time steps following a modification rule. Wecan characterize this class using a threshold value θ ∈ R+ for the maximumsmall severity value and the error rate ε ∈ R for the predictability:

∀1≤j≤n 0 < severityC T (Fj) ≤ θ, (4.6)∀1≤j≤n predictC T,ε(Fj) (4.7)

For a mere drifting landscape T = N is required and ε is supposed to be suf-ficiently small. In a bounded search space, the definition is usually weakenedby omitting a set of isolated time steps T ′ ⊂ N for the predict predicate wherethe direction of the movement may change (T = N \ T ′ in Equation 4.7).

Rotating landscapes: A special case of drifting landscapes where the modifica-tion rule is a rotation around a center in the search space. This can be char-acterized by Equations 4.6, 4.7 and the following predicate

∀1≤j≤n ∃T ′⊂T, T ′ 6=∅ repetitiveC T ′,ε′(Fj) (4.8)

with a sufficiently small error rate ε′ ∈ R and T’ being a subset of T inEquations 4.6 and 4.7. In order to rule out trivial repetition it should holdthat ε′ < θ − ε. Then, from predictability regarding T = t1, . . . , tk andrepetition for T ′ ⊂ T , a mere rotation follows as only possible coordinatetransformation for the period T .

54

Page 55: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

Randomly drifting landscapes: Unpredictable drifting component functions withsmall severity can be characterized by the following equations with regard toθ ∈ R+ and a sufficiently small ε ∈ R:

∀1≤j≤n 0 < severityC T (Fj) ≤ θ, (4.9)∀1≤j≤n ¬predictC T,ε(Fj) (4.10)

A subclass of problems returns frequently to previously found solutions wherein addition

∀1≤j≤n ∃T ′⊂T, |T ′|≥2 repetitiveC T ′,ε′(Fj) (4.11)

with a sufficently small error rate ε′ ∈ R holds (ε′ < θ − ε).

Fast drifting landscapes: Landscapes that drift into a certain direction with ratherbig steps can be described using the following equations:

∀1≤j≤n severityC T (Fj) > θ, (4.12)∀1≤j≤n predictC T,ε(Fj) (4.13)

∀1≤j≤n ∀T ′⊂T, |T ′|≥2 ¬repetitiveC T ′,ε′(Fj) (4.14)

with θ ∈ R+ and sufficiently small ε, ε′ ∈ R (ε′ < θ − ε).

Superimposing landscapes: A very fast rotation around a center in the searchspace equals the superposition of many landscapes. The global optima maydescribe a circular track in the search space. A pathological case is the posi-tioning of the global optima in the center. This class is described by

∀1≤j≤n severityC T (Fj) > θ, (4.15)∀1≤j≤n predictC T,ε(Fj) (4.16)

∀1≤j≤n ∃T ′⊂T, |T ′|≥2 repetitiveC T ′,ε′(Fj) (4.17)

with θ ∈ R+ and ε, ε′ ∈ R (ε′ < θ − ε).

Chaotic coordinate changes: Unpredictable big changes may lead in most casesto a rather chaotic behavior. This class is described by

∀1≤j≤n severityC T (Fj) > θ, (4.18)∀1≤j≤n ¬predictC T,ε(Fj) (4.19)

with θ ∈ R+ and ε ∈ R. A subclass on which an evolutionary algorithm isprobably applicable is characterized by the additional repetition property fora set of isolated time steps T ′ (|T ′| ≥ 2).

∀1≤j≤n repetitiveC T ′,ε′(Fj) (4.20)

55

Page 56: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Note, that this characterization assumes that the properties hold for all componentfunctions. There is an even bigger number of mixed classes where single compo-nent functions would fall into different classes. However, those mixed classes aremore complicated to analyze: single instances may have varying difficulty depend-ing on properties concerning the fitness rescalings and stretching factors.

Figure 4.6 gives an overview on the resulting classes.

fast drifting

superimposition

drifting

chaotic

random drifting

static

rotation

no severity

small severity

big severity

repetitivenot repetitive

notp

redi

ctab

le

pred

icta

ble

returning

Figure 4.6 Overview on the different problem classes generated by coordinatetransformations.

4.4.2 Fitness Rescalings

The following three definitions make statements on the properties of fitness rescal-ings. Since Example 4.3 uses no fitness rescalings, it is not considered in the dis-cussion of the properties.

56

Page 57: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

Definition 4.6 (Predictable fitness rescalings) Let F be a dynamic fitness func-tion consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤

n). Then, the fitness rescalings of Fj (1 ≤ j ≤ n) have constant dynamics withrespect to time steps T ⊆ N (T 6= ∅) iff

predictRT (Fj) ≡ ∀t∈T(r

(t)j < 1 ⇔ r

(t+1)j < 1

).

The time steps T are those moments where the optimum of Fj has almost identicalfitness. ♦

Example 4.1 is predictable since one hill is swelling and the other is shrinking atall time steps. Example 4.2 is not predictable with respect to the definition above.

Definition 4.7 (Severity of fitness rescalings) Let F be a dynamic fitness func-tion consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤

n). Then, the fitness rescalings of Fj (1 ≤ j ≤ n) have the following severity withrespect to time steps T ⊆ N (T 6= ∅)

severityRT (Fj) = maxt∈T

(r(t)j − 1)R

(t−1)j . ♦

The severity is the absolute (additive) change concerning the best fitness of a com-ponent function. In Example 4.1 the severity of hill 1 is −0.1 at time step 1, −0.09at time step 2, −0.081 at time step 3, and so on. The severity of hill 2 is 0.02 attime step 1, 0.024 at time step 2, 0.0288 at time step 3, and so on. In Example 4.2the severity of a component function Fcyc(i) is α(i+ t

5)mod8 − α(7+i+ t

5)mod8 for time

steps t = 5, 10, 15, . . ..

Definition 4.8 (Repetitive fitness rescalings) Let F be a dynamic fitness functionconsisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤ n).

Then, the fitness rescalings of Fj (1 ≤ j ≤ n) are repetitive with respect to timesteps T ⊆ N (|T | ≥ 2) and error ε ∈ R+

0 iff

repetitiveRT,ε(Fj) ≡ ∀t1,t2∈T,t1<t2 ∃ε′∈[1−ε,1+ε] R(t1,t2)j = ε′. ♦

Example 4.2 is repetitive for always five generations, e.g. T = 0, 1, 2, 3, 4 or T =5, 6, 7, 8, 9. Due to the cyclic movement also T = 0, 20, 40, . . . is repetitive.This is not the case with Example 4.1 where there is no repetition at all. Again,Example 4.3 is repetitive for trivial reasons.

57

Page 58: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Definition 4.9 (Homogenous fitness rescalings) Let F be a dynamic fitness func-tion consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤

n). Then, the fitness rescalings of Fj and Fk (1 ≤ j, k ≤ n) are homogeneous withrespect to time steps T ⊆ N (T 6= ∅) iff

homoRT (Fj, Fk) ≡ ∀t∈T r(t)j = r

(t)k .

F is homogeneous with respect to T iff homoRT (Fj, Fk) holds for all 1 ≤ j, k ≤n. ♦

Neither Example 4.1 nor Example 4.2 are homogeneous concerning the fitnessrescalings.

Another aspect of changing fitness rescalings is the fact that two components mayexchange their roles in a relation concerning the maximal height of the components.This may lead to a drastic change concerning the overall fitness landscape since theglobal optimum may be relocated in a different component function. This propertyis defined formally in the next definition.

This property is based on the fact that an alternation between two composite func-tions Fj and Fk takes place at time t if and only if either R(t)

k < R(t)j and R(t+1)

k >

R(t+1)j or R(t)

j < R(t)k and R(t+1)

j > R(t+1)k . Moreover, Definition 4.1 states that

there is at any time exactly one component function with maximal fitness, startingwith component function F1 at time t = 0.

Definition 4.10 (Alternating) Let F be a dynamic fitness function consisting of ncomponents Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤ n). Then, the best

fitness values of Fj and Fk (1 ≤ j, k ≤ n) are alternating with respect to time stepsT ⊆ N (T 6= ∅) iff

alterT (Fj, Fk) ≡ ∀t∈T

(1 <

R(t)k

R(t)j

<r

(t+1)j

r(t+1)k

∨ 1 <R

(t)j

R(t)k

<r

(t+1)k

r(t+1)j

).

Thus the sequence of time steps with changes concerning the global optimum andthe involved composite functions is defined by the series φi ∈ N×1, . . . , nwherethe first element contains the time when a change occurs and the second elementthe index of the new optimum component function.

φ0 = (0, 1)

φi =

(t, j), if alter t(Fj′ , Fj) holds and

∀t′<τ<t ∀1≤k≤n ¬alter τ(Fj′ , Fk)where φi−1 = (t′, j′)

undefined, otherwise

58

Page 59: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

The maximal and minimal time sequence of one dominating composite function isgiven by

mini∈N

(ti+1 − ti)

maxi∈N

(ti+1 − ti).

where φi = (ti, ki). Also the set of involved alternating composite functions maybe defined as

ki

∣∣∣ i ∈ N ∧ φi = (ti, ki)

. ♦

Note that the alternation is strongly related to the severity and the homogeneity asthe following lemma states.

Lemma 4.2 For any set of time steps T ⊂ N and T− = t − 1 | t ∈ T ∧ t 6= 0it holds that[

(∀1≤j≤n severityRT (Fj) = 0) ∨(∀1≤j,k≤n (k 6=j) homoRT (Fj, Fk)

)]⇒ 6 ∃1≤j,k≤n (k 6=j) alterT−(Fj, Fk) ♦

Proof: If the first condition concerning the severity holds, it follows immediatelythat r(t)

j = 1 for all 1 ≤ j ≤ n and for all t ∈ T . As a consequence the conditionfor the alternation can never be true for all 1 ≤ j, k ≤ n:

∀t∈T− 1 <R

(t)k

R(t)j

< 1.

Also, the second condition concerning the homogeneous fitness rescalings leads tothe same contradiction. q.e.d.

Figure 4.3 illustrates how for the moving hills problem (Example 4.1) the globaloptimum jumps from one hill to the other hill—this happens according to the def-inition exactly at time step t = 9. That means that both component functionsalternate. In Example 4.2 the component functions participating in the cycle ofcells are alternating at time steps 5, 10, . . ..

The following classes may be identified concerning the fitness rescalings.

static: The fitness values of the component functions are not changed by rescaling:

∀1≤j≤n severityRT (Fj) = 0.

The other properties are fulfilled trivially if the fitness is static.

59

Page 60: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

Unaffecting: As Lemma 4.2 states, the homogenous fitness functions are not al-ternating. As a consequence the class is described by

∀1≤j≤n 0 < severityRT (Fj)

∀1≤j,k≤n (j 6= k)⇒ ¬homoRT (Fj, Fk).

Swelling and shrinking: This class is characterized by predictable small fitnessrescalings.

∀1≤j≤n 0 < severityRT (Fj) ≤ θ

∀1≤j≤n predictRT (Fj)

∀1≤j,k≤n (j 6= k)⇒ ¬homoRT (Fj, Fk)

where θ ∈ R+ denotes the maximal acceptable small severity value.

A subclass of problems are those problems that alternate

∃1≤j,k≤n ∃T ′⊂T, |T ′|≥2 alterT ′(Fj, Fk).

Random: This class denotes those fitness rescalings which change slightly in arather random and unpredictable manner.

∀1≤j≤n 0 < severityRT (Fj) ≤ θ

∀1≤j≤n ¬predictRT (Fj)

∀1≤j,k≤n (j 6= k)⇒ ¬homoRT (Fj, Fk)

where θ ∈ R+ denotes the maximal acceptable small severity value.

A subclass of problems are those problems that alternate

∃1≤j,k≤n ∃T ′⊂T, |T |≥2 alterT ′(Fj, Fk).

Chaotic: This class is characterized by the following predicates

∀1≤j≤n severityRT (Fj) > θ

∀1≤j,k≤n (j 6= k)⇒ ¬homoRT (Fj, Fk)

where θ ∈ R+ denotes the maximal acceptable small severity value. In caseof the fitness rescalings, big changes imply immediately chaotic behaviorsince the domain of rescaling values is restricted to [0, 1].

Figure 4.7 gives a graphical overview on the different classes. Note that the repet-itive property is not considered in this classification. The reason is that the merefact that certain fitness values return exactly has not decisive influence on the dif-ficulty of the problem. Here the alternation, the predictability, and the severity arethe most important properties.

60

Page 61: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

predictablenot predictable

hom

ogen

eous

noth

omog

eneo

us

chaotic

not affecting alternationswelling and shrinking

alternating

alternating

random

static

no severity

small severity

big severity

Figure 4.7 Overview on the different problem classes generated by fitness rescal-ings.

4.4.3 Stretching Factors

The last dynamic modification applied to the components of fitness functions is thestretching factor. It differs from coordinate transformations and fitness rescalings inthe dynamics it creates. Obviously these modifications have no effect on the globaloptimum in the search space but on the morphology of the landscape. This impactis formalized in the following definition on the visibility of component functions.

Definition 4.11 (Visibility of component functions) Let F be a dynamic fitnessfunction consisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤

i ≤ n). Then, the best fitness value of Fj is visible at time stepst ∈ N

∣∣∣ ∀k∈1,...,n\j R(t)j > R

(t)k fk(C

(t)k

(S

(t)k (C

(t)j )−1(0)

))

61

Page 62: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

In the examples given only the moving hills problem in Example 4.1 may exhibitinvisibility of certain component functions. However, the values in the exampleare chosen such that the optimal value of both component functions is visible at alltime steps.

4.4.4 Frequency of Changes

Besides the actual modifications a very important factor is also the frequency withwhich the changes within the dynamic landscape occur. In many benchmark prob-lems for dynamic optimization there is no change for several generations and thelandscape stays fixed. A lot of optimization techniques rely on these static periods.Therefore, the following definition classifies non-stationary problems according totheir frequency of change.

Definition 4.12 (Frequency of change) Let F be a dynamic fitness function con-sisting of n components Fi =

(fi, (c

(t)i )t∈N, (s

(t)i )t∈N, (r

(t)i )t∈N

)(1 ≤ i ≤ n).

Then, for a time interval T ′ = t1, . . . , tl the frequency of change is defined as

frequencyT ′(F ) =#t ∈ T ′ | F is not constant at t

#T ′

where F is called constant at t iff

∀1≤j≤n severityC t(Fj) = 0,

∀1≤j≤n severityRt(Fj) = 0, and

∀1≤j≤n s(t)j = 1. ♦

In Examples 4.1 and 4.3 the frequency of change is 1, which means that in eachgeneration a change is occuring. In Example 4.2 the frequency of change is 1

5since

the landscape is modified once in five generations.

The frequency of change is probably a questionable concept as it is discussed here,since it is strongly related to the concept of a generation in the evolutionary algo-rithm. In general, the time steps of the problem and the number of fitness evalu-ations determining the time of one generation of the algorithm are independent ofeach other. This may be problematic if the problem changes within one generation.Therefore, we assume that the evolutionary algorithm is tailored to the problem inthe sense that the changes only occur in between the generations of the algorithm.

62

Page 63: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.4. PROBLEM PROPERTIES

4.4.5 Resulting Classification

In order to get a classification of dynamic problems in function optimization underconsideration of all problem properties, the different classes for coordinate trans-formations and fitness rescalings are combined in Figure 4.8.

swell/shrink

notaltalt.

notaltalt.

static drifting rotating rand. drift fast drift superpose chaotic

static

unaffecting

coordinate transformation

Drifting landscapes

fitne

ssre

scal

ing

Class 3Class 4

random

chaotic

Morphological changes (esp. with restr. visibility)

cyclic

cyclic cyclic

Class 1

Class 2

Figure 4.8 Graphical overview on the resulting classification. The areas with thicklines indicate where the general classes might be positioned. The fournumbered classes are used as examples in the next chapter.

The factors of the visibility of the component functions and the frequency of changeare not considered in Figure 4.8 since they do not affect the global optimum of thedynamic problem. These factors could be added in an additional dimension.

This new classification is rather complex. Therefore, it is important to embed pre-vious classifications from the literature. First, De Jong (2000) distinguishes ina rather coarse grained classification drifting landscapes, landscapes undergoingsignificant morphological changes, cyclic patterns, and abrupt and discontinuouschanges. The first two classes can be easily identified within the new framework.However, note that there is an overlap between both classes in Figure 4.8. This isprimarily due to the contrast between the existing rather inexact definitions of prob-lem classes and the classification presented here which is based on exact properties.According to De Jong (2000), the third class is characterized by a small number oflandscapes which may occur. This kind of dynamics may be introduced by different

63

Page 64: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4. A CLASSIFICATION OF DYNAMIC PROBLEMS

techniques in the presented framework but cannot be assigned to one combinationof properties. The fourth class, containing problems with abrupt changes, is pri-marily motivated by static landscapes where catastrophies happen and lead to a newlandscape. Where this kind of behavior can be easily modeled within the frame-work by many properties, it is slightly out of the focus of this thesis since it is moreconcerned with iteratively occuring changes. One of the main properties for theseproblems is the frequency of change—the characteristics of the coordinate trans-formations and rescalings appear to be of secondary importance. For this reason,this class was not included in Figure 4.8.

However, as we see in Figure 4.8, the classes from literature are very coarse-grainedand especially the cyclic problems can be generated by many different constella-tions of the meta rules. For the discussion of dynamic problems in the remainderof this thesis, the following four classes are introduced.

Class 1: pure tracking task with constant severity concerning the coordinates, i.e.the static fitness landscape is moving as a whole (homogeneous) or the partsare moving differently (inhomogeneous), the coordinate translation is pre-dictable and there is no fitness rescaling. As a consequence the problem isnot alternating.

Class 2: pure tracking task with constant severity in a changing environment, i.e.fitness rescalings and coordinate translations are inhomogeneous and pre-dictable, but the problem is not alternating.

Class 3: oscillation between several static optima, i.e. no coordinate translationbut inhomogeneous, predictable fitness rescalings take place. The problem isalternating.

Class 4: oscillating tracking task, i.e. fitness rescalings and coordinate translationsare inhomogeneous and predictable and the problem is not alternating.

In general we require the properties of the classes to hold for all time steps but therecan be a small fraction of singularly time steps which do not belong to the strictproperties. An example is the change of the direction of a linear translation withina restricted search space.

4.5 Discussion

This chapter introduced a new formal framework for analyzing, discussing, andcomparing dynamic problems. Using the framework single problem instances can

64

Page 65: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

4.5. DISCUSSION

be described in a very detailed manner which is not possible in any other avail-able classification scheme. This is a necessary foundation for the next chapters ofthe thesis where questions like the assignment of techniques to problem classes orthe measurement of performance are discussed in the light of properties of non-stationary problems.

65

Page 66: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,
Page 67: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 5

Measuring Performance in DynamicEnvironments

This chapter is devoted to the question what the term “good performance” meansin dynamic optimization. In the case of classical optimization tasks in static envi-ronments the performance can easily be measured using the best or average fitnessvalues—it indicates how good the optimum was approached. But in the case ofdynamic problems the task of finding one optimum shifts to an adaptive processthat approximates the best solution at each time step. Due to various kinds of dy-namics and the underlying component functions there are many possible aspects ofadaptation one might be interested in. The importance of each of these aspects isdirectly related to the goals one wants to achieve by dynamic optimization. There-fore, the suitability of a performance measure in a dynamic environment dependsprimarily on the goals and the problem characteristics. Section 5.1 deals with dif-ferent aspects concerning the goals of dynamic optimization. Possible performancemeasures are discussed in Section 5.2 and an empirical investigation is presentedin Section 5.3 before a short summary in Section 5.4. The chapter is based on theresults of a pre-published conference contribution (Weicker, 2002).

67

Page 68: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

5.1 Goals of Dynamic Optimization

In the previous chapter, a general framework and essential properties of dynamicfitness functions have been defined formally. This chapter is aiming at a relationbetween problem properties and performance measures to assess algorithms onthese problems. In order to do this, it is interesting to look closer at the goals thatare pursued by dynamic optimization.

However in the existing literature there is almost no discussion on the goals ofoptimization in a dynamic environment. This is somewhat surprising since thegoal is not anymore to find a static optimum only. It is rather an adaptation andestimation process. De Jong (1993) has already pointed out that, even in the caseof static problems, genetic algorithms are more than a mere optimization tool andpoints to the original intentions of Holland (1975). This is even more the case ina dynamic environment. As a consequence this section is concerned with variousaspects that are involved when talking about adaptation.

Adaptation is defined as the adjustment of organs or organisms to certain stimulantor environmental conditions. When transferring this definition to dynamic func-tion optimization such an adjustment can be measured best by the quality of theapproximation delivered by the algorithm over a set of generations. But this in-cludes not all aspects of adaptation. We might not only be interested in an averageapproximation quality but also in the stability of the quality and, if stability cannotbe guaranteed, the time necessary to reach again a certain approximation level.

In the following subsections, these aspects are defined more formally. These defi-nitions presume global knowledge on all aspects of the dynamic problem. Further-more they use only the observable behavior of the optimization algorithm, i.e. thebest approximation (fitness value) at each time step. Since this global knowledge isnot available in most applications for an assessment of the quality of an evolution-ary algorithm, the next section is concerned with performance measures to estimatethese characteristics of evolutionary algorithms on dynamic problems.

In order to enable a fair comparison of different algorithm with respect to the ques-tion how they reach the goal of dynamic optimization, it is useful to have a singlevalue that states how well an algorithm is doing. The standard technique for thisproblem was already considered by the performance measures of De Jong (1975):averaging over all generations (or a subset of generations).

Definition 5.1 (Averaging over time) Let T be a set of time steps and M (t) ∈ Rbe the performance value for the application of an algorithm to a function withrespect to generation t ∈ T .

68

Page 69: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.1. GOALS OF DYNAMIC OPTIMIZATION

Then, the average value or performance is defined as

Avg(M,T ) =1

| T |∑t∈T

M (t) (5.1) ♦

This technique will be used in the following discussion of goals as well as in theperformance measures.

5.1.1 Optimization accuracy

The primary overall goal of any (dynamic) function optimization is to achieve ahigh quality approximation for all considered time steps. This aim is resumed inthe following formal definition of the optimization accuracy.

Definition 5.2 (Optimization accuracy) Let F be a fitness function, Max(t)F ∈ R

the best fitness value in the search space, and Min(t)F ∈ R the worst fitness value

in the search space. Moreover, let EA be an optimization algorithm. Then, thealgorithm’s accuracy at time t ∈ TimeEA is defined as

accuracy(t)F,EA =

F (best(t)EA)−Min

(t)F

Max(t)F −Min

(t)F

(5.2)

where best(t)EA is the best candidate solution in the population at time t.

For a given set of relevant time steps T ⊆ TimeEA the average optimization accu-racy is defined as

AccF,T (EA) = Avg(accuracy(t)F,EA, T ) (5.3) ♦

Note, that the accuracy is only well defined if F is non-trivial at each time step,that means that the function is non-constant (Min

(t)F < Max

(t)F for t ∈ T ). The

optimization accuracy ranges between 0 and 1, where accuracy 1 is the best pos-sible value. It is also noteworthy that the optimization accuracy is independent offitness rescalings since it equals the percentage of the best fitness produced by thealgorithm with regard to the optimal fitness.

Trojanowski and Michalewicz (1999a) point out that this formula was already in-troduced by Feng et al. (1997) as a performance measure. However, there it wasonly applied to stationary fitness functions.

Example 5.1 Consider the following sequence of optimal fitness values, worst fit-ness values, and best approximations of an evolutionary algorithm.

69

Page 70: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

generation1 2 3 4

optimal fitness value 3.4 3.0 2.5 4.0worst fitness value 0.0 0.1 0.0 1.0best approximation 3.3 2.3 2.4 2.0

Then, the optimization accuracy results in

AccF,1,2,3,4(EA) =1

4

(3.3− 0.0

3.4− 0.0+

2.3− 0.1

3.0− 0.1+

2.4− 0.0

2.5− 0.0+

2.0− 1.0

4.0− 1.0

)=

1

4(0.9706 + 0.7586 + 0.96 + 0.3333)

= 0.7556. ♦

5.1.2 Stability

Stability is an important issue in optimization. Usually a candidate solution is re-ferred to as stable if slight variations of the candidate solution have a very similarfitness. In the context of dynamic optimization, I want to use the term “stability”with a slightly different meaning: an adaptive algorithm is called stable if changesin the environment do not affect the optimization accuracy severely. That meansthat a stable, adaptive algorithm is required to be prepared for changes in the en-vironment. Although the optimum can move slightly or severely or rather drasticchanges occur, an algorithm should be able to limit the respective fitness drop.

The following definition formalizes this aspect using the optimization accuracy.

Definition 5.3 (Stability) Let F be a fitness function, EA an optimization algo-rithm, and T ⊆ TimeEA the relevant time steps.

Then, the stability for time steps T is defined as

StabF,T (EA) = Avg(max0, accuracy(t−1)F,EA − accuracy

(t)F,EA, T ) (5.4) ♦

The stability ranges between 0 and 1. A value close to 0 implies a high stability.

Example 5.2 Consider the following sequence of optimal fitness values, worst fit-ness values, best approximations of an evolutionary algorithm, and the resultingaccuracy per generation.

70

Page 71: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.1. GOALS OF DYNAMIC OPTIMIZATION

generation1 2 3 4 5 6

optimal fitness value 4.4 4.4 4.4 3.0 3.0 3.0worst fitness value 0.0 0.0 0.0 0.0 0.0 0.0best approximation 4.0 4.1 4.1 1.0 0.9 1.1accuracy 0.9090 0.9318 0.9318 0.3333 0.3 0.3667

Then the accuracy difference between the succeeding generations is given in thefollowing table.

between generations1→ 2 2→ 3 3→ 4 4→ 5 5→ 6

accuracy difference -0.0228 0.0 0.5985 0.0333 -0.0667

The following stability results.

StabF,T (EA) =1

5(0 + 0 + 0.5985 + 0.0333 + 0)

= 0.1264

In this example the stability is rather high during the stable phases and the majorcontribution to the stability value stems from the drastic change in generation 4.However, note that the accuracy level after the change is rather low and accuracygains are not considered within the stability. ♦

Stability is especially in two scenarios of interest. In the case of drifting landscapesit is a very good means to gain insight into the ability to track the moving optimumby observing the stability over a period of time. However, as the example aboveshows, the stability must not serve as the sole criteria since it makes no statementon the accuracy level. Another scenario are chaotic landscapes with few changes.There it is of interest to examine the stability at the generations where changesoccur.

5.1.3 Reactivity

So far the optimization accuracy and the stability have been defined covering theaspects of quality and persistence. But there is still another aspect completely dis-regarded: the ability of an adaptive algorithm to react quickly to changes. Now,

71

Page 72: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

one could argue that a combined consideration of accuracy and stability covers thisaspect since unstable phases with a high overall accuracy implies good reactivity.Nevertheless, this aspect is formalized more exactly in the following definition.

Definition 5.4 (Reactivity) Let F be a fitness function, EA an optimization algo-rithm, and T ⊆ TimeEA a set of optimization time steps.

Then, algorithm EA’s average ε-reactivity for time steps T is defined as

ReactF,EA,ε = Avg(min recov(t)F,EA,ε, T ) (5.5)

where the time steps until ε-recovery for t ∈ T are defined as

recov(t)F,EA,ε =

t′−t

∣∣∣ t′ ∈ TimeEA and t′ > t and accuracy(t′)F,EA ≥ (1−ε)accuracy

(t)F,EA

(5.6)

and min ∅ =∞. ♦

The reactivity is a value of the set R+ ∪ ∞. A smaller value implies a higherreactivity.

This aspect of adaptation is especially of interest if the problem has short phasesof big severity alternating with extensive phases of no severity with regard to thecoordinate transformations or if the problem is alternating concerning the fitnessrescalings (with rather low severity for the coordinates).

Example 5.3 Consider the following sequence of optimal and worst possible fit-ness values, the best approximations of an evolutionary algorithm, and the resultingaccuracy per generation.

generation1 2 3 4 5 6

optimal fitness value 4.4 4.8 4.8 4.8 4.8 4.8worst fitness value 0.0 0.0 0.0 0.0 0.0 0.0best approximation 4.0 3.1 3.5 3.7 4.0 4.1accuracy 0.9090 0.6458 0.7292 0.7708 0.8333 0.8542

A 0.1-recovery is already met in generation 5, i.e. the minimum recovery time is 4.A 0.07-recovery is met in generation 6 with a minimum recovery time of 5. ♦

72

Page 73: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.2. PERFORMANCE MEASURES

5.1.4 Technical aspects of adaptation

Certain other aspects of adaptation are of a more technical nature. For example, itis possible to argue that an algorithm must detect changes in the problem to adaptwell. This ability will certainly improve the performance of an algorithm (if it istranslated into concrete and sensible actions), but it is not necessary for an evolu-tionary algorithm to guarantee a good adaptive behavior. Therefore, it is preferredto define in this context adaptation merely using the optimization accuracy.

Another example is the gaining of meta-knowledge in the case of predictable prob-lems. That means that the underlying rules of the dynamics should be predicted(e.g. coordinate transformations

(c

(t)i

)t∈N

and fitness rescalings(

r(t)i

)t∈N

). Again,used in an intelligent manner, it can boost the performance of an algorithm but isactually only a technique to improve the accuracy or stability.

These technical aspects are not characteristic properties of adaptive algorithms.Since there is no proof that detecting changes in a poor algorithm leads to betteradaptation than a very sophisticated problem-specific algorithm without this ability,the technical aspects are omitted in the discussion of the goals.

5.2 Performance Measures

Where the goals defined in the preceeding section capture different aspects of ap-proximation quality on dynamic problems, the global knowledge used in the def-initions is usually not available. As a consequence, this section is concerned withmeasures to estimate those globally defined values.

Before the special requirements of measurements in dynamic environments are dis-cussed, the two standard performance measures in static environments are quicklyreviewed. There, the set of best solutions is constant during the optimization.Therefore, the best approximation at the end of the search process is of primaryinterest. However, since the time to find an optimal value is an important issue,this factor may be considered in certain performance measures like the online oroffline performance by De Jong (1975). The online performance considers all func-tion evaluations in the search and is therefore well suited for expensive functions.Offline performance considers for each generation only the fitness of the best indi-vidual found so far. Naturally, these performance measures are the basis for manydynamic performance measures and, as a consequence, are revisited and formallydefined in the next section with a special emphasis on dynamic problems.

In the case of dynamic problems, measuring performance is even more difficult

73

Page 74: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

than in static environments. As it was already pointed out in the last section, theprocess of adaptation may involve various different aspects that are probably diffi-cult to cover with the usual performance measures. But even if only the optimiza-tion accuracy is considered, which equals a scaled fitness in static environments,certain problems may occur in the case of dynamic fitness functions. The coordi-nate transformations are not responsible for the difficulty in measuring—they arethe major challenge for adaptive algorithms. But the fitness rescalings aggravatethe measurement of performance for several reasons.

• If the fitness rescalings are not known, the best fitness value at a generationis not known. Because the best fitness can change, there is no common basisto talk about accuracy. A good fitness value at one time can be a bad fitnessvalue at another time—but this is not transparent for the user.

• To continue the argument of the previous item, a considerable drop in bestfitness does not necessarily imply worse accuracy—it could even mean thecontrary.

• Also in the case of alternating problems, a point with new best fitness canemerge where the hitherto optimum may have unaffected fitness. Then, astable fitness gives the wrong impression that the optimum is still observed.

These arguments will be revisited in the following subsections.

Before the concrete measures are discussed, two possible classifications of perfor-mance measures are presented. A first approach uses the information employed bythe measure. Here two classes can be distinguished, namely

• fitness based measures and

• genotypic or phenotypic measures.

A second distinction can be made by the knowledge which is available and used inthe measure:

• knowledge on the position of the optimum is available (which is usually onlythe case in academic benchmarks),

• knowledge on the best fitness value is available, and

• no global knowledge is available.

The following section argues how the aspects of adaptation could be assessed withthe different degrees of knowledge. These performance measures are evaluated forvarious dynamic problems in Section 5.3.

74

Page 75: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.2. PERFORMANCE MEASURES

5.2.1 Measures for optimization accuracy

For measuring the optimization accuracy with global knowledge, the definition ofaccuracy

(t)F,EA may be used. This makes the most evident statement possible since

it is unaffected by any fitness rescaling. Indeed, Mori et al. (1996) and Mori, Iman-ishi, Kita, and Nishikawa (1997) used this performance measure averaged over anumber of generations T

PerfAccF,T (EA) =1

| T |∑t∈T

αtaccuracy(t)F,EA.

For an exact average value the weights are set to αt = 1 for all t ∈ T . An almostsimilar performance measure was used by Trojanowski and Michalewicz (1999b)where the normalization using the worst fitness value was omitted—which is rea-sonable since the worst fitness value in the regarded problem is 0. They consideredalso only those time steps before a change in the environment occurs.

However, very often simple averaging might lead to misleading values. This canbe avoided by putting more emphasis on the detection of the optimum in each timestep. This was accomplished by Mori et al. (1998) using the weights αt. Theyused αt = 1 if accuracy

(t)F,EA = 1 and αt = 0.5 otherwise. Since this rewards only

those generations where the optimum was found exactly, a more gradual approachis the usage of the square error to the best fitness value as done by Hadad and Eick(1997).

However, this measure requires the best and worst possible fitness values. In mostapplications and problems, this information is not available. As a consequence,other measures are examined how well they approximate this exact measure. First,also fitness based measures are considered.

The majority of publications uses the best fitness value in a generation t to assessthe quality of the algorithm (e.g. Angeline, 1997; Back, 1999; Cobb, 1990; Cobb& Grefenstette, 1993; Collard et al., 1996, 1997; Dasgupta & McGregor, 1992;Dasgupta, 1995; Gaspar & Collard, 1997, 1999a, 1999b; Goldberg & Smith, 1987;Grefenstette, 1992, 1999; Hadad & Eick, 1997; Lewis et al., 1998; Liles & DeJong, 1999; Mori et al., 1996, 1997; Vavak et al., 1996a, 1998).

currentBest(t)F,EA = maxF (ω) | ω ∈ P (t)

EA

Another common possibility is the average fitness value of the population at gener-ation t (e.g. Cobb & Grefenstette, 1993; Dasgupta & McGregor, 1992; Dasgupta,

75

Page 76: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

1995; Goldberg & Smith, 1987; Lewis et al., 1998; Mori et al., 1996, 1997).

currentAverage(t)F,EA =

1

| P (t)EA |

∑ω∈P (t)

EA

F (ω)

Averaged over generations T those two measures lead to the classical performancemeasures of De Jong (1975). The online performance is defined as the average overall function evaluations since the start of the algorithm. Presumed that the popula-tion size is constant and the algorithm is generational, the online performance maybe defined as follows

PerfOnlineF,T (EA) = Avg(currentAverage(t)F,EA, T ) (5.7)

Online performance reflects the focusing of the search on optimal regions (cf.Grefenstette, 1992, 1999; Vavak et al., 1996a, 1996b, 1997, 1998). In the on-line performance actually each new created individual is supposed to contribute ahigh fitness value. However, Cobb (1990) argued that this conception might notbe suited for many dynamic problems because focusing too much on good fitnessvalues might have negative effects on the adaptability.

The offline performance is usually defined using the best fitness value found up toeach generation.

PerfOfflineF,T (EA) = Avg(

max1≤t′≤t

currentBest(t′)F,EA, T

)(5.8)

This measure allows a much higher degree of exploration since the performancecan revert to the best individual of previous generations. However, as Grefenstette(1999) and Branke (1999b) point out this measure is not suited for dynamic prob-lems, since it cannot be assumed that a very good solution from several generationsago is still valid. Therefore, the individuals of previous generations should not beconsidered when assessing the current generation. As a consequence the offlineperformance can be restricted to the individuals of each generation only.

PerfOffline∗F,T (EA) = Avg(

currentBest(t)F,EA, T

)(5.9)

This performance measure shows how well the optimum is approximated over thegenerations. Branke (1999a) uses a different approach to deal with this problem.There for each generation the best fitness value is used from those preceeding gen-erations in which no change in the environment occurred. Apparently, this requiresglobal knowledge on any possible change in the environment.

76

Page 77: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.2. PERFORMANCE MEASURES

Offline performance or variants have been used by Branke (1999c), Cobb (1990),Collard et al. (1997), Gaspar and Collard (1997), Grefenstette (1992), Vavak andFogarty (1996), Vavak et al. (1997), and (Vavak et al., 1998).

Another approach to measure the accuracy without knowing the actual best possiblefitness is based on the assumption that the best fitness value will not change muchwithin a small number of generations. As a consequence we introduce a localwindow of interest W ∈ N and measure the accuracy using the best fitness withinthe window as virtual target value.

windowAcc(t)F,EA,W = max

F (ω)− windowWorst

windowBest − windowWorst

∣∣∣ ω ∈ P (t)EA

with

windowBest = maxF (ω) | ω ∈ P (t′)EA , t−W ≤ t′ ≤ t

windowWorst = minF (ω) | ω ∈ P (t′)EA , t−W ≤ t′ ≤ t

Apparently, this measure has the same problems like the above fitness based ap-proximations: suboptimal convergence leads to values which cannot be recognizedas suboptimal. However, presumed that an algorithm does not completely fail on aproblem with fitness rescalings, this measure produces at least reasonable approxi-mations of the accuracy. This is especially of interest when the values are averagedover a set of generations. Here the previous measures can easily produce valuesthat are difficult to interpret. The averaged measure is defined as follows.

PerfWindow ∗F,T,W (EA) = Avg(

windowAcc(t)F,EA,W , T

)(5.10)

This window based measure has not been used in the experiments reported in theliterature. A similar technique with windows was used by Grefenstette (1986) forscaling fitness functions in order to improve fitness proportional selection.

Alternatively to the fitness based measures, genotype or phenotype based measuresmay be used to approximate the optimization accuracy. Although they are indepen-dent of fitness rescalings, they require full global knowledge of the position of thecurrent optimum. There are two variants of those measures in the literature. First,Weicker and Weicker (1999) used the minimal distance of the individuals in thepopulation to the current optimum ω∗ ∈ Ω. In the following version of the mea-sure the values are scaled such that a value close to 1 implies a close-to-optimumapproximation and a value close to 0 means a rather huge distance.

bestDist(t)F,EA = max

maxdist − d(ω∗, ω)

maxdist

∣∣∣ ω ∈ P (t)EA

(5.11)

Second, Salomon and Eggenberger (1997) used the distance of the mass center ofthe population to the current optimum’s position. Again the measure is scaled in

77

Page 78: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

the same way.

centerDist(t)F,EA =

maxdist − d(ωbest , ωcenter)

maxdistwith

ωcenter =1

µ

∑1≤i≤µ

A(i) where P (t)EA = 〈A(i)〉1≤i≤µ

Where the first approach seems to be straightforward to assess the approximationquality, the second performance measure is more difficult to interpret. Similarly tothe online performance each considered individual is important. It requires that thepopulation as a whole describes very closely the region of the optimum. As it willbe discussed shortly this may be difficult for certain dynamic problems.

Similarly to the fitness based performance measures, those measures may be ex-tended to sets of time steps by averaging. The according performance measures arecalled PerfBestDist and PerfCenterDist .

5.2.2 Measures for stability

Since the definition of stability uses the approximation accuracy the performancemeasures for stability also build on the performance measures for the accuracy.

If there is global knowledge available the stability can be computed directly. If thisis not possible, the accuracy in the definition of the stability

StabF,T (EA) =1

| T |∑t∈T

max0, accuracy(t−1)F,EA − accuracy

(t)F,EA (5.12)

can be replaced by an arbitrary measure discussed in the previous section. How-ever, the genotype/phenotype based measures do not seem to be adequate for thispurpose since they also require global knowledge and it does not make sense to usea distance to the optimum if the actual fitness as accuracy measure may be used.

If no global knowledge is available and the problem does not exhibit fitness rescal-ings, the best fitness values should be a good approximation for the accuracy. Alsothe stability using best fitness should still deliver reasonable performance values ifthere are moderate fitness rescalings.

The empirical investigation in Section 5.3 considers the performance measures forthe stability based on best fitness, average fitness, and the fitness within the window.

78

Page 79: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.2. PERFORMANCE MEASURES

5.2.3 Measures for reactivity

Analogously to the performance measures for stability, the measures for reactiv-ity may be defined in a first approach using the definition of reactivity and theperformance measures for the optimization accuracy. Depending on the availableknowledge most defined measures seem to be reasonable in the context of reactiv-ity. Only the measure using the distance of the population center to the optimumappears to be rather problematic. Good reactivity seems to be correlated to highdiversity and exploration in many situations—therefore the population center is aprobably inadequate basis for measuring it.

Again the performance measures based on best fitness, average fitness, and fitnesswithin the window are used in the empirical investigation in Section 5.3.

5.2.4 Comparing algorithms

Performance measures are necessary to compare different algorithms and serve asa decision criteria for the design of new algorithms. Admittedly, the usage of asingle averaged performance value for an algorithm over a period of several gener-ations might be difficult—especially in the case of dynamic fitness functions wherechanging conditions require separate evaluations. Therefore, it is not a big surprisethat the majority of publications relies on a visual comparison of performance val-ues of two (or more) algorithms.

The usage of statistical tests is still rather uncommon. To get empirical confi-dence that one method is better than another, it is necessary to execute a numberof independent experiments. Then the average performance of each experiment orthe performance in a certain generation may be used as independent samples of arandom variable. For example, Student’s t-test is one method to compare two al-gorithms. Other methods like the Scheffe test can be used to compare more thantwo algorithms. In general, statistical tests are used rather seldom in investigationsconcerning evolutionary algorithms. In the context of dynamic problems, they havebeen used by Angeline et al. (1996) to assess the hit rate with time series predic-tion, Vavak et al. (1997) compared operators, and other approaches may be found in(Pettit & Swigger, 1983; Grefenstette & Ramsey, 1992; Stanhope & Daida, 1998),to name just a few. Besides the independence of the samples, there are usuallycertain assumptions in statistical tests, e.g. the normal distribution of the samplevalues. Since those assumptions are usually not completely fulfilled, those resultsmust also be viewed critically. However the t-test appears to be rather conservativeand, therefore, tends to be more strict if the assumptions are not met.

79

Page 80: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

5.3 Examination of Performance Measures

This section considers the four dynamic problem classes that have been introducedin Section 4.4.5. The various performance measures are compared concerning theirutility on the different problem classes.

5.3.1 Considered problems

The four different problem classes are exemplified and instantiated using the fol-lowing basic non-stationary component function

f (t)(A) = max1≤j≤hills

0, if d(A, optj) > 150

maxfit j150−d(A,optj)

150, otherwise

withA ∈ Ω = [−500, 500]×[−500, 500], Euclidean distance d, and hills randomlychosen local optima optj ∈ Ω—each local optima corresponds to one component.The coordinate transformation for each component j is a linear translation of lengthcoordsev into a direction dirj which is randomly determined at the beginning andevery time a point outside of Ω would be created. The fitness rescaling is a factorfitchangej which is added tomaxfitj . Again, fitchangej ∈ [−fitsev, fitsev] ischosen randomly at the beginning and when maxfitj would leave the range [0, 1].In non-alternating problems the maximum hill with j = 1 must have maximalfitness in the range [0.5, 1] and all other hills have maximum fitness in the range[0,maxfit1]. For the four problem classes the following concrete values are used.

Class 1: coordinate translation, no fitness rescaling, no alternationProblem instance: hills = 5 , coordsev = 7.5, fitsev = 0Various hills are moving while their height remains constant and the best hillremains best.

Class 2: coordinate translation, fitness rescaling, no alternationProblem instance: hills = 5 , coordsev = 7.5, fitsev = 0.01Various hills are moving while their height is changing, but the best hill re-mains best.

Class 3: no coordinate translation, fitness rescaling, alternationProblem instance: hills ∈ 2, 5 , coordsev = 0, fitsev = 0.01The hills are not moving but changing their height leading to alternating besthills.

80

Page 81: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.3. EXAMINATION OF PERFORMANCE MEASURES

Table 5.1: Average accuracy and standard deviation for the genetic algorithm withand without hypermutation.

w/out hypermut. w/ hypermut.avg sdv avg sdv

class 1 0.45 0.023 0.87 0.0049class 2 0.45 0.018 0.87 0.0035class 3 0.82 0.035 0.96 0.0054class 3 (2 hills) 0.97 0.0029 0.99 0.00086class 4 0.46 0.025 0.87 0.0031class 4 (2 hills) 0.41 0.023 0.86 0.0019

Class 4: coordinate translation, fitness rescaling, alternationProblem instance: hills ∈ 2, 5 , coordsev = 7.5, fitsev = 0.01The hills are moving while changing their height and different hills take therole of the best hill at different generations.

The problem instances with 2 hills are chosen in such a way that there is at leastone alternation while both hills are changing their height into the same direction.This additional characteristic is supposed to be problematic when measuring theperformance. Note, that the fitness severity is chosen moderately in all classes.

5.3.2 Experimental Setup

To optimize the dynamic problems two genetic algorithms are used. Both algo-rithms are based on a standard genetic algorithm where each search space dimen-sion is encoded using 16 bits, the crossover rate is 0.6, the bit flipping mutation isexecuted with probability 1

32, a tournament selection with tournament size 2 is used,

and the algorithm runs for 200 generations. In addition to this standard algorithm,a version using hypermutation with a fixed rate of 0.2 is used (see Grefenstette,1999). Table 5.1 shows the accuracy averaged over 10 problem instances and 50independent experiments for each instance as well as the respective standard devi-ation. The GA with hypermutation performs superior—however the performanceof both algorithms should be expressed by a performance measure equally well.

5.3.3 Statistical examination of the measures

The goal of this investigation is to find some empirical evidence for the questionhow good the various measures approximate the exact adaptation characteristics.

81

Page 82: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

Table 5.2: Ranking based on pairwise hypothesis tests concerning the MSE of thecurves

standard GA GA with hypermutationC

lass

1

Cla

ss2

Cla

ss3

Cla

ss3

(2hi

lls)

Cla

ss4

Cla

ss4

(2hi

lls)

Cla

ss1

Cla

ss2

Cla

ss3

Cla

ss3

(2hi

lls)

Cla

ss4

Cla

ss4

(2hi

lls)

Accuracy:best fitness 1 1 1 4 1 1 1 2 1 4 1 1average fitness 2 2 2 3 2 2 4 3 2 3 3 3window based 3 3 5 5 3 5 2 1 5 5 2 2shortest distance 4 4 3 1 4 3 3 3 4 1 4 4distance of center 5 4 3 2 5 4 5 5 2 2 5 5Stability:best fitness 1 1 1 1 1 1 1 1 1 1 1 1average fitness 2 2 2 2 2 2 2 2 2 2 2 2window based 3 3 3 3 3 3 3 3 3 3 3 30.05-Reactivity:best fitness 1 1 1 1 1 1 1 1 1 1 2 1average fitness 3 3 3 3 3 3 3 3 3 3 3 3window based 2 2 2 2 2 2 2 1 1 1 1 1

A first approach is based on the assumption that the curves of the performancemeasures should match the curves of the respective exact values to guarantee ameaningful statement of the performance measure. So the essential question iswhether the values of the performance measure are a solid basis for comparingone generation of the algorithm with another generation. The second approachconsiders the averaged performance values only and tests how well they correlateto the averaged exact values.

In the first approach, the measurements are normalized (g(t) − Eg)/√Vg where

Eg is the expectancy value and Vg the variance of the whole curve. This makesthe values of different performance measures comparable since the values are in-dependent of the range of the values. To assess the similarity of the curves of theexact values h′ and the normalized performance measure g′, the mean square error

82

Page 83: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.3. EXAMINATION OF PERFORMANCE MEASURES

Table 5.3: Percentage of problem instances with a high correlation to the exactaveraged value

standard GA GA with hypermutationC

lass

1

Cla

ss2

Cla

ss3

Cla

ss3

(2hi

lls)

Cla

ss4

Cla

ss4

(2hi

lls)

Cla

ss1

Cla

ss2

Cla

ss3

Cla

ss3

(2hi

lls)

Cla

ss4

Cla

ss4

(2hi

lls)

Accuracy:best fitness 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0average fitness 1.0 1.0 1.0 1.0 1.0 1.0 0.8 1.0 1.0 1,0 1.0 1.0window based 0.9 0.9 0.6 0.0 0.8 0.4 0.9 0.9 0.5 0.0 1.0 1.0shortest distance 0.7 0.7 0.9 1.0 0.7 0.8 0.9 0.7 0.9 1.0 0.8 0.6distance of center 0.7 0.7 0.9 1.0 0.7 0.9 0.7 0.4 0.9 0.5 0.6 0.3Stability:best fitness 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0average fitness 1.0 1.0 0.4 0.0 1.0 1.0 0.0 0.2 0.0 0.0 0.0 0.0window based 0.4 0.4 0.2 0.2 1.0 0.5 0.1 0.5 0.4 0.2 0.5 0.30.05-Reactivity:best fitness 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.6 1.0 1.0 1.0 0.8average fitness 1.0 1.0 0.4 0.2 1.0 1.0 0.3 0.3 0.0 0.0 0.1 0.1window based 0.9 0.8 0.8 1.0 0.6 0.4 1.0 1.0 1.0 1.0 1.0 1.0

MSEg′,h′ =∑maxgen

t=1 (g′(t) − h′(t))2 is computed. In order to get a statisticalconfidence of one measure over another, a hypothesis test is carried out using the500 independent mean square errors of each performance measure. Those pairwisehypothesis tests are used to establish a ranking concerning the suitability of the per-formance measures. Student’s t-test is used as a hypothesis test with a significanterror probability of 0.05. Table 5.2 shows the results of this analysis.

In the second approach, the averaged measures at the end of a optimization run areused to determine how well the algorithm performed on the problem. Therefore,a statistical test is used to compute the correlation of the approximated measuresto the exact measures. The input data for the correlation analysis are the averagedperformance values of the 50 different runs of an algorithm on a problem instance.(Since the reactivity measures depend highly on the successive generations, thevalues up to generation 150 are used for those measures instead of the final perfor-

83

Page 84: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5. MEASURING PERFORMANCE IN DYNAMIC ENVIRONMENTS

mance values in generation 200). As statistical method Spearman’s rank correlationis used. A series of data is considered to be highly correlated if the Spearman’s rankcorrelation is positive and the two-sided significance level of its deviation from zerois less than 0.001. The correlation is computed for each of the ten instances of aproblem class. Table 5.3 shows the percentage of instances where a high correlationbetween exact value and performance measure could be identified.

5.3.4 Discussion of the Results

The interpretation of the results appears to be difficult and they should be inter-preted only very carefully. The reason for this problematic interpretation is the factthat there is actually no well founded methodology for a comparison. The statisticalexaminations used in this chapter are an attempt to reflect two aspects of good per-formance measures. However, it seems that a comparison of the averaged measuresmight show some misleading tendencies since certain deviations and problems areaveraged out.

According to the averaged values, best fitness is a quite good indicator for allclasses and both high and low quality algorithms. However, the examination ofthe MSE shows that all fitness based measures have severe problems with class 3(2 hills) where only fitness rescalings are occurring, in a misleading way. Espe-cially the windows based measure has severe problems with all class 3 instancesand also with low quality approximations of class 4 problems. The MSE of theGA with hypermutation on class 2 indicates that the window based measures canbe a better indicator than best fitness although this is not approved by the averagedperformance values.

Also, the stability is measured best using best fitness values. The average fitnessshows very poor results with the averaged performance values. The windows basedmeasure is insufficient regarding the MSE.

Concerning the reactivity, the window based measure proves to be equivalent orsuperior to the best fitness in case of the high quality experiments (GA with hy-permutation). Here, the measure based on best fitness shows problems in the MSEcurves for class 4 and in the averaged values on class 2 and class 4. The goodperformance of the window based measure can be explained presumably by recon-sidering the definition of the reactivity and the window based measure: both have avery strong relation to filters of data series which could be a hint why they interactso good with each other.

However, the results for the reactivity visualize also a serious concern: apparentlythe quality and usefulness of a performance measure depends on the quality that is

84

Page 85: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

5.4. SUMMARY

produced by an algorithm. However, this complicates the picture since we want tochoose a performance measure at hand to determine whether the algorithm showsgood or bad performance.

5.4 Summary

This chapter presents the first systematic approach to examine the usefulness ofperformance measures in time-dependent non-stationary environments. The goalsof an adaptation process are discussed in detail and accuracy, stability, and reac-tivity are proposed as key characteristics. Existing performance measures fromliterature are reviewed and a new window based performance measure is proposed.

On a wide set of dynamic problems the measures are examined for an algorithmwith high accuracy and an algorithm generating low accuracy. Altogether the bestfitness value proves to be the best performance measure for problems with moderatefitness severity—deficiencies exist for problems without coordinate transitions andas a basis for recovery measures. In the latter case, the window based measureexhibits a superior performance.

The mapping between problem classes and performance measures appears to bedifficult. However, certain problematic issues for instances of class 3 could be de-rived for example. Moreover, the performance measure seems to depend in additionon the quality of the approximation algorithm which raises many new questionsconcerning the measurement of algorithms in dynamic environments.

Furthermore, two rather basic concerns can be derived from this examination. First,a good methodology to examine performance measures is necessary. And second,the use of rather general problem generators should be reconsidered. The resultsfor the instances of class 3 and the more selective determined instances of class 3with two hills show, that there is a huge variance in the behavior of the different in-stances. Basically, this shows that problem generators are only useful if any aspectof the dynamics can be calibrated somehow using the parameters of the generator.This justifies probably the very strict definition of the problem framework in theprevious chapter and underlines the need for such a classification.

85

Page 86: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,
Page 87: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 6

Techniques for DynamicEnvironments

This chapter provides an overview on the techniques used to trim evolutionary al-gorithms for dynamic environments—some of the techniques are inherent in cer-tain standard algorithms. For this purpose, the major publications on the topic arereviewed. The techniques are classified using the following distinction:

• restarting techniques where the optimization is started again from scratch,

• local variation to adapt to changes in the environment,

• memorizing techniques where previous solutions (or parts of previous solu-tions) are reintroduced into the population,

• diversity preserving techniques to avoid the loss of adaptability,

• adaptive and self-adaptive techniques,

• algorithms with overlapping generations,

• learning of the dynamics rules, and

87

Page 88: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

• non-local encoding.

There are many algorithms that use various mechanisms and, therefore, can beassigned to more than one class.

The goal of this overview is to achieve a first mapping of the different techniquesto the problem classes and problem properties identified in Chapter 4. As a conse-quence, this chapter uses primarily references where the properties of the problemframework can be identified easily in the considered application. This is for ex-ample not the case in rescheduling problems or dynamic job shop scheduling (e.g.Biegel & Davern, 1990; Bierwirth et al., 1995; Bierwirth & Mattfeld, 1999; Hart &Ross, 1998). In those problems the properties of the dynamics are not as obviousand can even depend on the used representation and the operators.

6.1 Restarting

An intuitive approach to deal with premature convergence is the restart of the op-timizer as soon as a change in the environment or convergence in the populationis observed. However, it is advisable to initialize the population at the restart withfew individuals of the previous optimization run to guarantee continuity (e.g. inanytime learning, Grefenstette & Ramsey, 1992).

Although there is a broad range of applications using restarting most of the prob-lems is hardly categorizable within the proposed framework (e.g. Ramsey & Grefen-stette, 1993; Pipe, Fogarty, & Winfield, 1994). Only the work by Vavak et al.(1998) examines restarting on a moving hills problem which has no fitness rescal-ings, varying coordinate severity, a frequency of change between 24−1 and 6−1, andis highly inhomogeneous since only the maximum hill is changing its position. Theexamination shows that restarting is a feasible option if the frequency is less than15−1 where this value decreases with increasing coordinate severity.

Since restarting leads to a completely new optimization there should be no influ-ence of fitness rescalings or scaling factors. Therefore, the results of Vavak et al.(1998) should be valid for almost all problems.

6.2 Local variation

If only slight modifications occur, it is sensible to use local operators to createoffspring in the region of the current individuals. This is accomplished by standard

88

Page 89: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6.3. MEMORIZING PREVIOUS SOLUTIONS

mutation operators in evolution strategies with a lognormal update rule (see Back,1997, 1998; Salomon & Eggenberger, 1997) and in evolutionary programming withthe additive Gaussian update rule (see Angeline, 1997; Saleem & Reynolds, 2000).However, in genetic algorithms there is no standard operator searching within aphenotypic neighborhood of the real-valued search space. Therefore, a specialvariable local search operator has been introduced by Vavak et al. (1996a, 1996b)and Vavak et al. (1997, 1998) where a local search phase is started in which firstonly bits with low order are changed and with missing success the search range isextended by higher order bits. The multinational GA by Ursem (2000) promotes theintense local variation by linking the mutation rate to the distance of the individualto a special representative of the subpopulation (called “nation”). The closer anindividual is to its representative, the smaller the used mutation rate. This leads toan intense search around the representative.

Most problems where local variation has been used are characterized by no fit-ness rescaling and rather small coordinate severity. There are drifting landscapes(Angeline, 1997; Back, 1998; Salomon & Eggenberger, 1997; Saleem & Reynolds,2000), repetitive tracking (or rotating) problems (Angeline, 1997; Back, 1998), andrandom drifting (Angeline, 1997; Back, 1998). However, the work of Saleem andReynolds (2000) shows in the context of a moving hills problem that increasingseverity or decreasing frequency of change handicaps algorithms using local vari-ation. Very big (unpredictable) severity values lead to bad performance. As aconsequence successful reports on problems with rather big severity comes alongwith very low frequency of change (Back, 1997). This is confirmed by the studyof Vavak et al. (1996a, 1996b) which shows that variable local search is only betterthan a diversity increasing technique (e.g. hypermutation) if there is a rather smallcoordinate severity in the problem.

The multinational GA by Ursem (2000) is the only known application of localvariation to an alternating two hills problem with fitness rescaling, small severityvalues, and the meta dynamics following sinusoidal curves, straight lines as well ascircles.

6.3 Memorizing previous solutions

If the time-dependent changes in the environment create landscapes that are verysimilar to previous landscapes, it might be sensible to memorize previous solutionsto guide the evolutionary search to those regions in the search space. There are twodifferent concepts for memorization in the literature.

89

Page 90: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

6.3.1 Explicit memory

In the case of explicit memory there is a storage for previous candidate solutionsthat can be reevaluated or reinserted into the population if a change in the envi-ronment occurs. Mori et al. (1997) and Branke (1999c) use an external populationwhere the individuals are gathered. Kirley and Green (2000) add an external mem-ory to a fine-grained parallel population model. Trojanowski and Michalewicz(1999b) extend each individual by a FIFO queue where the ancestors are stored.When a change in the environment occurs, all candidate solutions are reevaluatedand the best may switch positions with the current individual if it is better. Andlast, the cultural algorithm of Saleem and Reynolds (2000) uses a memory for itssituational knowledge.

Those approaches that rely primarily on the explicit memory are applied to alternat-ing problems. The test function of Trojanowski and Michalewicz (1999b) alternatesbetween 8 different component functions (swelling and shrinking) and Mori et al.(1997) use a recurrently varying knapsack problem where three different weightlimits alternate. In both examples the frequency of change is rather high. Branke(1999c) applied the memorizing technique to a moving hills problem where thereare fitness rescalings and coordinate transformations with small severity (randomdrift). Also the cellular genetic algorithm by Kirley and Green (2000) is improveddrastically by an external memory if applied to a problem alternating between twostates. However, the performance of this algorithm on drifting or randomly drift-ing problems is not satisfying. Saleem and Reynolds (2000) applied their culturalalgorithm to moving hills problems drifting with small or medium severity or arather chaotic behavior. However, the memory mechanism is only one aspect inthis mixture of techniques.

6.3.2 Implicit memory

Contrary to explicit memory, implicit memory does not memorize complete candi-date solutions that are reinserted into the population but rather provides a mecha-nism where parts of previous solutions still participate in the evolutionary processbut do not undergo the selective pressure. The major techniques are diploid or poly-ploid representations where either depending on a dominance mechanism certainrecessive information persists in the individual or an encoded switch is used to fadeout and activate (parts of) candidate solutions. Diploid algorithms with dominancemechanisms have been used by Goldberg and Smith (1987), Smith and Goldberg(1992), Hadad and Eick (1997), Lewis et al. (1998), and Ng and Wong (1995).Ryan introduced several modifications like additive diploidity (Ryan, 1996), forced

90

Page 91: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6.4. PRESERVING DIVERSITY

mutation (Ryan, 1997), and perturb operators (Ryan & Collins, 1998). Also, Ryanand Collins (1998) introduced a triploid scheme called shades. As a generaliza-tion of diploidity polygenic inheritance was introduced (Ryan, 1996). Hadad andEick (1997) developed a tetraploid scheme and Dasgupta and McGregor (1992) andDasgupta (1995) presented an diploid algorithm with switch bits, called structuredGA.

All these algorithms have been applied primarily to recurrently varying knapsackproblems, i.e. alternating, repetitive problems with either high frequency changes(e.g. 15−1, Goldberg & Smith, 1987; Smith & Goldberg, 1992; Dasgupta & Mc-Gregor, 1992; Ng & Wong, 1995) or low frequency changes (e.g. 1500−1, Lewiset al., 1998). Most problems oscillate between two different weight limits. Excep-tions are three weight limits in the work of Dasgupta and McGregor (1992) andrandom new weight limits in one application of Lewis et al. (1998). The latterwork also showed implicitly by the definition of a general knapsack problem thatthis problem is equivalent to pattern tracking. Therefore, the pattern tracking prob-lem considered by Ryan (1996) adds nothing new to the problem classes above.

There is only one application to a drifting problem that is repetitive in a morerestricted way (Dasgupta, 1995).

6.4 Preserving diversity

As it was pointed out in Section 2.3, a huge fraction of the techniques orientedresearch focuses on the maintainance of the diversity to preserve the ability of analgorithm to adapt to a dynamic environment. As a consequence, there is a widevariety of techniques in the literature to support diversity in the population. Here,diversity increasing techniques, niching techniques to prevent convergence, andrestricted mating are distinguished.

6.4.1 Diversity increasing techniques

One very basic mechanism to increase diversity is to introduce random individ-uals into the population each generation. This method is referred to as randomimmigrants, partial hypermutation, or hypermutation with a fixed rate (e.g. Grefen-stette, 1992, 1999; Cobb & Grefenstette, 1993). The triggered hypermutation onlyintroduces new individuals when the performance, usually measured as best-of-generation fitness over 5 generations, worsens. In addition, the individuals are notcompletely random but obtained by a mutation with severely increased mutation

91

Page 92: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

rate (e.g. Cobb, 1990; Cobb & Grefenstette, 1993; Vavak et al., 1996a, 1996b;Morrison & De Jong, 2000). Grefenstette (1999) examined also a self-adaptivehypermutation rate and Smith and Vavak (1999) introduced hypermutation to al-gorithms with overlapping populations. Instead of using a hypermutation operatorCollard et al. (1996, 1997) introduced a dual GA where an additional bit in theindividual determines whether all bits are reversed when decoding the individual.In addition a mirroring operator inverses complete individuals. Based on this al-gorithm a set of different techniques was proposed. The dreamy GA restricts re-combination between normal and inversed individuals during certain phases of theevolutionary process (Escazut & Collard, 1997). The folding GA separates the in-dividuals into segments by introducing introns as seperators (or meta-genes). Thoseseperators determine whether the subsequent segment is inversed during decoding(Gaspar & Collard, 1997). Eventually, the dual sharing GA extends the dual GAby a mechanism to adapt the mirroring rate on the basis of the ratio of invertedindividuals (Gaspar & Collard, 1999b).

Most applications of diversity increasing techniques use problems without fitnessrescaling but with non-repetitive coordinate transitions. Usually the frequency ofchange is rather big—between 20−1 (Cobb & Grefenstette, 1993; Grefenstette,1992; Smith & Vavak, 1999; Vavak et al., 1996a, 1996b; Grefenstette, 1999) and 1(Collard et al., 1996, 1997; Gaspar & Collard, 1997; Grefenstette, 1999; Morrison& De Jong, 2000). However the severity can be rather small or rather big as inthe study of Grefenstette (1999) or it is even varying within one problem (Cobb &Grefenstette, 1993; Collard et al., 1996, 1997; Gaspar & Collard, 1997). The pat-tern tracking problem by Escazut and Collard (1997); Collard et al. (1997); Gasparand Collard (1997, 1999b) has quite similar dynamics. There is just one applica-tion where the problem is in addition returning to previous optimum positions inthe search space (Cobb, 1990).

In three publications, diversity increasing techniques are used to tackle alternat-ing problems without coordinate transitions (Cobb & Grefenstette, 1993; Smith &Vavak, 1999; Morrison & De Jong, 2000). In the first two cases the frequency ofchange is 20−1 resp. 200−1 where each change equals an alternation between com-ponent functions. The latter publication has a frequency of change of 1 leading torather irregular alternations.

6.4.2 Niching techniques

In the literature, there is a wide variety of sharing methods as niching techniquesavailable. However, suprisingly, most techniques are not yet used in the context

92

Page 93: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6.5. ADAPTIVE AND SELF-ADAPTIVE TECHNIQUES

of dynamic optimization (the author is aware of only one publication using a tra-ditional sharing method by Ursem, 2000). Cedeno and Vemuri (1997) introducedthe multi-niche crowding GA where a combination of mating and environmentalreplacement guarantees niching. Also, Liles and De Jong (1999) combined a re-stricted mating mechanism with sharing. In the thermodynamical GA by (Mori etal., 1996, 1998) a diversity measure is used in the selection to reach a good distri-bution of the search space. As already mentioned this technique was also combinedwith a memory (Mori et al., 1997).

All applications are alternating problems. Both Cedeno and Vemuri (1997) andLiles and De Jong (1999) use a swelling and shrinking problem without coordinatetranslations. (Ursem, 2000) add coordinate translations with small severity in hisstudy. The examinations of Mori et al. (1996, 1997, 1998) use dynamic knapsackproblems which are also alternating.

6.4.3 Restricted mating

This set of techniques preserves diversity by dividing the population into several“subpopulations” and restricting the recombination operator to individuals of thesame subpopulation. Two approaches using common distributed algorithms arethe cellular GA with a toroidal grid as topology (Kirley & Green, 2000) and thespatially distributed GA in a 15x15 grid (Sarma & De Jong, 1999). The taggedGA by Liles and De Jong (1999) assigns each individual a tag and restricts matingto individuals with equal tags. In the multinational GA by Ursem (2000) eachindividual is assigned to a subpopulation called nation. A valley detection can re-assign individuals to other or new nations.

Two of the applications are alternating swelling and shrinking problems: Liles andDe Jong (1999) use constant coordinates and Ursem (2000) coordinate translationswith small severity values. The other problems have no fitness rescalings but vary-ing coordinate severity and a frequency of change of 20−1 (Sarma & De Jong, 1999;Kirley & Green, 2000)

6.5 Adaptive and self-adaptive techniques

Adaptation and self-adaptation are very popular and successful in static optimiza-tion. Therefore, it is sensible to apply those techniques to a non-stationary environ-ment where also the characteristics of the search space keep changing. Adaptiveand self-adaptive EP has been the focus of a publication of Angeline (1997). Self-

93

Page 94: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

adaptive EP has also been investigated by (Saleem & Reynolds, 2000). Back (1997,1998) and (Salomon & Eggenberger, 1997) examine self-adaptive ES with the log-normal update rule. Genetic algorithms with a self-adaptive mutation rate havebeen the target of work by Back (1997, 1999) and Grefenstette (1999). The latterpublication also considered a self-adaptive hypermutation rate.

An adaptive technique was used in the thermodynamical GA by Mori et al. (1998)to update a parameter, called the temperature.

Since most adapation techniques are strongly related to local variation, the prob-lems have almost similar properties. Most problems are again characterized bymissing fitness rescaling and rather small coordinate severity. There are driftinglandscapes (Angeline, 1997; Back, 1998; Salomon & Eggenberger, 1997; Saleem& Reynolds, 2000), rotating problems (Angeline, 1997; Back, 1998), and randomlydrifting problems (Angeline, 1997; Back, 1998).

Applications on problems with rather big severity values have usually a very lowfrequency of change (Back, 1997, 1999). Also the non-repetitive dynamic knap-sack problem used in the work of Mori et al. (1998) has a frequency of change of0.01.

6.6 Algorithms with overlapping generations

Algorithms with overlapping populations have been a research topic especially inthe context of genetic algorithms (steady state GA). Vavak and Fogarty (1996) werethe first to use a steady state GA on a dynamic problem. Smith and Vavak (1999)examined various replacement strategies. They show that the steady state model isoften better suited to dynamic problems than the generational model. However, thereplacement strategy must be chosen appropriately to guarantee good performance.The empirical investigation of several strategies showed that deletion of the oldestor a random individual leads to poor performance. A good strategy seems to be amodification of the deletion of the oldest: each the parents is selected using a binarytournament beween a random individual and the oldest individual in the population.This scheme guarantees reevaluation of individuals and an implicit mechanism forelitism. Cedeno and Vemuri (1997) use a worst among most similar replacementmethod. A rather sophisticated adaptive replacement strategy was introduced byDozier (2000) for a path planning problem. A comparable approach for a specificapplication was also presented by Stroud (2001).

The problems tackled in these publications are very distinct. Since the method ofoverlapping generations does not seem to depend on the kind of the dynamics and

94

Page 95: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6.7. NON-LOCAL ENCODING

there are only few general publications, the classification of the problems is omittedhere.

6.7 Non-local encoding

Standard genetic algorithms are also often applied to dynamic problems—usuallyto demonstrate the superiority of a new technique. However, there are few studieswhere genetic algorithms are the main topic of an examination, e.g. in the publica-tions of Salomon and Eggenberger (1997), Stanhope and Daida (1998), and Vavakand Fogarty (1996).

Based on the results pre-published by Weicker and Weicker (2000), it can be de-duced that the good results of GAs on certain dynamic problems are rather due tothe encoding of the search space than some parallel schema/hyperplane search. Thegeneral details of the examination are resembled in Section 8.1. The particular ge-netic algorithm uses standard binary encoding of each search space dimension with16 bits, population size 100, crossover rate 0.6, mutation rate (Number of Bits)−1,and fitness proportional selection with linear scaling.

The results on a rotating Rastrigin function are shown in Figure 6.1 and demon-strate a continual convergence of the GA towards the optimum. Only in the caseof very fast rotations the performance is slightly worse. This result is in particu-lar interesting since the evolution strategy based on local variation is not able toconverge to the optimum (see Section 8.1). Apparently the good performance isnot due to recombination of “building blocks” since the GA without recombina-tion always outperforms the standard GA. Therefore the advantage seems to be inthe different neighbourhood structure induced by a binary encoding—supportinga very diffuse neighborhood for each point in the search space. However, smalldeteriorations after each quarter cycle of the rotaiton show that the encoding hasalso its disadvantages. Presumably due to huge Hamming cliffs the algorithm hassevere problems to follow the currently best known fitness region.

In our experiment only one example could be found where recombination has astatistically significant positive effect on the behavior of a genetic algorithm. Theexperiments use a rotating cone segment and the cycle time of the rotation is veryfast, i.e. after 5 generations the dycle is complete. The details of the experimentsare again described in Section 8.1. Figure 6.2 shows the results. Apparently therecombination helps to generalize from five different alternating landscapes with acommon optimum. This does not result in a significant advantage concerning thefitness values but in a significantly shorter distance to the optimum. However, the

95

Page 96: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

GAGA, mut.

1.2

1.6

2

2.4

2.8

3.2

0 40 80 120 160 200generation

cycle time 5

0

50

100

150

200

250

300

0 40 80 120 160 200generation

cycle time 50

0.5

1

1.5

2

2.5

3

3.5

0 40 80 120 160 200generation

cycle time 200

0

50

100

150

200

250

0 40 80 120 160 200generation

cycle time 100

0.5

1

1.5

2

2.5

3

3.5

0 40 80 120 160 200generation

cycle time 100

0.5

1

1.5

2

2.5

3

3.5

0 40 80 120 160 200generation

cycle time 50

fitn

ess

fitn

ess

dis

tance

toopti

mum

dis

tance

toopti

mum

dis

tance

toopti

mum

dis

tance

toopti

mum

Figure 6.1 Rotating Rastrigin function optimized by genetic algorithms. The ex-perimental setup is explained detailed in Section 8.1.1.

96

Page 97: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6.7. NON-LOCAL ENCODING

significance for rec.

significance for rec.

significance for no rec.

significance for rec.

GAGA, mut.

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 5

dis

tance

toopti

mum

0

1

2

3

4

5

6

7

8

0 40 80 120 160 200generation

avg.

dis

tance

toop

t.

-2

0

2

4

6

0 40 80 120 160 200generation

bes

tfitn

ess

1

2

3

4

5

6

7

8

0 40 80 120 160 200generation

bes

tdis

tance

toopt.

Figure 6.2 Rotating cone segment with fast severity (5 generations for one com-plete cycle) optimized by genetic algorithms. The Student’s t-testshows that the recombination improves the performance of the GA onthe rotating cone segment with cycle time 5 (as far as the distance to theoptimum is concerned). The experimental setup is explained detailedin Section 8.1.1.

properties of this dynamic problem are probably closer to a noisy function than toa usual dynamic problem.

Therefore, a conclusive indication for the usefulness of schema processing in dy-namic environments is still missing.

97

Page 98: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

6.8 Learning of the underlying dynamics

Algorithms that predict the next changes in a non-stationary environment are promis-ing. In a very broad sense we could even consider self-adaptive mechanisms (e.g.Angeline, 1997; Back, 1998) as one possible technique to derive knowledge con-cerning the dynamics. However, we will argue in the next chapter (Section 7.8.2)that the standard self-adaptation techniques are not suited to adapt a step-size pa-rameter in all possible dynamic situations. Also the cultural algorithms as they areused by Saleem and Reynolds (2000) are not suited to derive knowledge on thedynamics—they are, similar to self-adaptation, only concerned with useful param-eter values.

I am only aware of one consequent approach, namely the work by Munetomo etal. (1996), where the derivation of knowledge concerning the dynamics is realized.The environment is modeled as a stochastic learning automata. As a consequencethis approach is also only applicable on a rather narrow class of problems. Inthis thesis a technique to derive the meta-rule of dynamics of drifting problems isproposed in Section 8.4.

6.9 Resulting Problem-Techniques Mapping

Based on the above analysis of existing applications of dynamic optimization tech-niques to non-stationary problems, a first mapping between problem characteristicsand techniques can be derived.

Figure 6.3 shows the resulting map. Due to the rather facile character of the liter-ature analysis and the difficulties of categorizing problems described with varyingdegree of detail, the result is only a first impression how such a mapping could looklike.

Certain compromises have been made when drawing the table in Figure 6.3. Forexample the entry of implicit memory in the case of static fitness and rotating co-ordinate transformations is derived from a problem with quite similar propertiesstemming from a sinusoidal translation in one search space dimension.

The dynamic problem classes can be seen in the figure where certain techniques areprobably well suited for (or even designed for). However, the resulting Figure 6.3should not be considered as an overall truth since it only reflects where researchershave applied a certain technique successfully. Therefore, a missing entry does notimply that a technique should not be used in a specific context. As a consequence,a detailed interpretation of the resulting problem-techniques mapping is omitted

98

Page 99: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6.9. RESULTING PROBLEM-TECHNIQUES MAPPING

swell/shrink

unaffecting

random

returning

chaotic

static drifting rotating rand. drift fast drift superpose chaotic

coordinate transformation

static

fitne

ssre

scal

ing

expl. mem.

expl. mem.impl. mem.div. incr.nichingrestr.mating

niching

restricted mating

diversity increasing

local variation

restricted mating

big fr. of ch.

small/med. fr.ch.

impl. mem.

smallfr.ch.loc.var. small

fr.ch.loc.var.

schema?

non-localencoding

Figure 6.3 A first mapping between problem characteristics and techniques basedon a facile analysis of the literature.

99

Page 100: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

6. TECHNIQUES FOR DYNAMIC ENVIRONMENTS

here.

6.10 Discussion

This chapter demonstrates how the framework of Chapter 4 can be used to integratethe research of mannifold publications into a common knowledge basis. Also thenecessity of a systematic study becomes obvious to create a complete overview ontechniqes and problem classes. As soon as the different gaps of Figure 6.3 are filledin the map might serve as a decision criterion for the application or development ofalgorithms for certain problems.

In addition, future work should consider in depth the different reasons why certaintechniques are applied in a specific domain. There are three different tasks anevolutionary algorithm might face in a dynamic environment, namely

• the tracking of “good” regions in the search space,

• the ability to manage an inhomogenous dynamic search space, and

• the task to find an optimum in a changing environment.

We can expect that in different problem classes different techniques are sensiblefor the three tasks. Probably this is a good explanation for the success of manytechniques that mix various of the previously described techniques (Ursem, 2000;Saleem & Reynolds, 2000).

100

Page 101: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 7

Analysis of Local Operators forTracking

The literature analysis of the previous chapter has shown that local operators areprimarily used for problems with a drifting character. However there is no advice inthe literature how to choose and tailor a local operator to a specific problem. Thischapter analyzes an exemplary local operator concerning the parameter calibrationand the limits of its performance using a theoretical model. The findings of theanalysis result in a set of design rules that should simplify the development of newevolutionary algorithms for tracking tasks.

The local operator analyzed in this chapter takes up essential properties of the prob-ably most prominent and successful local operator, the evolution strategy mutation(see Section 2.2.2). This operator is characterized by the following principles:

1. zero-mean: an average neutrality implies that an object variable may be in-creased with the same probability as it may be decreased,

2. small changes occur with a higher probability than big changes,

3. each point in the search space is reachable by a mutation in one step with aprobability greater than zero,

101

Page 102: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

4. mostly a Gaussian distribution is used, but there have been also works withCauchy (Yao & Liu, 1996) and Laplace (Montana & Davis, 1989) distribu-tions, and

5. self-adaptive techniques are used to adjust the operator to the fitness land-scape.

For principles (1), (2), and (4) see also the book by Schwefel (1977). Principle(3) is of essential importance if a global convergence in static fitness landscapes isshown (see Rudolph, 1997).

The local operator in this chapter is defined on the discrete search space Z × Zin order to simplify the examinations. The operator is also zero-mean, preferssmall changes over big changes, and uses the binomial distribution which is closelyrelated to the Gaussian distribution. The reachability of any point in the searchspace by one application of the mutation operator is omitted here since the focus ison the local tracking of an optimum. Issues like escaping from a local optimum arenot considered because the area of interest in the search space is very narrow andthe probability of hitting a moving target by random probes can be ignored froma practical viewpoint. Also self-adaptation is difficult to model in our theoreticalframework. As a consequence the major part of the analysis is done with a fixedbut arbitrary value for the step size parameter.

Note that the analysis focuses on an as exact as possible analysis of a local operatorin a low dimensional search space. This is motivated by the dimensionality of mostexisting moving peaks problems. Also, rather complex problem dynamics wheredrifting is only one aspect are presumably only manageable in few dimensions. Forsimple high dimensional problems results may be obtained using the sphere modelfrom ES theory (Beyer, 2001). Arnold and Beyer (2002) presented a first result fora randomly drifting peak.

This chapter is organized as follows. Section 7.1 presents a few basic definitionsand derives two different Markov chain models for the optimization of an unimodalproblem and the local mutation operator. In Section 7.2 a worst-case analysis isused to derive minimal requirements for a successful tracking behavior. Optimalparameter settings concerning the tracking rate are investigated in Section 7.3. Inthe next two sections two of the underlying principles of ES-mutation are ques-tioned within the context of tracking problems: the zero-mean mutation in Sec-tion 7.4 and the preference of small steps over big steps in Section 7.5. The in-fluence of the population size on the performance in a non-stationary problem isexamined in Section 7.6. The combination of a local operator with a memoriz-ing technique is analyzed in Section 7.7. Section 7.8 discusses several issues of

102

Page 103: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

self-adaptation and adaptation related to the considered problem. And Section 7.9summarizes and concludes.

7.1 Theoretical framework

For the major part of this analysis, a unimodal problem is assumed where the fitnesscorresponds to the distance to the optimum. As a discrete two-dimensional searchspace Z× Z is chosen.

The distance within the model is defined as the number of vertical and horizontalcrossings of raster boundaries of the search space.

Definition 7.1 (Distance metric) For two points B = (B1, B2) and C = (C1, C2)in the search space Z× Z the distance is defined as

dist(B,C) = d(B1, C1) + d(B2, C2). (7.1)

with d(x, y) = |x− y|. ♦

This distance metric is used to define the non-stationary optimization problem.The dynamics are introduced by moving the optimum horizontally as stated in thefollowing definition.

Definition 7.2 (Tracking problem) The tracking problem is defined by the start-ing position of the optimum A∗(0), the severity of the dynamics s, and the trackingtolerance δ ∈ N. The position of the optimum at generation t ∈ N0 is defined as

A∗(t+ 1) = A∗(t) + S (7.2)

with S = (s, 0)T . The task for an optimization algorithm ALG is to produce ateach time step t a tolerable approximation A(t) for the position of the optimum:

dist(A(t), A∗(t)) ≤ δ (7.3)♦

Figure 7.1 illustrates the definition of the tracking problem. For severity 6 theoptimum and the respective tolerable points with a tracking tolerance 2 are shown.

To simplify the following computations and notations the number of points withina distance to a given point is introduced.

103

Page 104: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

5

6

6

6

7

7

7

8

8

severity = 6

8

8

8

4

Figure 7.1 The gray point marks the position of the optimum in the last genera-tion which was now moved horizontally six steps. The region on theright denotes those points which are within a distance 2 of the new op-timum’s position. The numbers in the squares denote the distance tothe previous optimum.

Definition 7.3 (Number of points) For an arbitrary point B ∈ Z×Z in the searchspace, the number of points with distance d and respective within distance d aredenoted by

NB(d) = # C ∈ Z× Z | dist(B,C) = dNB(d) = # C ∈ Z× Z | dist(B,C) ≤ d ♦

Note, that according to the following lemma the numbers are independent of pointB. As a consequence the subscript B is usually dropped (N(d) and N(d)).

Lemma 7.1 For any B ∈ Z× Z:

NB(d) =

4d, if d > 01, otherwise d = 0

(7.4)

NB(d) = 2d(d+ 1) + 1. (7.5)♦

Proof: For all points C with dist(B,C) = d, it follows from Definition 7.1 that

|B1 − C1|+ |B2 − C2| = d.

104

Page 105: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

Then there exists 0 ≤ a ≤ d with |B1 − C1| = a and |B2 − C2| = d − a. Allpossible first coordinates of C can be described by C1 ∈ B1 − d, . . . , B1 + d.Then, the following values C2 result immediately:

• C2 = 0 if C1 ∈ B1 − d,B1 + d and

• C2 ∈ B2 − (d − a), B2 + (d − a) for C1 ∈ B1 − a, . . . , B1 + a and0 ≤ a ≤ d− 1.

The proof for NB(D) is finished by counting all points. Also NB(d) follows im-mediately.

NB(d) =d∑i=0

NB(i)

= NB(0) +d∑i=1

NB(i) = 1 + 4d∑i=1

i

= 1 + 4d(d+ 1)

2q.e.d.

In the framework of this examination, a simple local search algorithm using a(1, λ)-selection strategy is investigated, i.e. λ new offspring are created and thebest offspring replaces the current individual. The mutation operator is definedin the discrete search space by the binomial pdf mimicking the ES-mutation withthe Gaussian pdf. How the binomial pdf is used in the mutation is illustrated inFigure 7.2.

Definition 7.4 (Local mutation) A local mutation, applied to individualA, resultsin B = A+X where X ∈ Z×Z is a random variable. The respective probabilitydensity function on the discrete search space is defined as

Prp[X = x] =

p(dist(~0, x))N(dist(~0, x))

(∑C∈R p(dist(~0, C))N(dist(~0, C))

)−1

,

if x ∈ R0, otherwise

(7.6)

where R = C | dist(~0, C) ≤ maxstep with maximal step size maxstep and abasic one-dimensional pdf p.

The local mutation operator is defined by p = plocal using the following binomialdistribution for 0 ≤ d ≤ maxstep:

plocal(d) =1

22maxstep

(2maxstep + 1

maxstep − d

)(7.7)♦

105

Page 106: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

Figure 7.2This figure illustrates the usage of thebinomial pdf to define the mutation op-erator. The pdf is used to assign a prob-ability to each point for being the resultof a mutation. It resembles the princi-ples of evolution strategies.

Figure 7.3 shows the resulting probability density function exemplary.

-100

10 -100

10

0

0.014

Figure 7.3 Resulting probability density function for the local standard mutation.

7.1.1 Exact Markov chain model

In this section, the dynamics of the described local search algorithm to the trackingproblem are modeled exactly using a Markov chain. This model is used to deriveexact results concerning the tracking behavior.

In this model as well as in the worst-case model described in the next section, themodel is simplified considerably by keeping the optimum at a stationary pointA∗ =~0 = (0, 0)T in the search space. The original problem’s movement of the optimumis transformed into a negative movement of the current best approximation. Bothdescriptions of the tracking problem are equivalent to each other. They are shown

106

Page 107: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

in Figure 7.4. A similar modeling approach was chosen independently by Droste(2002).

Model with static optimum:

A

δ1

S

X

A∗

δ2A− S +X

A− S

Moving optimum:

A A∗ S

A∗ + Sδ1

δ2A+X

X

Figure 7.4 The left diagram shows the situation according to the problem defini-tion: the optimum A∗ moves according to S, and the best approxima-tion A moves according to X . However, the Markov chain model iseasier to define if the optimum stays at the same position and the neg-ative severity S is applied to the approximation A, as it is shown in theright diagram.

The states of the Markov chain are defined as the relative position of the current bestsolution candidate to the optimum. However, only a window around the optimumis considered, i.e. those points in the search space that have a distance greater thanradius to the optimum are unified in an absorbing state absorb.

States =A ∈ Z2 | dist(A,~0) ≤ radius

∪ absorb.

The limitation of the number of States is necessary to keep the simulation of theEA dynamics feasible. The probability to be in the absorbing state is an indicatorwhether the algorithm stays close to the optimum A∗ and whether other derivationsof those simulations are accurate.

In the subsequent paragraphs, for an arbitrary point A in the search space the tran-sition probability is derived that the best individual of λ randomly created off-springs is positioned at A+X − S. Again X equals the effect of the mutation andS = (s, 0)T is the change of the optimum from one generation to the next.

As a first step, the probability is computed that an offspring has the exact distanceδ to the new optimum. This probability results as

Prp[dist(A+X − S,~0) = δ] =∑

x ∈ Z× Zdist(A+ x− S,~0) = δ

Prp[X = x].

107

Page 108: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

Note, that the x with Pr[X = x] > 0 are described by the set R in Definition 7.4.The probability to be further away than distance δ follows as

Prp[dist(A+X − S,~0) > δ] =

dist(A−S,~0)+maxstep∑i=δ+1

Prp[dist(A+X − S,~0) = i]

= 1−δ∑i=0

Prp[dist(A+X − S,~0) = i].

This probability may be used to compute the probability that the best of λ offspringof parent A is placed at A+X − S.

Lemma 7.2 When creating λ offspring of parent A under given severity S, theprobability that the offspring with minimal distance to A∗ = ~0 is positioned atA+X − S = A+ x− S = y with dist(y,~0) = δ is given by

Prp[best1≤j≤λ(A+Xj − S) = y | X1, . . . , Xλ] =λ∑i=1

[(λi

) (Prp[dist(A+Xi − S,~0) > δ]

)λ−i i∑k=1

((Prp[X = x])k(

Prp[dist(A+X − S,~0) = δ]− Prp[X = x])i−k (

i−1k−1

))]where selection is uniform among several offspring with minimal distance andX1, . . . , Xλ are independent and identical distributed as X . ♦

Proof: The probability that λ − i individuals (0 ≤ i < λ) are further away thandistance δ equals (

λi

) (Prp[dist(A+X − S,~0) > δ]

)λ−i.

Then the remaining i offspring individuals have distance δ to the optimum. Now,the probability that k of those offspring individuals (1 ≤ k ≤ i) are placed at thetarget spot in the search space equals

(Prp[X = x])k(Prp[dist(A+X − S,~0) = δ]− Prp[X = x]

)i−k (ik

).

The probability to choose one of those k individuals uniformly equals ki. And the

lemma follows immediately. q.e.d.

Then the exact Markov chain model of the dynamic optimization is given in thefollowing definition.

108

Page 109: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

Definition 7.5 (Exact local Markov chain model) The exact local Markov chainmodel for severity S = (s, 0)T , a mutation defined by pdf p, and λ offspring isgiven by the tuple (States , T ) where

States =A = (x, y) ∈ Z2 | |x|+ |y| ≤ radius

∪ absorb.

and the transition matrix T = States × States is given by the following equationswith A,B ∈ States .

T [A→ B] = Prp[best1≤i≤λ(A+Xi − S) = B | X1, . . . , Xλ]

T [A→ absorb] = 1−∑

B∈States\absorb

Prp[best1≤i≤λ(A+Xi − S) = B | X1, . . . , Xλ]

T [absorb → absorb] = 1

T [absorb → A] = 0. ♦

Example 7.1 The exact Markov chain model is illustrated with a small example.For simplicity the probability density function p in Figure 7.5 is used which doesnot correspond to the mutations defined above. The maximal step size maxstep =2, population size λ = 3, parent individual A = (−1,−1)T , and severity S =(1, 0)T is used. Figure 7.6 shows the best individual as a black frame, the current

Figure 7.5Exemplary mutation distribution. The square in thecenter is the current position.

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

optimum as a gray square and the movement of the optimum from one generationto the next.

The probability to produce an offspring at distance δ equals

Prp[dist(A+X − S,~0) = δ] =

0, if δ = 00.1, if δ = 10.2, if δ = 20.35, if δ = 30.2, if δ = 40.15, if δ = 50, if δ > 5

109

Page 110: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

Figure 7.6Scenario: The square in the center is the current positionof the best individual. The gray square is the optimumwhich moves one position to the right.

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

δ = 1

0.1

δ = 4

0.2

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

0.05

0.05

0.05

0.05

0.05

0.05

0.05

0.05 0.1

0.1

0.10.1 0.2

δ = 2

0.2

δ = 5

0.15

δ = 0

0

δ = 3

0.35

Figure 7.7 The computation of the probability to produce an offspring at distanceδ.

as it is also shown in Figure 7.7 and the probability to produce an offspring that isfurther away than δ equals

Prp[dist(A+X − S,~0) > δ] =

1, if δ = 00.9, if δ = 10.7, if δ = 20.35, if δ = 30.15, if δ = 40, if δ ≥ 5

.

The transition probability is calculated exemplary for the case that the offspringis placed at the same position like the parent, while the optimum moves one step

110

Page 111: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

further away. Using d = dist(A+X − S,~0) the probability equals

Prp[best((−1,−1)T +X − (1, 0)T ) = (−2,−1)T | X1, X2, X3]

=(

31

)Prp[d > 3]2

(Prp[X = ~0]1(Prp[d = 3]− Prp[X = ~0])0

(00

))+(

32

)Prp[d > 3]1

(Prp[X = ~0]1(Prp[d = 3]− Prp[X = ~0])1

(10

)+

Prp[X = ~0]2(Prp[d = 3]− Prp[X = ~0])0(

11

))+(

33

)Prp[d > 3]0

(Prp[X = ~0]1(Prp[d = 3]− Prp[X = ~0])2

(20

)+

Prp[X = ~0]2(Prp[d = 3]− Prp[X = ~0])1(

21

)+

Prp[X = ~0]3(Prp[d = 3]− Prp[X = ~0])0(

22

))= 3 · 0.352 · (0.21 · 0.150 · 1) +

3 · 0.351 · (0.21 · 0.151 · 1 + 0.22 · 0.150 · 1) +

1 · 0.350 · (0.21 · 0.152 · 1 + 0.22 · 0.151 · 2 + 0.23 · 0.150 · 1)

= 0.1715 ♦

All resulting transition probabilities are shown in Figure 7.8.

Figure 7.8The resulting probabilities for an offspring popu-lation size λ = 3.

0.043

0.001

0.172

0.193

0.043

0.02

0.193 0.136

0.1360.043

0.001 0.02

0.001

This exact Markov chain model is used in the remainder of this chapter for thederivation of exact results and for simulations how the state distribution is changingover several generations.

7.1.2 Worst-case Markov chain model

In this section a simplified Markov chain model is presented with the goal of aone-dimensional state space. The basis for this model is the assumption that the

111

Page 112: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

current individual is always situated at the horizontal line drawn through the opti-mum (see Figure 7.9), i.e. that there is no vertical deviation relative to the optimum.Independently from the actual position of a new individual, the mere distance is of

optimum

current individual

Figure 7.9 Simplified situation for the worst-case model: it is assumed that thenext individual is always chosen on the horizontal line through the op-timum. That implies that the probabilities to reach the light gray points(with distance 2 to the optimum) are unified in the light gray point atthe horizontal line.

interest and in the further steps of the approximation the point on the horizontalline is assumed. This situation is shown in Figure 7.10 where all points with thesame distance are mapped to the same transition from the current distance to thenew distance. At the bottom of the figure the resulting Markov chain is shown: Thestates States = N0 are the distances to the optimum and the transition probabilitiesare summarized probabilities as sketched in Figure 7.10.

The relevance of this modeling is pointed out by the following lemma which is thebasis for the consideration of the model as a worst-case scenario.

Lemma 7.3 If the considered mutation operator is zero-mean, the situation de-scribed above is a worst-case scenario with regard to the probability to get at leastas close as distance d to the optimum (d = 0, 1, 2, . . .). ♦

Proof: The proof is omitted here. Instead, Figure 7.11 illustrates a plausibilityargument that the probability to hit the optimum or get close to the optimum in-creases in any other possible scenario. It follows immediately from the figure thatthe probability to get at least as close as distance d increases in any other scenariofor arbitrary d. q.e.d.

112

Page 113: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

01234

Figure 7.10 This figure shows exemplarily which points of the probability den-sity function are summarized and assigned to which transition of theMarkov chain. The optimum moves one step to the right. Last gener-ations state of the Markov chain was 2.

For the subsequent discussion of this model we can distinguish four different sce-narios how distance d′ of the current approximation to the optimum, the severitys, the maximal step size maxstep, and the new distance d to the optimum relate toeach other.

(i) (s+ d′ > d) ∧ (maxstep ≥ s+ d′ − d)

(ii) (s+ d′ ≤ d) ∧ (maxstep > d− s− d′)(iii) (s+ d′ ≤ d) ∧ (maxstep ≤ d− s− d′)(iv) (s+ d′ > d) ∧ (maxstep < s+ d′ − d)

The four situations are described visually in Figure 7.12 The range of X = xleading to the new distance d to the optimum is restricted for the different cases asfollows.

(i) s+ d′ − d ≤ dist(x,~0) ≤ mins+ d′ + d,maxstep(ii) d− s− d′ + 1 ≤ dist(x,~0) ≤ mins+ d′ + d,maxstep

(iii) 0 ≤ dist(x,~0) ≤ mind− s− d′,maxstep(iv) none

113

Page 114: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

132231

33

33

132231

33

13

33

2231

3331

33

13

13

33

33

22

22

31

22

31

33

13

13

2231

2233

31

33

13

13

13

33 11

31

31

possible positions of theoptimum at distance 3 tothe current individual

possible new positionsof the optimum withseverity 1

worst-casescenario

2222

20

02

current individual

Figure 7.11 The upper left part of the figure shows the possible positions of anoptimum at distance 3 moved by severity 1. The other parts of thefigure show different scenarios for the optimum’s new position. Thepossible offspring with distance 1, 2, or 3 to the new optimum areshown. The number ij indicates an offspring with distance i to thenew optimum and distance j to the parental individual. Apparentlyfor all distances the numbers in the worst-case scenario in the upperright part are a subset of the numbers in any other scenario.

114

Page 115: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

A A∗ + S

d

(iv)

d

A∗ + SA

(ii)

d

A A∗ + S

(iii)

d

A∗ + SA

(i)

Figure 7.12 The four different cases used in the following computations: thedashed lines mark the possible values of the maximal step parame-ter of the mutation, the solid line refers to those points with distanced to the new optimum.

The following lemma makes a statement on the number of points within distanced to the optimum, that can be reached from a point with distance s + d′ from theoptimum by a mutation of exact step size δ.

Lemma 7.4 Given point A with distance d′ from optimum ~0. After the optimummoving by s and mutating A with X at exact step size dist(X,~0) = δ (accordingto the valid values given above), the number of distinct newly created points withindistance d from the optimum is

Ndist(A+Xδ−S,~0)≤d =

2d+ 1− 2⌊s+d′+d+1−δ

2

⌋, iff (i) or (ii)

N(δ), iff (iii)0, otherwise

115

Page 116: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

The number given in the lemma specifies the number of points on the dashed linesthat are within or at the borderline of the solid drawn rectangle in Figure 7.12.

Proof: In case (iii) all points with distance δ from A are within distance d to thenew optimum, resulting in N(δ). In case (iv) the range of the mutation and thepoints within distance d of the optimum do not overlap. As a consequence thereare no points. In case (i) the valid range of X = x is given as

s+ d′ − d ≤ dist(x,~0) ≤ mins+ d′ + d,maxstep

For δ = s− d′ − d it follows

Ndist(A+Xδ−S,~0)≤d = 2d+ 1− 2

⌊s+ d′ + d+ 1− (s− d′ − d)

2

⌋2d+ 1− 2d = 1

This corresponds to the most left dashed line in the figure for case 1 where there isan overlap of just one point. As one can easily verify increasing δ by 2 leads to anincrease of 2 for N . The upper bound follows similarly. The same considerationshold for case (ii) where the lower bound δ = d − s − d′ + 1 leads immediatelyto 2(d − s − d′) + 1 corresponding to the innermost dashed line in the figure ofscenario (ii). q.e.d.

Lemma 7.5 Given point A with distance d′ from optimum ~0. After moving theoptimum by s and mutating A with maximal step size maxstep, the probability tohit a point within distance d from the optimum, the hitting probability, results as

Pr[dist(A+X − S,~0) ≤ d] =

∑s+d′−d≤δ≤mins+d′+d,maxstep

(2d+ 1− 2

⌊s+d′+d+1−δ

2

⌋)Pr[dist(X,~0)=δ]

N(δ), if (i)∑

0≤δ≤d−s−d′ Pr[dist(X,~0) = δ]

+∑

d−s−d′+1≤δ≤mins+d′+d,maxstep(2d+ 1− 2

⌊s+d′+d+1−δ

2

⌋) Pr[dist(X,~0)=δ]N(δ)

,

if (ii)∑0≤δ≤mind−s−d′,maxstep Pr[dist(X,~0) = δ], if (iii)

0, otherwise

Proof: The formula results immediately if the possible values for |X| and the num-bers of Lemma 7.4 are substituted into

Pr[dist(A+X − S,~0) ≤ d] =∑

s+d′−d≤δ≤s+d′+d

Ndist(A+Xδ−S,~0)≤d

N(δ)Pr[dist(X,~0) = δ].

q.e.d.

116

Page 117: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

Figure 7.13Scenario adapted to the worst-case model: Thesquare in the center is the current position of the bestindividual and the gray square, situated at a horizon-tal line to the individual, is the optimum which movesone position to the right.

Example 7.2 This example uses again the exemplary mutation distribution in Fig-ure 7.5. The modified scenario for the worst-case model is shown in Figure 7.13.Then s = 1, d′ = 2, and maxstep = 2. In this example we write Pr[δ] as a shortcut for Pr[dist(X,~0) = δ].

For d = 0, case (iv) holds. As a consequence

Pr[dist(A+X − S,~0) ≤ 0] = 0.

For d = 1, case (i) holds and

Pr[dist(A+X − S,~0) ≤ 1] =∑

1+2−1≤δ≤min1+2+1,2

(2 + 1− 2

⌊1 + 2 + 1 + 1− δ

2

⌋Pr[δ]

N(δ)

)

= 2 + 1− 2

⌊1 + 2 + 1 + 1− 2

2

⌋Pr[2]

N(2)︸ ︷︷ ︸=0.05

= 0.05.

For d = 2, case (i) holds too and

Pr[dist(A+X − S,~0) ≤ 2] =∑

1+2−2≤δ≤min1+2+2,2

(4 + 1− 2

⌊1 + 2 + 2 + 1− δ

2

⌋Pr[δ]

N(δ)

)

=

(4 + 1− 2

⌊1 + 2 + 2 + 1− 1

2

⌋)Pr[1]

N(1)︸ ︷︷ ︸=0.1

+

(4 + 1− 2

⌊1 + 2 + 2 + 1− 2

2

⌋)Pr[1]

N(1)︸ ︷︷ ︸=0.05

= 0.15.

For d = 3, case (ii) holds and

Pr[dist(A+X − S,~0) ≤ 3]

117

Page 118: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

=∑

0≤δ≤3−1−2

Pr[δ] +∑

3−1−2+1≤δ≤min1+2+3,2

(6 + 1− 2

⌊1 + 2 + 3 + 1− δ

2

⌋)Pr[δ]

N(δ)

= Pr[0]︸ ︷︷ ︸=0.2

+

(6 + 1− 2

⌊1 + 2 + 3 + 1− 1

2

⌋)︸ ︷︷ ︸

=1

Pr[1]

N(1)︸ ︷︷ ︸=0.1

+

(6 + 1− 2

⌊1 + 2 + 3 + 1− 2

2

⌋)︸ ︷︷ ︸

=3

Pr[2]

N(2)︸ ︷︷ ︸=0.05

= 0.45.

For d = 4, case (ii) holds too and

Pr[dist(A+X − S,~0) ≤ 4]

=∑

0≤δ≤4−1−2

Pr[δ] +∑

4−1−2+1≤δ≤min1+2+4,2

(8 + 1− 2

⌊1 + 2 + 4 + 1− δ

2

⌋)Pr[δ]

N(δ)

= Pr[0]︸ ︷︷ ︸=0.2

+Pr[1]︸ ︷︷ ︸=0.4

+

(8 + 1− 2

⌊1 + 2 + 4 + 1− 2

2

⌋)︸ ︷︷ ︸

=3

Pr[2]

N(2)︸ ︷︷ ︸=0.05

= 0.75.

For d = 5 (and also d > 5) case (iii) holds and

Pr[dist(A+X − S,~0) ≤ 5] =∑

0≤δ≤min5−1−3,2

Pr[δ]

= Pr[0] + Pr[1] + Pr[2] = 1.0. ♦

Figure 7.14 illustrates the computations in the example.

118

Page 119: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

0.05

0.05

0.05

0.05 0.05

0.05

0.050.1

0.1

0.1

0.1 0.2

0.05

d = 0: 0.0d = 1: 0.05

d = 2: 0.15d = 3: 0.45

d = 4: 0.75d = 5: 1.0

Figure 7.14 The resulting probabilities for Pr[dist(A+X − S,~0) ≤ d].

Lemma 7.6 Given point A with distance d′ from optimum ~0. After the optimummoving by s and mutating A with maximal step size maxstep, the probability to beat exact distance d to the optimum ~0 results as

Pr[dist(A+X − S,~0) = d | dist(A,~0) = d′] =

119

Page 120: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

Pr[dist(A+X − S,~0) ≤ 0],iff d = 0

0, iff (maxstep < s+ d′ − d) ∧ (s+ d′ > d > 0)

Pr[maxstep]N(maxstep)

, iff (maxstep = s+ d′ − d) ∧ (s+ d′ > d > 0)

Pr[s+d′−d]N(s+d′−d)

+∑

s+d′−d+1≤δ≤maxstep 2χ(s+ d′ + d− δ)Pr[δ]N(δ)

,

iff (s+ d′ − d < maxstep < s+ d′ + d)∧ (s+ d′ > d > 0)

Pr[s+d′−d]N(s+d′−d)

+∑

s+d′−d+1≤δ≤s+d′+d−1 2χ(s+ d′ + d− δ)Pr[δ]N(δ)

+ (2d− 1)Pr[s+d′+d]

N(s+d′+d),

iff (maxstep ≥ s+ d′ + d) ∧ (s+ d′ > d > 0)

Pr[0] +∑

1≤δ≤maxstep 2χ(2d− δ)Pr[δ]N(δ)

,

iff (s+ d′ = d) ∧ (maxstep < 2d)

Pr[0] +∑

1≤δ≤2d−1 2χ(2d− δ)Pr[δ]N(δ)

+ (2d− 1)Pr[2d]N(2d)

,

iff (s+ d′ = d) ∧ (maxstep ≥ 2d)

(N(d− s− d′)− 2d+ 1 + 2s+ 2d′)Pr[d−s−d′]

N(d−s−d′)

+∑

d−s−d′+1≤δ≤maxstep 2χ(s+ d′ + d− δ)Pr[δ]N(δ)

,

iff (d− s− d′ < maxstep < s+ d′ + d) ∧ (s+ d′ < d)

(N(d− s− d′)− 2d+ 1 + 2s+ 2d′)Pr[d−s−d′]

N(d−s−d′)

+∑

d−s−d′+1≤δ≤s+d′+d−1 2χ(s+ d′ + d− δ)Pr[δ]N(δ)

+ (2d+ 1)Pr[s+d′+d]

N(s+d′+d),

iff (maxstep ≥ s+ d′ + d) ∧ (s+ d′ < d)

(2maxstep + 1)Pr[maxstep]N(maxstep)

, iff (maxstep = d− s− d′) ∧ (s+ d′ < d)

0, iff (d− s− d′ > maxstep) ∧ (s+ d′ < d)

where χ(x) =

1, iff bx+1

2c = bx

2c

0, otherwise and the short cut Pr[δ] is again used for

Pr[dist(X,~0) = δ]. ♦

Proof: The first case is true for trivial reasons. The remaining cases of the lemmafollow immediately from

Pr[dist(A+X − S,~0) = d]

= Pr[dist(A+X − S,~0) ≤ d]− Pr[dist(A+X − S,~0) ≤ d− 1]

for d ≥ 1. Here, we have to clarify which situations might occur for d and d − 1.When situation (iv) holds for d − 1, either (iv) is also true for d (second line) or

120

Page 121: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

(i) is true for d (third line). Lines four and five deal with the transitions from (i) to(i). Lines six and seven distinguish two cases for the transition from situation (i)to (ii). Lines eight and nine show the formulas for the transition from (ii) to (ii).Situation (ii) at d− 1 leading to situation (iii) at d is handled by line ten. The lastline takes care of situation (iii) for both d − 1 and (d). Substitutions and simpletransformations lead to the formula given in the lemma. q.e.d.

Example 7.3 This example uses again the exemplary mutation distribution in Fig-ure 7.5 and the scenario in Figure 7.13 with s = 1, d′ = 2, and maxstep = 2.

For d = 0, the first line of Lemma 7.6 holds.

Pr[dist(A+X − S,~0) = 0] = Pr[dist(A+X − S,~0) ≤ 0] = 0.

For d = 1, the case (maxstep = s+ d′ − d) ∧ (s+ d′ > d) holds and

Pr[dist(A+X − S,~0) = 1] =Pr[2]

N(2)= 0.05.

For d = 2, case (s+ d′ − d < maxstep < s+ d′ + d) ∧ (s+ d′ > d) holds and

Pr[dist(A+X − S,~0) = 2] =Pr[1]

N(1)+

∑1+2−2+1≤δ≤2

2χ(1 + 2 + 2− δ)Pr[δ]N(δ)

= 0.1 + 2χ(3)︸︷︷︸=0

Pr[2]

N(2)= 0.1.

For d = 3, case (s+ d′ = d) ∧ (maxstep < 2d) holds and

Pr[dist(A+X − S,~0) = 3] = Pr[0] +∑

1≤δ≤2

2χ(6− δ)Pr[δ]N(δ)

= 0.2 + 2χ(6− 1)︸ ︷︷ ︸=0

Pr[1]

N(1)+ 2χ(6− 2)︸ ︷︷ ︸

=1

Pr[2]

N(2)︸ ︷︷ ︸0.05

= 0.3.

For d = 4, case (d− s− d′ < maxstep < s+ d′ + d) ∧ (s+ d′ < d) holds and

Pr[dist(A+X − S,~0) = 4] = (N(4− 1− 2)︸ ︷︷ ︸=4

−8 + 1 + 2 + 4)Pr[4− 1− 2]

N(4− 1− 2)︸ ︷︷ ︸=0.1

+

∑4−1−2+1≤δ≤2

2χ(1 + 2 + 4− δ)Pr[δ]N(δ)

= 0.3 + 2χ(1 + 2 + 4− 2)︸ ︷︷ ︸=0

Pr[2]

N(2)= 0.3.

121

Page 122: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

For d = 5, case (maxstep = d− s− d′) ∧ (s+ d′ < d) holds and

Pr[dist(A+X − S,~0) = 5] = (N(2)︸ ︷︷ ︸=8

−10 + 1 + 2 + 4)Pr[2]

N(2)︸ ︷︷ ︸=0.05

= 0.25.

For d > 5, case (d− s− d′ > maxstep) ∧ (s+ d′ < d) holds resulting in

Pr[dist(A+X − S,~0) = d] = 0. ♦

Figure 7.15 illustrates the computations in the example.

0.05

0.05

0.05

0.05 0.05

0.05

0.050.1

0.1

0.1

0.1 0.2

0.05

d = 0: 0.0d = 1: 0.05

d = 2: 0.1d = 3: 0.3

d = 4: 0.3d = 5: 0.25

d = 6: 0.0

Figure 7.15 The resulting probabilities for Pr[dist(A+X − S,~0) = d].

Corollary 7.1 In the worst-case model, the probability to get from distance d′ todistance d within one generation is

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = d | X1, . . . , Xλ, dist(A,~0) = d′]

=λ∑i=1

[(λi

)(Pr[dist(A+X − S,~0) > d])λ−i(Pr[dist(A+X − S,~0) = d])i

]where

Pr[dist(A+X − S,~0) > d] = 1− Pr[dist(A+X − S,~0) < d+ 1] ♦

122

Page 123: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.1. THEORETICAL FRAMEWORK

Definition 7.6 (Worst-case Markov chain model) The worst-case Markov chainmodel for severity S = (s, 0)T , a mutation defined by pdf p, and λ offspring isgiven by the tuple (States , T ) where

States = N0

and the transition matrix T = States × States is given by

T [d′ → d] = Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = d | X1, . . . , Xλ, dist(A,~0) = d′].♦

Example 7.4 Given the numbers computed in Examples 7.2 and 7.3, the probabil-ities for a (1, λ)-selection model result as follows for λ = 3, s = 1, and d′ = 2.

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = 0 | X1, . . . , Xλ, dist(A,~0) = 2] = 0

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = 1 | X1, . . . , Xλ, dist(A,~0) = 2]

=(

31

)(Pr[dist(A+X − S,~0) > 1])2(Pr[dist(A+X − S,~0) = 1])1 +(

32

)(Pr[dist(A+X − S,~0) > 1])1(Pr[dist(A+X − S,~0) = 1])2 +(

33

)(Pr[dist(A+X − S,~0) > 1])0(Pr[dist(A+X − S,~0) = 1])3

= 3 · 0.952 · 0.051 + 3 · 0.951 · 0.052 + 1 · 0.950 · 0.053

= 0.142625

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = 2 | X1, . . . , Xλ, dist(A,~0) = 2]

=(

31

)(Pr[dist(A+X − S,~0) > 2])2(Pr[dist(A+X − S,~0) = 2])1 +(

32

)(Pr[dist(A+X − S,~0) > 2])1(Pr[dist(A+X − S,~0) = 2])2 +(

33

)(Pr[dist(A+X − S,~0) > 2])0(Pr[dist(A+X − S,~0) = 2])3

= 3 · 0.852 · 0.11 + 3 · 0.851 · 0.12 + 1 · 0.850 · 0.13

= 0.24325

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = 3 | X1, . . . , Xλ, dist(A,~0) = 2]

=(

31

)(Pr[dist(A+X − S,~0) > 3])2(Pr[dist(A+X − S,~0) = 3])1 +(

32

)(Pr[dist(A+X − S,~0) > 3])1(Pr[dist(A+X − S,~0) = 3])2 +(

33

)(Pr[dist(A+X − S,~0) > 3])0(Pr[dist(A+X − S,~0) = 3])3

= 3 · 0.552 · 0.31 + 3 · 0.551 · 0.32 + 1 · 0.550 · 0.33

= 0.44775

123

Page 124: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

0134 25

0.015625

0.243250.0

0.142625

0.15075

0.44775

Figure 7.16 The resulting transitions for state 2 in the worst-case Markov model.

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = 4 | X1, . . . , Xλ, dist(A,~0) = 2]

=(

31

)(Pr[dist(A+X − S,~0) > 4])2(Pr[dist(A+X − S,~0) = 4])1 +(

32

)(Pr[dist(A+X − S,~0) > 4])1(Pr[dist(A+X − S,~0) = 4])2 +(

33

)(Pr[dist(A+X − S,~0) > 4])0(Pr[dist(A+X − S,~0) = 4])3

= 3 · 0.252 · 0.31 + 3 · 0.251 · 0.32 + 1 · 0.250 · 0.33

= 0.15075

Pr[ min1≤i≤λ

dist(A+Xi − S,~0) = 5 | X1, . . . , Xλ, dist(A,~0) = 2]

=(

31

)(Pr[dist(A+X − S,~0) > 5])2(Pr[dist(A+X − S,~0) = 5])1 +(

32

)(Pr[dist(A+X − S,~0) > 5])1(Pr[dist(A+X − S,~0) = 5])2 +(

33

)(Pr[dist(A+X − S,~0) > 5])0(Pr[dist(A+X − S,~0) = 5])3

= 3 · 0.02 · 0.251 + 3 · 0.01 · 0.252 + 1 · 0.00 · 0.253

= 0.015625

Figure 7.16 shows the transition for state 2 in the resulting Markov model. Com-paring these values to the values of the exact model in Figure 7.8 the worst-casecharacter of the model presented in this section can be seen again. ♦

The worst case model is used in the next section for the derivation of a necessarycriterion concerning feasible tracking.

7.2 Feasible tracking

This section is concerned with the definition of a criterion on how the parametersof a local search algorithm have to be chosen to guarantee feasible tracking. In this

124

Page 125: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.2. FEASIBLE TRACKING

section we are only marginally interested in the accuracy of tracking or optimalparameter settings.

A first attempt to define such a criterion could be the statement that “the survivingindividuals in a (1, λ)-strategy should stay rather close to the moving optimum”.This is, however, only partially true. The state distribution of the exact Markovchain model around the optimum can be compared best to an electron cloud aroundthe atomic nucleus. This cloud cannot be bound since in an evolutionary algorithmthere will be always a probability greater than zero for k steps leading away fromthe optimum. Instead, the evolutionary algorithm should guarantee that there isalways a higher probability to get closer towards the optimum than to continuemoving away from the optimum.

This is examined using the worst-case Markov chain model. It can be easilyobserved that the relative transition probabilities are constant for all states d >maxstep. It is enough to analyze the transition probabilities of those states whereit must be guaranteed that outliers are very likely shifted back into the center of thecloud. The behavior of the other states is much more diverse.

In order to find a useful criterion for this behavior, we consider the expected dis-tance change, i.e. the change concerning the distance to the optimum that can beexpected in one generation,

E[ min1≤i≤λ

dist(A+Xi − S,~0)− d′ | X1, . . . , Xλ, dist(A,~0) = d′]

=∑

−maxstep≤δ≤maxstep

δ T [dist(A,~0)→ dist(A,~0) + δ].

This value is required to be less than zero, that means on average the distance tothe optimum is decreased.

Figure 7.17 shows the criteria for severity values s = 1, 2, and 3, maximal stepsize between 1 and 40, and λ = 5 offspring each generation. For the (1, λ)-strategysuccessful tracking is unlikely with maximal step sizes smaller than 7 in the case ofseverity 1. This lower bound for the maximal step size increases to 30 for severity2. And for severity 3, the smallest value of the maximal step size is 67 that enablesfeasible tracking (not shown in the Figure). The latter step size value implies a verylow accuracy with huge deviations during tracking.

However, a negative expected state change can be reached for any severity—implyingsuccessful tracking. Figure 7.18 provides an argument for this statement. There thesolid line marks the distance to the optimum of the current parental individual. Theoptimum is to the right and the dashed line marks the shift of the problem causedby the dynamics. The area of the displayed pdf to the right of the dashed line is the

125

Page 126: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40maximal step size

exp

ecte

ddis

tance

chan

ge

severity 1

severity 2

severity 3

Figure 7.17 Using offspring population size λ = 5, this figure shows the expecteddistance change. For severity 1 the criterion is met from maximal stepsize 7 on; for severity 2 with maximal step size 30; for severity 3 withmaximal step size 68 (not shown here).

severity

no change to parentalindividual

severity

Figure 7.18 This figure shows two pdfs for changing the distance to the optimum,where the optimum is situated to the right of the pdf’s. It illustrateshow increasing the maximal step size decreases the impact of theseverity.

probability to move closer to the optimum. By increasing the maximal step size,the impact of the severity is getting smaller, that means, the dashed line in the figureis moving closer to the solid line—relatively to the range of possible step sizes. Asa consequence the proportion of the area to the right of the dashed line to the areaof the complete pdf is increasing. For any ε > 0, maxstep can be chosen suitably

126

Page 127: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.2. FEASIBLE TRACKING

big that

maxstep − s2maxstep + 1

>1

2− ε.

Possible values are

maxstep ≥ s+ 1

2ε.

Since in addition the pdf is always skewed towards the optimum because of theselective pressure, the criterion defined above can always be met by an accordinglyhigh maximal mutation step size.

These considerations show that it is always possible to guarantee successful track-ing by increasing the maximal step size. A bigger maximal step size leads to anexpanded cloud around the optimum. As a consequence, bigger severity comesalways along with decreased accuracy for the considered local mutation (see Defi-nition 5.2 with the fitness defined by the distance to the optimum).

-4

-2

0

2

0 10 20 30 40maximal step size

severity 1severity 2severity 3

exp

ecte

ddis

tance

chan

ge

Figure 7.19 For an offspring population size λ = 20, the expected distance changeis shown for severity values 1, 2, and 3.

However, when increasing the offspring population size to λ = 20, the worst-case Markov chain model exhibits an interesting behavior. As it can be seen inFigure 7.19 the required step sizes are reduced to achieve stable tracking. This canalso explained using Figure 7.18. Since an increase in the population size createsa higher selective pressure in a (1, λ)-strategy, the pdf for one generational stepis more skewed towards the optimum. As a consequence, a smaller step size is

127

Page 128: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

0

5

10

15

20

25

30

20 30 40population size

bal

ance

dm

ax.

step

size

10

Figure 7.20 For severity 2 and various offspring population sizes, the expectedmaximal step size values are shown where a stable tracking can bederived from the worst-case model.

required to achieve an expected change towards the moving optimum. Figure 7.20shows how the boundary for feasible tracking approaches the minimal requiredmaximal step size, namely the severity of the problem, with increasing offspringpopulation size.

Design rule 1 By increasing the maximal step size parameter and/or the offspringpopulation size tracking becomes feasible for any severity value. Increasing themaximal step size decreases the accuracy. Increasing the population size can de-crease the minimal required value for the maximal step size parameter. ♦

In the following sections, we are primarily concerned with problems associatedwith a very restricted time resource. In this case increasing the population size isno choice, since there is not enough time available, and bigger maximal step sizevalues are probably also no option, since the tracking accuracy decreases by thismethod. Various possible solutions to this problem are examined.

7.3 Optimal parameter settings

On the basis of the results of the previous section, we are concerned with the choiceof optimal parameter settings for the local mutation operator and small severityvalues in this section. As an exemplary value population size 5 is chosen in this

128

Page 129: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.3. OPTIMAL PARAMETER SETTINGS

1

10

100

1000

2 4 6 8 10 12expected step size

par

am

eter

Figure 7.21 The values for the parameter “maximum step size” that are necessaryto reach a particular expected step size (see Equation 7.8).

Figure 7.22Local Mutation: Expected distance tothe optimum for severity 1, populationsize 5, and maximal step sizes as indi-cated.

0.5

1

1.5

2

2.5

3

3.5

4

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

79

15

35

examination. But how is an “optimal” parameter setting defined? This term refersto parameter values that are chosen in such a way that tracking is feasible and thehighest possible accuracy is guaranteed.

For this analysis the exact Markov chain model is used with a radius of at most30 (see Definition 7.5), which means that any probability to be at a distance ofmore than 30 to the optimum is summarized in the absorbing state. From thiscomputation of the exact probability distributions in the search space for the first20 generations with the first individual starting at the optimum, we derive a 3-dimensional graph on the course of the distribution as well as a comparison ofdifferent mutation step sizes concerning the expected distance to the optimum.

129

Page 130: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

4 8 12 16 20 04

812

00.150.3

0.45

frequency

generation distance

Figure 7.23 Local Mutation: Change in the distribution of the distance to the op-timum for severity 1 and population size 5 using the optimal maximalstep size 9.

4 8 12 16 20 04

812

00.150.3

0.45

frequency

generation distance

Figure 7.24 Local Mutation: Change in the distribution of the distance to the op-timum for severity 1 and population size 5 using maximal step size3.

4 8 12 16 20 04

812

00.10.20.30.4

frequency

generation distance

Figure 7.25 Local Mutation: Change in the distribution of the distance to the op-timum for severity 1 and population size 5 using maximal step size15.

130

Page 131: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.3. OPTIMAL PARAMETER SETTINGS

Due to limited time and memory resources, a complete analysis is only possiblefor the severity value 1. This is in particular due to the exponential increase in theparameter value for the maximal step size needed to achieve a certain average stepsize (or expected step size)

Ep[X] =∑

x∈Z2, dist(~0,x)<maxstep

Prp[X = x]dist(~0, x). (7.8)

These numbers are shown for the local mutation operator in Figure 7.21.

The expected distance to the optimum is shown for severity 1 and for selectedstep sizes in Figure 7.22. After 20 generations the maximal step size 9 shows thebest performance with an expected distance of 1.831 in generation 20. Probably,maximal step size 9 is not completely stable toward infinity, but it represents thebest possible value for tracking over 20 generations. The respective distribution ofthe values is shown in Figure 7.23 and appears to be a very stable tracking behavior.This result underpins the experimental findings using evolution strategies with slowdynamics (cf. Back, 1998, 1999).

The drawback of a suboptimally chosen maximal step size can be illustrated againby the course of the changing probability distribution. On the one hand, the effectof a step size chosen too small is shown in Figure 7.24 (with maximal step size 3for severity 1) where the distribution is flattening out with advancing generations.This results in an increasing expected distance to the optimum. On the other hand,a too big maximal step size results in a broader distribution. This can be seen inFigure 7.25 (with maximal step size 15 for severity 1) compared to the optimalmaximal step size 9 shown in Figure 7.23. This leads also to a slightly increasedexpected distance to the optimum. However the optimal step size value 9 is veryclose to the step size 7 of the worst-case analysis in Section 7.2 where a break evenconcerning the distance to the optimum (see Figure 7.17).

All in all the results of this section can be summarized in the following recommen-dation, that will be also underlined by examinations in the following sections.

Design rule 2 Optimal step size parameters can be roughly estimated by the breakeven expected distance of the worst-case analysis in the previous section. Theconsequences of too big parameter values are less severe than too small valueswith regard to the accuracy for a fixed number of generations. ♦

131

Page 132: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

7.4 Non-zero-mean mutation

In any evolutionary algorithm the operators have a certain underlying conception ofhow the next individuals should be created using previous individuals. These con-ceptions can be made explicit by operator-defined fitness landscapes (Jones, 1995)in the case of discrete search spaces and by probability density functions (pdf) inthe case of continuous search spaces, which is the case with evolution strategies.In order to get good results there should be a high correlation between the char-acteristics of the problem and this conception of the operator. One example is theGaussian mutation applied to real-valued, smooth, stationary problems. A smooth,partially monotonous search space (e.g. sphere model) guarantees that even a smallstep in the right direction is a good step. This fits perfectly to the internal model ofthe zero-mean Gaussian pdf. The additional self-adaptation mechanisms of evolu-tion strategies enable a quick adaptation to quite different problem spaces.

⊗ ⊗

Time t

Time t+1

Figure 7.26 This figure shows three different landscapes at time t (upper row)and t + 1 (middle row). By the change of the landscape there arisesa region of discontinuity shown in the lower row where the fitnessshould be equal or better to a point situated to the left but where thefitness decreases caused by the dynamics.

However, reconsidering the worst-case analysis in Section 7.2 which implied a de-creasing accuracy with increasing severity, the correlation between mutation andproblem characteristics is disturbed by the introduction of considerable dynamic,time-varying aspects into the problem. Figure 7.26 illustrates this effect schemat-ically. The upper row shows three different one-dimensional fitness landscapeswhere the circled cross marks the current position of a candidate solution. Themiddle row shows how the fitness landscape is shifted from one generation to the

132

Page 133: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.4. NON-ZERO-MEAN MUTATION

next. And the lower row shows the arising discontinuity from this shift as the dif-ference between the fitness values we could expect if the problem was stationaryand the fitness values we encounter in the next generation. The smoothness wecould actually expect gets disarranged from generation t to generation t + 1. Attime t, any step to the right improves the fitness—the mutation cannot do wrong ifa small step is chosen instead of a bigger step. But this is not true anymore at timet+ 1 since the small step reaches a worse fitness than at time t. Here a bigger stepis desirable. As a consequence the hitting probability decreases considerably withincreasing severity and the algorithm is not able to track the optimum anymore.Thus dynamics introduce a new difficulty in the optimization which is probablynot met by a local mutation operator fulfilling the five principles mentioned at thebeginning of this chapter (page 101).

Where principles (3) and (5) have been completely disregarded in this examinationand principles (4) is mimicked to some extend, the principle of zero-mean mutation(1) and the principle concerning smaller changes (2) are the main characteristics ofa local mutation. As a consequence, this section and the next section examine towhat extend it is useful to break with those principles. This section is dedicated tonon-zero-mean mutations.

There have been several proposed local mutations involving the preference of acertain direction. In the real-valued domain, there are examinations by Ghozeil andFogel (1996) and Hildebrand, Reusch, and Fathi (1999) concerning the stationaryfitness functions. In this section, we introduce a direct mutation by skewing theprobability function according to the following definition.

Definition 7.7 (Directed mutation) The directed version of a mutation operatorwhere positive values of search space dimension x1 are favored is defined usingpdf p

Prdirp [X = x] =

(

32− 2d(0,x2)

N(dist(~0,x))

)Prp[X = x], if x1 ≥ 0 ∧ dist(~0, x) > 0(

12

+ 2d(0,x2)

N(dist(~0,x))

)Prp[X = x], if x1 < 0 ∧ dist(~0, x) > 0

Prp[X = x], if dist(~0, x) = 0

(7.9)♦

Figure 7.27 sketches how the probabilities are modified. This modification ofthe mutation operators interferes with the usual property of zero-mean mutations.However, this definition does not change the general scheme of prefering smallchanges over big changes as is shown in the following lemma.

Lemma 7.7 The probability to make a step of size d (for 0 ≤ d ≤ maxstep) isidentical for the undirected and the directed version of a mutation using an arbi-trary pdf p. ♦

133

Page 134: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

12Pr[d] d

Pr[d]

Pr[d]

32Pr[d]

+ 2N(d)

Figure 7.27 Transformation of an undirected mutation into a directed mutation bymodification of the probabilities.

Proof: The case d = 0 holds trivially because of the last case of Definition 7.7. Ford > 0 the following transformations hold (using N(d) = 4d).∑x∈Z2, dist(~0,x)=d

Prdirp [X = x]

=

(∑0≤i<d

2

(3

2− 2(d− i)

4d

)+

3

2+∑

0<i<d

2

(1

2+

2(d− i)4d

)+

1

2

)Prp[X = x]

= 4dPrp[X = x]

=∑

x∈Z×Z, dist(~0,x)=d

Prp[X = x].

q.e.d.

The application of Definition 7.7 to the previously defined local mutation (usingplocal ) results in the probability density function shown in Figure 7.28.

The following investigation of the directed local mutation relies completely on theexact Markov chain model since the worst-case model is not applicable to non-zero-mean mutations. Furthermore we assume that the direction of the mutation

134

Page 135: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.4. NON-ZERO-MEAN MUTATION

-100

10 -100

10

0

0.02

Figure 7.28 Resulting probability density function for the directed local mutation.

Figure 7.29Directed local mutation: Expecteddistance to the optimum for severity1, population size 5, and maximal stepsizes as indicated.

0.4

0.6

0.8

1

1.2

1.4

1.6

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

3457

1015

4 8 12 16 20 04

812

00.150.3

0.45

frequency

generation distance

Figure 7.30 Directed local mutation: Change in the distribution of the distance tothe optimum for severity 1 and population size 5 using the best foundstep size.

is optimal set in the first generation and is not changing in the complete simula-tion/computation.

Figure 7.29 shows the expected distance for severity 1, population size 5, and var-

135

Page 136: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

0

1

2

3

4

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

611162125

Figure 7.31 Directed local mutation: Expected distance to the optimum for sever-ity 2, population size 5, and maximal step sizes as indicated.

4 8 12 16 20 04

812

00.10.20.30.4

frequency

generation distance

Figure 7.32 Directed local mutation: Change in the distribution of the distance tothe optimum for severity 2 and population size 5 using the best foundstep size 21.

ious selected values for the maximal step size. The discovered best maximal stepsize is 5 leading to an expected distance of 1.079 at the end of generation 20. Therespective course of the distribution is shown in Figure 7.30.

Compared to the results of the undirected mutation in Figure 7.22 the negativeeffects of too small maximal step sizes can be reduced considerably. The effects oftoo large maximal step sizes appear to be unaffected however.

For severity value 2, Figure 7.31 shows the expected distance to the optimum forselected maximal step sizes. The optimal parameter setting could be identified asstep size 21 with an expected distance of 2.3825. The respective course of the dis-

136

Page 137: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.5. PROPOSITION OF BIGGER STEPS

Figure 7.33This figure illustrates the usage of thebinomial pdf to define the mutation op-erator promoting bigger steps. The pdfis used to assign a probability to eachpoint that this point is created as an off-spring. It advocates a certain step sizewith a smaller standard deviation.

tribution is shown in Figure 7.32 with a considerably broader deviation than forseverity 1. These results indicate, however, that the technique of directed muta-tions is not a significant remedy for the identified problems when applying localmutations to bigger severity values.

All in all the findings of this section can be summarized in the following recom-mendation.

Design rule 3 For small population sizes and predictable dynamics with smallseverity, a well-orientated directed local mutation is able to reduce the divergentbehavior of local mutations with small maximal step sizes. It does not solve theproblems with higher severity values. ♦

7.5 Proposition of bigger steps

The motivation at the beginning of the last section has been the need to overcomethe discrepancy between local mutation and problem characteristics. There, break-ing with the underlying principles has been argued to be a proper means for iden-tifying properties of better suited mutation operators. In the last section non-zero-mean mutations have been examined. In this section the promotion of bigger stepsis analyzed as well as the combination of bigger steps with non-zero-mean muta-tion.

Therefore, it has to be achieved somehow that smaller steps are more unlikely tooccur than bigger steps. In this section, this is implemented by a different assign-ment of the underlying binomial pdf to the pdf of the mutation operator as it is

137

Page 138: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

undirected directed

-100

10 -100

10

0

0.01

-100

10 -100

10

0

0.014

Figure 7.34 Resulting probability density functions for the ring-like mutationproposing bigger step sizes.

shown in Figure 7.33. A similar mutation with a more crisp pdf has been intro-duced by Weicker and Weicker (1999). The following definition introduces themutation formally.

Definition 7.8 The ring-like mutation is defined using the following pdf

pring(d) =

0, if d = 0

12maxstep−1

(maxstep−1

d−1

), if d > 0

(7.10)

with 0 ≤ d ≤ maxstep in Definition 7.4. ♦

Since this definition only specifies the assignment of probability values to step-width d, the zero-mean principle is untouched by the definition. However, bycombining Definitions 7.7 and 7.8 a mutation breaking with both principles canbe defined. Both mutation probability density functions are shown in Figure 7.34.

Since the mere ring-like mutation is zero-mean, the worst-case Markov chain modelis applicable. The result of the computation concerning the feasibility of trackingis shown in Figure 7.35. Due to the different shape of the ring-like mutation’s pdf,the maximal step size needed to guarantee feasible tracking grows almost linearwith increasing severity. This can also be seen in the dependence of the expectedstep size on the maximal step size parameter shown in Figure 7.36. As a conse-quence the mutation step size can be derived better from a given severity value andthe optimal step size parameters are closer together for varying severity than in thecase of the local mutation.

Another conclusion that can be drawn from the worst-case Markov chain model isthe fact that analogously to the considerations in Section 7.2 any too big maximalstep size parameter will lead to successful tracking but with a smaller accuracy (see

138

Page 139: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.5. PROPOSITION OF BIGGER STEPS

-7-6-5-4-3-2-101234

0 5 10 15 20 25 30 35 40maximal step size

severity 1severity 2severity 3

exp

ecte

ddis

tance

chan

ge

Figure 7.35 Ring-like mutation: This figure shows the expectancy value of chang-ing states for number of offsprings λ = 5. For severity 1 the criterionis met from maximal step size 4 on; for severity 2 with maximal stepsize 9 and for severity 3 with step size 15.

Gauss likeRing like

1

10

100

1000

2 4 6 8 10 12expected step size

par

amet

er

Figure 7.36 The values for the maximum step size parameter that are necessary toreach a particular expected step size.

Figure 7.37). This is probably astonishing since the shape of the ring-like mutationcould imply a different behavior.

The exact Markov chain model is used for computations concerning severity val-ues 1, 2, and 3. Again the number of offspring individuals is λ = 5. The expected

139

Page 140: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

severity severity

no change to parentalindividual

Figure 7.37 Schematic illustration how increasing the maximal step size parame-ter of the ring-like mutation helps to increase the fraction of improv-ing mutations (leading to smaller distance to the optimum).

distance values are shown for severity 1 and selected maximal step sizes in Fig-ure 7.38. The best maximal step size for the undirected ring-like mutation is 3causing an expected distance of 1.724 after 20 generations. The best maximal stepsize for the directed version is 2 with an expected distance of 0.913. The respectivecourses of the distribution are shown in Figure 7.39. In accordance to the resultsusing the local mutation, the directed version is able to soften the negative effectsof too small mutations.

The results for severity 2 shown in Figure 7.40 and Figure 7.41 are along the samelines as well as the results for severity 3 in Figure 7.42 and Figure 7.43.

Table 7.1 gives an overview on the optimal parameter values and the respective ex-pected distances to the optimum in generation 20 for all considered severity valuesand mutation types.

In case of the ring-like mutations, the worst case scenario yields again good ap-proximations of these values. In the scenario using population size 5, the expecteddistance to the optimum (being an indicator for the accuracy) is approximately lin-ear with the severity value (see also Figure 7.44 for severity 4). This aspect makesthe directed ring-like mutation very appealing. However, two restrictions must beconsidered: first, the maximal step size has to be chosen accordingly and, second,the orientation of the directed mutation is crucial. The first restriction is discussedin the next paragraph and the second restriction is considered in Section 7.8.

When comparing the expected distances in Figure 7.22 and Figure 7.38, it appearsthat the ring-like mutation is much more sensitive to inappropriately big maximalstep size parameters. This leads to a more severe decline in the tracking accuracy.Partially this can be explained using the different effect of the maximal step size

140

Page 141: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.5. PROPOSITION OF BIGGER STEPS

generation

directed

generation

undirected

exp

ecte

dd

ista

nce

exp

ecte

dd

ista

nce

0

1

2

3

4

5

6

7

8

0 4 8 12 16 20

12346

0

0.5

1

1.5

2

2.5

3

3.5

0 4 8 12 16 20

1

2

3

4

6

Figure 7.38 Ring-like mutation: Expected distance to the optimum for severity 1,population size 5, and maximal step sizes as indicated.

4 8 12 16 20 04

812

0

0.2

0.4

0.6

frequency

generation distance4 8 12 16 20 0

48

12

0

0.2

0.4

0.6

frequency

generation distance

undirected directed

Figure 7.39 Ring-like mutation: Change in the distribution of the distance to theoptimum for severity 1 and population size 5 using the optimal stepsize.

141

Page 142: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

0.5

1.5

2.5

3.5

4.5

5.5

0 4 8 12 16 20generation

directed

0

2

4

6

8

10

12

0 4 8 12 16 20generation

undirected

exp

ecte

dd

ista

nce

exp

ecte

dd

ista

nce

3457

10

3457

10

Figure 7.40 Ring-like Mutation: Expected distance to the optimum for severity 2and population size 5.

4 8 12 16 20 04

812

00.10.20.30.4

frequency

4 8 12 16 20 04

812

00.150.3

0.45

frequency

generationdistance distance

undirected directed

Figure 7.41 Ring-like Mutation: Change in the distribution of the distance to theoptimum for severity 2 and population size 5 using the optimal stepsize.

142

Page 143: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.5. PROPOSITION OF BIGGER STEPS

69

121824

0

2

4

6

8

10

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

undirected

0

2

4

6

8

10

0 4 8 12 16 20generation

4

exp

ecte

dd

ista

nce

directed

24

6

15

9

Figure 7.42 Ring-like mutation: Expected distance to the optimum for severity 3and population size 5.

4 8 12 16 20 04

812

00.10.20.3

frequency

generation distance

4 8 12 16 20 04

812

0

0.1

0.2

frequency

generation distance

undirected directed

Figure 7.43 Ring-like Mutation: Change in the distribution of the distance to theoptimum for severity 3 and population size 5 using the optimal maxi-mal step size value.

143

Page 144: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

best maximal step size final expected distanceseverity 1:undirected local 9 1.831directed local 5 1.079undirected ring-like 3 1.724directed ring-like 2 0.913severity 2:undirected Gaussian ≈ 30 –directed Gaussian – –undirected Ring 7 3.459directed Ring 5 2.104severity 3:undirected Gaussian ≈ 68 –directed Gaussian – –undirected Ring 12 5.225directed Ring 9 3.203

Table 7.1: Overview of maximal step size for the four mutation operators usingpopulation size 5.

69

121824

0

2

4

6

8

10

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

Figure 7.44 Ring-like mutation: Expected distance to the optimum for the di-rected mutation, severity 4 and population size 5.

144

Page 145: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.5. PROPOSITION OF BIGGER STEPS

parameter on the expected step size shown in Figure 7.36.

A more compelling explanation can be found by looking at the hitting probability(see Lemma 7.5) for a parental individual placed at the current optimum, i.e. wecompute the probability to get as close as distance 5 to the optimum within onegeneration where the optimum has moved the severity distance s. The results areshown for the local mutations as well as the ring-like mutations in Figure 7.45.

maximum step size

undirected ring-like

10 20 30 40 500

0.2

0.4

0.6

0.8

1

maximum step size

directed ring-like

10 20 30 40 500

0.2

0.4

0.6

0.8

1

1356

10

100 200 300maximum step size

directed local

0

0.2

0.4

0.6

0.8

1

100 200 300maximum step size

undirected local

hit

tin

gp

rob

.

severity

Figure 7.45 The diagram shows the probabilities to hit a point within distance 5of the optimum. On the x-axis the maximum step size is shown anddifferent line types indicate different severity of dynamics.

There are three primary facts that we can conclude from this figure.

1. Like in the examinations over 20 generations a substantial higher accuracy ofthe ring-like mutation over the local mutation as well as the directed versionsover the undirected versions is obvious.

2. However, the higher hitting probability of the ring-like mutation holds onlyfor a very narrow window of step size parameters. As a consequence, apoorly calibrated local mutation can easily outperform a poorly calibratedring-like mutation.

3. In addition, the narrow windows of the ring-like mutation are shifted andoverlap only partially. This may cause problems if the severity value is notknown in advance or varies during optimization.

145

Page 146: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

To sum it up it can be said that the proposition of bigger steps has certain consid-erable advantages. However, the risk of the disadvantages must be analyzed andweighed carefully. The important recommendations of this section are summarizedin the following design rules.

Design rule 4 The proposition of bigger steps enables better accuracy rates, butrequires proper calibration of the step size parameter. For problems with varyingseverity values, the latter point should be guaranteed. Otherwise the accuracyrates may drop below the accuracy of the local mutation. ♦

Design rule 5 Even with small population sizes, combining non-zero mean muta-tion with the proposition of bigger steps may lead to very precise tracking accu-racy. ♦

7.6 Dependence on the population size

As it was already discussed in Section 7.2 (Design rule 1), an increase in the pop-ulation size can be used to diminish the expected distance to the optimum. Thereit was also argued that a change in the population size comes along with a changein the required optimal value of the step size parameter. This section is devoted toa more exhaustive analysis of the influence of the population size.

The dependence on the population size and the optimal value of the step size param-eter is examined closer using the exact Markov chain model. Figure 7.46 shows theexpected distance to the optimum in generation 20 for severity 1, varying maximalstep size (in the x-axis), and different population sizes as indicated. Figures 7.47and 7.48 show the analogous computations for severity values 2 and 3. In all threefigures the dependence between maximal step size and population size can be seen.Also the effect on the expected distance becomes obvious. Rather small populationsizes (between 2 and 5) affect the negative effect of suboptimally chosen maxi-mal step size values on the distance to the optimum more significantly than biggerpopulation sizes (e.g. 8 and above).

However, the probably obvious conclusion to increase the population size for im-proving the tracking behavior contradicts the limited time resources assumed inthis chapter. Therefore, in the remainder of this section the severity depends on thenumber of evaluations (or the number of offspring per generation) anymore. Up tonow, the severity was rather conceived as an external characteristic of the problem.In this analysis, we correlate the population size with the severity: each evaluationcontributes to the severity as it is conceived by the evolutionary algorithm in onegeneration.

146

Page 147: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.6. DEPENDENCE ON THE POPULATION SIZE

258

12

0

2

4

6

8

10

12

14

0 4 8 12 16maximal step size

fin

al

exp

ecte

dd

ist.

Figure 7.46 Severity 1, ring-like mutation: Expected distance to the optimum ingeneration 20 for several maximal step sizes shown in the x-axis andpopulation sizes as indicated.

258

12

0

5

10

15

20

25

0 4 8 12 16maximal step size

fin

alex

pec

ted

dis

t.

Figure 7.47 Severity 2, ring-like mutation: Expected distance to the optimum ingeneration 20 for several maximal step sizes shown in the x-axis andpopulation sizes as indicated.

147

Page 148: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

258

12

0

5

10

15

20

25

30

35

2 4 6 8 10 12 14 16maximal step size

fin

al

exp

ecte

dd

ist.

Figure 7.48 Severity 3, ring-like mutation: Expected distance to the optimum ingeneration 20 for several maximal step sizes shown in the x-axis andpopulation sizes as indicated.

This correlation is modeled using two distinct time factors,

• the time α to create and evaluate an individual and

• the time β as the processing time of one generation that is independent of thenumber of individuals in the population.

Then, the total computation time for one generation results in

T = β + αn

where n is the population size. Furthermore, we assume that the computation timeT equals the severity of the problem. For example, the time factors α = 1

3and

β = 0 imply that a population size 3 leads to severity 1. For α = β = 13

apopulation size 5 leads to severity 2.

In the following computations the directed ring-like mutation is used since it ex-hibits the best tracking performance and the computations using the exact Markovmodel are still feasible for severity value 4. The population sizes and the respectiveoptimal values for the step size parameter are shown in Table 7.2 for the used (α,β)-configurations and the considered severity values.

First, the dependence of the severity on the population size without additional timecosts (β = 0) is examined. Figure 7.49 shows the computations of the expected

148

Page 149: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.6. DEPENDENCE ON THE POPULATION SIZE

48

1216

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 4 8 12 16 20generation

α = 14

exp

ecte

dd

ista

nce

2468

0

1

2

3

4

0 4 8 12 16 20generation

α = 12

exp

ecte

dd

ista

nce

0

0.5

1

1.5

2

2.5

0 4 8 12 16 20generation

α = 13

exp

ecte

dd

ista

nce

369

12

5101520

0

0.2

0.4

0.6

0.8

1

0 4 8 12 16 20generation

α = 15

exp

ecte

dd

ista

nce

Figure 7.49 Results for β = 0 and directed ring-like mutation. In each graph fourdifferent population sizes are used where the smallest population sizecorresponds to severity 1, the next to severity 2, severity 3, and thebiggest to severity 4.

149

Page 150: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

5101520

0

0.2

0.4

0.6

0.8

1

0 4 8 12 16 20generation

α = 15 , β = 0

exp

ecte

dd

ista

nce

38

1318

0

0.5

1

1.5

2

2.5

0 4 8 12 16 20generation

α = 15 , β = 2

5

exp

ecte

dd

ista

nce

27

1217

0

1

2

3

4

0 4 8 12 16 20generation

α = 15 , β = 3

5

exp

ecte

dd

ista

nce

49

1419

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 4 8 12 16 20generation

α = β = 15

exp

ecte

dd

ista

nce

Figure 7.50 Results for α = 15, β > 0, and directed ring-like mutation. In each

graph four different population sizes are used where the smallest pop-ulation size corresponds to severity 1, the next to severity 2, to sever-ity 3, and the biggest to severity 4.

150

Page 151: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.6. DEPENDENCE ON THE POPULATION SIZE

severity 1 severity 2 severity 3 severity 4

α = 12, β = 0:

population size 2 4 6 8maximal step size 4 6 8 10

α = 13, β = 0:

population size 3 6 9 12maximal step size 3 5 7 9

α = 14, β = 0:

population size 4 8 12 16maximal step size 2 4 6 8

α = 15, β = 0:

population size 5 10 15 20maximal step size 2 4 6 8

α = 15, β = 1

5:

population size 4 9 14 19maximal step size 2 4 6 8

α = 15, β = 2

5:

population size 3 8 13 18maximal step size 3 4 6 8

α = 15, β = 3

5:

population size 2 7 12 17maximal step size 4 4 6 8

Table 7.2: Population sizes and the respective optimal values for the maximal stepsize parameter used to achieve a certain severity for the given (α, β) combinations(directed ring-like mutation).

distance in the exact Markov model for α ∈ 12, 1

3, 1

4, 1

5. The value α = 1

2reflects

a very strong influence of the number of offspring on the severity. Surprisingly,increasing the offspring population size up to 8 improves still the expected distancealthough the severity increases. With α = 1

3the picture is still the same. But with

decreasing influence of the population size, optimal values for the population sizeare found: for α = 1

4population size 12 is optimal and for α = 1

5population size

10 is optimal (restricted by the coarse granularity of the values for the populationsize). This result can be explained as follows. For small numbers of offspring, aslight increase in offspring population size leads to rather big positive effects on the

151

Page 152: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

performance (see Figures 7.46, 7.47, and 7.48). This is due to the improvements ofa bigger population size in general as well as the decreasing optimal value for thestep size parameter. Apparently this positive effect outweighs the negative effectsof an increasing severity. With decreasing α, the influence due to the decreasingoptimal value of the step size parameter becomes smaller. As a consequence thenegative effects of an increasing severity value are bigger than the mere effects ofincreasing the population size.

When we introduce independent time cost β > 0, the results for α = 15

are shownin Figure 7.50. There the optimal population size increases from 10 with β = 0 to14 with β = 1

5. For bigger values of β the optimal population size increases further

although the optimal values are not shown in the figure. Like with the dependentcost α, we see that increasing independent cost β affects the optimal populationsize considerably too. In particular the figure shows how sensitive the optimalpopulation size is affected by small irritations, e.g. due to administrative tasks.

The findings of this section are summarized in the following design rule.

Design rule 6 If the severity depends on the number of evaluations there is anoptimal offspring population size. This optimal number of offspring per generationincreases if the value α or the value β increase. This implies especially for veryrestricted time resources rather big offspring population sizes. For the examinedmutation operator and β = 0, the population size 10–15 could serve as a rule ofthumb. A detailed analysis is necessary for concrete recommendations. ♦

7.7 Memorizing techniques

Memorizing techniques are rather seldom used in tracking tasks. However, thereare scenarios like the examination of Branke (1999c) where at least situations witha tracking character might occur in an oscillating framework. In those situationsadding an external memory to the optimizer might be a useful idea. The externalmemory stores a fixed number of previous solutions. Also the algorithm of Kirleyand Green (2000) uses an external memory and is applied to a drifting landscape.An internal memory (using a polyploid representation) was applied to a trackingtask by Dasgupta (1995).

This section examines the use of an external memory for a tracking task. Theframework is similar to the previous sections, however, it is assumed that the pathof the optimum returns from time to time to former positions of the optimum. Theinvestigation is concerned with what we gain by this technique in the successfulcase and what we loose if the memory fails to save the right solutions.

152

Page 153: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.7. MEMORIZING TECHNIQUES

This is modeled by a slight modification of the exact Markov-model from Sec-tion 7.1.1. In the model the organization of the memory and the actual path ofthe optimum are not considered in detail. Instead, it is simply assumed that ineach generation one individual is selected from the memory and is inserted intothe population. With a certain probability psuccess the individual corresponds to thecurrent position of the optimum. Furthermore, it is assumed that the memory storesrather distinct solutions. Therefore, the model simplifies the new dynamics of thememorizing technique by the assumption that either of the following cases holds:

1. The optimum is hit with the probability psuccess .

2. The optimum is not hit by the new individual and the introduced individual isso far away from the optimum that the probabilities are not affected to reachany other point of the exact model distinct from the optimum.

Since the random variables for hitting the optimum by the individual from the mem-ory and by the individuals created by the local operator are independent from eachother, the probability to hit the optimum is expressed by

Prp[best(A+Xi − S,Z) = ~0 | X1, . . . , Xλ, Z]

= Prp[best(A+Xi − S) = ~0 | X1, . . . , Xλ] + psuccess

−Prp[best(A+Xi − S) = ~0 | X1, . . . , Xλ] psuccess

whereZ is a random variable associated with the individual selected from the mem-ory.

The probability that another point distinct from the optimum created as best resultfrom the local operators application must be modified too. It is only effective if theindividual from the memory does not correspond to the optimum. This probabilityis expressed by the following formula for y 6= ~0 since both random variables areagain independent.

Prp[best(A+Xi − S,Z) = y | X1, . . . , Xλ, Z]

= Prp[best(A+Xi − S) = y | X1, . . . , Xλ] (1− psuccess)

Then a Markov chain model for the optimizer including the memory is given in thefollowing definition.

Definition 7.9 (Markov chain model for algorithm with memory) The Markovchain model for a local optimizer using an external memory is defined by the model

153

Page 154: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

succ. memory (0.01)succ. memory (0.05)succ. memory (0.1)

no memoryunsucc. memory

0.6

0.7

0.8

0.9

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

0.6

0.7

0.8

0.9

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

0.6

0.7

0.8

0.9

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

0.6

0.8

1

1.2

1.4

1.6

0 4 8 12 16 20generation

exp

ecte

dd

ista

nce

λ = 10, severity 2 λ = 10, severity 3

λ = 20, severity 4λ = 15, severity 3

Figure 7.51 Comparison of the tracking accuracy of evolutionary algorithms withand without external memory. The unsuccessful use of the memoryassumes a success probability psuccess = 0.

of Definition 7.5 with the following modifications.

T [A→ B] = Prp[best(A+Xi − S, Z) = ~0 | X1, . . . , Xλ, Z]

T [A→ absorb] = 1−∑

v∈States\absorb

Prp[best(A+Xi − S, Z) = v | X1, . . . , Xλ, Z]♦

154

Page 155: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.7. MEMORIZING TECHNIQUES

This model is now used to examine at which expected success rate for choosing anoptimal individual from the memory it is useful to invest one fitness evaluation inthe individual from the memory. For an offspring population size λ, the EA with anexternal memory creates λ − 1 individual with the mutation operator and choosesone individual from the memory. This algorithm is compared to an EA that createsλ individuals with the mutation operator.

The results of the comparison are shown in Figure 7.51 for population size 10 withseverity values 2 and 3, population size 15 with severity 3, and population size 20with severity 4.

The graph for offspring population size λ = 10 and severity 2 shows that witha rather small population size the success rate of the memorizing technique mustreach a value of approximately psuccess = 0.1 that the tracking accuracy is im-proved. However, that means that each 10 generations a close-to-optimal solutionis selected from the memory. This is a very unrealistic assumption. This thresh-old can be lowered by choosing bigger population sizes as the other graphs imply(even for bigger severity values). But still for all examined setups, a success rateof psuccess = 0.01 is very close to the unsuccessful case with psuccess = 0. Andfor the relevant range of “optimal” offspring population sizes the investment in anadditional offspring is always an advantage over the use of an external memory ina realistic scenario.

However, an external memory is still useful if the problem combines a trackingcharacter with alternating and repetitive coordinate dynamics. Then the successrate psuccess is probably big enough that the investment into the memory evaluationpays off.

Design rule 7 For a mere tracking task the usage of an external memory shouldbe avoided if the individuals from the memory have a success rate of psuccess ≤0.01. However, introducing one individual from the memory into the populationaffects the tracking accuracy moderately such that an external memory is usefulfor tracking problems with non-predictable, repetitive phases or alternations withlow severity. ♦

The examination of this section is based on a preliminary investigation of the hittingprobability within a similar framework (Weicker, 2000). The findings go along thesame lines than the empirical results of Kirley and Green (2000).

155

Page 156: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

7.8 Issues of adaptation and self-adaptation

In the previous sections various techniques and approaches are discussed that aimat improving the tracking behavior of a local search algorithm. However, all exam-inations and comparisons rely on the assumption that all algorithms are calibratedoptimally, which means that they perform at their best. Whether this assumptioncan be met by the different algorithms in a dynamic environment is investigated inthis section.

Usually the properties of the dynamics in a non-stationary problem are not knownat hand. As a consequence, it appears to be useful to combine such an operatorwith a mechanism to adapt the parameter settings such that the algorithm is ableto tune itself for reaching an optimal tracking behavior or to react on changes inthe dynamics of the environment. Examples for adaptation and self-adaptationmechanisms have already been presented in Section 2.2.2. However, the decisionfor or against an adaptation mechanism must consider the following factors that areexamined in the remainder of this section.

1. Severity: Is the characteristic of the tackled problem rather stable (constantseverity)? Can we assume in advance that the severity is rather big or small?Is the severity rather unstable, i.e. small and big severity values alternate?

2. Accuracy: Do we require the best possible accuracy or is the successful track-ing at any accuracy sufficient?

3. Number of parameters: How many parameters are used in the adaptationmechanism and need to be calibrated?

7.8.1 Evaluation of the presented operators

This subsection focuses on the self-adaptive potential of the local and the ring-likemutation as well as the tension between undirected and directed mutations.

In Section 7.5 the characteristics of the two different mutation operators have beencompared. As it is shown in Figure 7.45 the higher accuracy of the ring-like mu-tation is due to a smaller window of maximum step size values that are close tothe optimal value. This is not the case with the local mutation where the hittingprobability flattens smoothly with increasing maximal step size. And what is evenmore important: The optimal parameter range of the ring-like mutation shifts withincreasing dynamics with the consequence that there is no overlap between the op-timal ranges, e.g. for severity values 3 and 10. Therefore, a self-adaptation mech-

156

Page 157: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.8. ISSUES OF ADAPTATION AND SELF-ADAPTATION

anism has to react very fast in case of the ring-like mutation when the severityof the problem is changing drastically. Due to the different structure of the pdf,this appears to be only a subordinate problem for the local mutation. It is onlyof interest if a small maximal step size is used (e.g. for severity value 3) and theseverity value increases (e.g. to 10) such that a big maximal step size (above 50)is required. However, a self-adaptation mechanism like in evolution strategies usesa multiplicative factor with the probability density function shown in Figure 7.52(see also Equation 2.1). As a consequence considerable increased values of the

multiplicative factor

prob

abili

ty

0

0.1

0.2

0.3

0.4

0.1 1 10

Figure 7.52 Probability density function of the multiplicative change of the strat-egy variable in evolution strategy mutation.

self-adaptation parameter occur with a rather high probability. Since the windowof good parameter values for the local operator is bound only single-sided and thereis no severe problem for too big maximal step sizes, it is enough to create an in-dividual with a big enough step size. This is less critical than hitting the correctrange of step size values in case of the ring-like mutation.

Design rule 8 For problems with drastically changing dynamics, the use of the lo-cal mutation together with a self-adaptation mechanism promises a better trackingbehavior. ♦

If a directed mutation has to be supported by a self-adaptation mechanism, at leastn strategy variables are necessary to represent a vector for the direction in an ndimensional search space. However, this does not consider the step size of themutation. Therefore, the directed mutation e.g. by Hildebrand et al. (1999) needsn+1 strategy variables. However, to adjust n+1 parameters we need at least n+1

157

Page 158: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

useful, distinct evaluations of the search space. In a higher dimensional searchspace this requires either many evaluations per generation or the adaptation speedis very slow. This is underpinned by experimental investigations for undirectedmutation schemes with n and n(n+1)

2strategy variables. There it was shown that the

adaptation mechanism is often too inert and fails to track an optimum (Weicker &Weicker, 1999).

Design rule 9 If the direction of the dynamics is expected to change considerably,it is advisable to use an undirected mutation operator with few strategy variablesto ensure quick adaptation. ♦

7.8.2 Limits of self-adaptation: uncentered tracking

This subsection questions whether the self-adaptation technique is applicable forany kind of tracking problem. Self-adaptation was invented in the context of evo-lution strategies to enable the mutation operator to adapt to different problem land-scapes. Analogously to the argumentation concerning the local (Gaussian) muta-tion in Section 7.4 (see Figure 7.26), the self-adaptation is tailored to stationarylandscapes. But in non-stationary problems the dynamics add a completely newdimension to the problem. And the question arises whether self-adaptation is stillable to handle those problems.

To analyze this problem closer, we drop the assumption that our algorithm is closeto the optimum and tries to track its position only. Rather it is assumed that theoptimum still needs to be detected in the changing landscape: tracking is combinedwith optimization. In order to explore the limits, a pathological setup is constructedwhere the tracking direction is arranged orthogonally to the optimization direction.The resulting “moving corridor problem” is sketched in Figure 7.53. Inside thecorridor the fitness is increasing in one direction. Outside the corridor the fitnessvalue is assumed to be constantly bad—if an algorithm fails to track the corridor inany generation the tracking target is lost.

For an exemplary setup shown in Figure 7.54, we investigate the hitting probability(to hit the corridor) and the expected fitness improvement if an improvement takesplace. For a severity value 3, a corridor of width 5, and the assumption that in thecurrent generation the individual is located in the middle of the corridor, the com-putations are carried out for values of the maximal step size parameter between 5and 550. From the examination of the worst-case model (Figure 7.17), the opti-mal setting of the maximal step size parameter is known to be approximately 68for a mere tracking task and population size λ = 5. Figure 7.55 shows the com-puted values of the hitting probability and the expected fitness improvement in the

158

Page 159: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.8. ISSUES OF ADAPTATION AND SELF-ADAPTATION

Figure 7.53Arrangement of the moving corridor problemwhere the tracking direction and the optimiza-tion direction are arranged orthogonally.

DynamicsTrackingregion

Optimum

case that the corridor is hit. Again there is an optimal value concerning the hit-ting probability. For higher values of the maximal step size the hitting probabilitydecreases. On the other hand the expected improvement is constantly increasingwith increasing maximal step size. This has the following consequences: if theself-adaptation mechanism chooses randomly a rather high value for the step sizeparameter (which happens rather often because of the multiplicative self-adaptationrule) and by chance the modified object value of the individual is inside the cor-ridor and better than the current individual’s fitness, then the individual with thebigger step size parameter can be expected to have a better fitness than an offspringwith lower step size parameter (since the expected fitness improvement increasesconstantly when the fitness increases). As a consequence the individual with thebigger value of the step size parameter is accepted for the next generation. Iter-ating this scheme the maximal step size parameter will increase until the hittingprobability has decreased so much that no offspring within the corridor is foundanymore. During this process the self-adaptation will create also individuals withsmaller step sizes but because of their smaller expected improvement the selectionwill usually prefer the individual with the bigger step size parameter.

This is an example where the self-adaptation mechanism is not able to master bothtracking and optimization. Because of the focus of the selection on the fitness asthe optimization criterion, the trackability is lost in the long run.

Design rule 10 Sole dependence of a self-adaptation of parameters on the fitnessvalue is not advisable. The tracking rate should be reflected as a control quantityin any component of the algorithm. ♦

Moreover, the moving corridor problem should be used as a benchmark for all

159

Page 160: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

part of thecorridor withbetter fitness

part of thecorridor withequal orworse fitness

Figure 7.54 Supposed the current individual is centered in the corridor, the fig-ure shows the corridor of the next generation with severity 3 and acorridor of width 5. The shaded area marks the corridor.

adaptation techniques in dynamic environments to ensure a stable behavior even inrather extreme situations.

7.8.3 Alternative adaptation mechanisms

As the previous subsection has shown, the standard self-adaptation mechanismsmight be deceived by the fitness values in the moving corridor problem. This sub-section raises the question how different adaptation mechanisms might be condi-tioned.

The adaptation technique like in the 1/5-success rule (see page 2.2.2) uses statisticsto adapt a strategy parameter. The main differences to the self-adaptation mecha-nism are

160

Page 161: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.8. ISSUES OF ADAPTATION AND SELF-ADAPTATION

expectedimprovement

hitting probability

0.2

0.3

0.4

0 100 200 300 400 500

0.2

0.4

0.6

0.8

1

maximal step size

hit

ting

pro

bab

ilit

y

exp

ecte

dim

pro

vem

ent

Figure 7.55 For the example in Figure 7.54, the lower curve shows the hittingprobability and the upper curve shows the expected improvement ifan improvement takes place.

• that the change to the strategy parameter is not random but derived using arule and

• that the strategy parameter is applied to all individuals in the populationequally.

The second issue is probably an advantage since the dynamics are identical for allpoints in the search space as long as homogenous problems with linear coordinatetransformations are considered. Also, Angeline (1997) reported that there are in-dications in his experiments that adaptation might be superior to self-adaptation.However, how a sensible rule can be defined to guarantee successful adaptationconcerning the moving corridor problem is not yet known and a topic of futurework.

An even more sophisticated method to adapt an evolutionary algorithm to a dy-namic problem would be the derivation of the meta-rule of the dynamics from theevaluated individuals. If the meta-rule can be approximated sufficiently, the dy-namics can be eliminated, and a standard algorithm may be used. In addition thisapproach has the advantage that arbitrary small accuracy can be reached indepen-dent of the population size. But this method is still unexplored like the adaptationmethods, too. In Section 8.4, a first technique is presented to derive the meta-rulein 2-dimensional dynamic problems.

161

Page 162: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7. ANALYSIS OF LOCAL OPERATORS FOR TRACKING

7.9 Conclusion

This chapter has investigated the local mutation in a dynamic environment thor-oughly. The focus has been on the question whether the principles of ES mutation,i.e. a zero-mean and preferably small change, is a good choice for arbitrary trackingproblems. Furthermore the adjustment of the parameters and necessary additionaltechniques are examined.

The examination results in a set of design rules that are summarized in the followinglist.

1. By increasing the maximal step size parameter and/or the offspring popu-lation size tracking becomes feasible for any severity value. Increasing themaximal step size decreases the accuracy. Increasing the population size candecrease the minimal required value for the maximal step size parameter.

2. Optimal step size parameters can be roughly estimated by the break evenexpected distance of the worst-case analysis. The consequences of too bigparameter values are less severe than too small values with regard to theaccuracy for a fixed number of generations.

3. For small population sizes and predictable dynamics with small severity, awell-orientated directed local mutation is able to reduce the divergent behav-ior of local mutations with small maximal step sizes. It does not solve theproblems with higher severity values.

4. The proposition of bigger steps enables better accuracy rates, but requiresproper calibration of the step size parameter. For problems with varyingseverity values, the latter point should be guaranteed. Otherwise the accuracyrates may drop below the accuracy of the local mutation.

5. Even with small population sizes, combining non-zero mean mutation withthe proposition of bigger steps may lead to very precise tracking accuracy.

6. If the severity depends on the number of evaluations there is an optimal off-spring population size. This optimal number of offspring per generation in-creases if the value α or the value β increase. This implies especially for veryrestricted time resources rather big offspring population sizes. For the exam-ined mutation operator and β = 0, the population size 10–15 could serve asa rule of thumb. A detailed analysis is necessary for concrete recommenda-tions.

162

Page 163: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

7.9. CONCLUSION

7. For a mere tracking task the usage of an external memory should be avoidedif the individuals from the memory have a success rate of psuccess ≤ 0.01.However, introducing one individual from the memory into the populationaffects the tracking accuracy moderately such that an external memory isuseful for tracking problems with non-predictable, repetitive phases or alter-nation with low severity.

8. For problems with drastically changing dynamics, the use of the local mu-tation together with a self-adaptation mechanism promises a better trackingbehavior.

9. If the direction of the dynamics is expected to change considerably, it is ad-visable to use an undirected mutation operator with few strategy variables toensure quick adaptation.

10. Sole dependence of a self-adaptation of parameters on the fitness value is notadvisable. The tracking rate should be reflected as a control quantity in anycomponent of the algorithm.

Besides these rules that might help to construct algorithms that are better suited totracking problems, the moving corridor problem is introduced as a new benchmarkproblem for self-adaptation and adaptation techniques. This problem was used toidentify a new type of hard problems in the domain of dynamic environments.

163

Page 164: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,
Page 165: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 8

Four Case Studies Concerning theDesign Rules

This chapter links the findings of Chapter 7 together with standard evolution strat-egy. In four small case studies the validity of (a subset of) the design rules in acontinuous search space is shown. Since evolution strategies are usually used withself-adaptation or adaptation techniques, the used step width changes too. As aconsequence, this chapter here is not concerned with concrete optimal parametervalues. It is rather intended to underline the general applicability of the design rulesto optimization with evolution strategies.

Section 8.1 is concerned with the various adaptation and self-adaptation mecha-nisms. Concerning the proposition of bigger steps a new mutation operator is de-rived from the previous chapter in Section 8.2. Section 8.3 examines the movingcorridor problem as a pathological combination of tracking with optimization inthe context of evolution strategies. And, eventually, Section 8.4 explores the possi-bilities to add a new adaptation scheme by building a global model of the dynamicsduring optimization.

165

Page 166: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

8.1 Adapting and self-adapting local operators

8.1.1 Experimental setup

All the experiments in this section use the rotation shown in Example 4.3 whereonly one static component function is rotated around a point in the search space.The optimum is always positioned in the center of the rotation. And the task is todetect the optimum in spite of the changing environment. This kind of problemwas in particular chosen for this examination since it reveals certain difficulties forthe optimizer:

• the circular movement is more pretentious than a linear movement concern-ing more advanced adaptation schemes,

• points close to the optimum have a smaller severity than points with a biggerdistance to the optimum, and

• it resembles the characteristics of the moving corridor problem since againthe algorithm has to keep up tracking of certain points where the optimizationdirection is orthogonally to the tracking direction.

The following fitness functions are used as static component functions. Their gen-eral structure is visualized in Figure 8.1 for the two dimensional case. All problemsmust be minimized.

• Weighted Sphere: for xi ∈ [−30, 30] (1 ≤ i ≤ n) and n = 30

f(~x) =n∑i=1

ix2i

This is a unimodal problem. This problem is used to determine how well thealgorithm optimizes in a problem with small fitness changes.

• Rastrigin: for xi ∈ [−1, 1] (1 ≤ i ≤ n) and n = 30

f(~x) = 10n+n∑i=1

(x2i − 10 cos(2πxi)

)Note the restricted definition range of the Rastrigin function. However stillthe problem is multimodal with several local optima situated around theglobal optimum. This problem faces the optimizer with static difficulty –overcoming a local optimum – embedded into a dynamic environment.

166

Page 167: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

• Cone segment: for xi ∈ [−10, 10] (1 ≤ i ≤ n) and n = 30

f(~x) =

√∑n

i=1 x2i , if arccos

(x1√

x21+...+x2n

)≤ α√∑n

i=1 102, otherwise

with α = π2

in this investigation. This problem is also unimodal. However,only in one direction from the optimum fitness information is available thatleads to the optimum. All other directions are part of a plateau with con-stant “bad” fitness. This problem helps to investigate the ability to follow asmall tracking region and requires further optimization within the changinglandscape.

• Windmill: for xi ∈ [−1, 1] (1 ≤ i ≤ n), n = 2, and the following fitnessfunction described in polar coordinates (r, ϕ)

f(r, ϕ) =

r, P0(r, ϕ) ∨ Pπ

2(r, ϕ) ∨ Pπ(r, ϕ) ∨ P 3π

2(r, ϕ)

2.0, otherwise

with the predicate

Prot(r, ϕ) ≡ (ϕ+ rot)4√

2

π−√

2

9≤ r ≤ (ϕ+ rot)

4√

2

π

This function results in four segments of width 5 = π36

that look like a wind-mill. It is a two-dimensional version of the cone segment where a more accu-rate tracking is necessary. However, the existence of four segments releasesthe punishment if one segment is lost since the next approaching segmentoffers a next chance to gain information leading to the optimum.

The rotation must be supplied with a predetermined rotation time τ , which equalsthe number of generations until previous landscapes are repeated again. The ro-tation is managed by the multiplication of single rotations around two coordinateaxes. The rotation is given exactly in the following definition.

Definition 8.1 (Rotation matrix) The rotation matrix for an n-dimensional searchspace is defined by a permutation Perm of the dimensions 1, . . . , n and the ro-tation time τ . In addition, n is required to be even. Then the rotation matrix isdefined as the multiplication

R = Rp1,p2 ·Rp3,p4 · · ·Rpn−1,pn

167

Page 168: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

-30 -20 -10 0 10 20 -30-20

-100

1020

30-3000

-2000

-1000

0

-1-0.5

00.5 -1

-0.50

0.51

-40

-20

0

Weighted sphere

Rastrigin

-10-5

05 -10

-50

510

-12

-6

0

-1 -0.6 -0.2 0.2 0.6 -1-0.6

-0.20.2

0.61

-2

-1

0

Cone segment Spiral

Figure 8.1 Inverse diagrams of the used static fitness functions. In all functionsthe optimum is positioned at (0, 0).

with Perm = (p1, . . . , pn) and the pairwise rotations Ri,j defined by

Ri,j(k, l) =

cos(2π

τ), if (k = i ∧ l = i) ∨ (k = j ∧ l = j)

sin(2πτ

), if k = i ∧ l = j− sin(2π

τ), if k = j ∧ l = i

1, if k = l ∧ k 6= i ∧ k 6= j0, otherwise

for 1 ≤ i, j, k, l ≤ n. The complete rotation matrix at generation g > 0 may becomputed as Rg. ♦

Example 8.1 The matrix for a rotation in a 4-dimensional search space R4 deter-mined by the permutation (1, 3, 4, 2) is constructed using the following two basic

168

Page 169: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

cycle time5 25 50 100 200

1/5-success rule 0.225 0.272 0.217 0.0315 0.0self-adaptive isotropic 0.0928 0.027 0.0117 0.0 0.0self-adaptive non-isotropic 0.179 0.0758 0.082 0.0696 0.0736

Table 8.1: Fraction of lost generations where there is no valid individual in thepopulation (averaged over all respective runs and generations).

rotations

R1,3 =

cos(2π

τ) 0 −sin(2π

τ) 0

0 1 0 0sin(2π

τ) 0 cos(2π

τ) 0

0 0 0 1

R4,2 =

1 0 0 00 cos(2π

τ) 0 sin(2π

τ)

0 0 1 00 −sin(2π

τ) 0 cos(2π

τ)

resulting in the overall rotation matrix

R =

cos(2π

τ) 0 sin(2π

τ) 0

0 cos(2πτ

) 0 sin(2πτ

)sin(2π

τ) 0 cos(2π

τ) 0

0 −sin(2πτ

) 0 cos(2πτ

)

In the experiments presented here the values for τ are 5, 25, 50, 100, and 200for the weighted sphere, the cone segment, and the Rastrigin function. And thepermutations Perm are created randomly. For the windmill function τ equals 72and 144. Also note, that in case of the windmill function also the static landscapewith τ = 2π was examined.

The evolution strategy as an optimization algorithm is used as a (15, 100)-strategywithout recombination. There was no further tuning of this parameters. The off-spring population size was investigated closer for the two-dimensional windmillfunction where a (1, λ)–strategies with λ ∈ 10, 15, 20, 25, 30, 35, 50, 75, 100was used. In order to adapt the step-size of the mutation the following mechanismsare used:

169

Page 170: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 5

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 200

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 50

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 25

dis

tan

ceto

op

tim

um

dis

tan

ceto

op

tim

um

dis

tan

ceto

opti

mu

m

dis

tan

ceto

opti

mu

m

isotropicnon-isotropic

1/5 rule

Figure 8.2 Rotating weighted sphere optimized by evolution strategies

• the 1/5-success rule by Rechenberg (1973) as a global adaptation mechanism,

• self-adaptation of the isotropic mutation with one strategy parameter whichis applied to all search space dimensions (see Schwefel, 1981),

• step-size self-adaptation for the non-isotropic mutation with n strategy pa-rameters (see Schwefel, 1981), and

• the covariance matrix adaptation (cma) by Hansen and Ostermeier (1996,2001) for the two-dimensional windmill function only.

170

Page 171: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

1/5 ruleisotropic

non-isotropic

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 5

00.10.20.30.40.50.60.70.80.9

0 40 80 120 160 200generation

cycle time 5

dis

tan

ceto

op

tim

um

fract

ion

of

lost

run

s

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 25

dis

tan

ceto

opti

mu

m

0

0.1

0.2

0.3

0.4

0.5

0 40 80 120 160 200generation

cycle time 25

frac

tion

oflo

stru

ns

Figure 8.3 Rotating cone segment with fast severity optimized by evolution strate-gies

All experiments are averaged over 100 – or in the case of the windmill function200 – independent runs. As a basis for comparison the best fitness value and thedistance of the best individual to the optimum of each generation are used. In caseof the rotating cone segment also the percentage of generations is considered wherethe segment with “valid” or “good” fitness was completely lost.

The results are displayed in Figures 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, and Table 8.1.They are described and discussed in the following sections in various different

171

Page 172: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 100

0

5

10

15

20

25

30

0 40 80 120 160 200generation

cycle time 200

0

0.05

0.1

0.15

0.2

0.25

0 40 80 120 160 200generation

cycle time 100

0

0.05

0.1

0.15

0.2

0.25

0 40 80 120 160 200generation

cycle time 200

frac

tion

oflo

stru

ns

dis

tan

ceto

opti

mu

md

ista

nce

toop

tim

um

fract

ion

of

lost

run

s1/5 ruleisotropic

non-isotropic

Figure 8.4 Rotating cone segment with low severity optimized by evolution strate-gies

contexts.

8.1.2 Limitations of local operators

In this subsection, we are concerned with the question whether the evolution strat-egy is actually able to solve the dynamic optimization problems at all. A compari-son of the different techniques follows in the next two subsections.

172

Page 173: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

2.62.72.82.9

33.13.23.33.43.5

0 40 80 120 160 200generation

cycle time 5

1/5 ruleisotropic

non-isotropic

2.6

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

0 40 80 120 160 200generation

cycle time 100

2.6

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

0 40 80 120 160 200generation

cycle time 200

2.6

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

0 40 80 120 160 200generation

cycle time 50

dis

tan

ceto

opti

mu

m

dis

tan

ceto

opti

mu

m

dis

tan

ceto

opti

mu

m

dis

tan

ceto

opti

mu

m

cycle time 50

260280

cycle time 100

120140160180200220240260280300

0 40 80 120 160 200generation

100120140160180200220240

0 40 80 120 160 200generation

fitn

ess

fitn

ess

Figure 8.5 Rotating Rastrigin function optimized by evolution strategies

173

Page 174: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

static case, population size 15

isotropicnon-isotropic

cma

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

static case, population size 30

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

cycle time 144, population size 30

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

cycle time 144, population size 15

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

cycle time 72, population size 15

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

cycle time 72, population size 30

avg.

dis

tan

ceto

opti

mu

m

avg.

dis

tan

ceto

opti

mu

mav

g.d

ista

nce

toop

tim

um

avg.

dis

tan

ceto

opti

mu

mav

g.d

ista

nce

toop

tim

um

avg.

dis

tan

ceto

op

tim

um

Figure 8.6 Windmill function: average convergence for small populations and var-ious degrees of rotation.

174

Page 175: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

cycle time 72, population size 50

isotropicnon-isotropic

cma

0

0.2

0.4

0.6

0.8

1

40 80 120 160 200generations

cycle time 72, population size 100

avg.

dis

tan

ceto

op

tim

um

avg.

dis

tan

ceto

op

tim

um

Figure 8.7 Windmill function: average convergence for big populations and cycletime τ = 72.

Operators that are local concerning the phenotypic search space are very effectiveon rather smooth static landscapes. As the experiments on the rotating weightedsphere (Figure 8.2) show, this is also the case on a smooth unimodal function rotat-ing around the optimum.

Also the evolution strategy is able to optimize the rotating cone segment as the re-sults in Figures 8.3 and 8.4 show. Although this function is the only function wherea successful tracking of a certain region is strictly required (like in the moving cor-ridor problem of the previous chapter), this task is solved adequately.

But, similar to static optimization, as soon as multi-modal problems are involvedthe picture is different. The rotating Rastrigin function cannot be solved by allvariants of the evolution strategy (Figure 8.5). Apparently the algorithm is not ableto overcome the local optima by mere local search. The reasons for this behaviorhave not been investigated further. But since the evolution strategy is able to followthe cone segment in the previous problem, we assume that the interplay of thedistance between local and global optima, the severity of the dynamics, and thetechniques for step-size adaptation prevent the algorithm from detecting the basinof the global optimum.

175

Page 176: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

cma, run 91, 60 generations

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

isotr., run 99, 100 generations

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

non-isotr., run 93, 110 generations

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

non-isotr., run 95, 130 generations

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

isotr., run 28, 200 generations

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

cma, run 99, 60 generations

Figure 8.8 Windmill function: path of the best individuals for exemplary runs withcycle time τ = 72 and population size 15. The points where the visiblesegments are lost are marked with a +. Note that the visible segmentsrotates clockwise.

176

Page 177: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

020406080

100120140160180

0 0.4 0.8 1.2

freq

uen

cy

distance to optimum

Rotation 0.0, population size 15

isotropicnon-isotropic

cma

020406080

100120140160180200

0 0.4 0.8 1.2

freq

uen

cy

distance to optimum

Rotation 0.0, population size 30

isotropicnon-isotropic

cma

020406080

100120140160180200

0 0.4 0.8 1.2

freq

uen

cy

distance to optimum

Rotation 0.5, population size 30

isotropicnon-isotropic

cma

0

20

40

60

80

100

120

140

0 0.4 0.8 1.2

freq

uen

cy

distance to optimum

Rotation 0.5, population size 15

isotropicnon-isotropic

cma

0

20

40

60

80

100

120

0 0.4 0.8 1.2

freq

uen

cy

distance to optimum

Rotation 1.0, population size 15

isotropicnon-isotropic

cma

020406080

100120140160180

0 0.4 0.8 1.2

freq

uen

cy

distance to optimum

Rotation 1.0, population size 30

isotropicnon-isotropic

cma

Figure 8.9 Windmill function: distribution of the final fitness values of the 200experiments for each parameter setting for a qualitative comparison ofthe reliability of the different algorithms.

177

Page 178: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

8.1.3 Adaptation

As an adaptive technique the 1/5-success rule was used for the adaptation of thestep size in evolution strategy mutation. This technique is known to work well forunimodal static problems but gets trapped easily in the case of multimodal prob-lems.

In the first experiments using the rotating weighted sphere (Figure 8.2), the 1/5-success rule performs superior as long as the rotation time is very fast. It wors-ens slowly and with the rotation time τ = 200 certain problems become visible.This result is somewhat surprising since one would expect a better performancewith slower rotation. But apparently, in case of slow rotation, certain points inthe landscape are conceived as if they were local optima during certain periods ofthe optimization since all points in this region have degrading fitness. Then, the1/5-success rule reduces the step size considerably and impedes the optimizationprocess. In case of fast rotation, the 1/5-success rule is not able to reduce the stepsize comparably. As a consequence, the optimization process is not affected andthe 1/5-success rule outperforms all other algorithms. The fast rotation of an uni-modal search space around the optimum helps to soften the effects of prematureconvergence occurring with slow speeds of rotation.

However, these results are not transferable to arbitrary dynamic problems. In thecase of the slowly rotating cone segment (Figure 8.3 and Table 8.1) the 1/5-successrule shows an insufficient tracking behavior which leads to mediocre performance.One reason for this behavior is the very high fraction of lost runs which underlinesthat the 1/5-success rule is not able to adapt the operator appropriately and to fol-low the moving cone segment. These effects vanish with increasing cycle time, i.e.decreasing severity. With cycle time 200, the algorithm is even able to follow thecone segment in all runs and the 1/5-success rule outperforms the self-adaptingtechniques. As a consequence, it seems that the 1/5-success rule is a good choiceif the dynamics are very slow and the tracking area is distinct from the remainingsearch space.

The investigation of the 1/5-success rule shows that results of algorithms in dy-namic environments must be examined and interpreted very carefully since variousreasons and interactions can lead to good or poor performance.

8.1.4 Self-adaptation

Contrary to the 1/5-success rule as an adaptation mechanism, the results indicatethat self-adaptation is able to yield good results on a wider range of problems. In

178

Page 179: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.1. ADAPTING AND SELF-ADAPTING LOCAL OPERATORS

particular, this is true for the step-size adaptation of the isotropic mutation. Here avery stable behavior can be observed on the rotating weighted sphere (Figure 8.2)where the performance even seems to be independent of the cycle time of the prob-lem. In case of the rotating cone segment the step-size adaptation of the isotropicmutation shows also a good performance (Figures 8.3 and 8.4)—however with in-creasing severity (or decreasing cycle time) the performance worsens slightly. AsTable 8.1 indicates, this is also due to an increase of lost generations, i.e. no indi-vidual could be placed in the cone segment. But still the fraction of lost generationsis very small compared to the other used techniques.

The self-adaptation using n strategy parameters for the non-isotropic mutationproves to be less adequate than the adaptation of the isotropic mutation. In particu-lar this is due to the rotating movement which requires the self-adaptation mecha-nism to adapt all separate strategy parameters simultaneously in a correlated man-ner. As the fraction of lost runs for cycle time 100 shows in Figure 8.4, there is apeak of lost runs after each quarter rotation—these are the points where the sign ofhalf of the strategy parameters has to change. Apparently this is rather problematicand is at least one reason for the inappropriate performance.

The self-adaptation for the isotropic mutation uses only one strategy parameterand, therefore, shows a much more stable behavior. This might be a hint that self-adaptation involving less strategy variables are able to react more promptly—as itis indicated by Design rule 9.

This hypothesis is examined more closely with the rotating windmill function withdimension 2. On this function self-adaptation for both types of mutation as well asthe covariance matrix adaptation cma are investigated.

Figure 8.6 shows the results for small population sizes by using the distance to theoptimum as convergence criterion. The covariance matrix adaptation outperformsboth other self-adaptation techniques in the static case. However, with increasingseverity, cma performs considerably worse. Also the self-adaptation of the non-isotropic mutation with n strategy parameters worsens. The self-adaptation of theisotropic mutation is the only technique that shows with an offspring populationsize of 30 individuals a stable performance in the static problem and the two dy-namic problems. However, the differences between 15 and 30 offspring indicatesalready that increasing the offspring population can improve the performance ofcma considerably. Figure 8.7 shows that an offspring population size of 100 indi-viduals suffices for cma to beat the self-adaptive isotropic mutation again. How-ever, such an offspring populations size seems to be inadequate for a problem witha two-dimensional search space.

By examining the windmill experiments closer, a few explanations for the insuffi-

179

Page 180: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

cient behavior of more complex self-adaptive techniques are found. Obviously thesimple adaptation technique is able to adapt quickly within the dynamically chang-ing environments where the more sophisticated mechanisms are too inert to adapt.The information is not steady enough to enable successful adaptation. In the caseof covariance matrix adaptation two exemplary runs in Figure 8.8 show an oftenoccurring behavior for cycle time τ = 72 with a small populations size: the muta-tion is able to track the visible segment for some generations but then gets lost andcan recover only very seldom, though visible information is passing by. Figure 8.9reflects this behavior and shows that a high percentage of the experiments got lostfar from the optimum.

In the case of the self-adaptive non-isotropic mutation with n strategy parameters,the exemplary runs in Figure 8.8 show that, although the visible segment is trackedfor periods, the behavior is rather erratic. In one example, it is not attracted by theoptimum and only moves in a big circle around it. In the other example, it gets closeto the optimum but moves away again. Moreover, it seems as if the self-adaptationonly adapts in one dimension at a time.

Figure 8.8 shows also two typical runs for the self-adaptive isotropic mutation inwhich the visible segment is tracked very well and the search is drawn towards theoptimum. However, in the right example the algorithm is not able to stay closeto the optimum and tends to move again slightly away from it. Still, a very goodaccuracy of the optimization is also reflected in the final fitness distributions inFigure 8.9.

This comparison illustrates that sometimes the simple self-adapting mutations aremore successful than complex, smart adaptation mechanisms. The reason for thisbehavior is the underlying supposition for those mechanisms that the fitness land-scape remains firm until the adaptation takes place. In the case of dynamic land-scapes this supposition is seldom true.

8.1.5 Discussion

The experiments confirm the consideration of the previous chapter for the self-adaptation. A more simple self-adaptation mechanism seems to be advisable (De-sign rule 9). However adaptation in form of the 1/5-success rule is not a generalenough adaptation mechanism.

The investigation of the offspring population size also confirms Design rule 1 con-cerning the influence of the population size.

The consequences concerning optimization and tracking in Design rule 10, could

180

Page 181: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.2. PROPOSITION OF BIGGER STEPS

not be discovered in the examination of the rotating cone segment. Here, at least thestep-size adaptation with one strategy parameter shows a good adaptive behavior.

The results of this section are based on the experiments of two previously publishedarticles (Weicker & Weicker, 1999, 2000).

8.2 Proposition of bigger steps

This section examines whether Design rule 4 is applicable, i.e. the proposition ofbigger steps leads to an improved tracking accuracy.

For this investigation, a mere tracking problem was chosen: a decentralized sphererotating around the center. The static component function is defined as follows.

• Decentralized Sphere: for xi ∈ [−30, 30] (1 ≤ i ≤ n) and n = 2

f(~x) =n∑i=1

(xi −

√302

n

)2

Again the rotation from Definition 8.1 is applied to turn the static problem intoa dynamic one. For the cycle time τ ∈ 50, 100 is considered. Note, that thedistance of the optimum to the center of the rotation is always 30, which meansthat the optimum is always inside or at the border of the search space.

For this examination, a (1, 15)-evolution strategy is used. The standard isotropicGaussian mutation with step-size self-adaptation is compared to the following ring-like mutation which is derived from the analytical investigation in Section 7.5.

Where in the theoretical investigation the ring-like mutation was parameterized bythe maximal step size maxstep, this is not possible in a real-valued search spaceif the ring-like mutation is based on the Gaussian pdf. However, for conveniencewe still call the strategy parameter of the real-valued ring-like mutation maxstep.When mutating an individual A = 〈A1, . . . , An,maxstep〉 we proceed as follows.

1. The maximal step size is update like the standard deviation in isotropic step-size adaptation of evolution strategies

maxstep ′ ← maxstep e1√lN (0,1)

2. Randomly a unity vector v = 〈v1, . . . , vn〉 is chosen for the direction of themutation.

181

Page 182: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

3. A random number x ∼ N (0, 1) is chosen which is mapped to the step size

stepsize ← 4 + x

8maxstep ′

4. each component Ai (1 ≤ i ≤ n) is updated using the following rule

A′i ← Ai + stepsize vi

The new individual is A′ = 〈A′1, . . . , A′n,maxstep ′〉.Figure 8.10 shows how the random variable x is mapped to the distance range[0,maxstep]. However maxstep is not the maximal occurring step size, with aprobability of approximately 0.000317 a bigger step may occur (see p.20, Bron-stein & Semendjajew, 1991). And similarly steps with a negative steps size mayoccur with probability 0.000317 too, i.e. the opposite direction is used. Besidesthose restrictions, this mutation imitates the discrete ring-like mutation as close aspossible.

Gaussianpdf

mapped to distances [0,maxstep]

probability togo further thanmaxstep

probabilityto step inoppositedirection

prob

abili

ty

random variable x for the step size

0

0.1

0.2

0.3

0.4

-4 -2 0 2 4

Figure 8.10 The probability of the step size in the ring-like mutation is explainedin this figure.

For each set-up of problem and algorithm 100 experiments have been executed for200 generations using different initial random seeds. Student’s t-test is applied tothe best fitness values of each generation. A difference between the mean best

182

Page 183: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.2. PROPOSITION OF BIGGER STEPS

GaussianRing

-4

-2

0

2

4

0 40 80 120 160 200generation

sig. for Gauss

sig. for Ring

t-va

lue

0

10

20

30

40

50

0 40 80 120 160 200generation

bes

tfi

tnes

s(o

nav

erage

)

Figure 8.11 This figure shows the results of the Gaussian and the ring-like mu-tation on the decentralized, rotating sphere function with cycle time50. The best fitness value averaged over 100 experiments is shown onthe left. The t-value of the statistical hypothesis test is shown on theright.

GaussianRing

0

2

4

6

8

10

0 40 80 120 160 200generation

bes

tfi

tnes

s(o

nav

erag

e)

-4

-2

0

2

4

0 40 80 120 160 200generation

sig. for Gauss

sig. for Ring

t-va

lue

Figure 8.12 This figure shows the results of the Gaussian and the ring-like mu-tation on the decentralized, rotating sphere function with cycle time100. The best fitness value averaged over 100 experiments is shownon the left. The t-value of the statistical hypothesis test is shown onthe right.

fitness values of the two algorithms is considered to be significant if the error rate

183

Page 184: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

is less than 0.05, i.e. t-values above 1.96 or below -1.96.

Figure 8.11 shows the results for cycle time 50. If we exclude the initial phasefor finding the optimum, there is in almost all generations a better average bestfitness of the ring-like mutation and in many generations the difference is evensignificant. For a smaller severity (cycle time 100), the results in Figure 8.12 showthat the advantage of the ring-like mutation over the Gaussian mutation decreases.Altogether we can conclude that there is an obvious trend in favor of the ring-likemutation with increasing severity which validates the statement of Design rule 4.

8.3 Self-adaptation for the moving corridor

Section 7.8.2 of the last chapter has argued that in a set-up where the trackingdirection and the optimization direction are arranged orthogonally a self-adaptationmechanism of a local operator will increase its step size until the step size is so bigthat no accurate tracking is possible anymore. This statement is reconsidered forevolution strategies.

The problem is defined very similar to the discrete version of the problem with afew particularities. Again the dimension of the problem is 2. Since we considerboth isotropic and non-isotropic mutation with self-adaptation the direction of thedynamics are arranged in a 30 angle from one axis of the search space to rule outeffects due to an alignment. The width of the corridor is 1.0 and the severity ismeasured as the fraction of the area of the corridor that is not covered anymore bythe corridor of the next generation. In this investigation a severity of 0.6 is used.Both the set-up of the corridor and the definition of the severity are displayed inFigure 8.13.

Because of the very narrow corridor that must be tracked, a (1, 40)-evolution strat-egy is used. As indicated already above, mutation operators with one and n strategyparameters are used. For each set-up of algorithm and problem 100 independentexperiments are executed using different initial random seeds. As a performancemeasure the average best fitness over all experiments, the percentage of invalid in-dividuals created, and the percentage of experiments which got completely lost, i.e.they could not track the tracking region, are considered.

The results are shown in Figure 8.14 exemplary for dynamics 0.6. Other severityvalues reveal a similar behavior and are not shown here. The step size adaptationof the isotropic mutation shows an astonishingly good behavior. However as thetheoretical analysis in the previous chapter predicted, this is accompanied by ahuge increase in completely lost runs and invalid individuals. At the end of 200

184

Page 185: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.3. SELF-ADAPTATION FOR THE MOVING CORRIDOR

degree ofdynamics

30°

1.00.6

0.20.0

trackingregion

dynamics

optimum

Figure 8.13 The orthogonal arrangement of tracking and optimization of the mov-ing corridor problem is shown at the left as well as the orientation ofthe tracking direction. On the right the definition of the severity issketched.

generations more than 50 percent of all experiments have lost the moving corridor.This is due to the inevitable increase of the values in the strategy parameter, shownfor the isotropic mutation in the left part of Figure 8.15.

In order to avoid such a behavior Design rule 10 proposes to break up the soledependence of the self-adaptation on the fitness value. Since the suggested con-sideration of the tracking rate is difficult to realize, a different technique was cho-sen here to take care of the discrepancy between the greedy behavior concerningthe optimization and the needs for successful tracking. The primary idea is toselect object values and strategy values with different mechanisms. The objectvalues are selected with the usual selection using the best fitness (comma selec-tion). However, the strategy values are determined as follows. For all currentoffspring individuals that have a better fitness than the parent’s fitness, the distance√∑n

i=1(schildi − sparenti )2 of their strategy variables ~schild and the parent’s strategyvariables ~sparent are computed. Those strategy variables with a minimal deviationfrom the parents strategy values are selected. If there are no offspring with betterfitness, the strategy variables are selected with the best fitness selection too. Thenew individual is composed of the selected values. This selection mechanism isreferred to as distinct selection.

The distinct selection is applied to the isotropic mutation. The results concerningthe moving corridor problem are displayed in Figures 8.14 and 8.15. The distinctselection dampens the explosion of the strategy variables. As a consequence, the

185

Page 186: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

1

0.9

0.8

0.7

0.6

0 40 120 160 20080generation

all runs

0

0.2

0.4

0.6

0.8

1

0 40 80 120 160 200generation

all runs

frac

tion

ofco

mp

lete

lylo

stru

ns

frac

tion

ofin

vali

din

ds.

9300

9400

9500

9600

9700

9800

9900

10000

0 40 80 120 160 200generation

successful runs

avg.

bes

tfi

tnes

s

isotropic

isotr. (dist. sel.)non-isotropic

Figure 8.14 Results for the moving corridor problem with strength of dynamics0.6: the upper row shows the best fitness values of all successfulexperiments on average (with two differently scaled ordinates), thelower row shows the fraction of completely lost experiments (left)and the fraction of invalid generated individuals (right).

created invalid individuals can be kept on an almost constant level which leads to asignificantly smaller increase of the completely lost runs.

For a test how the distinct selection behaves on a mere tracking task, the movingcircle problem is considered where inside the circle the distance to the center serves

186

Page 187: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.3. SELF-ADAPTATION FOR THE MOVING CORRIDOR

isotropic mutation isotropic mutation

with distinct selection

0

200

400

600

800

1000

1200

1400

0 40 80 120 160 200generation

all runs

avg.

bes

tst

rate

gy

var

00.5

11.5

22.5

33.5

44.5

5

0 40 80 160 200generation

all runs

120av

g.

bes

tst

rate

gy

vars

0

Figure 8.15 Analysis of the adaptation of strategy variables for the moving cor-ridor problem: the left picture shows the values for the standardisotropic mutation with one strategy parameter and the right pictureshows the values for isotropic mutation with distinct selection.

severity1.0

0.60.20.0

0.2

0.630°

regiontracking

dynamics

Figure 8.16 This figure shows the moving circle problem where the dynamics andthe severity of the dynamics are similarly defined to the moving cor-ridor problem.

as fitness value and outside of the circle a constant “bad” fitness value is assumed.The dynamics are introduced analogously to the dynamics in the moving corridorproblem. They are sketched in Figure 8.16. Figure 8.17 shows the results of the

187

Page 188: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0 40 80 120 160 200generation

Dynamics 0.2, successful runs

0

0.1

0.2

0.3

0.4

0.5

0.6

0 40 80 120 160 200generation

Dynamics 0.2, all runs

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0 40 80 120 160 200generation

Dynamics 0.6, successful runs

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0 40 80 120 200generation

Dynamics 0.6, all runs

160

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0 40 80 120 160 200generation

Dynamics 1.0, successful runs

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

0 40 80 120 160 200generation

Dynamics 1.0, all runs

per

centa

geof

inva

lid

ind

s.p

erce

nta

ge

of

inva

lid

ind

s.p

erce

nta

geof

inva

lid

ind

s.

avg.

bes

tfi

tnes

sav

g.b

est

fitn

ess

avg.

bes

tfi

tnes

s

standard isotr.isotr. (dist. sel.)

Figure 8.17 Impact of the distinct selection on the behavior in a mere trackingtask.

188

Page 189: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.4. BUILDING A MODEL OF THE DYNAMICS

experiments for severity of dynamics 0.2, 0.6, and 1.0. In case of this mere trackingtask, the performance of the algorithm is not affected by the new selection. These,experiments indicate that this selection technique is a useful means for dynamicproblems. The basic idea of considering most similar individuals is used in thenext section to define a more global adaptation mechanism.

The results in this section are based on the experiments of an article previouslypublished at a conference (Weicker, 2001).

8.4 Building a model of the dynamics

As Design rule 10 and the case study in the previous section have shown, soledependence on self-adaptive mechanisms can be harmful. In the previous section,distinct selection of strategy and object variables has been proposed as one meansto reduce the negative effects. The approach in this section goes one step furthersince it tries to implement a small-scale version of the vision noted in Section 7.8.3:the derivation of the underlying rules of dynamics during the optimization process.

The idea of learning something about the problem during optimization is not new.This is the basis for all self-adaptation techniques as well as the approach to buildstatistical models of the fitness distribution of the problem (Baluja, 1994; Pelikan,Goldberg, & Cantu-Paz, 1999; Muhlenbein & Mahnig, 2001; Sebag, Schoenauer,& Ravise, 1997a, 1997b). However, as it was already noted in Section 6.8, inthe field of dynamic environments, the approach of Munetomo et al. (1996) usingstochastic learning automata is unique. But, it is not applicable to drifting prob-lems. As a consequence this section proposes a new method for the purpose ofshowing the feasibility of such a technique.

In Section 4.3 the possible coordinate transformations in the framework have beenidentified as rotations and linear translations as well as combinations of both. It wasargued in Section 4.4.1 that especially the latter case makes a simple derivationof the dynamics from the movement of a few points very hard. This problem iscomplicated by the fact that the evolutionary algorithm cannot derive the exactmovement of certain points in the search space.

For the two-dimensional case any dynamics of a predictable problem can be de-scribed by the following two formulas.

x ← a+ u+ cos(α)(x− a)− sin(α)(y − b)y ← b+ v + sin(α)(x− a)− cos(α)(y − b)

It describes a linear translation if a = b = α = 0 where the translation equals

189

Page 190: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

0

5

10

15

20

25

30

35

40

0 40 80 120 160 200generation

GaussianPredictive

-2

-1

0

1

2

3

4

5

0 40 80 120 160 200generation

sig. for Gauss

t-va

lue

bes

tfi

tnes

s(o

nav

erag

e)

sig. for Predictive

Figure 8.18 This figure shows the results of the Gaussian and the predictive muta-tion using a global model applied to the decentralized, rotating spherefunction with cycle time 50. The best fitness value averaged over 100experiments is shown on the left. The t-value of the statistical hypoth-esis test is shown on the right.

the vector (u, v). With u = v = 0 a rotation around the center (a, b) and rotationangle α is described. If now the parameter values a, b, u, v, α can be learnt cor-rectly during the optimization process, the two formulas can be used to eliminatethe dynamics of the problem with the consequence that—in spite of probably bigcoordinate severity—arbitrary small accuracy during tracking is possible.

The basic algorithm is a (1, 15)-evolution strategy. Each offspring individual iscreated by mutation only—no recombination operator is used.

The mechanism to adapt the dynamics model works as follows. In each generationthe fitness of the created offspring is compared to the fitness of the parent individ-ual. And like in the distinct selection, a comparison concerning the most similarindividuals is used. However, here not the strategy variables but the most similarfitness values are used to approximate the movement of the position associated withthe parent in the search space. The position of the parent and the vector from theparent to the most similar offspring are stored together in a FIFO list. In the currentimplementation the list holds at most 50 positions and vectors. After generation30, it is assumed that enough data is collected and the learning process starts.

Currently a rather expensive learning procedure has been chosen. At the end ofeach generation the center of a rotation is approximated in a first step which isdescribed in the next paragraph. If the approximated center lies within the search

190

Page 191: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.4. BUILDING A MODEL OF THE DYNAMICS

0

2

4

6

8

10

12

14

0 40 80 120 160 200generation

-2

-1

0

1

2

3

4

5

6

0 40 80 120 160 200generation

sig. for Gauss

sig. for Predictive

PredictiveGaussian

bes

tfi

tnes

s(o

nav

erage

)

t-va

lue

Figure 8.19 This figure shows the results of the Gaussian and the predictive muta-tion using a global model applied to the decentralized, rotating spherefunction with cycle time 100. The best fitness value averaged over100 experiments is shown on the left. The t-value of the statisticalhypothesis test is shown on the right.

GaussianPredictive

-2

0

2

4

6

8

10

0 40 80 120 160 200generation

sig. for Gauss

sig. for Predictive

t-va

lue

0

0.1

0.2

0.3

0.4

0 40 80 120 160 200generation

bes

tfi

tnes

s(o

nav

erag

e)

Figure 8.20 This figure shows the results of the Gaussian and the predictive muta-tion using a global model applied to the linear translation of a spherefunction. The best fitness value averaged over 100 experiments isshown on the left. The t-value of the statistical hypothesis test isshown on the right.

space boundaries it replaces the values (a, b) of the current model, otherwise a =

191

Page 192: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8. FOUR CASE STUDIES CONCERNING THE DESIGN RULES

b = 0 are used. Then in a second step, a self-adaptive (1 + 1)-evolution strategy isexecuted for 2000 steps which optimizes all model parameters a, b, u, v, α.

The estimation of the center uses three points and vectors from the FIFO queue,namely the last entry, the oldest entry, and the entry in the middle. For all vectorsa normal vector is determined. Then the normal vectors at the respective pointsare intersected pairwise. This results in three different intersection points. And theestimated center is computed as the arithmetic mean of the three points.

After generation 30 the mutation operator uses the model and translates the parentalindividual accordingly. The usual self-adaptive mutation is applied to the translatedpoint in the search space.

This algorithm is applied to the rotating decentralized sphere introduced in Sec-tion 8.2 and compared to the standard evolution strategy with similar parametersettings. Again for each pair of problem and algorithm 100 independent experi-ments have been executed for 200 generations. Student’s t-test is applied to thebest fitness values of each generation. A difference between the mean best fitnessvalues of the two algorithms is considered to be significant if the error rate is lessthan 0.05, i.e. t-values above 1.96 or below -1.96.

The results are shown in Figure 8.18 for a cycle time of 50 and in Figure 8.19 fora cycle time of 100. As it can be seen clearly there is a strong tendency that thealgorithm with the dynamics model produces better results. For the majority ofgenerations there is even a statistical significance for this algorithm. At no timethere is statistical significance for the Gaussian mutation. As a consequence, wecan conclude that the algorithm is able to derive a useful model of the rotatingdynamics and to improve its performance by using this model.

In order to investigate whether linear translations can be derived by the algorithmssimilarly good, the following problem of a linearly moving sphere is investigated.The problem is defined as

f (t)(~x) =n∑i=1

(xi − z(t)

i

)2

where xi ∈ [−30, 30] (1 ≤ i ≤ n) and n = 2 and z(t) follows the schedule

z(0)1 = −30

z(0)2 = −15

z(t+1)1 = z

(t)1 + 0.3

z(t+1)2 = z

(t)2 + 0.15

192

Page 193: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

8.4. BUILDING A MODEL OF THE DYNAMICS

for generations 1 ≤ t ≤ 200. The experimental setup and the assessment is identi-cal to the rotating sphere.

The results are shown in Figure 8.20. Apparently the linear translation can be usedeven better by the model of the dynamics.

There has been made no attempt to improve or tune the algorithm that uses theglobal dynamic model. As a consequence the computation time of this algorithm issignificantly bigger than the time needed by the standard evolution strategy. In thecurrent version the global model is only sensible in problems where each fitnessevaluation is extremely expensive. Nevertheless the comparison is still valid sincethis case study only aims at a feasibility study that it is possible to derive a globalmodel of the dynamics—independently of the self-adaptive control of the step size.Future work has to investigate how the computational cost of the algorithm canbe reduced and whether there is a simplified model of the dynamics possible thatscales with increasing dimensionality.

193

Page 194: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,
Page 195: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

CHAPTER 9

Conclusions and Future Work

This chapter summarizes the results of this thesis, re-evaluates their utility, andgives an outlook at future improvements and topics which I believe to be the focusof future research in the field.

Chapter 1 gives a short motivation for the topic of this thesis and Chapter 2 presentsa short summary of the knowledge presupposed in the thesis. It also provides a sur-vey of the major advances in evolutionary dynamic optimization. This survey is thebasis for the discussion of open problems in Chapter 3 and an overview on the con-tributions of this thesis and how they integrate into the existing research. The lastsection of this chapter is also devoted to a short discussion on the methodologicalapproach of the thesis.

Chapter 4 proposes a mathematical framework to classify and compare dynamicproblems. This is the first detailed classification for non-stationary environmentsand should serve as a proper basis for the integration of results of many differentresearchers. The main drawback of the framework is the exclusion of problemsdefined on a binary search space. Also the usage of the framework in the thesis—especially in Chapter 6—has shown that there is still a high variance how certainproblems can be fit into the framework. Future work should focus on this aspect,probably redefine certain aspects of the framework, and develop strict guidelineshow the framework should be applied.

195

Page 196: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

In Chapter 5 properties of “good” or “successful” evolutionary processing in dy-namic environments are discussed. This leads to the definition of stability andreactivity as alternative or additional goals to the mere accuracy of an approxima-tion. The focus of the chapter is on the empirical investigation how the differentgoals may be measured in various problem classes. This is the first study devotedto performance measures only in dynamic optimization. Probably most surprisingis the difference between high and low quality algorithms for some measures andproblem classes since performance measures should be used to determine the qual-ity of an algorithm—if the measure depends on the quality this is not possible in anobjective way. For measuring the recovery a new window based measure appearsto be promising for high quality approximations. Still this first study presented hereis only based on four different problem classes. A large-scale investigation shouldbe carried out to get more insight into the utility of performance measures. Alsothe methodology of how to assess the quality of performance measures should bere-evaluated. It appears that averaging favors certain measures that are doing fairlywell on most problem instances. Future work should consider the aspect in moredepth to what extent a measure can guarantee exactness.

Chapter 6 deals with the various different techniques in dynamic environments. Itanalyzes a major part of the existing experimental research and classifies the usedtechniques. Furthermore, the tackled problems are classified using the frameworkof Chapter 4. As a consequence, both classifications lead to a mapping betweentechniques and problems. This demonstrates well the potential utility of the pro-posed framework. However, the current mapping reflects only the attempts of cer-tain techniques on certain problems. Most research articles are not concerned witha proper comparison of techniques and there are many combinations of problemsand techniques nobody has investigated yet. In addition, many compromises havebeen taken into bargain to categorize the tackled problems within the proposedframework. Future work should concentrate on a systematical investigation withthe primary focus on the comparison of the different techniques.

The first part of the thesis (including Chapter 6) endeavors to lay a broad foundationfor the whole field of dynamic optimization. Due to time restrictions it was notpossible to apply the framework intensively for filling in the missing results to buildup the holistic comprehensive understanding of dynamic evolutionary optimizationthe framework aims at. The systematic construction of integrated results on thisfoundation is only sketched. Numerous extensive investigations will be necessaryto fill the scientific empty spaces created by this foundation. Eventually, thesefuture results will confirm or jeopardize the presented framework.

Chapter 7 concentrates on one special class of problems, namely unimodal drift-ing problems without fitness rescaling. It models instances of this problem class

196

Page 197: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

on a discrete, two dimensional domain. With two Markov chain models of the EAdynamics on this problem several results are derived. Both the impact of the param-eters and the relevance of the principles underlying the ES mutation are examined.From these investigations ten design rules are derived for tackling dynamic prob-lems with local variation. This is the most profound and rigorous part of the thesissince all possible aspects of the subfield are considered. The very abstract modelcould be criticized which is not directly related to any existing evolutionary algo-rithm. However, the principles of the evolution strategy mutation are reflected verywell. Therefore, the results should be transferable easily to evolution strategies.This statement is supported by the case studies presented in Chapter 8.

The thesis closes with four different case studies in Chapter 8 where some of thedesign rules of Chapter 7 are applied. The validity of the design rules is confirmedby the empirical investigations. On the basis of Chapter 7 two new techniques havebeen developed for tracking a drifting problem, distinct selection and the usage ofa global model of the dynamics.

197

Page 198: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

References

Angeline, P. J. (1997). Tracking extrema in dynamic environments. In P. J. Ange-line, R. G. Reynolds, J. R. McDonnell, & R. Eberhart (Eds.), EvolutionaryProgramming VI (pp. 335–345). Berlin: Springer.

Angeline, P. J. (1998). Evolving predictors for chaotic time series. In S. Rogers,D. Fogel, J. Bezdek, & B. Bosacchi (Eds.), Proc. of SPIE (Volume 3390): Ap-plication and Science of Computational Intelligence (pp. 170–180). Belling-ham, WA.

Angeline, P. J., & Fogel, D. B. (1997). An evolutionary program for the identi-fication of dynamical systems. In S. Rogers (Ed.), Proc. of SPIE (Volume3077): Application and Science of Artificial Neural Networks III (pp. 409–417). Bellingham, WA.

Angeline, P. J., Fogel, D. B., & Fogel, L. J. (1996). A comparison of self-adaptationmethods for finite state machines in dynamic environments. In L. J. Fogel,P. J. Angeline, & T. Back (Eds.), Evolutionary Programming V: Proc. ofthe Fifth Annual Conf. on Evolutionary Programming (pp. 441–449). Cam-bridge, MA: MIT Press.

Arnold, D. V., & Beyer, H.-G. (2002). Random dynamics optimum tracking withevolution strategies. In J. J. Merelo Guervos, P. Adamidis, H.-G. Beyer, J.-L. Fernandez-Villacanas, & H.-P. Schwefel (Eds.), Parallel problem solvingfrom nature – PPSN VII (pp. 3–12). Berlin: Springer.

Back, T. (1997). Self-adaptation. In T. Back, D. B. Fogel, & Z. Michalewicz (Eds.),Handbook of Evolutionary Computation (pp. C7.1:1–15). Bristol, New York:Institute of Physics Publishing and Oxford University Press.

Back, T. (1998). On the behavior of evolutionary algorithms in dynamic envi-ronments. In IEEE Int. Conf. on Evolutionary Computation (pp. 446–451).Piscataway, NJ: IEEE Press.

Back, T. (1999). Self-adaptive genetic algorithms for dynamic environments withslow dynamics. pp. 142–145, GECCO Workshops, A. Wu (ed.).

Baluja, S. (1994). Population-based incremental learning: A method for inte-grating genetic search based function optimization and competitive learning(Tech. Rep. No. CMU-CS-94-163). Pittsburgh, PA: Carnegie Mellon Uni-versity.

Beasley, J. E., Krishnamoorthy, M., Sharaiha, Y. M., & Abramson, D. (1995). Dy-namically scheduling aircraft landings - the displacement problem. (ImperialCollege, London, England)

Bersini, H. (1998). Fuzzy-evolutionary systems. In E. H. Ruspini, P. P. Bonissone,& W. Pedrycz (Eds.), Handbook of Fuzzy Computation (pp. D3.1:1–D3.6:2).Bristol: Institute of Physics Publishing.

198

Page 199: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

Beyer, H.-G. (2001). The theory of evolution strategies. Berlin: Springer.Biegel, J. E., & Davern, J. J. (1990). Genetic algorithms and job shop scheduling.

Computers and Industrial Engineering, 19(1–4), 81–91. (Proc. of the 12thAnnual Conf. on Computers and Industrial Engineering)

Bierwirth, C., Kopfer, H., Mattfeld, D. C., & Rixen, I. (1995). Genetic algorithmbased scheduling in a dynamic manufacturing environment. In Proc. of 1995IEEE Conf. on Evolutionary Computation. Piscataway, NJ: IEEE Press.

Bierwirth, C., & Mattfeld, D. C. (1999). Production scheduling and reschedulingwith genetic algorithms. Evolutionary Computation, 7(1), 1–17.

Bonissone, P. P. (1997). Soft computing: the convergence of emerging reasoningtechnologies. Soft Computing, 1(1), 6–18.

Box, G. E. P. (1957). Evolutionary operation: A method for increasing industrialproductivity. Applied Statistics, 6(2), 81–101.

Box, G. E. P., & Muller, M. A. (1958). A note on the generation of random normaldeviates. Annals. Math. Stat., 29, 610–611.

Branke, J. (1999a). Evolutionary algorithms for dynamic optimization problems:A survey (Tech. Rep. No. 387). Karlsruhe, Germany: Institute AIFB, Uni-versity of Karlsruhe.

Branke, J. (1999b). Evolutionary approaches to dynamic optimization problems:A survey. pp. 134–137, GECCO Workshops, A. Wu (ed.).

Branke, J. (1999c). Memory enhanced evolutionary algorithms for changing op-timization problems. In 1999 Congress on Evolutionary Computation (pp.1875–1882). Piscataway, NJ: IEEE Service Center.

Branke, J. (2002). Evolutionary optimization in dynamic environments. Boston:Kluwer.

Branke, J., & Mattfeld, D. (2000). Anticipation in dynamic optimization: Thescheduling case. In M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton,J. J. Merelo, & H.-P. Schwefel (Eds.), Parallel problem solving from nature– PPSN VI (pp. 253–262). Berlin: Springer.

Bremermann, H. J. (1962). Optimization through evolution and recombination. InM. C. Yovitis & G. T. Jacobi (Eds.), Self-organizing systems (pp. 93–106).Washington, D.C.: Spartan.

Bronstein, I. N., & Semendjajew, K. A. (1991). Taschenbuch der Mathematik.Stuttgart: Teubner.

Cartwright, H. M., & Tuson, A. L. (1994). Genetic algorithms and flowshopscheduling: towards the development of a real-time process control system.In T. C. Fogarty (Ed.), Proc. of the AISB Workshop on Evolutionary Comput-ing (pp. 277–290). Berlin: Springer.

Caruana, R. A., & Schaffer, J. D. (1988). Representation and hidden bias: Grayversus binary coding in genetic algorithms. In J. Leard (Ed.), Proc. of the

199

Page 200: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

5th Int. Conf. on Machine Learning (pp. 153–161). San Mateo, CA: MorganKaufmann.

Cedeno, W., & Vemuri, V. R. (1997). On the use of niching for dynamic landscapes.In Int. Conf. on Evolutionary Computation (pp. 361–366). Piscataway, NJ:IEEE Press.

Cobb, H. G. (1990). An investigation into the use of hypermutation as an adaptiveoperator in genetic algorithms having continuous, time-dependent nonsta-tionary environments (Tech. Rep. No. 6760 (NLR Memorandum)). Wash-ington, D.C.: Navy Center for Applied Research in Artificial Intelligence.

Cobb, H. G., & Grefenstette, J. J. (1993). Genetic algorithms for tracking changingenvironments. In S. Forrest (Ed.), Proc. of the Fifth Int. Conf. on GeneticAlgorithms (pp. 523–530). San Mateo, CA: Morgan Kaufmann.

Coker, P., & Winter, C. (1997). N-sex reproduction in dynamic environments. InP. Husbands & I. Harvey (Eds.), Fourth european conference on artificial life(p. ?). Cambridge, MA: MIT Press.

Collard, P., Escazut, C., & Gaspar, A. (1996). An evolutionary approach for timedependant optimization. In Int. Conf. on Tools for Artificial Intelligence 96(pp. 2–9). IEEE Computer Society Press.

Collard, P., Escazut, C., & Gaspar, A. (1997). An evolutionary approach fortime dependant optimization. International Journal on Artificial IntelligenceTools, 6(4), 665–695.

Dasgupta, D. (1995). Incorporating redundancy and gene activation mechanisms ingenetic search for adapting to non-stationary environments. In L. Chambers(Ed.), Practical handbook of genetic algorithms, vol.2 – new frontiers (pp.303–316). Boca Raton: CRC Press.

Dasgupta, D., & McGregor, D. R. (1992). Nonstationary function optimizationusing the structured genetic algorithm. In R. Manner & B. Manderick (Eds.),Parallel Problem Solving from Nature 2 (Proc. 2nd Int. Conf. on ParallelProblem Solving from Nature, Brussels 1992) (pp. 145–154). Amsterdam:Elsevier.

De Jong, K. A. (1975). An analysis of the behavior of a class of genetic adaptivesystems. Unpublished doctoral dissertation, University of Michigan, AnnArbor, MI.

De Jong, K. A. (1993). Genetic algorithms are not function optimizers. In L. D.Whitley (Ed.), Foundations of genetic algorithms 2 (pp. 5–17). San Mateo,CA: Morgan Kaufmann.

De Jong, K. A. (2000). Evolving in a changing world. In Z. Ras & A. Skowron(Eds.), Foundation of intelligent systems (pp. 513–519). Berlin: Springer.

Dozier, G. (2000). Steady-state evolutionary path planning , adaptive replacement,and hyper-diversity. In M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lut-

200

Page 201: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

ton, J. J. Merelo, & H.-P. Schwefel (Eds.), Parallel problem solving fromnature – PPSN VI (pp. 561–570). Berlin: Springer.

Droste, S. (2002). Analysis of the (1 + 1) EA for a dynamically changing onemax-variant. In Congress on Evolutionary Computation (CEC 2002) (pp. 55–60).Piscataway, NY: IEEE Press.

Eigen, M. (1971). Selforganization of matter and the evolution of biological macro-molecules. Die Naturwissenschaften, 58(10), 465–523.

Escazut, C., & Collard, P. (1997). Genetic algorithms at the edge of a dream. In J.-K. Hao, E. Lutton, E. Ronald, M. Schoenauer, & D. Snyers (Eds.), ArtificialEvolution – Third European Conference (pp. 69–80). Berlin: Springer.

Fadali, M. S., Zhang, Y., & Louis, S. J. (1999). Robust stability analysis of discrete-time systems using genetic algorithms. IEEE Trans. on Systems, Man, andCybernetics—Part A, 29(5), 503–508.

Fang, H.-L., Ross, P., & Corne, D. (1993). A promising genetic algorithm approachto job-shop scheduling, re-scheduling, and open-shop scheduling problems.In S. Forrest (Ed.), Proc. of the Fifth Int. Conf. on Genetic Algorithms (pp.375–382). San Mateo, CA: Morgan Kaufmann.

Feng, W., Brune, T., Chan, L., Chowdhury, M., Kuek, C. K., & Li, Y. (1997).Benchmarks for testing evolutionary algorithms (Tech. Rep. No. CSC-97006). Glasgow, UK: Center for System and Control, University of Glas-gow.

Fogarty, T. C., Vavak, F., & Cheng, P. (1995). Use of the genetic algorithm for loadbalancing of sugar beet presses. In L. J. Eshelman (Ed.), Proc. of the SixthInt. Conf. on Genetic Algorithms (pp. 617–624). San Francisco, CA: MorganKaufmann.

Fogel, D. B. (1992a). An analysis of evolutionary programming. In D. B. Fo-gel & W. Atmar (Eds.), Proc. of the first annual conference on evolutionaryprogramming (pp. 43–51). La Jolla, CA.

Fogel, D. B. (1992b). Evolving artificial intelligence. Unpublished doctoral dis-sertation, University of California, San Diego, CA.

Fogel, D. B. (1995). Evolutionary computation: Toward a new philosophy ofmachine intelligence. New York, NY: IEEE Press.

Fogel, L. J., Owens, A. J., & Walsh, M. J. (1965). Artificial intelligence through asimulation of evolution. In M. Maxfield, A. Callahan, & L. J. Fogel (Eds.),Biophysics and Cybernetic Systems: Proc. of the 2nd Cybernetic SciencesSymposium (pp. 131–155). Washington, D.C.: Spartan Books.

Fogel, L. J., Owens, A. J., & Walsh, M. J. (1966). Artificial intelligence throughsimulated evolution. New York, NY: Wiley & Sons.

Fraser, A. S. (1957). Simulation of genetic systems by automtic digital computersi. introduction. Australian Journal of Biological Sciences, 10, 484–491.

201

Page 202: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

Friedberg, R. M. (1958). A learning machine: Part I. IBM Journal of Researchand Development, 2(1), 2–13.

Friedberg, R. M., Dunham, B., & North, J. H. (1959). A learning machine: Part II.IBM Journal of Research and Development, 3(3), 282–287.

Friedman, G. J. (1956). Selective feedback computers for engineering synthesisand nervous system analogy. Unpublished master’s thesis, University of Cal-ifornia, Los Angeles, CA.

Futuyma, D. J., & Slatkin, M. (Eds.). (1983). Coevolution. Sunderland, MA:Sinauer Associates.

Gaspar, A., & Collard, P. (1997). Time dependent optimization with a foldinggenetic algorithm. In Int. Conf. on Tools for Artificial Intelligence 97 (pp.207–214). IEEE Computer Society Press.

Gaspar, A., & Collard, P. (1999a). From GAs to artificial immune systems: Im-proving adaptation in time dependent optimization. In 1999 Congress onEvolutionary Computation (pp. 1859–1866). Piscataway, NJ: IEEE ServiceCenter.

Gaspar, A., & Collard, P. (1999b). There is ALife beyond convergence: using adual sharing to adapt in time dependent optimization. In 1999 Congress onEvolutionary Computation (pp. 1867–1874). Piscataway, NJ: IEEE ServiceCenter.

Ghozeil, A., & Fogel, D. B. (1996). A preliminary investigation into directedmutations in evolutionary algorithms. In H. Voigt, W. Ebeling, & I. Rechen-berg (Eds.), Parallel Problem Solving from Nature – PPSN IV (Berlin, 1996)(Lecture Notes in Computer Science 1141) (pp. 329–335). Berlin: Springer.

Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machinelearning. Reading, MA: Addison-Wesley.

Goldberg, D. E., & Smith, R. E. (1987). Nonstationary function optimization usinggenetic algorithms with dominance and diploidy. In J. J. Grefenstette (Ed.),Proc. of the Second Int. Conf. on Genetic Algorithms (pp. 59–68). Hillsdale,NJ: Lawrence Erlbaum Associates.

Granlund, T. (1996). GNU MP: The GNU multiple precision arithmetic library.Boston, MA.

Grefenstette, J. (1986). Optimization of control parameters for genetic algorithms.IEEE Trans. on Systems, Man, and Cybernetics, SMC-16(1), 122–128.

Grefenstette, J. J. (1992). Genetic algorithms for changing environments. InR. Manner & B. Manderick (Eds.), Parallel Problem Solving from Nature2 (Proc. 2nd Int. Conf. on Parallel Problem Solving from Nature, Brussels1992) (pp. 137–144). Amsterdam: Elsevier.

Grefenstette, J. J. (1999). Evolvability in dynamic fitness landscapes: A geneticalgorithm approach. In 1999 Congress on Evolutionary Computation (pp.

202

Page 203: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

2031–2038). Piscataway, NJ: IEEE Press.Grefenstette, J. J., & Ramsey, C. L. (1992). An approach to anytime learning. In

Proc. of the ninth int. machine learning workshop (pp. 189–195). San Mateo,CA: Morgan Kaufmann.

Hadad, B. S., & Eick, C. F. (1997). Supporting polyploidy in genetic algorithmsusing dominance vectors. In P. J. Angeline, R. G. Reynolds, J. R. McDonnell,& R. Eberhart (Eds.), Evolutionary Programming VI (pp. 223–234). Berlin:Springer. (Lecture Notes in Computer Science 1213)

Hansen, N., & Ostermeier, A. (1996). Adapting arbitrary normal mutation distri-butions in evolution strategies: the covariance matrix adaptation. In Proc.of the 1996 IEEE Int. Conf. on Evolutionary Computation (pp. 312–317).Piscataway, NJ: IEEE Service Center.

Hansen, N., & Ostermeier, A. (2001). Completely derandomized self-adaptationin evolution strategies. Evolutionary Computation, 9(2), 159–195.

Hart, E., & Ross, P. (1998). A heuristic combination method for solving job-shop scheduling problems. In A. E. Eiben, T. Back, M. Schoenauer, & H.-P.Schwefel (Eds.), Parallel Problem Solving from Nature – PPSN V (pp. 845–854). Berlin: Springer. (Lecture Notes in Computer Science 1498)

Hartley, A. R. (1999). Accuracy-based fitness allows similar performance tohumans in static and dynamic classification environments. In W. Banzhaf,J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, & R. E. Smith(Eds.), Proc. of the Genetic and Evolutionary Computation Conf. GECCO-99 (pp. 266–273). San Francisco, CA: Morgan Kaufmann.

Hildebrand, L., Reusch, B., & Fathi, M. (1999). Directed mutation – A newselfadaptation for evolutionary strategies. In 1999 Congress on EvolutionaryComputation (pp. 1550–1557). Piscataway, NJ: IEEE Service Center.

Hirst, T. (1997). Evolutionary signal processing: A preliminary study. In P. Hus-bands & I. Harvey (Eds.), Fourth european conference on artificial life (p. ?).Cambridge, MA: MIT Press.

Holland, J. H. (1969). A new kind of turnpike theorem. Bulletin of the AmericanMathematical Society, 75(6), 1311–1317.

Holland, J. H. (1973). Genetic algorithms and the optimal allocation of trials.SIAM Journal on Computing, 2(2), 88–105.

Holland, J. H. (1975). Adaptation in natural and artifical systems. Ann Arbor, MI:University of Michigan Press.

Holland, J. H. (1992). Adaptation in natural and artifical systems. Cambridge,MA: MIT Press.

Jones, T. (1995). Evolutionary algorithms, fitness landscapes and search. Unpub-lished doctoral dissertation, The University of New Mexico, Albuquerque,NM.

203

Page 204: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

Karr, C. L. (1991). Design of an adaptive fuzzy logic controller using a genetic al-gorithm. In R. K. Belew & L. B. Booker (Eds.), Proc. of the Fourth Int. Conf.on Genetic Algorithms (pp. 450–457). San Mateo, CA: Morgan Kaufmann.

Karr, C. L. (1997). Fuzzy-evolutionary systems. In T. Back, D. B. Fogel, &Z. Michalewicz (Eds.), Handbook of Evolutionary Computation (pp. D2.1:1–2:9). Bristol, New York: Institute of Physics Publishing and Oxford Univer-sity Press.

Karr, C. L. (1999). An architecture for adaptive process control systems. pp. 146–148 in Evolutionary Algorithms for Dynamic Optimization Problems (eds.Jurgen Branke, Thomas Back), part of GECCO Workshops, A. Wu (ed.).

Kirley, M., & Green, D. G. (2000). An empirical investigation of optimisation indynamic environments using the cellular genetic algorithm. In D. Whitley,D. Goldberg, E. Cantu-Paz, L. Spector, I. Parmee, & H.-G. Beyer (Eds.),Proc. of the genetic and evolutionary computation conf. (gecco-2000) (pp.11–18). San Francisco, CA: Morgan Kaufmann.

Knuth, D. E. (1981). Seminumerical algorithms. Reading, MA: Addison-Wesley.(2nd ed., vol. 2 of The Art of Computer Programming)

Koza, J. R. (1989). Hierarchical genetic algorithms operating on populations ofcomputer programs. In N. S. Sridharan (Ed.), Proc. of the 11th Joint Conf. onGenetic Algorithms (pp. 786–774). San Francisco, CA: Morgan Kaufmann.

Koza, J. R. (1992a). The genetic programming paradigm: Genetically breedingpopulations of computer programs to solve problems. In B. Soucek (Ed.),Dynamic, genetic, and chaotic programming (pp. 203–321). New York: JohnWiley.

Koza, J. R. (1992b). Genetic programming: On the programming of computers bymeans of natural selection. Cambridge, MA: MIT Press.

Lewis, J., Hart, E., & Ritchie, G. (1998). A comparison of dominance mechanismsand simple mutation on non-stationary problems. In A. E. Eiben, T. Back,M. Schoenauer, & H.-P. Schwefel (Eds.), Parallel Problem Solving from Na-ture – PPSN V (pp. 139–148). Berlin: Springer.

Liles, W., & De Jong, K. A. (1999). The usefulness of tag bits in changing envi-ronments. In 1999 Congress on Evolutionary Computation (pp. 2054–2060).Piscataway, NJ: IEEE Press.

Lin, S.-C., Goodman, E. D., & Punch III, W. F. (1997). A genetic algorithmapproach to dynamic job shop scheduling problems. In Proc. of the SeventhInt. Conf. on Genetic Algorithms (pp. 481–488). Morgan Kaufmann.

Littman, M. L., & Ackley, D. H. (1991). Adaptation in constant utility non-stationary environments. In R. K. Belew & L. B. Booker (Eds.), Proc. ofthe Fourth Int. Conf. on Genetic Algorithms (pp. 136–142). San Mateo, CA:Morgan Kaufmann.

204

Page 205: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

Mattfeld, D. C., & Bierwirth, C. (1999). Adaptation and dynamic optimizationproblems: A view from general system theory. pp. 138–141, GECCO Work-shops, A. Wu (ed.).

Montana, D. J., & Davis, L. (1989). Training feedforward neural networks us-ing genetic algorithms. In N. S. Sridharan (Ed.), Proc. of the eleventh int.joint conf. on artificial intelligence IJCAI-89 (pp. 762–767). San Mateo,CA: Morgan Kaufmann.

Mori, N., Imanishi, S., Kita, H., & Nishikawa, Y. (1997). Adaptation to changingenvironments by means of the memory based thermodynamical genetic algo-rithm. In T. Back (Ed.), Proc. of the Seventh Int. Conf. on Genetic Algorithms(pp. 299–306). San Francisco, CA: Morgan Kaufmann.

Mori, N., Kita, H., & Nishikawa, Y. (1996). Adaptation to a changing environmentby means of the thermodynamical genetic algorithm. In H. Voigt, W. Ebeling,& I. Rechenberg (Eds.), Parallel Problem Solving from Nature – PPSN IV(pp. 513–522). Berlin: Springer.

Mori, N., Kita, H., & Nishikawa, Y. (1998). Adaptation to a changing environ-ment by means of the feedback thormodynamical geentic algorithm. In A. E.Eiben, T. Back, M. Schoenauer, & H.-P. Schwefel (Eds.), Parallel ProblemSolving from Nature – PPSN V (pp. 149–158). Berlin: Springer. (LectureNotes in Computer Science 1498)

Morrison, R. W., & De Jong, K. A. (1999). A test problem generator for non-stationary environments. In 1999 Congress on Evolutionary Computation(pp. 2047–2053). Piscataway, NJ: IEEE Service Center.

Morrison, R. W., & De Jong, K. A. (2000). Triggered hypermutation revisited. InProc. of the 2000 Congress on Evolutionary Computation (pp. 1025–1032).Piscataway, NJ: IEEE Service Center.

Muhlenbein, H., & Mahnig, T. (2001). Evolutionary algorithms: From recombina-tion to search distributions. In L. Kallel, B. Naudts, & A. Rogers (Eds.), The-oretical aspects of evolutionary computing (pp. 135–173). Berlin: Springer.

Munetomo, M., Takai, Y., & Sato, Y. (1996). Genetic-based dynamic load balanc-ing: Implementation and evaluation. In H.-M. Voigt, W. Ebeling, I. Rechen-berg, & H.-P. Schwefel (Eds.), Parallel problem solving from nature – PPSNIV (pp. 920–929). Berlin: Springer.

Narendra, K. S., & Thathachar, M. A. L. (1989). Learning automata: An introduc-tion. Englewood Cliffs, NJ: Prentice Hall.

Neubauer, A. (1996). A comparative study of evolutionary algorithms for on-line parameter tracking. In H.-M. Voigt, W. Ebeling, I. Rechenberg, & H.-P.Schwefel (Eds.), Parallel problem solving from nature – PPSN IV (pp. 624–633). Berlin: Springer.

Neubauer, A. (1997). Prediction of nonlinear and nonstationary time-series using

205

Page 206: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

self-adaptive evolution strategies with individual memory. In T. Back (Ed.),Proc. of the Seventh Int. Conf. on Genetic Algorithms (pp. 727–734). SanFrancisco, CA: Morgan Kaufmann.

Ng, K., & Wong, K. C. (1995). A new diploid scheme and dominance changemechanism for non-stationary function optimization. In L. Eshelman (Ed.),Proc. of the Sixth Int. Conf. on Genetic Algorithms (pp. 159–166). San Fran-cisco, CA: Morgan Kaufmann.

Odetayo, M. O., & McGregor, D. R. (1989). Genetic algorithm for inducingcontrol rules for a dynamic system. In J. D. Schaffer (Ed.), Proc. of the ThirdInt. Conf. on Genetic Algorithms (pp. 177–182). San Mateo, CA: MorganKaufmann.

Oussedik, S., Delahaye, D., & Schoenauer, M. (1999). Dynamic air traffic planningby genetic algorithms. In 1999 Congress on Evolutionary Computation (pp.1110–1117). Piscataway, NJ: IEEE Service Center.

Papadimitriou, C. H. (1977). Euclidean TSP is NP-complete. Theoretical Com-puter Science, 4, 237–244.

Papadimitriou, G. I., & Pomportsis, A. S. (2000). On the use of stochastic estimatorlearning automata for dynamic channel allocation in broadcast networks. InProc. of the 2000 Congress on Evolutionary Computation (pp. 112–116).Piscataway, NJ: IEEE Service Center.

Park, S. K., & Miller, K. W. (1988). Random number generators: Good ones arehard to find. Communications of the ACM, 31(10), 1192–1201.

Pelikan, M., Goldberg, D. E., & Cantu-Paz, E. (1999). BOA: The Bayesian op-timization algorithm. In W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon,V. Honavar, M. Jakiela, & R. E. Smith (Eds.), Proc. of the Genetic and Evo-lutionary Computation Conf. GECCO-99 (pp. 525–532). San Francisco, CA:Morgan Kaufmann.

Pettit, E., & Swigger, K. M. (1983). An analysis of genetic-based pattern trackingand cognitive-based component tracking models of adaptation. In Proc. ofthe National Conf. on Artificial Intelligence (AAAI-83) (pp. 327–332). ?

Pipe, A. G., Fogarty, T. C., & Winfield, A. (1994). Hybrid adaptive heuristiccritic architectures for learning in mazes with continuous search spaces. InY. Davidor, H.-P. Schwefel, & R. Manner (Eds.), Parallel Problem Solvingfrom Nature – PPSN III (pp. 482–491). Berlin: Springer. (Lecture Notes inComputer Science 866)

Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numeri-cal recipes in C: the art of scientific computing. Cambridge: Press Syndicateof the University of Cambridge. (2nd ed.)

Ramsey, C. L., & Grefenstette, J. J. (1993). Case-based initialization of geneticalgorithms. In S. Forrest (Ed.), Proc. of the Fifth Int. Conf. on Genetic Algo-

206

Page 207: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

rithms (pp. 84–91). San Mateo, CA: Morgan Kaufmann.Rana-Stevens, S., Lubin, B., & Montana, D. (2000). The air crew scheduling sys-

tem: The design of a real-world, dynamic genetic scheduler. In D. Whitley(Ed.), Late breaking papers at the 2000 genetic and evolutionary computa-tion conference (pp. 317–324). ?

Rechenberg, I. (1964). Kybernetische Losungsansteuerung einer experimentellenForschungsaufgabe [Cybernetic solution path of an experimental problem].presented at the Annual Conference of the WGLR at Berlin in September1964.

Rechenberg, I. (1973). Evolutionsstrategie: Optimierung technischer Systeme nachPrinzipien der biologischen Evolution. Stuttgart: frommann-holzbog.

Rechenberg, I. (1994). Evolutionsstrategie ’94. Stuttgart: frommann-holzbog.Reed, J., Toombs, R., & Barricelli, N. A. (1967). Simulation of biological evolution

and machine learning: I. selection of self-reproducing numeric patterns bydata processing machines, effects of hereditary control, mutation type andcrossing. Journal of Theoretical Biology, 17, 319–342.

Rixen, I., Bierwirth, C., & Kopfer, H. (1995). A case study of operational just-in-time scheduling using genetic algorithms. In J. Biethan & V. Nissen (Eds.),Evolutionary algorithms in management applications (pp. 113–123). Berlin:Springer.

Ronnewinkel, C., Wilke, C. O., & Martinetz, T. (2000). Genetic algorithms in time-dependent environments. In L. Kallel, B. Naudts, & A. Rogers (Eds.), Theo-retical aspects of evolutionary computing (pp. 263–288). Berlin: Springer.

Rowe, J. E. (1999). Finding attractors for periodic fitness functions. In W. Banzhaf,J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, & R. E. Smith(Eds.), Proc. of the Genetic and Evolutionary Computation Conf. GECCO-99 (pp. 557–563). San Francisco, CA: Morgan Kaufmann.

Rowe, J. E. (2001). Cyclic attractors and quasispecies adaptability. In L. Kallel,B. Naudts, & A. Rogers (Eds.), Theoretical aspects of evolutionary comput-ing (pp. 251–259). Berlin: Springer.

Rudolph, G. (1997). Convergence properties of evolutionary algorithms. Hamburg:Kovac.

Ryan, C. (1996). The degree of oneness. In 1st online workshop on soft computing(pp. 100–105). Nagoya, Japan.

Ryan, C. (1997). Diploidy without dominance. In J. T. Alander (Ed.), Proc. ofthe Third Nordic Workshop on Genetic Algorithms and their Applications(3NWGA) (pp. 63–70). Vaasa, Finnland.

Ryan, C., & Collins, J. J. (1998). Polygenic inheritance – a haploid scheme thatcan outperform diploidy. In A. E. Eiben, T. Back, M. Schoenauer, & H.-P. Schwefel (Eds.), Parallel Problem Solving from Nature – PPSN V (pp.

207

Page 208: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

178–187). Berlin: Springer. (Lecture Notes in Computer Science 1498)Saleem, S., & Reynolds, R. (2000). Cultural algorithms in dynamic environ-

ments. In Proc. of the 2000 Congress on Evolutionary Computation (pp.1513–1520). Piscataway, NJ: IEEE Service Center.

Salomon, R., & Eggenberger, P. (1997). Adaptation on the evolutionary timescale: A working hypothesis and basic experiments. In J.-K. Hao, E. Lutton,E. Ronald, M. Schoenauer, & D. Snyders (Eds.), Artificial Evolution: ThirdEuropean Conf., AE’97 (pp. 251–262). Berlin: Springer.

Sarma, J., & De Jong, K. (1999). The behavior of spatially distributed evolutionaryalgorithms in non-stationary environments. In W. Banzhaf, J. Daida, A. E.Eiben, M. H. Garzon, V. Honavar, M. Jakiela, & R. E. Smith (Eds.), Proc. ofthe Genetic and Evolutionary Computation Conf. GECCO-99 (pp. 572–578).San Francisco, CA: Morgan Kaufmann.

Schwefel, H.-P. (1975). Evolutionsstrategie und numerische Optimierung. Unpub-lished doctoral dissertation, Technische Universitat Berlin, Berlin.

Schwefel, H.-P. (1977). Numerische Optimierung von Computer-Modellen mittelsder Evolutionsstrategie [Numeric optimization of computer models using theevolution strategy]. Basel, Stuttgart: Birkhauser.

Schwefel, H.-P. (1981). Numerical optimization of computer models. Chichester:John Wiley & Sons, Ltd.

Schwefel, H.-P. (1995). Evolution and optimum seeking. New York, NY: Wiley &Sons.

Sebag, M., Schoenauer, M., & Ravise, C. (1997a). Inductive learning of mu-tation step-size in evolutionary parameter optimization. In P. J. Angeline,R. G. Reynolds, J. R. McDonnell, & R. Eberhart (Eds.), Evolutionary Pro-gramming VI (pp. 247–261). Berlin: Springer. (Lecture Notes in ComputerScience 1213)

Sebag, M., Schoenauer, M., & Ravise, C. (1997b). Toward civilized evolution:Developing inhibitions. In T. Back (Ed.), Proc. of the Seventh Int. Conf. onGenetic Algorithms (pp. 291–298). San Francisco, CA: Morgan Kaufmann.

Slatkin, M. (1983). Models of coevolution: Their use and abuse. In M. H. Nitecki(Ed.), Proc. of the Fifth Annual Spring Systematics Symposium: Coevolution(pp. 339–370). Chicago, IL: The University of Chicago Press.

Smith, J. E., & Vavak, F. (1999). Replacement strategies in steady state geneticalgorithms: dynamic environments. Journal of Computing and InformationTechnology, 7(1), 49–60.

Smith, R. E., & Goldberg, D. E. (1992). Diploidy and dominance in artificialgenetic search. Complex Systems, 6, 251–285.

Spalanzani, A., & Kabre, H. (1998). Evolution, learning and speech recognition inchanging acoustic environments. In A. E. Eiben, T. Back, M. Schoenauer, &

208

Page 209: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

H.-P. Schwefel (Eds.), Parallel Problem Solving from Nature – PPSN V (pp.663–670). Berlin: Springer. (Lecture Notes in Computer Science 1498)

Stanhope, S. A., & Daida, J. M. (1998). Optimal mutation and crossover ratesfor a genetic algorithm operating in a dynamic environment. In V. W. Porto,N. Saravanan, D. Waagen, & A. E. Eiben (Eds.), Evolutionary ProgrammingVII (pp. 693–702). Berlin: Springer. (Lecture Notes in Computer Science1447)

Stanhope, S. A., & Daida, J. M. (1999). (1 + 1) genetic algorithm fitness dynamicsin a changing environment. In 1999 Congress on Evolutionary Computation(pp. 1851–1858). Piscataway, NJ: IEEE Service Center.

Stroud, P. D. (2001). Kalman-extended genetic algorithm for search in nonsta-tionary environments with noisy fitness evaluations. IEEE Transactions onEvolutionary Computation, 5(1), 66–77.

Thierens, D., & Vercauteren, L. (1991). A topology exploiting genetic algorithmto control dynamic systems. In H.-P. Schwefel & R. Manner (Eds.), Parallelproblem solving from nature: 1st Workshop, PPSN I (pp. 104–108). Berlin:Springer.

Trojanowski, K., & Michalewicz, Z. (1999a). Evolutionary algorithms for non-stationary environments. In Proc. of 8th workshop: Intelligent informationsystems (pp. 229–240). ?: ICS PAS Press.

Trojanowski, K., & Michalewicz, Z. (1999b). Searching for optima in non-stationary environments. In 1999 Congress on Evolutionary Computation(pp. 1843–1850). Piscataway, NJ: IEEE Service Center.

Ursem, R. K. (2000). Multinational GAs: Multimodal optimization techniques indynamic environments. In D. Whitley, D. Goldberg, E. Cantu-Paz, L. Spec-tor, I. Parmee, & H.-G. Beyer (Eds.), Proc. of the Genetic and EvolutionaryComputation Conf. (GECCO-00) (pp. 19–26). San Francisco, CA: MorganKaufmann.

Vavak, F., & Fogarty, T. C. (1996). A comparative study of steady state andgenerational genetic algorithms for use in nonstationary environments. InProc. of the Society for the Study of Artificial Intelligence and Simulation ofBehaviour Workshop on Evolutionry Computing 96 (pp. 301–307). ?

Vavak, F., Fogarty, T. C., & Jukes, K. (1996a). A genetic algorithm with variablerange of local search for adaptive control of the dynamic systems. In Proc.of the 2nd Int. Mendelian Conf. on Genetic Algorithms (pp. 181–186). Brno:PC-DIR Publishing.

Vavak, F., Fogarty, T. C., & Jukes, K. (1996b). A genetic algorithm with variablerange of local search for tracking changing environments. In H.-M. Voigt,W. Ebeling, I. Rechenberg, & H.-P. Schwefel (Eds.), Parallel problem solv-ing from nature – PPSN IV (pp. 376–385). Berlin: Springer.

209

Page 210: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

9. CONCLUSIONS AND FUTURE WORK

Vavak, F., Jukes, K., & Fogarty, T. C. (1997). Adaptive combustion balancingin multiple burner boiler using a genetic algorithm with variable range oflocal search. In T. Back (Ed.), Proc. of the Seventh Int. Conf. on GeneticAlgorithms (pp. 719–726). San Mateo, CA: Morgan Kaufmann.

Vavak, F., Jukes, K. A., & Fogarty, T. C. (1998). Performance of a genetic al-gorithm with variable local search range relative to frequency of the en-vironmental changes. In J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb,M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, & R. Riolo(Eds.), Proc. of the Third Int. Conf. on Genetic Programming (pp. 602–608).San Mateo, CA: Morgan Kaufmann.

Weicker, K. (2000). An analysis of dynamic severity and population size. InM. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, &H.-P. Schwefel (Eds.), Parallel problem solving from nature – PPSN VI (pp.159–168). Berlin: Springer.

Weicker, K. (2001). Problem difficulty in real-valued dynamic problems. InB. Reusch (Ed.), Computational Intelligence: Theory and Applications (pp.313–325). Berlin: Springer.

Weicker, K. (2002). Performance measures for dynamic environments. In J. J.Merelo Guervos, P. Adamidis, H.-G. Beyer, J.-L. Fernandez-Villacanas, &H.-P. Schwefel (Eds.), Parallel problem solving from nature – PPSN VII (pp.64–73). Berlin: Springer.

Weicker, K., & Weicker, N. (1999). On evolution strategy optimization in dynamicenvironments. In 1999 Congress on Evolutionary Computation (pp. 2039–2046). Piscataway, NJ: IEEE Service Center.

Weicker, K., & Weicker, N. (2000). Dynamic rotation and partial visibility. InProc. of the 2000 Congress on Evolutionary Computation (pp. 1125–1131).Piscataway, NJ: IEEE Service Center.

Wilke, C. O. (1998). Evolution in time-dependent fitness landscapes (Tech. Rep.Nos. 98–09). Bochum, Germany: Ruhr-Universitat Bochum, Institut furNeuroinformatik.

Wilke, C. O. (1999). Evolutionary dynamics in time-dependent environments.Aachen, Germany: Shaker Verlag.

Wilke, C. O., & Ronnewinkel, C. (2001). Dynamic fitness landscapes: Expansionsfor small mutation rates. Physica A, 290(3-4), 475–490.

Wilke, C. O., Ronnewinkel, C., & Martinetz, T. (1999). Molecular evolution intime-dependent environments. In D. Floreano, J.-D. Nicoud, & F. Mondada(Eds.), Advances in artificial life, proc. of the tth european conf. on artificiallife (p. ??). Berlin: Springer.

Wilke, C. O., Ronnewinkel, C., & Martinetz, T. (2001). Dynamic fitness landscapesin molecular evolution. (accepted by Physics Reports)

210

Page 211: Evolutionary Algorithms and Dynamic Optimization Problemsweicker/publications/disskarsten.pdf · Summary This thesis examines evolutionary algorithms, a universal optimization method,

REFERENCES

Wolpert, D. H., & Macready, W. G. (1995). No free lunch theorems for search(Tech. Rep. No. SFI-TR-95-02-010). Santa Fe, NM: Santa Fe Institute.

Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimiza-tion. IEEE Trans. on Evolutionary Computation, 1(1), 67–82.

Yao, X., & Liu, Y. (1996). Fast evolutionary programming. In L. J. Fogel, P. J. An-geline, & T. Back (Eds.), Proc. 5th Ann. Conf. on Evolutionary Programming(p. ?). Cambridge, MA: MIT Press.

211