Technische Universität...

TECHNISCHE UNIVERSITÄT ILMENAU

Fakultät für Elektrotechnik und Informationstechnik

In Zusamennarbeit mit

Kazan National Research Technical University named afterA.N.Tupolev

Evaluation of Multi-Criteria Handover Algorithms inMulti-Terminal and Multi-Application Scenarios

Masterarbeitzur Erlangung des akademischen Grades Master of Science

vorgelegt von: Radion Bikmukhamedov

geboren:

Studiengang: Communications and Signal Processing

Studienrichtung: keine

Anfertigung im Fachgebiet: Kommunikationsnetze

Fakultät für Elektrotechnik und Informationstechnik

Verantwortlicher Professor: Prof. Dr. rer. nat. Jochen Seitz

Wissenschaftlicher Betreuer: Dipl.-Ing. Yevgeniy Yeryomin

Datum: August 5, 2016

To the memory of my father.

i

ACKNOWLEDGMENTS

I would like to express my gratitude to my supervisor Dipl.-Ing. Yevgeniy Yeryomin,for his invaluable support during writing of this thesis. Also, I owe many thanks toProf. Nadeev, who was giving me helpful advises during my master studies.

I can not forget those people, who have been supporting me outside of the universitywalls – my family. Without them the whole tuition would be more complicated andless motivated.

Thanks to all of you.

Master thesis Radion Bikmukhamedov

ii

ABSTRACT

Increasing complexity of the modern mobile networks forces using of more sophisticatedmechanisms for handover procedure. One of the most widely-recognized solutions fornetwork selection problem is usage of Multi-Criteria Decision Making (MCDM), alsoknown as Multi-Criteria Decision Analysis (MCDA). However, there is plenty of openedissues, such as construction of optimal MCDM-algorithm in terms of user’s experienceand evaluation of results produced by the algorithms.

Basing on results of literature review and own investigations, an optimal schemeof the MCDM-algorithm was proposed and its optimality was further confirmed bymeans of computer simulations. In this scheme, Enhanced Max-Min method servedon normalization stage, Weighted Least Squares was used for weighting of criteria andGRA was chosen as a ranking algorithm.

Known evaluation approaches was analyzed and verified, own modular method forevaluation of MCDM-algorithms was proposed. This approach is designated to solveproblems of incomparability of evaluation results produced by researchers in this areadue to use of di�erent evaluation criteria. The approach allows classification of the cri-teria by field of application on sets (e.g. Handover, Quality of Service, Device). Than,depending on needs, algorithm’s evaluation scores from a single set may be used, whatcreates opportunity to use evaluation results of those sets, which a researcher is in-terested in. Moreover, examples and suggestions regarding method’s parametrizationprocedure were provided and applied in practical part of the work with help of OM-NeT++ software. Superiority of the proposed evaluation method was shown throughcase studies.


yevgeniy

was -> were

yevgeniy

problem of incomparability of evaluation results

Contents iii

Contents

Acknowledgments i

Abstract ii

Contents iii

List of Figures vi

List of Tables vii

1 Introduction 11.1 Overview of the structure . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Multi-Criteria Handover Algorithms 32.1 Criteria selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Weighting algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3.1 Subjective techniques . . . . . . . . . . . . . . . . . . . . . . . . 82.3.1.1 Eigenvector . . . . . . . . . . . . . . . . . . . . . . . . 92.3.1.2 Weighted Least Squares . . . . . . . . . . . . . . . . . 92.3.1.3 TRUST . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.1.4 Comparison of Eigenvector and WLS . . . . . . . . . . 9

2.3.2 Objective techniques . . . . . . . . . . . . . . . . . . . . . . . . 102.3.3 Hybrid technique . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4 MCDA algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4.1 SAW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4.2 GRA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4.3 VIKOR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.4.4 TOPSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.5 Comparison of the algorithms . . . . . . . . . . . . . . . . . . . . . . . 152.6 Construction of MCHA . . . . . . . . . . . . . . . . . . . . . . . . . . . 16


Contents iv

3 Evaluation approaches for MCHAs 183.1 Existing approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.1.1 Standard deviation from Optimum interface approach . . . . . . 183.1.2 Criticality analysis approach . . . . . . . . . . . . . . . . . . . . 203.1.3 Subjective Criticality Analysis approach . . . . . . . . . . . . . 23

3.2 New evaluation approach . . . . . . . . . . . . . . . . . . . . . . . . . . 233.2.1 Parametrization of the assessing sets . . . . . . . . . . . . . . . 25

3.2.1.1 Parametrization of the Handover set . . . . . . . . . . 263.2.1.2 Parametrization of the Network-related set . . . . . . . 26

3.2.2 Quality of Experience . . . . . . . . . . . . . . . . . . . . . . . . 273.2.2.1 Subjective QoE-methods . . . . . . . . . . . . . . . . . 273.2.2.2 Objective QoE-methods . . . . . . . . . . . . . . . . . 293.2.2.3 QoS-to-QoE mapping methods . . . . . . . . . . . . . 323.2.2.4 The option for implementation in a real system . . . . 33

3.2.3 Aggregation of the sets . . . . . . . . . . . . . . . . . . . . . . . 333.2.3.1 Description of the procedure . . . . . . . . . . . . . . . 333.2.3.2 Discussion of the aggregation strategies . . . . . . . . . 343.2.3.3 Discussion of approaches for construction of an evalu-

ation scheme . . . . . . . . . . . . . . . . . . . . . . . 363.2.3.4 Discussion of some real-life scenarios . . . . . . . . . . 37

3.2.4 Case study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.2.4.1 Decision stage . . . . . . . . . . . . . . . . . . . . . . . 383.2.4.2 Evaluation stage . . . . . . . . . . . . . . . . . . . . . 39

4 Simulation 424.1 Simulation setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.1.1 Simulation scenarios . . . . . . . . . . . . . . . . . . . . . . . . 434.1.2 The network topology . . . . . . . . . . . . . . . . . . . . . . . 454.1.3 Tra�c generation . . . . . . . . . . . . . . . . . . . . . . . . . . 454.1.4 Collection of decision criteria . . . . . . . . . . . . . . . . . . . 474.1.5 Implementation of the MCDM-algorithms . . . . . . . . . . . . 484.1.6 Configuration of the MCHAs . . . . . . . . . . . . . . . . . . . 48

4.1.6.1 The normalization stage . . . . . . . . . . . . . . . . . 484.1.6.2 The weighting stage . . . . . . . . . . . . . . . . . . . 494.1.6.3 The ranking stage . . . . . . . . . . . . . . . . . . . . 50

4.1.7 Processing of results . . . . . . . . . . . . . . . . . . . . . . . . 50


Contents v

4.2 Results of the simulations . . . . . . . . . . . . . . . . . . . . . . . . . 514.2.1 Collected simulation values . . . . . . . . . . . . . . . . . . . . . 514.2.2 Application of the proposed evaluation algorithm . . . . . . . . 55

4.2.2.1 QoS evaluation focus . . . . . . . . . . . . . . . . . . . 564.2.2.2 Cost evaluation focus . . . . . . . . . . . . . . . . . . . 56

5 Conclusions 59

A Simulation setup in OMNeT++ 62A.1 The network topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . 62A.2 Configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

B Implementation of the MCHAs 78

C Processing of simulation results 102

D Simulation results 110

Acronyms 114

Bibliography 116

Erklärung 120


List of Figures vi

List of Figures

2.1 Numerical comparison of Eigenvector and WLS [1] . . . . . . . . . . . . 11

4.1 A screenshot of the OMNeT++ graphical interface . . . . . . . . . . . 434.2 A screenshot of an AP’s structure from the INET package . . . . . . . 444.3 A screenshot of a STA’s structure from the INET package . . . . . . . 454.4 Visual description of integration of the MCDM-algorithms . . . . . . . 484.5 Visual description of processing of the simulation results . . . . . . . . 514.6 Collected normalized values in the low load scenario . . . . . . . . . . . 534.7 Collected normalized values in the high load scenario . . . . . . . . . . 544.8 The aggregated output of the proposed evaluation algorithm with QoS

focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574.9 The aggregated output of the proposed evaluation algorithm with Cost

focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

A.1 The network topology for the low load scenario . . . . . . . . . . . . . 62A.2 The network topology for the high load scenario . . . . . . . . . . . . . 63


List of Tables vii

List of Tables

2.1 Examples of criteria for MCHA . . . . . . . . . . . . . . . . . . . . . . 42.2 Normalization algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Examples of feasible ranges for the decision criteria . . . . . . . . . . . 72.4 Weighting algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.5 Algorithmic complexity of the MCDA algorithms . . . . . . . . . . . . 152.6 Comparison of the MCDA algorithms . . . . . . . . . . . . . . . . . . . 162.7 The constructed MCHA algorithm . . . . . . . . . . . . . . . . . . . . 17

3.1 Valuation levels of evaluation parameters . . . . . . . . . . . . . . . . . 213.2 Assessing sets for evaluation of MCHA . . . . . . . . . . . . . . . . . . 243.3 Examples of configuration of evaluation focuses for the Handover and

Network sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.4 Suggested objective QoE-methods for assessment of voice, video and

web-browsing applications . . . . . . . . . . . . . . . . . . . . . . . . . 303.5 Examples of the aggregation strategies of the Device, Network and Ap-

plication sets from the user’s perspective . . . . . . . . . . . . . . . . . 353.6 Methods for construction of an evaluation scheme . . . . . . . . . . . . 373.7 Example of decision values for demonstration of MCHAs . . . . . . . . 393.8 Example of the best-path values for evaluation of MCHAs . . . . . . . 40

4.1 The common simulation properties for all scenarios in the OMNeT++setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.2 The simulation scenarios in OMNeT++ . . . . . . . . . . . . . . . . . 474.3 The simulation sub-scenarios in OMNeT++ . . . . . . . . . . . . . . . 474.4 The normalization parameters for the Enhanced Max-Min method . . . 494.5 Classification of the simulated evaluation parameters . . . . . . . . . . 55


1 Introduction 1

1 Introduction

Proper selection of serving node is especially important for modern wireless networkswith heterogeneous structure. Therefore, consideration of multiple criteria for han-dover decision has become a desired option, and this requires a sophisticated solu-tion. The one may be using of MCDA algorithms or, more specifically, Multi-CriteriaHandover Algorithms (MCHAs). Also, the progress puts additional requirements onoperators, due to high user’s demands to service quality, while keeping low service’scost. Due to this reason, the main concern of this work is dedicated to construction ofdecision and evaluation strategies from user’s perspective.

HandOver (HO) process involves 3 steps [2]: Collection of information, HO Decision,HO Execution. This work is focused on the Decision step. Moreover, it is assumed,that available alternatives for handover at the current moment of decision process arefeasible (e.g. a signal is strong enough for reliable communication).

The researches in this field usually focus on selection of certain MCDA algorithmor normalization technique, rather than on the entire picture, where MCDA methodsshould be selected along with normalization and weighting algorithms. Despite thiswas reviewed in my previous work [1], here it is discussed in a more detailed way, whereissues of the algorithms’ coexistence are also addressed.

Another axis of this work – development of an evaluation approach, which allowse�ective evaluation of consequences of the decisions produced by MCHAs. The rea-sons for construction of own evaluation approach are: inflexibility of existing ones,furthermore, the results produced by those are not really applicable and comparabledue to usage of specific unstructured criteria-sets. Therefore, these main issues shouldbe addressed during development of own approach. More precisely, comprehensiveclassification of the sets should be presented with regard to source of criteria and itsapplication field. Among possible criteria, Quality of Experience (QoE) and practicalapproaches for its retrieving are to be considered especially in details. Moreover, somerealistic evaluation scenarios will be discussed, as well as influence of subjective user’sfactors on them.

During practical part, MCHAs are to be implemented in Objective Modular Network


1 Introduction 2

Testbed in C++ (OMNeT++), where realistic tra�c and network scenarios should beconsidered as well, with further assessing of the results by means of the proposedevaluation scheme in Matlab.

1.1 Overview of the structureChapter 2 introduces most popular MCDA algorithms in handover context. Key com-ponents of the algorithms are analyzed and considered for including in decision-schemefor di�erent scenarios. Some part of work introduced in this chapter has already beenpublished in [1].

Chapter 3 describes existing techniques for evaluation of MCHA from user perspec-tive. The new evaluation approach is developed, where classification of evaluationcriteria is considered.

Chapter 4 deals with practical implementation of di�erent decision schemes in sim-ulation environment OMNeT++ with further evaluation by means of the proposedmethod.

Then, conclusions regarding obtained results are presented, some future researchdirection are also briefly discussed.


yevgeniy

HAVE aready been

2 Multi-Criteria Handover Algorithms 3

2 Multi-Criteria Handover Algorithms

The major application of MCDA in this thesis is multi-criteria optimization task. Thisis a process of determining of best optimal solution among available alternatives bymeans of established criteria. This problem can be formalized as follows:

mcoi

{(fij(Di), i = 1, . . . , M), j = 1, . . . , N}, (2.1)

where mco operator denotes decision making procedure,fij – the value of alternative Di for j-th criterion,M – number of feasible alternatives,N – number of criteria.Usually, solution of any optimization task with MCDA in the handover context

includes the following stages:

1. Criteria selection

2. Normalization of values

3. Weight assignment for criteria

4. Ranking of alternatives with help of MCDA algorithm

2.1 Criteria selectionThere are a lot of possible criteria that may be used during handover decision process[2],[3]. Some of them are presented in Table 2.1. For the sake of completeness, criteriawhich may be used for evaluation purposes are also presented (further details aboutevaluation approaches are presented in the next chapter). Type field in the tabledemonstrates higher-the-better or lower-the-better property for upward (noted as +)and downward (�) criteria correspondingly. Several classifications for criteria arepossible. I’ve classified them by the source of originating. Note, that some criteria arenot available during Decision process. Even though all criteria from the decision set


yevgeniy

Note that


can be used during evaluation procedure, it doesn’t always make sense to use someof them for that and vice-versa. For instance, there is no need to consider ReceivedSignal Strength Indicator (RSSI) during evaluation process, since it is hard to judgeregarding the connection’s quality basing on this parameter in case of heterogeneousnetworks.

Table 2.1: Examples of criteria for MCHACriterion Type Decision Evaluation SourceHO Delay � 7 3 NetworkHO failure probability � 3 3 NetworkPing-Pong rate � 7 3 NetworkSecurity + 3 3 NetworkCost � 3 3 NetworkNetwork load � 3 3 NetworkReceived SignalStrength Indicator + 3 7 Device

Signal toInterference-and-NoiseRatio

+ 3 7 Device

Throughput + 7 3 DeviceDelay � 3 3 DeviceJitter � 3 3 DevicePacket loss � 3 3 DeviceBattery level + 3 3 DeviceSpeed of movement � 3 7 DeviceQuality of Experience + 7 3 DeviceComplexity � 7 3 AlgorithmScalability + 7 3 Algorithm

During decision process, the set of chosen criteria should be the same for all decisionscenarios. Since in a real system the process involves criteria-collection proceduresin every wireless node, it should be unified for each and with no regard to scenario,otherwise it may lead to case, when, for example, collected criteria are unsuitable forgiven decision scenario. Since we collect criteria for di�erent scenarios simultaneously,decision-profile should be switched fast, otherwise, we have to collect other criteriauntil we will be able to make decision. Possible disadvantage of such unified approachis increased amount of signaling tra�c. For example, streaming-tra�c scenario benefits


yevgeniy

„For instance, there is no need to consider Received Signal Strength Indicator (RSSI) during evaluation process, since it is hard to judge regarding the connection’s quality basing on this parameter in case of heterogeneous networks.“it can be done when using normalization

yevgeniy

why does the security belong to „Network“?


from usage of Network Load (NL) and Packet Jitter (J) attributes for handover decisionin a higher degree than from Packet Delay (D), as it is in the case of conversationalscenario.

We should take into account the fact, that the process of collection of parametersrelated to link-quality is quite complicated in a real system, since both terminal andAccess Point (AP) are not aware of them, until transmission session is established [4].Examples of such criteria are D, J, Packet Loss (PL), etc. Nevertheless, I assume thatthese criteria are available during the decision process.

It is important to consider the criteria which have the highest influence on userexperience, in order to decrease signaling and avoid spreading of attention on unim-portant ones. Basing on these concerns, the following criteria are considered as themost valuable:

• RSSI. It is one of the most important criterion for network selection, if theone falls below threshold, communications won’t be possible. Alternatively, thecriterion may be replaced by Signal to Interference-and-Noise Ratio (SINR), ifavailable, since it provides better representation of a radio channel.

• D has significant influence on QoS in voice and interactive applications.

• J is important for real-time applications and a�ects on smoothness of user expe-rience.

• PL is an important one for its influence on user’s experience.

• NL may be useful for load-balancing strategies, or to ensure that there are enoughresources to perform HO and provide required performance for the user.

• Cost (C). A�ects on user’s service policy, e.g. if the one is concerned on savinghis money, this criterion should be prioritized.

• Security (S) level of the AP may be also an important factor for some users.

2.2 NormalizationThe aim of this stage is to eliminate dimensionality of collected information and inorder to calculate correct distances to ideal and/or anti-ideal solutions for distance-based algorithms (e.g. TOPSIS, VIKOR in section 2.4). Applicability of normalizationstage for di�erent ranking algorithms is briefly discussed in section 2.6.



Table 2.2: Normalization algorithms

Name Equations for upward/downwardcriterion Applied in

Normalization withp-norm [5] dij = cij

(qMi=1 |cij|p)1/p

, doesn’t exist (2.2) TOPSIS (p = 2)

Max method [6] dij = cij

maxi{cij}, dij = mini{cij}

cij(2.3) VIKOR, SAW

Max-Min method[7]

dij = maxi{cij} ≠ cij

maxi{cij} ≠ mini{cij}, (2.4)

dij = cij ≠ mini{cij}maxi{cij} ≠ mini{cij}

GRA

Enhanced Max-Minmethod [7] dij = uj ≠ cij

uj ≠ lj, dij = cij ≠ lj

uj ≠ lj(2.5) Modified GRA

Since our concern is processing of only feasible alternatives, the feasible ranges foreach decision criterion should be discussed. The ranges may be di�erent and must bedetermined with regard to current application type (e.g. Voice over Internet Protocol(VoIP) and Internet Protocol TeleVision (IPTV)) and access technology (i.e. 802.11g).The examples of the ones can be found in Table 2.3, where values for conversationaland streaming applications are presented in accordance with recommendation ITU-TG.114, [8] and [9]. Although the range for D is provided, the one for J depends ondevice properties (e.g. bu�er size), therefore, it should be small enough, such thatbu�er could handle it, while being much less than feasible range of the D parameter.

It is supposed, that if there are no alternatives available after checking on feasibility,the procedure should be relaxed to checking only of RSSI (or SINR) parameter, inorder to provide some service.

After collection of decision criteria of feasible alternatives, the data is placed in


yevgeniy

definition of feasible alternatives is missed


Table 2.3: Examples of feasible ranges for the decision criteria

Criterion Streaming tra�c Conversationaltra�c

RSSI, dBm > ≠85 > ≠85D, ms < 2000 < 250PL, % < 0.1 < 1

NL possibly to provide120 Kbps

possibly to provide6.4 kbps

a matrix C œ RM◊N , where M represents a number of feasible alternatives, N – anumber of criteria. In this work, matrix D serve as normalized version of C. Also,normalization stage may be a reason for rank-reversal [7].

Rank reversal (or ranking abnormality) is an e�ect of rank-changing after removal oradding of an alternative [10]. Algorithms behaves di�erently in terms of rank reversal.This phenomena may cause ping-pong e�ect in the handover context.

Methods described in Table 2.2 (taken from [1]) can be applied for various MCDAalgorithms. Notation "doesn’t exist" in 2.2 means, that there is no equation for down-ward criterion. Usually, algorithms, in which this normalization technique is used,consider upward/downward properties of criterion inside itself.

The disadvantage of mentioned methods is that after removal of an alternative,especially if it is maximal or minimal value among all columns, the scale of the otherswill be changed. To solve the problem, improvement for 2.4 was proposed in [7], wherevariables maxi{cij} , mini{cij} are replaced with constant predefined values uj andlj. The constants are maximal and minimal values for j-th criterion correspondingly.These constants will be defined on the simulation stage (see section 4.1.6.1), becauseinappropriate selection of boundaries my lead to incorrect results (e.g. normalizedvalues may be > 1 or << 1). For GRA algorithm rank reversal e�ect was significantlyreduced [7] with this enhancement.

Another option for normalization stage is use of the utility functions. The onescan be considered as non-linear transformation of input values onto normalized scale– degree of utility. This parameter reflects user’s satisfaction by particular service.Therefore, the utility function must be configured for each criterion and for each servicetype. Sigmoid-functions are considered as the most suitable in the handover context[11]. Next, the functions with appropriate weights are to be a aggregated into totalutility for decision-making process, what is generalized under term Multi-AttributeUtility Theory (MAUT). However, this approach require proper parametrization of


yevgeniy

c, u_j, l_j are not introduced properly


the functions, otherwise it may lead to non-optimal decisions.Basing on arguments provided above, Enhanced Max-Min 2.2 is recommended as

normalization technique in this work.

2.3 Weighting algorithmsThe next stage is appliance of weighting algorithm, where configuration of weightsis the most important step, although it a�ects on decision-making process implicitly.Therefore, we can use di�erent decision strategies according to the user’s requirementsand/or his current tra�c pattern. Examples are QoS-oriented, Cost-saving, Battery-saving, Load-balancing strategies. For instance, in the case of QoS-oriented one, theQoS-related criteria are accented, while considering current tra�c-scenario. Cost-oriented strategy allows user saving of money, emphasizing C parameter. Other onesmay be explained in a similar way. Examples of such parametrization are provided insections 3.2.4, 4.1.6.2.

This implies the fact, that di�erent weight-sets are to be created. However, this isdoable only with subjective methods, since they are able to consider user preferences.In contrast, objective approach can’t do that, but its weights are changing dynami-cally, according to variance or entropy of criteria in matrix D. The methods, theirclassification and equations are provided in Table 2.4.

2.3.1 Subjective techniquesIn order to express user preferences, matrix of pair-wise comparisons A must be con-structed [12]:

A =

S

WWWWWWWWU

a11 a12 . . . a1N

a21 a22 . . . a2N

... ... . . . ...

aN1 aN2 . . . aNN

T

XXXXXXXXV

, where

Y_]

_[

aii = 1,

aji = 1a

ij

.(2.6)

Value to aij is assigned as an integer number in range {1, 9}, so the higher value is,the higher relative importance of i-th criterion over j-th.

After construction, matrix A must be processed by either Eigenvector or WeightedLeast Squares method.



2.3.1.1 Eigenvector

The most widely used algorithm for weighting stage is Eigenvector method. It is usedas a basis for Analytical Hierarchy Process (AHP) and Analytic Networking Process(ANP) algorithms [13].

In Eigenvector method, weights w are obtained through solving of system of linearequations 2.8, where ⁄max – maximal Eigenvalue of A; IN – identity matrix N ◊ N ;w – seeking weight-vector.

As we can see, solution of the system is the dominant eigenvector.

2.3.1.2 Weighted Least Squares

Another weighting algorithm is Weighted Least Squares (WLS), which has been com-prehensively discussed in the handover context in [1]. Objective function is defined in2.9, where aij is ij-th element of matrix A; wi – i-th element of sought-for vector w.

Method’s objective was expressed in matrix form and its closed-from solution (ob-tained by method of Lagrangian multipliers) was provided in [1]:

w = (diag(ATA) ≠ A ≠ AT + NIN )≠1e

eT (diag(ATA) ≠ A ≠ AT + NIN )≠1e(2.7)

where e =51 . . . 1

6T

œ RN◊1, the operator diag{·} nulls out all elements of amatrix except ones on the main diagonal.

2.3.1.3 TRUST

There is one more subjective weighting approach – TRUST [14], but on contrary toabove-mention methods, this one has dynamic nature. Depending on scenario, weightsare generated automatically from predefined pool of parameters. However, such ap-proach complicates initial configuration. Moreover, this method produce weight-vectors,which are highly correlated with the ones obtained by Eigenvector method. Therefore,the algorithm will not be further considered in this work.

2.3.1.4 Comparison of Eigenvector and WLS

Detailed numerical comparison of these 2 algorithms was conducted in [1], where bothperformance and resulting weight-vectors were analyzed.

• Performance evaluation: it was shown, that computational latency for WLS islower when number of criteria goes roughly over 20 (see Figure 2.1a).



Table 2.4: Weighting algorithmsName Expression

Eigenvector(AHP, ANP) (A ≠ ⁄maxIN) · w = 0 (2.8)

Weighted LeastSquares min

w

Nÿ

i=1

Nÿ

j=1(aij · wj ≠ wi)2 s.t.

Nÿ

i=1wi = 1, (2.9)

Variance-based Vj =ııÙ

qMi=1(dij ≠ dj)2

Mdj

=∆ wobj(j) = VjqN

j=1 Vj

(2.10)

Entropy-based Ej = ≠qM

i=1 dij ln(dij)ln M

=∆ wobj(j) = 1 ≠ EjqN

j=1(1 ≠ Ej)(2.11)

Hybrid w = k · wsubj

+ (1 ≠ k) · wobj

(2.12)

• Weight-vectors: the di�erence between obtained vectors is presented on Figure2.1b. It vanishes with increasing number of criteria, although noticeable whenthe number is low.

According to demonstrated results, WLS has been chosen as a reference weightingalgorithm in this work.

2.3.2 Objective techniquesSince objective approach is not the focus of this work, it will be mentioned briefly.However, this approach may be useful in case of lacking of a-priori knowledge for theuser or if there are no special requirements. For example, usage of such technique



Number of criteria0 20 40 60 80 100

T WLS

/ T Ei

g

0

0.5

1

1.5

2

2.5

3

(a) The ratio of computational latencies ofEigenvector and WLS

Number of criteria0 20 40 60 80 100

norm

(wEi

g - w

WLS

)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

(b) The distance between Eigenvector’s andWLS’s weight-vectors

Figure 2.1: Numerical comparison of Eigenvector and WLS [1]

allows more fair distribution of weights according to variance or entropy of criteria-values (explained below), in comparison with assigning of equal weights by a subjectivemethod. Let’s say, given 2 decision criteria for 2 alternatives – Received Signal Strength(RSS) (e.g. -50 and -80 in dBm) and C (e.g. 25 and 20). It is obvious without anycalculations, that relative di�erence of RSS parameter is much higher than one for C.Therefore, more attention should be dedicated to the first criterion, since there are nosubjective preferences, what may be useful in case of Background tra�c-scenario (seesection 3.2.1 for examples of parametrization of di�erent scenarios).

Variance-based method [3] utilize variance of criteria, such that weights are assignedproportionally to this parameter and can be calculated via 2.10, where Vj is the varianceof normalized values for j-th column of D, dj represents mean value for j-th column.

In contrast to Variance-based method, Entropy-based method [3] assigns weightsproportionally to entropy of the criterion as in 2.11.

2.3.3 Hybrid techniqueAnother approach called "Hybrid" has been developed in [1]. The idea is to combineobjective and subjective weights via particular factor k 2.12. This can be useful ifdetermination of current tra�c type is complicated or if there is some uncertainnessabout preferences.


yevgeniy

Why higher relative difference should mean higher importance? What is the benefit of that?


2.4 MCDA algorithmsAccording to research results of [1], the following algorithms are to be analyzed as themost suitable in the handover context: SAW, GRA, TOPSIS, VIKOR.

Prerequisite for appliance of a MCDA algorithm is preprocessed data (values arenormalized) in the input matrix.

2.4.1 SAWThe most simple algorithm is Simple Additive Weighting (SAW) or sometimes referredto as Weighted Sum Model (WSM) [15]. The score of alternatives can be calculatedas follows:

SAWi =Nÿ

j=1wjdij, i = 1 . . . M, (2.13)

where dij – an element of normalized matrix D,wj – the weight of j-th criterion.After, alternatives are to be ranked in decreasing order.

2.4.2 GRAThe Grey Relational Analysis (GRA) algorithm exploits the theory of the same name todeal with decision-making problem [16]. In fact, this technique allows to demonstratedegree of similarity to ideal solution, what involves next steps:

1. Determination of the ideal d+j solution:

d+j = maxi dij for upward criteria ;

d+j = mini dij for downward criteria;

2. Calculation of Grey Relational Coe�cient:

GRCi = 1qN

j=1 wj|dij ≠ d+j | + 1

(2.14)

3. The alternative (network) with the largest coe�cient is the most desirable.



2.4.3 VIKORThe VIsekriterijumska optimizacija i KOmpromisno Resenje (VIKOR) algorithm wasdeveloped as MCDA algorithm for solving of problems with conflicting criteria [6]. Thealgorithm has the following steps:

1. Determination of the ideal d+j and anti-ideal d≠

j solutions:

d+j = maxi dij, d≠

j = mini dij for upward criteria;

d+j = mini dij, d≠

j = maxi dij for downward criteria;

2. Computation of the values Si and RI , i = 1, . . . , M :

Si =Nÿ

j=1wj(d+

j ≠ dij)/(d+j ≠ d≠

j ), (2.15)

Ri = maxj

(wj(d+j ≠ dij)/(d+

j ≠ d≠j )), (2.16)

where wj is the weight of j-th criterion.

3. Computation of the score of i-th alternative by relation:

Qi = v(Si ≠ S≠)/(S+ ≠ S≠) + (1 ≠ v)(Ri ≠ R≠)/(R+ ≠ R≠), (2.17)

where S+ = maxi Si, S≠ = mini Si, R+ = maxi Ri, R≠ = mini Ri,

v – weight for the strategy of maximum group utility, where (1≠v) is the weightof the individual regret.

4. Ranking of the alternatives by sorting the values Qi, Si, Ri in decreasing order.

5. Selection of the alternative AiÕ as a compromise solution, i�:

a) Acceptable advantage: Q(AiÕÕ)≠Q(AiÕ) >= DQ, where Q(AiÕÕ) is the secondalternative on ranked list by Q; DQ = 1/(M ≠ 1).

b) Acceptable stability: the alternative is also the best in terms of ranked listsby S, R. Results are significantly influenced by decision strategy: maximumgroup utility (v > 0.5), consensus (v ¥ 0.5) or with veto (v < 0.5).

If one of these conditions isn’t satisfied, a set of solutions is proposed:

a) AiÕÕ and AiÕ if there is no acceptable stability.


yevgeniy

Ai′′ and Ai′ should be A´ and Ai´´


b) AiÕ , AiÕÕ . . . AiS , as long as Q(AiS ) ≠ Q(AiÕ) < DQ – in case of absence ofacceptable advantage.

In the handover context, this additional check may lead to reduced Ping-pong e�ect.

2.4.4 TOPSISThe Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is adistance-based algorithm as well as VIKOR.

Unlike to previously described algorithms, normalization method 2.2 a�ects on thenumber of steps involved in the algorithm [5]. The most common in literature optionis p = 2 for p-norm method.

TOPSIS was defined as follows (for p = 2):

1. Construction of the weighted normalized decision matrix V , where vij:

vij = dij · wj (2.18)

2. Determination of the best v+j and the worst v≠

j values among all criteria N :

v+j = maxi vij, v≠

j = mini vij for upward criteria;

v+j = mini vij, v≠

j = maxi vij for downward criteria;

3. Calculation of the similarity to the ideal and anti-ideal solutions:

S+i =

ıııÙNÿ

j=1(v+

j ≠ vij)2, i = 1, . . . , M (2.19)

S≠i =

ıııÙNÿ

j=1(vij ≠ v≠

j )2, i = 1, . . . , M (2.20)

4. Finding of the relative closeness to the ideal solution:

RCi = S≠i

S+i + S≠

i

, i = 1, . . . , M (2.21)

5. Ranking of alternatives according to decreasing order of RCi.



2.5 Comparison of the algorithmsThe algorithms are to be analyzed against such criteria as complexity, scalability, rank-ing reversal and performance in the handover context. Results of the analysis aremostly based on [1] and composed into Table 2.6. The number of stars in the tablerepresents a mark of algorithms in range from bad to excellent.

The column named Complexity represents how much resource-demanding the algo-rithms are in terms of computation procedure. Additionally to the retrieved resultsfrom [1], analytic evaluation of the complexity of the MCDM-algorithms has beenperformed in this work.

According to the computational complexity theory [17], the algorithms were ana-lyzed against worst-case performance parameter. Assuming sorting, matrix-matrix andmatrix-vector multiplications were involved and there were no optimizing techniquesfor those operations, obtained results are represented in Table 2.5. As before, M isthe number of alternatives, N – the number of criteria.

Table 2.5: Algorithmic complexity of the MCDA algorithmsName Algorithmic complexitySAW O(MN) / O(N3)GRA O(MN) / O(N3)TOPSIS O(N3)VIKOR O(N3)

SAW algorithm consist of the following procedures: array access (O(1)) + matrix-vector multiplication (O(D · w) = O(M · N · 1)) + array sorting (O(log(N))). Thisasymptotically yields to O(M ·N) algorithmic complexity. GRA’s complexity is calcu-lated similarly to SAW and it is equal. However, complexities of TOPSIS and VIKORare higher – O(N3), since operations of multiplications are involved in a loop. Algorith-mic complexity for SAW and GRA would be O(N3) in case of scalar-form calculation,due to scalar-multiplication (O(N2)) in a loop.

Henceforth, all results in this section for mentioned algorithms have been taken from[1].

Numerical evaluation showed, that SAW and GRA are least demanding in terms ofcomputational time. For the sake of evaluation, Matlab’s function "Tic-Toc" was usedfor determination of computational latency. In fact, column "Complexity" in Table 2.6represents scores, which involve results of both analytic and numerical evaluations.



The next assessing criteria is scalability. It defines, how flexible is algorithm in termsof computational latency, if the number of criteria in matrix A changes. On practice,the algorithms were evaluated against scalability in Matlab. For that, computationaldelay of algorithms was measured for di�erent dimensionalities of A (in range from 2 to100). The lower the increase in latency is, the higher score is assigned to an algorithm.SAW is the most scalable (see Table 2.6), due to its straight-forward nature. GRAdemonstrated slightly weaker results.

Rank-reversal score shows how often this phenomena occurs in particular algorithm.As we can see in Table 2.6, VIKOR and GRA are the most robust ones against thisphenomena.

The performance (in the handover context) parameter in last 2 columns of the tableis based on di�erent basises. The first one exploits results gained from comprehensiveliterature review conducted in [1], where VIKOR and GRA are top-2 algorithms andTOPSIS is an outsider. However, practical evaluation, conducted in that work, showed,that SAW is on the same level of performance as VIKOR, while GRA was the bestone.

Table 2.6: Comparison of the MCDA algorithms

Name Complex-ity

Scalabil-ity

Rankingreversal

Perfor-mance

(literature)

Perfor-mance

(practical)[1]

SAW ***** ***** ** ** ****GRA **** **** ***** **** *****TOPSIS *** *** ** *** **VIKOR ** *** ***** ***** ****

2.6 Construction of MCHAKnowing procedure of ranking for all mentioned MCDM-algorithms, now we can dis-cuss whether normalization procedure is mandatory for given ranking algorithm. Forinstance, unnormalized values for SAW lead to unpredictable results due to arbitraryscale of input values. The same is true for GRA, where normalization allows to calcu-late correct distances (define grey gradations) to ideal (or white) solution. Moreover,weighted normalized values are defined as part of TOPSIS algorithm, where thesevalues are involved in obtaining of ideal and anti-ideal solutions.



Table 2.7: The constructed MCHA algorithmStage Conversational scenario Streaming scenarioNormalization Enhanced Max-Min 2.2 Enhanced Max-Min 2.2Weighting Eigenvector 2.8/WLS 2.9 WLS 2.9Ranking GRA GRA

At the same time, the situation is di�erent for VIKOR, where normalization stagemay be omitted at all, since during calculation of Si and Ri values (see 2.15 and 2.16),the results are obtained as a weighted sum of ratios of i-th values di�erences. Exactlyin this place we see, that there is no need for normalization stage, since the ratioswill be the same for both normalized and unnormalized input values. However, bettervisibility may become additional motivation for normalization of input values, becauseit will be easier to compare values of di�erent criteria if the are of the same scale.

Due to mentioned reasons, Enhanced Max-Min is recommended for SAW, GRA,TOPSIS, while for the VIKOR it is optional.

Research results of [1] have shown, that some of weighting and MCDA algorithmsare more suitable for di�erent scenarios. Therefore, we can construct an algorithmthat shows the best performance in particular decision scenario:

• Streaming tra�c: according to [1], WLS is preferred as a weighting algorithm.GRA looks as a reasonable choice as ranking algorithm.

• Conversational tra�c: despite both WLS and Eigenvector demonstrate verysimilar performance, the latter consumes much less computational time for low-scale problems [1], what is the case for us. As before, GRA holds leading positionas MCDA algorithm.

Resulting composite MCHA algorithm is represented in Table 2.7.


yevgeniy

they are

3 Evaluation approaches for MCHAs 18

3 Evaluation approaches for MCHAs

Despite plenty of MCHAs has been developed, there is no standard methodology fortheir evaluation. It is hard to underrate importance of this fact, since a generalizedapproach allows objective assessment of the algorithms, minimizing influence of sidefactors, which may arise during building of evaluation setup.

Typically, we assess decisions and properties of an algorithm, e.g. resulting QoSparameters or a number of handovers along the chosen path.

3.1 Existing approachesNevertheless, several approaches have been developed for algorithm’s assessment pur-pose. The most recognizable ones are:

• Standard deviation from Optimum interface (SDOI) [18];

• Criticality analysis [19], [20].

The ones exploit 1-tier hierarchy of assessing criteria. Limitations of such structurewill be explained later on.

3.1.1 Standard deviation from Optimum interface approachThe approach introduced in [18] exploits quantitative metric for assessment of algo-rithms’ decisions and based on the concept of standard deviation. The idea can beexplained by the steps below.

1. An evaluation matrix E œ RH◊K (with H is a number of algorithms beingassessed and K – a number of evaluating parameters) is to be constructed. Thismatrix is filled with results retrieved from simulations or a test-bed. The authorsrefer to H as to a number of network interfaces, where di�erent MCHAs areactivated.


yevgeniy

Which „side factors“ exactly are meant here? This motivation doesn’t correlate with the motivation for the evaluation phase from Chapter Introduction

yevgeniy

But also properties of the device …


For the sake of explanation, I developed a simple example, where 2 algorithms(Alg1 and Alg2 ) are being assessed against 2 criteria (Crit1, Crit2 – downwardtype). The resulting evaluation matrix comprises from arbitrary chosen valuesof the best path:

E =

S

WU35 0.5

100 0.1

T

XV

2. Definition of ideal interface, which is a virtual entity, composed from the bestpath-values of evaluated algorithms (ideal-solution from E):

o =5max{e1}, max{e2}, . . . , max{eK}

6(3.1)

where ej – j-th column of the matrix E,

o – resulting ideal-interface-vector, if the parameters are of downward type, themax operator must be changed to min.

In our case, it is:

o =535 0.1

6

3. Calculation of the standard deviation for j-th decision parameter should be per-formed at t-th time instance for every algorithm i = 1 . . . H:

sdoi,j =ııÙ

ATÿ

t=1(ei,j,t ≠ oj,t)2

B

/H (3.2)

Since there is only one data snapshot in our example, expression 3.2 can besimplified to:

sdoi,j =Ò

(ei,j ≠ oj)2/H, (3.3)

what results to:

sdo =

S

WU0 0.28

65 0

T

XV

4. After that, resulting score for each i-th algorithm can be calculated as:



SDOIi =Kÿ

j

ln(sdoi,j) (3.4)

where ln is used to decrease influence of non-normalized values.

Since logarithm of 0 is an undefined value, 0-entries will be omitted. This leads to:

SDOI =5≠1.27 4.17

6T

The lower SDOIi index is, the better performance the algorithm have, therefore,the first alternative has been determined as the best one. However, this approach isnot designated for use with post-processed (e.g. averaged) values, since results will besomewhat incorrect, as was shown above. Also, normalization of matrix E is desired,although it is not considered in the algorithm.

This approach performs objective assessment of di�erent algorithms, since there isno parameters to change by user, what might be a serious limitation during evaluationprocedure.

3.1.2 Criticality analysis approachThe methodology proposed in [19] exploits a concept of criticality. This concept allowsestablishing priorities of processes or parameters in a system in order to define the mostcrucial ones for operational reliability. In the handover context, we need to obtaincriticality index for each algorithm being assessed. The one with the highest index isconsidered as the best.

The method’s structure can be formulated as follows:

1. First of all, we need to define evaluation parameters, this is the ones which thewhole algorithm is analyzed against. For instance, it can be the obtained user’sD or Complexity of the evaluated algorithm. The number of such parametersmay be defined according to research needs. According to out example, they areCrit1, Crit2.

2. Then, valuation levels (as integer numbers in range from 1 to 9) for values of eachcriterion to be assigned. An example of values for such mapping procedure isshown in Table 3.1, where c is a value of an evaluation parameter. All parametersmust be transformed to downward type. It may be done via taking of inverse.


yevgeniy

„However, this approach is not designated for use with post-processed (e.g. averaged) values, since results will be somewhat incorrect, as was shown above.“Where was it shown?

yevgeniy

Would it be a good idea to assign a weight vector to the sdo matrix from the point 3?


3. Composition of the evaluation matrix E œ RH◊K (as in the previous section).It consists of above-mentioned parameters (columns) which belong to particularalgorithms (rows). For the sake of example, let’s consider the same matrix E asin the previous section.

E =

S

WU35 0.5

100 0.1

T

XV

In case of several simulations, the matrix’ data may be "filtered" by "3 sigmarule", assuming that data distribution is normal. After this procedure, averagevalues must be calculated and inserted in the matrix.

4. Afterwards, the weight for each parameter must be calculated. Despite the au-thors mentioned, that evaluation matrix may be normalized by column-basis inorder to eliminate dimensionality of input values, we’ll see that this doesn’t workwith described weighting method.

This method is of the objective nature and can be described as follows: wecalculate the sum of values in each column and take inverse of this sums (1/(100+35) = 0.007 and 1/(0.5+0.1) = 1, 667). The weights are obtained as the inversesdivided by theirs sum:

weval =

S

WU0.007/(0.007 + 1.667)

1.667/(0.007 + 1.667)

T

XV =

S

WU0.004

0.996

T

XV

If the values are normalized, the column-sum will be the same, which leads tothe equal weights for the parameters.

Formally, the weight-vector can be represented:

weval = [weval1 , weval

2 , . . . , wevalK ]T , (3.5)

Table 3.1: Valuation levels of evaluation parameters

Very low=1 Low=3 Medium=5 High=7 Veryhigh=9

c>80% of themax value

60%<cÆ80%of the max

value


value


value

cÆ20% of themax value



where wk – the weight for k-th parameter and calculated as follows:

wevalk = 1/(qH

h=1 eh,k)qK

k=1 1/(qHh=1 eh,k)

, (3.6)

where eh,k – is the element of matrix E, which represent k-th parameter of h-thalgorithm.

5. Next, we map values in E with help of Table 3.1 in order to construct criticalitymatrix.

Emap =

S

WU7 1

1 9

T

XV

6. Finally, we calculate criticality index of the algorithms using this equation:

IC = 100 · Emapweval

L · qKk=1 weval

k

, (3.7)

where L – maximum valuation level (= 9).

Then, this vector is sorted in descending order to obtain rank of algorithms undertest.

In our example:

IC =

S

WU11.38

99.64

T

XV

Thus, Alg2 has the best performance.

As we can see, this approach has certain drawbacks:

• Input data is not normalized, therefore weighting is influenced by di�erent scalesof parameters.

• Weighting stage has limited e�ciency, as it was mentioned above.

• Use of the mapping procedure from the 5th step causes loss of input information.

To cope first two problems, authors of [20] proposed using of di�erent weightingmethod – AHP (Eigenvector). Therefore, this method will be further referred to asSubjective Criticality analysis approach.



3.1.3 Subjective Criticality Analysis approachIn this approach, the first 3 stages are identical to above-mentioned one. However,evaluation matrix has normalized version – E, although it has no sense, since themapping procedure is still used afterwards. The normalization procedure is done withMax method 2.3.

The 4th stage is di�erent – instead of native weighting method (described by 3.5,3.6),Eigenvector 2.8 is used. Of course, it is desirable to construct corresponding matricesof pair-wise comparisons A for di�erent evaluation scenarios (see equation 2.6). In ourexample, let’s assume that the criteria are equally important:

weval =50.5 0.5

6T

,

Further steps are the same as in the previous approach. This lead to the followingcriticality indexes:

IC =

S

WU44.44

55.56

T

XV

So, the best algorithm is Alg2. But it is obviously, in case of accenting on thefirst criterion, the best algorithm would be Alg1. Using of subjective method gives usopportunity to emphasize particular criteria during evaluation process.

However, even this modified approach su�ers from the following limitations:

1. The mapping procedure leads to loss of information, what may be crucial duringthe evaluation.

2. There is no classification of evaluation criteria by group of interest. Let’s say, aresearcher, who uses some set of evaluation parameters, will get incomparableresults with the one, who use di�erent set even if some criteria within are thesame. Expanded motivation is provided in the next section.

These facts put some limitations on this approach as an evaluation method.

3.2 New evaluation approachThe idea of proposed evaluation approach is to classify criteria according to groupof interest (e.g. QoS parameters, HO parameters). If we incrementally add desiredparameters to the same hierarchy level of assessing parameters, as before, we will loose


yevgeniy

the section is confusing„who uses some set of evaluation parameters, will get incomparable results with the one, who use different set even if some criteria within are the same.“what do you propose to solve it?


flexibility in this case, since adding or removing of particular parameter make theevaluation results incomparable with another ones. Since every researcher cannot beforced to use the same set of the evaluation parameters, I suggest replacing of suchflat structure of criteria by a multilevel one, in order to make possible reusing andcomparison of required evaluation results gained within the classified sets. Therefore,we can parameterize certain sets of criteria and think of them as "building blocks" forevaluating of algorithm. Whenever we need a one, we add it to our model, aggregatingthem in a special way. This allows us to choose only those assessing sets what we want,constructing mixed evaluation scenarios (see 3.2.3).

Possible examples of the sets with corresponding evaluation focuses are shown inTable 3.2.

Table 3.2: Assessing sets for evaluation of MCHA

Handover Device Network Algorithm Applica-tion

Parameters

HO Latency,Ping-Pongrate, HOFailure

probability

Battery . . .

Security, Networkload, Cost, QoS{Throughput,

Delay, Packet loss,Jitter}

Complexity,Scalability

Quality ofExperience

Suggestedweighting Subjective — Subjective Objective —

Examples ofevaluation

focuses

HOrobustness,

HOperformance

Energyconsumption

Network loading,Money spendings,Secureness, QoS

{Streaming,Conversational,

Background,Interactive}

— Applicationperformance

Perspectiveof interest Operator User Operator and User Operator User

During the stage of assessment we may want to focus our preferences on certainparameters in the sets. In that case, some subjective weighting method is requiredfor further definition of evaluation focuses (examples of which have been mentionedin Table 3.2). Note, that term evaluation focus shouldn’t be confused with decisionscenario, which have an e�ect on decision process via corresponding configuration ofweighting stage (see section 2.3). Evaluation parameters in the table may be di�erentfrom decision parameters as well (see Table 2.1), because some parameters can notbe a�ected by an algorithm’s decision. For example, Speed of the terminal usuallydepends only on the user and handover decisions can’t influence on that.

Another concern is relevance of the focuses to either operator or user perspective.



From the user’s point of view, such focuses as Energy consumption, Money spend-ings, Secureness, QoS-related and Application performance are the most importantdue to explicit impact on the one’s experience. On contrary, Network loading, HO-and Algorithm-related focuses are more interesting from operator’s perspective, e.g.for performance evaluation of load-balancing schemes.

3.2.1 Parametrization of the assessing setsAfter weighting approach type for each set is defined, parametrization is required forthe ones with subjective weighting type. During this parametrization, some additionalfactors may be considered, such as environmental, subjective, etc. For that, we needto discuss importance of some of these factors for evaluation process from a userperspective:

• Tra�c type. The knowledge of this property is mandatory for applying of correctdecision and evaluation methods considering QoS- and QoE as parameters.

• Battery charge level has influence on the user’s priorities. For instance, if thebattery is discharged the main user’s concern would be about reducing energyconsumption during communication procedure. I believe, that importance of thisparameter isn’t linearly changed, the lower is the charge, the higher importanceis, so a utility function may be used for modeling of this. This parameter maybe important for both decision and evaluation stages.

• Low balance may also change the user’s behavior, especially if he is out of prepaidtra�c. Hence, Money spendings scenario should be emphasized during aggrega-tion procedure.

• In case of emergency, the importance of all parameters but QoS and/or QoE dis-appears instantly. This term may include range of sub-scenarios from "importantcall" to "disaster".

• Despite Network conditions have huge influence on resulting user’s satisfaction,it is of implicit nature and more relevant to the operator’s side. Therefore, itmay be better to consider QoS and QoE as an explicit metrics from the user’sside, since they are a�ected by the network conditions.

• Locational and time of day properties (e.g. downtown/uptown, morning/evening)might have certain impact on the user’s perception of service as well as type of


yevgeniy

What does it mean?

yevgeniy

What is „Network conditions“?


device being used, what has been proved in [21]. Let’s say, a user being hun-gry and tired is more influenced by irritating factors like long-response time.Therefore, a bit more attention should be paid to these factors during evaluationprocess.

Of course, if these factors cannot be estimated, we won’t consider them at all.Moreover, there may be an option to avoid consideration of those factors, which cannotbe influenced by algorithm’s decisions.

Configuration examples for di�erent sets are presented in Table 3.3, where accordingto selected sets and criteria within, I defined some possible evaluation focuses (as inTable 3.2) with corresponding subjective weights for their parameters. The weights arecalculated via WLS from matrices of pair-wise comparisons, which are also present in-there. In case of absence of particular criteria (or existence of additional ones) withinthe sets, set re-parametrization is required.

3.2.1.1 Parametrization of the Handover set

All priorities are described in Table 3.3. The vector of weights is described as:

wevalHO =

5HO Latency Ping-pong rate Failure probability

6T

(3.8)

The HO robustness focus allows assessment of reliability of handover process duringgiven path. Obviously, HO Failure probability has the highest influence on it, however,Ping-pong rate significantly influence on that parameter as well, while HO latencycan negatively a�ect performance (see the HO performance focus) and robustnessproperties as well, especially in case of high user’s velocity.

3.2.1.2 Parametrization of the Network-related set

Description of parametrization if placed in Table 3.3. The weight-vector includesparameters in following order:

wevalNetwork =

5Security Network load Cost QoS

6T

(3.9)

Depending on evaluation focus, corresponding parameters are prioritized within theset. However, in case of absence of some criteria (but from QoS-set), the evaluationprocedure may be simplified by using of only desired ones (e.g. use only S parameterfor the Secureness focus).


yevgeniy

why can’t these factors be estimated? localization? time of day? traffic type?


Here, QoS set is defined as a subset of the Network set for better consistency andflexibility of the evaluation process.

Parametrization of the QoS setThe focuses for the parameters within QoS set are incorporated in accordance with

definition of the tra�c classes [22] for such scenarios as Streaming, Conversational,Interactive, Background from Table 3.2. This parametrization is summarized in Table3.3. Weight-vector looks as follows:

wevalQoS =

5T D J PL

6T

(3.10)

In real life situation, it may happen that there will not be "pure" type of tra�c, butwe often can distinguish the major one and to give priority to the dominating type oftra�c (e.g. the one having the biggest importance).

For Background-tra�c scenario, an objective method may be used, since there areno preferences regarding criteria (see section 2.3.2). However, equal weights have beenassigned for the sake of coherence of parametrization procedure.

3.2.2 Quality of ExperienceAmong above-mentioned evaluation criteria, QoE requires special consideration. QoEis a holistic metric, which evaluates the user’s experience from overall service, anddepends not only on the network-related parameters, but also on subjective prefer-ences and environmental properties of a particular user [21]. Majority of researchesare dedicated to resource-demanding types of applications, such as streaming and con-versational.

3.2.2.1 Subjective QoE-methods

The process of QoE estimation involves subjective factors, therefore, explicit assess-ment of the score (i.e Mean Opinion Score (MOS)) by users provides the best possibleresults. In general, there are 2 approaches for subjective evaluation of a service [23]:

• Absolute-based rating methods – there is no reference signal to compare thereceived signal with. The most widely used examples are:

– Absolute Category Ranking (ACR) is a well-known single stimulus method,where media-sequences are shown only once during 10s interval per each,with consequent 10s intervals for assessment procedure by the user via 5-point opinion scale.



Table 3.3: Examples of configuration of evaluation focuses for the Handover and Net-work sets

Set Evaluation focusPriorities of the

parameters(importance)

Matrix of priorities AWeightsby WLS

Han

dove

r HO robustnessHO latency (low),

Ping-pong rate (high),Failure probability (high)

S

WU1 1/9 1/99 1 19 1 1

T

XV

S

WU0.060.470.47

T

XV

HO performance

HO latency (high),Ping-pong rate

(moderate), Failureprobability (low)

S

WU1 3 9

1/3 1 91/9 1/9 1

T

XV

S

WU0.680.260.06

T

XV

Network loadingSecurity (low), Networkload (high), Cost (low),

QoS (low)

S

WWWU

1 9 9 91/9 1 1 11/9 1 1 11/9 1 1 1

T

XXXV

S

WWWU

0.760.080.080.08

T

XXXV

Net

wor

k Money spendingsSecurity (low), Networkload (low), Cost (high),

QoS (low)

S

WWWU

1 1/9 1 19 1 9 91 1/9 1 11 1/9 1 1

T

XXXV

S

WWWU

0.080.760.080.08

T

XXXV

SecurenessSecurity (high), Networkload (low), Cost (low),

QoS (low)

S

WWWU

1 1 1/9 11 1 1/9 19 9 1 91 1 1/9 1

T

XXXV

S

WWWU

0.080.080.760.08

T

XXXV

QoSSecurity (low), Networkload (low), Cost (low),

QoS (high)

S

WWWU

1 1 1 1/91 1 1 1/91 1 1 1/99 9 9 1

T

XXXV

S

WWWU

0.080.080.080.76

T

XXXV

Streaming T (high), D (low), J(high), PL (moderate)

S

WWWU

1 3 2 31/3 1 1/9 1/31/2 9 1 31/3 3 1/3 1

T

XXXV

S

WWWU

0.480.050.330.14

T

XXXV

Net

wor

k{Q

oS} Conversational

T (low), D (high), J(moderate), PL

(moderate)

S

WWWU

1 1/7 1/7 1/37 1 3 37 1/3 1 53 1/3 1/5 1

T

XXXV

S

WWWU

0.060.580.260.10

T

XXXV

Interactive T (low), D (moderate), J(low), PL (low)

S

WWWU

1 1/2 1 12 1 2 11 1/2 1 21 1 1/2 1

T

XXXV

S

WWWU

0.210.360.230.20

T

XXXV

Background Equally non-important

S

WWWU

1 0 0 00 1 0 00 0 1 00 0 0 1

T

XXXV

S

WWWU

0.250.250.250.25

T

XXXV



– Quantal-Response Detectability (QRD) suits well for detection and ratingof disturbances of a sound via detectability opinion scale.

• Comparison-based ranking methods – both reference and degraded signals arecompared by a human subject. The following methods may be used for that:

– Degradation Category Ranking (DCR) or double stimulus impairment scalemethod allows to estimate how much the degraded sample is worse than theoriginal, given the reference is always the first available for testing.

– Comparison Category Rating (CCR) is similar to the DCR, except the fact,that the order of test-samples is chosen randomly.

These methods are defined in ITU-T recommendation P.800 and P.910 for audio andvideo correspondingly.

The assessment procedure may be taken either in a laboratory or remotely viaInternet. Internet-based one is supposed to be more precise, since the process is usuallyperformed in realistic conditions. However, such subjective approaches are expensive,time-consuming and online evaluation is not possible.

3.2.2.2 Objective QoE-methods

Voice and video applications. Objective nature of a method can be exploited var-iously depending on type of tra�c (voice or video). In general, there are 2 approachesfor objective assessment – signal layer processing and parametric-model:

• Signal-layer or (media-layer) methods involve processing of the media-signal it-self exploiting its spatial-temporal features. They can be further classified asFull-Reference (FR), Reduce-Reference (RR) and Non-Reference (NR) for video-signals. FR methods usually require source signal to be presented unlike to NRmethods, which exploit only human vision system properties. RR approachesexploit much fewer amount of information from the source (e.g. statistics ofwavelet coe�cients). Despite FR methods usually demonstrate the highest cor-relation with MOS, some RR methods have nearly the same correlation levels,while requiring much less side-information. At the same time, NR techniques arenot so good due to quite low or even negative correlation with MOS [24].

For voice-signals, Perceptual Evaluation of Speech Quality (PESQ) method is agood example of reference model.



Table 3.4: Suggested objective QoE-methods for assessment of voice, video and web-browsing applications

Type of signal Media-layer method/Type Packet-layermethod

Voice Perceptual Evaluation of SpeechQuality (PESQ)/FR Q-Model

Video

Motion based Video IntegrityEvaluation (MOVIE)/FR,Spatio-temporal ReducedReference Entropic Di�erences(STRRED)/RR

ITU-T G.1070,Joskowicz et al. [25]

Web-browsing Egger et al. [27] + G.1070

• Parametric model (or packet-layer method) is generally considered as less ac-curate than signal layer methods, but it also requires much less computationalresources. For VoIP tra�c, the model takes a set of parameters such as propertiesof a room, J, PL, Coding scheme, Echo duration, etc. as the input. Examples ofsuch models are E-Model and Q-Model, where the latter one is improved versionof E-Model due to usage of e�cient window-based approach. Moreover, boththese models were improved with ability to process packet burstiness parameter,what led to noticeable improvement [23].

Regarding video-signals, there are also several parametric methods for the as-sessment procedure. The most common one is ITU-T recommendation G.1070.However, authors of [25] proposed an extension to this model, which takes intoaccount additional parameters related to display format and codec performance.This enhancements provide better correlation with MOS. A survey [26] conductedlater by the same authors demonstrated, that proposed model works better forsystems with week encoding impairments, while G.1070 – with transmission im-pairments. Therefore, both these models may be used with certain awareness ofproperties of the system-under-test.

Taking into account those facts and results of the surveys [23], [24], [26], recom-mended objective methods for video- and audio-content assessment are presented inTable 3.4.

Also, current sophisticated objective approaches demonstrate high correlation withsubjective assessments. However, their application scope may be limited by di�cultiesin gathering of additional assessing criteria (parametric models) and high computa-



tional complexity together with excess signaling tra�c (for signal-layer methods).Web-browsing application. For this case the situation somewhat di�ers. Al-

though above-described ACR subjective method can be applied, comparison-basedmethods don’t work here as well as objective methods. On contrary to voice andvideo applications, the most QoE-a�ecting metric is Page Load Wait Time (PLWT).However, the di�culties during obtaining of this parameter arise due to huge num-ber of parameters which has influence on it. Network characteristics are not the onlyones, there also are device-related (e.g. type of operating system, browser, etc.) andweb-page-related (size, contents...) parameters. This is obvious, there is no chance toexplicitly obtain all these parameters on the operator’s side.

It was shown in [28], that PLWT and QoE-score are related logarithmically and thisholds for almost every interactive application, including di�erent tasks with waiting(e.g. file downloading). Since measurement of time-to-load is very challenging forweb-browsing, the PLWT cannot be mapped directly from Throughput (T) due tolarge number of small pieces which a web-page consist of, and perceptual properties ofend-users. The main output of the paper is that QoE-score for web-browsing activitycannot be mapped from T and other QoS-parameter, since it is necessarily to considerwider range of parameters (e.g. page rendering speed, response time).

The authors further developed a method for extraction of those parameters viameasuring engagement of a user in [27]. This measure is considered as the mostimportant one for QoE-estimation. In turn, engagement can be estimated throughsession length and abandonment parameters. For that, correct identification of userclicks is required, therefore, authors developed an approach which allows the operatore�ectively estimate this events via classification of web-pages basing on their sourcedomains by machine learning algorithm. Moreover, they developed an approach forprediction of QoE basing on radio-network parameters (e.g. RSSI, number of verticalhandovers) via linear regression model.

Similarly, authors of [29] proposed an approach, where web-page-related propertiesare obtained through preliminary classification of the pages. For that they propose toselect representative groups of pages – popular and important ones, and then processthem with help of machine learning algorithm to build a statistical model, by meansof which non-representative pages should be classified. After, isolation of some otherparameters is suggested, e.g. to select the most common version of operating system.However, explicit model wasn’t proposed there.

Fortunately, given the key performance parameter is known (i.e. PLWT), QoE canbe gained through a model defined in ITU-T recommendation G.1030. Unlike video



and voice QoE approaches, the classification scheme "Media-layer"-"Packet-layer" doesnot really apply, since required parameters may be placed on Packet-layer (like QoS-parameters) and on Media-layer (e.g. class of web-page) as well. Therefore, proposedscheme do not belong to any of these in Table 3.4.

3.2.2.3 QoS-to-QoE mapping methods

The motivation for researchers to develop a mapping technique is implied by necessityto provide QoE-score with reduced number of input parameters in comparison withobjective methods. The idea behind this exploits the fact, that there is some kind ofcorrelation between QoS parameters and QoE score. Therefore, numerous researchestry to figure out this dependency using parametrized non-linear functions.

Authors of [30] proposed approach with exponential function being used for VoIP-tra�c scenario. They conducted series of measurements for MOS as a function of PLor type-p reordered ratio (related to J parameter). The generalized form of proposedfunction is:

QoE = – · e≠—·QoS + “, (3.11)

where –, —, “ are fitting parameters, with – = 3.01, — = 4.473, “ = 1.065 for QoE asa function of PL for VoIP, and with – = 2.482, — = 10.453, “ = 1.141 as a function oftype-p reordered ratio. The most critical restriction of the model – QoE is a functionof only one QoS parameter.

Another QoE-evaluation model, which uses sigmoid mapping function of QoS wasproposed in [31]. The function is defined as follows:

QoE = K{ eQoS≠– + e≠QoS+–

eQoS≠– + e≠QoS+– + —+ 1} (3.12)

Authors suggest to parametrize the function via K, –, — variables according to servicequality ("Guaranteed", "Premium","Best E�ort") basing on QoS-score. The score iscalculated as a sum of parameter’s scores (e.g. D, J, PL). The main drawback of thework is configuration of parameters for the sigmoid function wasn’t performed. Thus,another research is required for this.

The authors extended previous model and performed parametrization for the caseof IPTV service in [32]. PL, Burst Level, J, D and T were used as QoS-criteria. Allof them are normalized and weighted in order to obtain the QoS-value. The mappingfunction for QoE of video had been changed and looks like:



QoE = Qr ◊ (1 ≠ QoS(X))QoS(X)◊A

R (3.13)

where Qr – variable, which depends on external factors (i.e. display size),A, R – denote service class and properties of the video-frames structure correspond-

ingly.This allows e�ective mapping of QoS-to-QoE parameters for streaming tra�c class.Despite all above mentioned correlation models propose mapping functions, proper

parametrization should be performed for each kind of tra�c. This requires certain ex-pertize level, some resources and, therefore, that will be omitted in the work. Moreover,since some input parameters (irrelevant to network ones) are omitted by purpose, cor-relation with subjective assessment will always be lower in comparison with advancedobjective parametric models.

3.2.2.4 The option for implementation in a real system

WiFi Scheduled Multimedia (WSM) is an optional part of IEEE 802.11e standard. Itsfeature is to require for associated devices to send their tra�c profiles, which describeneeds of current application type and consist of such parameters as T, D and J. Then,required QoE can be obtained from these parameters with help of above-mentionedQoS-to-QoE mapping techniques. This allows calculation of QoE score on the AccessPoint and consequent relaxation of the QoS-requirements, if the required QoE-scorewill not be changed significantly. All this may lead to improved network e�ciency,while keeping QoE at nearly the same level. To the best my knowledge, there are noany researches, where QoE is considered as decision criterion for handover process. Ibelieve this is not impossible, since reliable assessment of this parameter can not beachieved before an actual connection is set up.

3.2.3 Aggregation of the setsThe final step is aggregation of evaluation sets in case of mixed evaluation scenarios.This may be useful, if desired evaluating criteria belong to di�erent sets or evaluationscenario requires several evaluation focuses within the sets. The aggregation strategydefines the way in which the sets are being aggregated.

3.2.3.1 Description of the procedure

This procedure involves the following steps:



1. Definition of aggregating matrix E œ RH◊K , where K = qSs=1 Ks, H is a number

of algorithms being assessed, S – the number of sets being aggregated, K – thenumber of criteria in s-set:

E =5E1w

eval1 E2w

eval2 . . . ESw

evalS

6, (3.14)

where Es is s-th evaluation set with ws – weight-vector, which defines evaluationfocus for the set. There won’t be any weight-vector (just scalar 1), if the set hasonly one parameter.

2. Calculation of weights w for E with help of appropriate weighting method. Al-though subjective methods should be preferred (e.g. WLS 2.7), there is an optionto use an objective one (e.g. Variance-based 2.10) in case of absence of prefer-ences regarding the aggregation strategy. This completely depends on aims andobjectives of an engineer.

3. Find resulting evaluation score using obtained w and E via modified version ofcriticality index 3.7:

SE = Eweval (3.15)

Note, in comparison with expression 3.7 the mapping step is omitted, due to loosingof some amount of information during this procedure. Also, due to usage of thesubjective method, the sum of weight-vector is will always be 1, therefore, there is noneed in the denominator from the original expression, the constant in the numeratoris removed as well. In fact, this evaluation score is a weighted sum of normalizedevaluation values from E, what is very convenient in terms of retrieving of intermediateresults for the used sets.

In such way, we can easily construct evaluation scheme using pre-defined criteria-sets,assigning appropriate weights via subjective method. Thus, we can prioritize selectedevaluation sets in accordance to our preferences. Alternatively, objective method maybe used if there is no preferences for the evaluation process.

3.2.3.2 Discussion of the aggregation strategies

During evaluation process from user’s perspective the following question may arise:"Is it enough to use only Application set, since it contains the most holistic metric forevaluation of user experience – QoE?". The answer would probably be "No", since such



Table 3.5: Examples of the aggregation strategies of the Device, Network and Applica-tion sets from the user’s perspective

Aggregationstrategy

Set/Evaluation focus(importance) Matrix of priorities A

Weights byWLS

Battery savingperformance-1

Device (high), Network/QoS(low), Application (low)

S

WU1 7 71 1 01 0 1

T

XV

S

WU0.740.130.13

T

XV

Battery savingperformance-2

Device (low), Network/QoS(high), Application (high)

S

WU1 1 17 1 07 0 1

T

XV

S

WU0.080.460.46

T

XV

Performance Network/QoS (high),Application (high)

S

WU1 0 00 1 00 0 1

T

XV

C0.50.5

D

Cost-and-Batterysaving

Network/Cost (high), Device(high)

S

WU1 0 00 1 00 0 1

T

XV

C0.50.5

D

important from user’s point of view parameters as C, S and Battery are not explicitlyconsidered in QoE-models. Therefore, if this is the case, either modification of QoE-models or combining of these parameters from Application, Device and Network setsmay be desired. The choice of the latter approach looks more preferably.

In order to correctly perform aggregation procedure, environmental, subjective andnetwork properties of the evaluation scenario should be determined as it was discussedin section 3.2.1.

Generally speaking, we can define several strategies for aggregation of this sets. Thewhole thing depends on a bunch of external factors and priorities, which the userselects in particular scenario. Examples of the strategies are provided in Table 3.5,while some evaluation scenarios, where they are applicable, will be discussed later on.If both of applied sets are equally important, then compromise strategy for weightingis applied (e.g. Cost-and-Battery saving strategy from Table 3.5), where equal weightsare assigned to each set. Moreover, it may be desired to use an objective weightingfor such cases. Generally speaking, a number of possible strategies may be significant,and this is responsibility of the operator to define suitable one, which fulfills its needs.

Also, we have to consider relationships between the sets during the evaluation pro-cess. For example, changing of HO-parameters implicitly a�ects QoS of a user, inturn, mobility of the user has influence on the parameters in Handover-set. But themain concern should be dedicated to QoE and QoS parameters, since they are stronglyinterdependent. For instance, if we obtain QoE-score with help of mapping methods



from QoS, there will be no sense to consider both parameters during the process ofevaluation. However, QoE gained through objective methods may serve as a use-ful complementary metric in addition to QoS set. While evaluating from the user’sperspective, the priority should be given to QoE, if available, due to its more com-prehensive assessing nature (see 3.2.2) oriented on subjective properties of the user.Nevertheless, both these metrics are considered during aggregation in Table 3.5 as mu-tually supportive ones. Hence, plenty of nuances should be taken into account duringconstruction of an evaluation method.

3.2.3.3 Discussion of approaches for construction of an evaluation scheme

Since I have already proposed an evaluation algorithm, some ways for construction ofthe one in accordance with evaluation scenario should be discussed. First of all, let’ssummarize the terms which have been introduced:

• Evaluation criteria, which on contrary to decision ones, are used during evalua-tion process.

• The evaluation criteria are classified by field of application into evaluation sets.

• Within the evaluation sets, we can define evaluation focuses, if subjective weight-ing of the evaluation criteria is preferred.

• Evaluation score is retrieved via multiplication of evaluation matrix with weight-vector, which presents current evaluation focus.

• If it is desired to consider several evaluation focuses (or evaluation sets), we haveto define aggregation strategy, where weights to the evaluation scores, obtainedfrom the aggregated sets, should be assigned via either subjective or objectivemethod in order to get resulting evaluation score.

• Evaluation scenario defines how and which focuses/sets/criteria should be usedduring evaluation process.

In fact, there may be defined 2 approaches for construction of an evaluation scheme– inductive and deductive. They are summarized in Table 3.6, where represented pro-cedures are self-explanatory. Nevertheless, both these approaches lead to the sameresults, the di�erence comprises in a way of achieving of them. I would say, that theinductive approach is more realistic, since we firstly define what we have and thenthink what we would like to do with that. On contrary, in case of the deductive one,


yevgeniy

this is a good explanation of design of a evaluation process

yevgeniy

fine :)


Table 3.6: Methods for construction of an evaluation schemeInductive method Deductive method

• Analysis of available evaluationcriteria

• Classification of the criteria intoevaluation sets

• Definition of possible evaluationfocuses

• Aggregation of the sets, if thereis more than one

• Definition of possible evaluationscenario

• Definition of required evaluationscenario

• Definition of necessity of an ag-gregation procedure

• Definition of required evaluationfocuses

• Definition of required evaluationsets for the focuses

• Analysis of available criteria inthe sets

we firstly define what we want, and than check whether we have it or not. How-ever, the procedures are not that strict on practice, the order of the steps is usuallyinterchangeable and they may be resolved in parallel by an engineer.

3.2.3.4 Discussion of some real-life scenarios

Up to now, approaches for construction of the decision and evaluation schemes havebeen discussed in some general cases. So, let’s discuss some specific scenarios withoutnumeric examples. Let’s say, our user is running out of the battery’s charge. Inthis case, the main user’s concern would be saving of the battery level. And thisshould be reflected in his decision and evaluation profiles (equivalent to strategy). Forinstance, among decision parameters the Energy consumption one should be prioritized.The parameter can be estimated basing on e.g. type of access technology and RSSI.Therefore, an AP, which leads to the lowest power consumption, will be selected,thus saving user’s battery. The same applies to the evaluation stage, where the mainconcern is Battery, and its levels of discharge should be evaluated for every algorithm-under-test.

Another possible scenario – an important call (e.g. job interview). In this case, theuser doesn’t care about the phone’s balance, only QoS-parameters matters, with con-stant monitoring of the battery’s level. Therefore, the QoS profile should be activated,with further activation of the Battery-saving one, for painless handling of the situationwhen the charge level goes below a threshold.

Nevertheless, the operator cannot construct decision and evaluation profiles for every


yevgeniy

this is a good descriptive presentation of eval. alg. design strategies


real-life scenarios, what is obvious. Hence, it is likely that only few basic decisionand evaluation profiles must to be created, which are suitable for majority of real-lifesituations. Then there may be 2 options:

1. The user will be able to choose an appropriate decision strategy in a specialapplication from his terminal, what should be reflected on the operator’s sidefor proper support of the handover-decisions. This manual style of controlling istedious for the user, but explicit.

2. A more sophisticated way is usage of machine-learning algorithms on the oper-ator’s side, supplied with complementary information from the mobile terminal,such as battery level, identifiers of caller and callee etc. for more e�cient controlof handover process via adjusting of the call importance, billing-plan and otherprofile’s attributes.

To add, the latter option may be more desirable due to development of self-organizednetworks, where such mechanisms as handover-based load balancing optimization andmobility robust optimization have already been defined in ETSI’s paper [33] for Long-Term Evolution networks.

Also, it often happens, that decision and evaluation strategies are related (the accentsare on the same parameters and/or sets). However, if the decision one cannot bechanged or we want them to be di�erent by purpose (e.g. to evaluate QoS-parameterswith Cost-oriented decisions), we can apply di�erent evaluation strategy and it willnot make any problems.

3.2.4 Case studyFor the sake of the visibility, let’s define an example of the decision-making procedurefollowed by evaluation process with mixed evaluation scenario.

3.2.4.1 Decision stage

Given 2 MCDM-algorithms: SAW and GRA, which are noted as Alg1 and Alg2 corre-spondingly, in order to represent generalization of the approach. The decision strategycorresponds to the conversational scenario. As decision criteria RSSI, D, J, NL andCost are used. Corresponding weight-vector looks as follows (parameterized in section4.1.6.2):

wconversational =50.248 0.307 0.295 0.103 0.047

6T

,



where weights are in the same order as the criteria are listed. For simplicity, let’sconsider only one snapshot of data from 2 available feasible APs (see Table 3.7).

Table 3.7: Example of decision values for demonstration of MCHAsRSSI,dBm D, ms J, ms NL, % C

AP1 -74 64 10 63 33AP2 -56 35 21 35 41

Corresponding decision-matrix is:

C =

S

WU≠74 64 10 63 33

≠56 35 21 35 41

T

XV

In order to normalize the matrix, let’s use, for example, Max method 2.3, takinginto account whether a criterion is of upward or downward type:

D =

S

WU0.88 0.55 1 0.56 1

1 1 0.48 1 0.8

T

XV

After that, we are ready to apply ranking algorithms and here are the resultingscores:

Alg1 =

S

WU0.79

0.84

T

XV , Alg2 =

S

WU0.82

0.86

T

XV

As we see, both of the algorithms selected the 2nd AP is an optimal one for givendecision-scenario.

3.2.4.2 Evaluation stage

For evaluation purposes, let’s consider those 2 algorithms – Alg1 and Alg2 with onesnapshot of averaged best-path values, which are a�ected by decisions made by thealgorithms. Among available and collected network criteria are NL, S, C, D, J, PL,T, Battery discharge level (presented in Table 3.8) and also Complexity and Scala-bility parameters of the algorithms (known in advance). Assessment of QoE-score isnot possible. There is no information regarding subjective (e.g. user’s mood) andenvironmental (e.g. time of day, location) properties which can additionally a�ect theevaluation process. Active VoIP connection is established in the device-under-test.


yevgeniy

unit for C is missing


The aim is to assess influence of the algorithms’ decisions from the user perspectivemaking an accent on quality of the communications.

Table 3.8: Example of the best-path values for evaluation of MCHAsNetwork Network{QoS} Device Algorithm

NL,% S C QoS T,

MbpsD,ms

J,ms

PL,% Battery Com-

plexityScala-bility

Alg1 43 High 0.5 QoS1 3 54 15 0.23 5.1 5 5Alg2 39 High 0.6 QoS2 3.1 60 25 0.13 5.3 4 4

According to available criteria, we have 3 assessing sets – Device-, Network- andAlgorithm-related ones. Conversational tra�c is active type of scenario for QoS-set,which should be prioritized according to the task. Therefore, evaluation focus fornetwork-set is QoS{Conversational} according to Table 3.2. Despite the Algorithmset is available, there is no need to consider it, since it is not important from the userperspective. Hence, only 2 sets should be considered, prioritizing the Network one.

For start, input data is normalized by 2-norm method (as the most conventional)on column-wise basis to upward type and grouped in following matrices according toevaluation sets:

ENetwork =

S

WU0.67 0.71 0.76 QoS1

0.74 0.71 0.64 QoS2

T

XV ,EQoS =

S

WU0.69 0.74 0.86 0.55

0.72 0.67 0.51 0.84

T

XV ,

EDevice =

S

WU0.72

0.69

T

XV

Note, that QoSi values in ENetwork are to be calculated in accordance with tra�c-type (conversational) with help of weight-vector defined in Table 3.3:

QoS = EQoS · wevalconversational =

50.75 0.65

6T

This is already evaluation score for the QoS set according to expression 3.15! Thiscan be considered as final results, if someone has no interest in other evaluation setsexcept this one. The same applies for the Device set, since there is only one parameterand no weighting is required.

Let’s continue, resulting evaluation matrix of the Network set is:



ENetwork =

S

WU0.67 0.71 0.76 0.75

0.74 0.71 0.64 0.65

T

XV

Next, we have to aggregate obtained evaluation matrices as in 3.14 and via weight-vector from Table 3.3 corresponding to QoS evaluation scenario for Network set:

wevalNetwork =

50.08 0.08 0.08 0.76

6T

E =5ENetworkw

evalNetwork EDevice

6=

S

WU0.74 0.72

0.66 0.69

T

XV

Since Device-set involves only one parameter, no weighting is required.In order to fulfill task requirements, we have to prioritize the Network set – assign

larger weight, for that let’s define the following matrix A:

A =

S

WU1 7

1 1

T

XV

Note, that Battery saving performance strategy from Table 3.5 can not be used,due to absence of criteria from the Application. Corresponding weight-vector for E iscalculated via WLS:

weval =50.85 0.15

6T

,

where the first element corresponds to the Device set, the last – to the Network set.Finally, resulting evaluation scores (see 3.15) for these 2 algorithms are:

SE =

S

WU0.74

0.66

T

XV

As we see, the evaluation score for Alg1 algorithm is the highest, hence, according toconditions of the problem, this one is better than Alg2.


4 Simulation 42

4 Simulation

The goal of the practical part is to analyze behavior of di�erent decision schemes,where Enhanced Max-Min serves as a normalization method, WLS – as a weightingone, while as options for MCDM-algorithm are SAW, GRA, TOPSIS and VIKOR.Moreover, a common for a Wireless Local Area Network (WLAN) handover algorithm(based on RSS parameter) will be verified additionally (further referred as RSS). Also,applicability of the evaluation approach, proposed in the previous chapter should beproven on practice. For that, I will apply 2 di�erent evaluation focuses to outcome ofsimulation data.

As simulation environment OMNeT++ (v. 4.6) with INET framework (v. 3.0) [34]is chosen due to popularity and ease of extensibility.

In order to correctly accomplish simulation of the decision algorithms, the followingtasks should be fulfilled:

• Definition of network scenarios

• Definition of network’s topology and parameters

• Definition of tra�c properties

• Collection of decision criteria in OMNeT++

• Implementation and integration of the MCHAs into OMNeT++

• Processing and analyzing of the simulation outputs

4.1 Simulation setupFor the sake of simulation, OMNeT++ is used (Fig. 4.1). This software is posed asa well-written framework for construction of event-driven network simulations. It iswritten in C++ and has modular structure which can be easily extended with owncode. INET framework can be considered as an OMNeT++ extension, which comprisesof models suitable for building of wireless, mobile and wired network simulation. The


4 Simulation 43

modular structures of an AP and a STAtion (STA) from INET library are presentedon Figures 4.2 and 4.3.

Figure 4.1: A screenshot of the OMNeT++ graphical interface

4.1.1 Simulation scenariosFirst of all, common parameters for each scenario should be defined. These parametersare related to network and simulation properties (see Table 4.1). Note, some mobilityproperties are written in a way they appear in OMNeT++. Omitted properties eitherhave no significant influence on the simulation behavior or used with default values.The full version of configurations can be found in file MCDM_common.ini in AppendixA.

Since handover decision are made on STAs (in case of 802.11 network), the algo-rithms are active on each one. In the same time, only one terminal is being analyzed– so-called STA-under-test. Also, one instance of application, which generates tra�c,is active per STA, due to the following reasons:

• use of multiple number of di�erent kinds of application would change decisionstrategy only if 2 types of demanding applications is being used simultaneously,such as streaming and conversational, what is not typical in real-life scenarios.Otherwise, decision strategy is determined by the most demanding type of ap-plication, e.g. between background and conversational, the latter one dominates.


4 Simulation 44

Figure 4.2: A screenshot of an AP’s structure from the INET package

• since adding of additional application would not lead to changing of the decisionstrategy, it influences only via creation of additional tra�c volumes and somechanges in probabilistic properties of the tra�c. Despite these facts, the mainconcern is related to possible changes of the decision strategies, therefore, thefact of changing of tra�c-properties can be neglected.

I defined simulation scenarios and di�erentiated them by tra�c patterns and networkload conditions. Their summary is provided in Table 4.2. Further details regardingsimulation scenarios can be found in the OMNeT++ configuration files in AppendixA.2.

From own observations, network-performance degrades significantly with increasingnumber of STAs in OMNeT++. Therefore, the number of STAs, that correspondsto high-load case, is a compromise keeping network’s performance on adequate level,while the di�erence with low load scenario is still quite distinct.

These 4 scenarios are to be carried on for all 5 algorithms (SAW, GRA, VIKOR,TOPSIS, RSS), what gives us 20 simulation cases. These cases also comprise from so-called sub-scenarios. The di�erence between these sub-scenarios comprises in di�erentdistribution of C and Max Bitrate (fixed during simulation) parameters among APs anddi�erent random generator’s seeds (see Table 4.3). These parameters are obtained fromrandomly sorted set of values. Despite the seeds are di�erent for the sub-scenarios, theset of used ones for the scenarios is the same. This gives us comparable simulations,even with MassMobility mobility pattern for background STAs.


4 Simulation 45

Figure 4.3: A screenshot of a STA’s structure from the INET package

4.1.2 The network topologyThe topology is provided in Appendix A.1. It consists of 6 APs, with minimallyrequired backbone set: a switch for interconnection of APs and 2 servers dedicated forgeneration of tra�c. The network has 2 versions: the first one creates low backgroundload and comprises of 5 wireless STAs in total, the latter one is dedicated to producerelatively high load created by 10 wireless STAs. This relatively small number ofSTAs may be explained by limitations of OMNeT++ to correctly handle more realisticnumber of users. The code of corresponding .ini and .ned files is listed in AppendixA.2.

4.1.3 Tra�c generationIn order to produce realistic load by a terminal, probabilistic tra�c models were usedfor packet generation in OMNeT++.

Method of generation of VoIP tra�c was proposed in [35]. Authors claim, thatamong ITU’s codecs, the G.729 is considered as the most widely used. The codec hasthe following possible rates: 6.4 kbps, 8 kbps and 11.8 kbps. The second one is usedfor simulation as the most frequently occurring among examined sessions. Resultsshowed that packet size property tends to be stable during call session, and the onewith size of 144 bytes occurs in majority of the sessions, while inter-packet intervalfollows Gaussian distribution with µ = 20 ms and ‡ ¥ 0.25 ms.


4 Simulation 46

Table 4.1: The common simulation properties for all scenarios in the OMNeT++ setup

Name of parameter Property

Simulation duration, s 100

Start-time fortra�c-generation, s 3

Dimensions of thesimulation area, m 700x700

Number of APs 6

Communication standard IEEE 802.11g

Transmitter power (for allradio-devices), mW 2.0

Mobility properties of theSTA-under-test

TractorMobility: x1 = 100 m, x2 = 650 m,y1 = 180 m, y2 = 600 m, rowCount = 3,

speed = 20 mps

Mobility properties of thebackground STAs

MassMobility: changeInterval = truncnormal(2,0.5) s, changeAngleBy = normal(0, 30) deg,

speed = truncnormal(15, 5) mps

When there is a need to generate IPTV tra�c, things become more complicateddue to several reasons mentioned in [36], [37]: this kind of tra�c usually has variablebitrate, there are noticeable properties of self-similarity, and tra�c has heterogeneousstructure due to presence of di�erent categories of frames (e.g. I-,P-,B-frames). Sincethe goal of modeling is to produce realistic tra�c volumes in the network, while exactcharacteristics are not so important, I defined the following properties for the gen-eration process: normal distribution of packet sizes with µ = 1200 bytes, ‡ = 30,while inter-packet interval also has Gaussian distribution with µ = 4 ms, ‡ = 0.2 msin order to produce ¥ 0.3 Mbps load per stream (due to limitations of OMNeT++performance).

Both these tra�c-types are generated with help of UDPApp modules on source anddestination nodes under assumption, that there is no significant influence of TCPsignaling-packets on the tra�c volumes.


4 Simulation 47

Table 4.2: The simulation scenarios in OMNeT++

Name of scenario Number ofSTAs

Generated tra�c (loadKbps/STA)

Conversational, low load 5 VoIP (≥8)

Streaming, low load 5 IPTV (≥300)

Conversational, high load 10 VoIP (≥8)

Streaming, high load 10 IPTV (≥300)

Table 4.3: The simulation sub-scenarios in OMNeT++

Numberof sub-

scenario

Cost values ofAP1..AP6

Max Bitrate valuesof AP1..AP6

1 4 95 58 27 70 24 6 12 9 11 18 1

2 4 95 24 70 58 27 18 5.5 1 11 9 12

3 95 4 70 58 24 27 6 12 5.5 11 1 9

4 27 4 24 70 58 95 1 2 6 12 11 5.5

5 24 70 58 27 4 95 2 6 1 12 18 5.5

4.1.4 Collection of decision criteriaAccording to criteria selected in Section 2.1, and the ones that are available in OM-NeT++, the following criteria were chosen: RSSI, D, J, Available Bitrate (AB) and C.AB is more specific one than Network load. They will be interchangeable and inverselyproportional if the network nodes have only one operational mode (e.g. 802.11g withfixed static modulation), what is the case in this simulation setup.

The RSSI criterion is initially collected from the Radio module (see the right sideof Figure 4.3). After, RSSI, D, J criteria were gathered in Ieee80211MgmtAP. Addi-tionally, C and AB criteria were implemented in-there. Further they are transmittedto Ieee80211AgentSTA in broadcast management frame. All parameters are gatheredin Ieee80211AgentSTA module, where decisions regarding handover are usually done,and the parameters are further converted into variable of string type.


yevgeniy

„They will be interchangeable“ Are you sure?

yevgeniy

in-there -> in there

4 Simulation 48

4.1.5 Implementation of the MCDM-algorithmsThe string-variable with collected decision values serves as an input for the MCHAs,which are implemented as C++ library for Ieee80211AgentSTA module (see Figure4.4). It is written in object-oriented way, and consists of several modules: the Matrixclass – a basis for linear-algebra functions, which in turn serves as building blocksfor implementation of the MCDM-algorithms, where transformation from string todecision matrix C is also performed (see section 2.2). The code is provided in AppendixB.

OMNeT++:

Ieee80211AgentSTA

string decisionValues, string algorithmName, string trafficType, int criteriaNumber

best APCreation of weight-vector w

with preferencesfrom the

'A$trafficType.dat' file

The implemented library

Selection of the best APs with help of $algorithmName

Creation of object D --normalized version of the

matrix C with parameters from the'enhMaxMin.dat' file

Parsing of decisionValuesto object C of

the Matrix class,considering criteriaNumber

Figure 4.4: Visual description of integration of the MCDM-algorithms

4.1.6 Configuration of the MCHAsAfter implementation of the algorithm’s structure, the ones must be parameterized.

4.1.6.1 The normalization stage

Since Enhanced Max-Min has been selected as a normalization procedure, values uj andlj must be defined. Usually, they should be selected as maximal and minimal feasiblevalues correspondingly in accordance with used access technology. So, the definition ofthese values should be performed for each potential technology individually. Neverthe-less, they could be defined with su�cient gap, but then the scale of normalized valueswould be uneven, what could lead to "bad" decisions in case of simple algorithms, e.g.SAW. In accordance with observations of desired criteria in OMNeT++, the valuesare defined in Table 4.4 and in file enhNorm.dat from Appendix C. Despite such val-ues may be unrealistic in real systems (e.g. uj for D), that fits for given simulationconditions. Also, lj = ≠100 for the RSSI criterion was chosen due to handling of case


4 Simulation 49

with -inf values (produced by OMNeT++) via replacement with ≠100 value. Theimplemented MCDM-algorithms use the above-mentioned file, which is placed in thesame folder, for configuration purposes.

Table 4.4: The normalization parameters for the Enhanced Max-Min method

Name ofcriteria

RSSI,dBm Delay, s Jitter, s

Availablebitrate,

bit/sCost

uj 0 0.1 0.1 60000000 100

lj -100 0 0 0 0

Type Upward Downward Downward Upward Downward

4.1.6.2 The weighting stage

Assuming available criteria, generalized form of the weight-vector looks as follows:

w =5RSSI Delay Jitter Available bitrate Cost

6T

(4.1)

For the simulation stage, 2 decision strategies were chosen in accordance with application-type – conversational and streaming (see section 2.3). Forth, corresponding matricesof pair-wise comparisons must be constructed (see expression 2.6 for reference) . Thiscorresponds to configuration files Aconv.dat and Astream.dat in Appendix C, whichare placed in the same location as own implementation of the MCDM-algorithms.

The accents during decision stage are made in accordance with [22]. In such decisionstrategies, C parameter is not that important, therefore it gets small weights. Note,on contrary to evaluation focuses, where RSSI parameter may not even exist, decisionstrategy requires this criterion to be taken into account due to its crucial importancefor the connection maintenance.

Conversational tra�c demands D and J to be prioritized along with RSSI:

Aconversational =

S

WWWWWWWWWWWWU

1 1 1 3 1

1 1 1 3 7

1 1 1 3 5

1/3 1/3 1/3 1 5

1 1/7 1/5 1/5 1

T

XXXXXXXXXXXXV


yevgeniy

unit/dimension for Cost is missing

4 Simulation 50

In case of streaming tra�c, AB and J criteria gets the highest priority, while RSSIalso has high importance. The matrix A looks as follows:

Astreaming =

S

WWWWWWWWWWWWU

1 9 1 1 1

1/9 1 1/9 1/7 1/7

1 9 1 1 5

1 7 1 1 5

1 7 1/5 1/5 1

T

XXXXXXXXXXXXV

The weight-vectors are calculated from these matrices in accordance with WLSmethod 2.9:

wconversational =50.248 0.307 0.295 0.103 0.047

6T

wstreaming =50.260 0.031 0.321 0.312 0.076

6T

4.1.6.3 The ranking stage

All algorithms but VIKOR require no configuration. As for the latter one, there isa weight-parameter, which defines decision strategy – v (see section 2.4.3). Duringsimulation, this parameter was set as v = 0.4 due to ability to specify own decisionstrategy, what was induced by the authors of the algorithm.

4.1.7 Processing of resultsThe process involves 3 steps:

1. Conversion of simulation output .vec files into .csv format by means of scavetool– provided with OMNeT++ package.

2. Extraction of desired parameters from .csv via Matlab software into evaluationmatrices E, with preliminary averaging of the criteria values.

3. Processing of the matrices with help of the proposed evaluation algorithm inMatlab.

The process is displayed on Figure 4.5. The code of used Bash and Matlab scriptsis provided in Appendix C.


4 Simulation 51

OMNeT++simulations

Conversionto .csvformat

Processing of the resultsin Matlab

Evaluationof MCHAs

.vec .csv

SE

Figure 4.5: Visual description of processing of the simulation results

4.2 Results of the simulationsAfter the simulation setup stage, I simulated all above-mentioned scenarios and theirresults are presented in this section. Firstly, I selected among collected parameters theones of the most interest, which were further visualized on charts in order to figureout influence of the algorithms on the evaluation criteria. Then these criteria are tobe evaluated by means of the proposed evaluation approach with particular evaluationfocuses, with an eye to reveal the best MCDM-algorithm in the handover context.

According to results presented in section 2.5, I expect GRA and VIKOR to takeleading positions in the results of the evaluation.

4.2.1 Collected simulation valuesSince there are several dozens criteria available in .vec-files after simulation, we usuallydon’t need them all. The most valuable among collected criteria are: AB (on AP),T, D, J, C and Number of HO. As we see, this set of parameters di�ers from theone for the decision stage. Resulting simulation values were averaged and normalized(dividing by the maximal value) for better visibility, they are presented on Figures4.6 and 4.7. For consistency of the evaluation process, PL parameter was additionallygenerated as a random value with small variance (see parse function in Appendix C).The unnormalized, but averaged, collected values can be found in Appendix D.


yevgeniy

what does the empty block mean?

4 Simulation 52

Figure 4.6 demonstrates normalized collected criteria for conversational and stream-ing tra�c scenario with low load. To begin, in both scenarios amount of AB wasnearly the same almost for all algorithms, one of possible reasons is averaging of theresults of the sub-scenarios. The T parameter was at one-third higher for RSS andVIKOR than for others during VoIP scenario, but the IPTV one bringed TOPSIS tothe leaders, putting it on the same level as RSS, but still much lower than VIKOR. Dand J criteria are better (lower) for MCHAs in mostly all cases, except TOPSIS withVoIP tra�c scenario. As we see J values are in several times larger than D ones, whatis not true in a real system. However, additional investigations showed, that this canbe explained by incorrect retrieving of J parameter in OMNeT++ software (at leastin the used version). There is no need to analyze PL criteria since it was generated ina random way. As for C criterion, RSS and VIKOR showed the highest values amongthe others. At the same time, HO number was the highest for RSS in VoIP scenario,while for the case of IPTV tra�c TOPSIS and VIKOR performed more HOs than theothers.

High loaded scenario is shown on Figure 4.7. The tendency was mostly the samefor AB and C parameters. However, T values were not so spreaded as before, theywent much closer for the algorithms. D and J were much higher for RSS, TOPSIS andVIKOR in case of voice tra�c. At the same time, RSS, SAW and GRA demonstratedthe maximal values for D in the video one. Going further, I noticed that VIKORexecuted more HOs than others, while RSS and TOPSIS were showing the secondhighest number of HOs in VoIP and IPTV scenarios correspondingly.

After all, I can say that level of AB was decreasing from scenario to scenario dueto increasing amount of generated tra�c, this is logical. Moreover, larger numberof STAs creates more uniform load on APs in comparison with the low load scenario,what leaves less opportunities for optimization of some criteria (e.g. T) during networkselection procedure. Also, I need to admit high correlation of D and J criteria in allscenarios. Therefore, it might have sense to omit one of them during collection processfor the sake of simplification.

As you see, analyzing of the charts in such way might lead us to wrong judgments,due to big amount of data to consider. Therefore, an evaluation approach, whichsimplifies this work, should be applied.


4 Simulation 53

Availablebitrate

/53.21 Mbps

Troughput/0.61 Mbps

Delay/7.0 ms

Jitter/31.9 ms

Packetloss

/0.138 %

Cost/43.09

HO number/12.4

Normalized

values,VoIPlow

load

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RSS SAW GRA TOPSIS VIKOR

(a) Conversational tra�c

Availablebitrate

/53.09 Mbps

Troughput/0.67 Mbps

Delay/12.2 ms

Jitter/74.4 ms

Packetloss

/0.134 %

Cost/50.52

HO number/17.2

Normalized

values,IP

TV

low

load

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


(b) Streaming tra�c

Figure 4.6: Collected normalized values in the low load scenario


4 Simulation 54

Availablebitrate

/52.45 Mbps

Troughput/0.62 Mbps

Delay/7.2 ms

Jitter/32.9 ms

Packetloss

/0.134 %

Cost/51.92

HO number/20.4

Normalized

values,VoIPhighload

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


(a) Conversational tra�c

Availablebitrate

/52.36 Mbps

Troughput/0.54 Mbps

Delay/7.1 ms

Jitter/36.9 ms

Packetloss

/0.135 %

Cost/53.09

HO number/14.2

Normalized

values,IP

TV

highload

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


(b) Streaming tra�c

Figure 4.7: Collected normalized values in the high load scenario


4 Simulation 55

4.2.2 Application of the proposed evaluation algorithmIn order to correctly and precisely estimate results produced by the algorithms, anevaluation approach is required. Therefore, the evaluation of the results presented inthe previous section is performed below. I will use the approach proposed and describedin section 3.2. For that, exact definition of evaluation focuses and classification ofexisting parameters are required.

The main focus of the work lies in evaluation from the user perspective. For that,Device, Network and Application sets (see Table 3.2) should be composed and analyzedwith further aggregation, if needed. Therefore, di�erent aggregation strategies may beapplied from Table 3.5, as well as di�erent evaluation focuses in these sets (see Table3.3 as an example for Network set). However, this is doable only if correspondingcriteria are available. The ones are presented and classified by per-set basis in Table4.5.

Table 4.5: Classification of the simulated evaluation parametersSet Collected criteria

Network Available bitrate, Cost, QoSNetwork{QoS} Throughput, Delay, Jitter, Packet loss

HO Number of HODevice —

Application —

As we see from the table, there are no available criteria for Device and Applicationsets, although using of QoE would be a desirable option (reasons are mentioned insection 3.2.2). Even though HO set is available, it is not interesting for assessmentfrom the user perspective. Of course, this set may a�ect the user’s experience implicitly,but the set influence on QoS set as well. Therefore, I consider HO set as a redundantone for achieving of the above-mentioned goal.

Due to definition of the evaluation approach, we can immediately retrieve evaluationscores for the QoS set (see Appendix D).

Having only Network set and its QoS subset, there still are several options forevaluation focuses: Network loading, Money spendings, Secureness and QoS (accordingto Table 3.3). And again, the S parameter has not been simulated, hence, the weight-vector for the Network set must be reconfigured. A general representation of weight-vector for the Network set in this case:

wevalNetwork =

5Available bitrate Cost QoS

6T

(4.2)


4 Simulation 56

So, 3 options left: Network loading, Money spendings and QoS. Moreover, Networkloading isn’t of the user’s concern, thus it won’t be further considered. Since, withoutany doubts, decision strategy and evaluation focuses are correlated to some extent, theevaluation focus on QoS set is our primal interest (due to streaming and conversationaldecision strategies defined in section 4.1.6.2). However, there may be a case, whendecision and evaluation focuses are di�erent. This situation should not be treated assomething unusual. For instance, we want to estimate how average Money spendingswill be changed, if QoS-oriented decision strategy is changed to Cost-oriented andvice-versa. Another example, we want simply to estimate how the assessed algorithmswill a�ect on Money spendings, if the decision strategy is quality-oriented. And thiswill be discussed later on.

4.2.2.1 QoS evaluation focus

Here, I define the evaluation focus on the QoS-set. The corresponding matrix of pair-wise comparisons for vector 4.2 is:

AQoS =

S

WWWWU

1 0 1/9

0 1 1/9

9 9 1

T

XXXXV,

which yields to the following weight-vector:

wevalNetwork =

50.09 0.09 0.82

6T

Finally, evaluation score for all 4 scenarios are represented on Figure 4.8. As wesee from the chart, SAW and GRA demonstrated the highest criticality scores in highload conditions, while in voice scenario with low load they shared the leading positionwith VIKOR. Only in IPTV low-loaded scenario, TOPSIS and VIKOR pushed themdown. At the same time, RSS was an obvious outsider.

4.2.2.2 Cost evaluation focus

Since on the decision stage priority was given to QoS parameters, evaluation resultsfor the MCDM-algorithms are not expected to be better than for the RSS algorithmin case of focus on the C parameter.

Nevertheless, I constructed the following matrix of pair-wise comparisons with anaccent on C:


4 Simulation 57

VoIP (low load) IPTV (low load) VoIP (high load) IPTV (high load)

TheevaluationscoreSE

0

0.1

0.2

0.3

0.4

0.5

0.6


Figure 4.8: The aggregated output of the proposed evaluation algorithm with QoSfocus

ACost =

S

WWWWU

1 1/9 0

9 1 9

0 1/9 1

T

XXXXV,

with corresponding weight-vector:

wevalNetwork =

50.09 0.82 0.09

6T

Obtained evaluation scores for all 4 scenarios are shown on Figure 4.9. The situationwas changed in comparison with QoS focus, since VIKOR demonstrated similar scoresto RSS, and both were outsiders. While SAW and GRA were taking leading position,TOPSIS accompanied them in IPTV with low load and VoIP with high load cases.

The results demonstrated, that multi-criteria algorithms are superior to conventionalone in all verified cases. The MCDM-algorithms are aware of network’s current ap-plication and this allows formation of proper decision strategy, which leads to betteruser experience. It can be confidently said, that a MCDM-algorithm will show thehighest evaluation results if the same criteria are prioritized during both decision andevaluation stages.

SAW and GRA shows themselves as the best algorithms in 3 out of 4 scenarios with


4 Simulation 58

VoIP (low load) IPTV (low load) VoIP (high load) IPTV (high load)

TheevaluationscoreSE

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7


Figure 4.9: The aggregated output of the proposed evaluation algorithm with Costfocus

QoS focus and in all scenarios with Cost focus. These results match with my initialsuggestion of the decision scheme. SAW and GRA show mostly identical results in allcases, what testifies about their similar behavior during handover decision procedure.At the same time, TOPSIS and VIKOR produced various evaluation scores, thus stillbetter than RSS. Moreover, the superiority of GRA and SAW was also shown in myprevious work [1]. However, the results, what were gained in this work, can not bedirectly compared to others, due to usage of mixed evaluation criteria in-there andnovelty of the proposed evaluation approach.


yevgeniy

The results in the sense which algorithms fits better for which scenario can and shall be compared with other research works.

5 Conclusions 59

5 Conclusions

I reviewed and analyzed the main axes in multi-criteria decision making in handovercontext: decision and evaluation stages.

Firstly, I provided discussion of criteria selection problems, applicability of di�erentnormalization techniques, approaches for weighting of criteria and ranking algorithmswith help of MCDM-algorithms in Chapter 2. As the chapter’s output I proposedthe following decision scheme: Enhanced Max-Min, WLS, GRA as normalization,weighting and ranking algorithms correspondingly.

Secondly, For the sake of assessing of MCDM-algorithms, I developed an evaluationapproach in Chapter 3. Classification of evaluation criteria by set basis is provided inthe approach. Also, examples of parametrization for set-weighting was provided, aswell as possible methods for estimation of QoE for di�erent types of tra�c. Despite,resulting structure of the proposed method is more complex, it is more functional incomparison with criticality analysis and SDOI methods. Finally, among the improve-ments are removing of mapping procedure, usage of classified structure of the criteriaand simplified expression for calculation of evaluation scores. This leads to abilityto use precise per-set evaluation results and construct of own flexible scheme fromrequired criteria-set.

Finally, in the practical part (see Chapter 4), I verified the proposed decision schemewith several options for ranking stage (SAW, GRA, TOPSIS and VIKOR) by means ofdeveloped evaluation approach with accents on QoS and Cost parameters sequentially.The simulation results demonstrated, that SAW and GRA have better performance inmajority of the evaluation scenarios than others, what plays in support of my initialproposal. Although SAW demonstrated high performance, it should be used withcaution, since improper parametrization of the Enhanced Max-Min method may leadto wrong results. Thus, I recommend GRA as a ranking algorithm. However, itshould be stressed, that results of algorithm’s evaluation are usually unpredictable,since investigators in this field use di�erent simulation test-beds, di�erent evaluationapproaches. Moreover, there are too much a�ecting factors, even within the sametest-bed, which can lead to completely di�erent outputs, since we are dealing with an


yevgeniy

Why do you think GRA delivered the best results?

5 Conclusions 60

extremely complex system. Therefore, the matter of selection of the "best" MCDM-algorithm should not be top-priority during implementation in a real system. Muchmore attention should be dedicated to other aspects, such as correct selection andcollection of decision criteria, designing of suitable weights for the criteria with regardto user’s needs. In fact, yet the level of importance of influencing factors and thesystem’s aspects during network-selection process is to be determined.


61

Appendix


A Simulation setup in OMNeT++ 62

A Simulation setup in OMNeT++

A.1 The network topologies

Figure A.1: The network topology for the low load scenario



Figure A.2: The network topology for the high load scenario

A.2 Configuration filesThe MCDM_handover_low.ned file contains description of the MCDM_handover_low net-work, where 5 users are active://

// Copyright (C) 2006 Andras Varga

//

// This library is free software, you can redistribute it and/or

// modify it under the terms of the GNU Lesser General Public License

// as published by the Free Software Foundation;

// either version 2 of the License, or any later version.

// The library is distributed in the hope that it will be useful,

// but WITHOUT ANY WARRANTY; without even the implied warranty of

// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

// See the GNU Lesser General Public License for more details.

//

package inet.examples.wireless.handover;

import inet.examples.adhoc.hostautoconf.Host;

import inet.examples.inet.ipv4hook.MyRouter;

import inet.examples.wireless.dynamic.DynamicHost;

import inet.node.ethernet.Eth100G;



import inet.node.wireless.AccessPoint;

import inet.node.xmipv6.CorrespondentNode6;

import inet.node.xmipv6.MobileHost6;

import inet.physicallayer.contract.packetlevel.IErrorModel;

import inet.physicallayer.ieee80211.packetlevel.Ieee80211ScalarRadioMedium;

import inet.networklayer.configurator.ipv4.IPv4NetworkConfigurator;

import inet.node.ethernet.EtherSwitch;

import inet.node.inet.Router;

import inet.node.inet.StandardHost;

import inet.node.inet.WirelessHost;

import inet.physicallayer.common.packetlevel.RadioMedium;

import ned.DatarateChannel;

//import inet.examples.power

network MCDM_Handover_low

{

@display("bgb=700,700");

//string errorModelType = default("");

types:

channel fiberline extends DatarateChannel

{

delay = 1us;

datarate = 512Mbps;

}

channel ethline extends DatarateChannel

{

delay = 50ns;

datarate = 100Mbps;

}

submodules:

server: StandardHost {

@display("p=296,625;i=device/server_l");

}

configurator: IPv4NetworkConfigurator {

@display("p=48,23");

}

routerServerFarm: Router {

@display("p=353,521");

}

server2: StandardHost {

@display("p=386,625;i=device/server;is=l");

}

radioMedium: Ieee80211ScalarRadioMedium {

@display("p=148,38");

}

switch: EtherSwitch {

@display("p=353,390");

}

AP1: AccessPoint {

@display("p=222,112");

}

AP2: AccessPoint {

@display("p=222,362");

}



AP3: AccessPoint {

@display("p=222,612");

}

AP4: AccessPoint {

@display("p=472,112");

}

AP5: AccessPoint {

@display("p=472,362");

}

AP6: AccessPoint {

@display("p=472,612");

}

wirelessClient: WirelessHost {

@display("p=138,161");

}

wirelessClient1: WirelessHost {

@display("p=293,566");

}


@display("p=197,280");

}


@display("p=442,253");

}


@display("p=472,479");

}

connections:

switch.ethg++ <--> Eth100G <--> routerServerFarm.ethg++;

server.ethg++ <--> Eth100G <--> routerServerFarm.ethg++;

server2.ethg++ <--> Eth100G <--> routerServerFarm.ethg++;

AP1.ethg++ <--> ethline <--> switch.ethg++;

switch.ethg++ <--> ethline <--> AP2.ethg++;





}

The MCDM_handover_high.ned file contains description of the MCDM_handover_high

network, where 10 users are active://

// Copyright (C) 2006 Andras Varga

//

// This library is free software, you can redistribute it and/or

// modify it under the terms of the GNU Lesser General Public License

// as published by the Free Software Foundation;

// either version 2 of the License, or any later version.

// The library is distributed in the hope that it will be useful,

// but WITHOUT ANY WARRANTY; without even the implied warranty of

// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

// See the GNU Lesser General Public License for more details.

//



package inet.examples.wireless.handover;

import inet.examples.adhoc.hostautoconf.Host;

import inet.examples.inet.ipv4hook.MyRouter;

import inet.examples.wireless.dynamic.DynamicHost;

import inet.node.ethernet.Eth100G;

import inet.node.wireless.AccessPoint;

import inet.node.xmipv6.CorrespondentNode6;

import inet.node.xmipv6.MobileHost6;

import inet.physicallayer.contract.packetlevel.IErrorModel;

import inet.physicallayer.ieee80211.packetlevel.Ieee80211ScalarRadioMedium;

import inet.networklayer.configurator.ipv4.IPv4NetworkConfigurator;

import inet.node.ethernet.EtherSwitch;

import inet.node.inet.Router;

import inet.node.inet.StandardHost;

import inet.node.inet.WirelessHost;

import inet.physicallayer.common.packetlevel.RadioMedium;

import ned.DatarateChannel;

//import inet.examples.power

network MCDM_Handover_high

{

@display("bgb=700,700");

//string errorModelType = default("");

types:

channel fiberline extends DatarateChannel

{

delay = 1us;

datarate = 512Mbps;

}

channel ethline extends DatarateChannel

{

delay = 50ns;

datarate = 100Mbps;

}

submodules:

server: StandardHost {

@display("p=296,625;i=device/server_l");

}

configurator: IPv4NetworkConfigurator {

@display("p=48,23");

}

routerServerFarm: Router {

@display("p=353,521");

}

server2: StandardHost {

@display("p=386,625;i=device/server;is=l");

}

radioMedium: Ieee80211ScalarRadioMedium {

@display("p=148,38");

}

switch: EtherSwitch {



@display("p=353,390");

}

AP1: AccessPoint {

@display("p=222,112");

}

AP2: AccessPoint {

@display("p=222,362");

}

AP3: AccessPoint {

@display("p=222,612");

}

AP4: AccessPoint {

@display("p=472,112");

}

AP5: AccessPoint {

@display("p=472,362");

}

AP6: AccessPoint {

@display("p=472,612");

}

wirelessClient: WirelessHost {

@display("p=151,152");

}


@display("p=239,252");

}


@display("p=338,283");

}


@display("p=452,250");

}


@display("p=261,407");

}

wirelessClientX: WirelessHost {

@display("p=452,489");

}

wirelessClientX1: WirelessHost {

@display("p=222,540");

}


@display("p=344,440");

}


@display("p=353,192");

}


@display("p=464,311");

}

connections:

switch.ethg++ <--> Eth100G <--> routerServerFarm.ethg++;

server.ethg++ <--> Eth100G <--> routerServerFarm.ethg++;

server2.ethg++ <--> Eth100G <--> routerServerFarm.ethg++;




switch.ethg++ <--> ethline <--> AP2.ethg++;





}

The MCDM_common.ini file contains common configuration for both MCDM_handover_low

and MCDM_handover_high networks:[General]

sim-time-limit = 100s #100 s

cpu-time-limit = 800s

total-stack = 200MiB

tkenv-plugin-path = ../../../etc/plugins

record-eventlog = false

debug-on-errors = true

**.wirelessClient.**.vector-recording = true

**.vector-recording = false

**.scalar-recording = false

seed-set=100

**.constraintAreaMinX = 0m

**.constraintAreaMinY = 0m

**.constraintAreaMinZ = 0m

**.constraintAreaMaxX = 700m

**.constraintAreaMaxY = 700m

**.constraintAreaMaxZ = 0m

**.updateInterval = 0.1s # test with 0s too, and let getCurrentPosition update the display string from a test

module

#**.mobility.initFromDisplayString = false

#-----------------------------------------------------

[Config AppStream]

**.server.numUdpApps = 1

**.server.udpApp[0].typename = "UDPSink"

**.server.udpApp[0].localPort = 1000

**.wirelessClient*.numUdpApps = 1

**.wirelessClient*.udpApp[0].typename = "UDPBasicApp"

**.wirelessClient*.udpApp[0].destAddresses = "server"

**.wirelessClient*.udpApp[0].destPort = 1000

**.wirelessClient*.udpApp[0].messageLength = normal(1200 B, 30 B) #to review

**.wirelessClient*.udpApp[0].sendInterval=normal(4 ms, 0.2 ms)

**.wirelessClient*.udpApp[0].startTime = 3s

extends = Basic

[Config AppConv]

**.server.numUdpApps = 1

**.server.udpApp[0].typename = "UDPSink"

**.server.udpApp[0].localPort = 1000

**.wirelessClient*.numUdpApps = 1

**.wirelessClient*.udpApp[0].typename = "UDPBasicApp"



**.wirelessClient*.udpApp[0].destAddresses = "server"

**.wirelessClient*.udpApp[0].destPort = 1000

**.wirelessClient*.udpApp[0].messageLength = 144 B

**.wirelessClient*.udpApp[0].sendInterval=normal(20 ms, 0.25 ms)

**.wirelessClient*.udpApp[0].startTime = 3s

extends = Basic

#-----------------------------------------------------------------------------------

[Config Basic]

**.wirelessClient*.wlan[*].agent.critNumb = 5

# channel physical parameters

**.mgmt.numChannels = 14

#access points

# supported bitrates for 802.11g: 1,2,5.5,6,9,11,12,18 ,24,34,48,54

#Random sorted bitrate:

#https://www.random.org/lists/

#6 12 9 11 18 1 5.5 2

#18 5.5 1 11 9 12 2 6

#6 12 5.5 11 1 9 2 18

#1 2 6 12 11 5.5 9 18

#2 6 1 12 18 5.5 11 9

#Random prices

#4 95 58 27 70 24

#4 95 24 70 58 27

#95 4 70 58 24 27

#27 4 24 70 58 95

#24 70 58 27 4 95

**.AP1.wlan[*].mac.address = "01:00:00:00:00:00"

**.AP1.wlan[*].mgmt.ssid = "MYSSID"

**.AP1.wlan[*].mgmt.cost=24

**.AP1.wlan[*].mgmt.bitrate=2 Mbps

#**.AP1.wlan[*].radio.channelNumber = 1

**.AP1.wlan[*].radio.channelNumber = 5






























**.AP6.wlan[*].mgmt.bitrate=5.5 Mbps



#**.AP*.wlan[*].mgmt.beaconInterval = 100ms

**.AP*.wlan[*].mgmt.beaconInterval =50ms

#**.wlan[*].mgmt.numAuthSteps = 4

**.mgmt.frameCapacity = 10

#**.wlan[*].agent.activeScan = true

**.wlan[*].agent.default_ssid = "MYSSID"

#**.wlan[*].agent.default_ssid = ""

**.wlan[*].agent.channelsToScan = "1 5 9 13" # "" means all

#**.wlan[*].agent.probeDelay = 0.1s

**.wlan[*].agent.minChannelTime = 60ms

**.wlan[*].agent.maxChannelTime = 70ms

**.wlan[*].mac.throughputTimePeriod = 2s

#**.wlan[*].agent.authenticationTimeout = 5s

#**.wlan[*].agent.associationTimeout = 5s

# EEW

**.wirelessClient*.wlan[*].agent.regularScanTimeout = 1

**.radio.transmitter.power = 2.0mW

#mobile node-----------------------------------------------------------

**.wirelessClient.mobilityType = "TractorMobility"

**.wirelessClient.mobility.x1 = 100m

**.wirelessClient.mobility.y1 = 180m

**.wirelessClient.mobility.x2 = 650m

**.wirelessClient.mobility.y2 = 600m

**.wirelessClient.mobility.rowCount = 3

**.wirelessClient.mobility.speed = 20mps

**.wirelessClient*.mobilityType = "MassMobility"

**.wirelessClient*.mobility.initFromDisplayString = true

**.wirelessClient*.mobility.changeInterval = truncnormal(2s, 0.5s)

**.wirelessClient*.mobility.changeAngleBy = normal(0deg, 30deg)

**.wirelessClient*.mobility.speed = truncnormal(15mps, 5mps)

#**.wirelessClient*.mobility.speed = truncnormal(150mps, 50mps)

#**.wirelessClient*.mobilityType = "RandomWPMobility"

#**.wirelessClient*.mobility.speed = 10*uniform(20mps,50mps)



#**.wirelessClient*.mobility.waitTime = uniform(3s,8s)

#**.wirelessClient*.mobilityType = "GaussMarkovMobility"

#**.wirelessClient*.mobility.alpha = 0.9

#**.wirelessClient*.mobility.speed = 100mps

#**.wirelessClient*.mobility.angle = 0deg

#**.wirelessClient*.mobility.variance = 40

#**.wirelessClient*.mobility.margin = 30m

#**.numPcapRecorders = 1

#**.wirelessClient.pcapRecorder[0].pcapFile="results/clientLog.pcap"

#**.server*.pcapRecorder[0].pcapFile="results/serverLog.pcap"

# Ethernet switch

**.switch*.relayUnitType = "MACRelayUnit"

**.relayUnit.addressTableSize = 100

**.relayUnit.agingTime = 120s

**.relayUnit.bufferSize = 1MiB

**.relayUnit.highWatermark = 512KiB

**.relayUnit.pauseUnits = 300 # pause for 300*512 bit (19200 byte) time

**.relayUnit.addressTableFile = ""

**.relayUnit.numCPUs = 2

**.relayUnit.processingTime = 2us

The MCDM_handover_low.ini file contains specific configuration for the MCDM_handover_low network:include MCDM_common.ini

[General]

network = MCDM_Handover_low

[Config RSS-stream-low]

**.wirelessClient*.wlan[*].agent.decisionAlgorithm = "RSS"

extends = AppStream

[Config RSS-conv-low]


extends = AppConv

[Config SAW-stream-low]

**.wirelessClient.wlan[*].agent.decisionAlgorithm = "SAW"

**.wirelessClient*.wlan[*].agent.decisionAlgorithm = "SAW"

**.wirelessClient*.wlan[*].agent.trafficType = "stream"

extends = AppStream

[Config SAW-conv-low]



**.wirelessClient*.wlan[*].agent.trafficType = "conv"

extends = AppConv

#--------------------------------------------------------------

[Config GRA-stream-low]

**.wirelessClient.wlan[*].agent.decisionAlgorithm = "GRA"

**.wirelessClient*.wlan[*].agent.decisionAlgorithm = "GRA"




extends = AppStream

[Config GRA-conv-low]




extends = AppConv

#--------------------------------------------------------------

[Config TOPSIS-stream-low]

**.wirelessClient.wlan[*].agent.decisionAlgorithm = "TOPSIS"

**.wirelessClient*.wlan[*].agent.decisionAlgorithm = "TOPSIS"


extends = AppStream

[Config TOPSIS-conv-low]




extends = AppConv

#--------------------------------------------------------------

[Config VIKOR-stream-low]

**.wirelessClient.wlan[*].agent.decisionAlgorithm = "VIKOR"

**.wirelessClient*.wlan[*].agent.decisionAlgorithm = "VIKOR"


extends = AppStream

[Config VIKOR-conv-low]




extends = AppConv

The MCDM_handover_high.ini file contains specific configuration for the MCDM_ handover_high

network:include MCDM_common.ini

[General]

network = MCDM_Handover_high

[Config RSS-stream-high]


extends = AppStream

[Config RSS-conv-high]


extends = AppConv

[Config SAW-stream-high]




extends = AppStream



[Config SAW-conv-high]




extends = AppConv

#--------------------------------------------------------------

[Config GRA-stream-high]




extends = AppStream

[Config GRA-conv-high]




extends = AppConv

#--------------------------------------------------------------

[Config TOPSIS-stream-high]




extends = AppStream

[Config TOPSIS-conv-high]




extends = AppConv

#--------------------------------------------------------------

[Config VIKOR-stream-high]




extends = AppStream

[Config VIKOR-conv-high]




extends = AppConv

The simul.sh bash-script for automated simulation of the Omnet++ scenarios wasdeveloped, where conversion of the .vec results to .csv format was also implemented.There was an an error occurring during the simulations, for example:<!> Error in module (inet::ieee80211::Ieee80211MgmtSTA) MCDM_Handover_high.wirelessClientX4.wlan[0].mgmt (id

=431) at event #443408, t=5.700099400412: handleAssociationResponseFrame: AP not known: address

=02-00-00-00-00-00.

So, I implemented some workaround, where console output was being monitored,and in case of error-message the simulation was restarted with slightly changed initial



position of the problematic wirelessClient** on a random value in the range from 0to 50 meters. The simulation was looped until it was finished successfully. The erroroccurred in every, roughly, second simulation and from 1 to 5 repetitions was enough.The code of the script is provided below.

1 #!/bin/bash

2

3 cd /home/radion/omnetpp-4.6/inet/examples/wireless/handover

4 # 0 1 2 3 4

5 algos=(RSS SAW GRA TOPSIS VIKOR )

6 numberAlgos=${#algos[*]}

7

8 trafficType=(conv stream)

9 numberTraff=${#trafficType[*]}

10

11 load=(low high)

12 numberLoad=${#load[*]}

13

14 #update 2 scenario!!

15 #define bitrate in Mbps and cost for each AP as an array for each sub-scenario

16 AP1_cost=(4 4 95 27 24)

17 AP1_bit=(6 18 6 1 2)

18 AP2_cost=(95 95 4 4 70)

19 AP2_bit=(12 5.5 12 2 6)

20 AP3_cost=(58 24 70 24 58)

21 AP3_bit=(9 1 5.5 6 1)

22 AP4_cost=(27 70 58 70 27)

23 AP4_bit=(11 11 11 12 12)

24 AP5_cost=(70 58 24 58 4)

25 AP5_bit=(18 9 1 11 18)

26 AP6_cost=(24 27 27 95 95)

27 AP6_bit=(1 12 9 5.5 5.5)

28 # seeds for each subscenario

29 seedsSim=(18 21 3 4 100)

30

31 numberSim=${#seedsSim[*]}

32

33 function changeIniFile

34 {

35 #change seed number

36 sed -i -e "s/seed-set.*/seed-set=${seedsSim[$1]}/g" MCDM_common.ini

37 #change cost of APs

38 sed -i "s/.*AP1.*cost.*/**.AP1.wlan[*].mgmt.cost=${AP1_cost[$1]}/" MCDM_common.ini






44 #change bitrate of APs

45 sed -i "s/.*AP1.*bitrate.*/**.AP1.wlan[*].mgmt.bitrate=${AP1_bit[$1]} Mbps/" MCDM_common.ini








51 }

52

53 for (( currentLoad=0; currentLoad<$numberLoad; currentLoad++ ))

54 do

55

56 #let�s make copy of original .ned file

57 if [ ! -d "backUp" ]; then

58 echo Creating BackUps

59 mkdir backUp

60 cp MCDM_Handover_${load[$currentLoad]}.ned backUp/MCDM_Handover_${load[$currentLoad]}

61 else

62 echo BackUp already exists

63 cat backUp/MCDM_Handover_${load[$currentLoad]} > MCDM_Handover_${load[$currentLoad]}.ned

64 fi

65

66 for (( currentTraff=0; currentTraff<$numberTraff; currentTraff++ ))

67 do

68

69 for (( currentAlg=0; currentAlg<$numberAlgos; currentAlg++ ))

70 do

71

72 for (( simul=0; simul<$numberSim; simul++ ))

73 do

74

75 position=()

76 nS=0 #check number of repetitions

77 simulOk=false

78 #reinitilize arrays

79 failedSTA=()

80 #--------------------------------------------------

81 #adjust conf in accordance with simulation number

82 changeIniFile $simul

83 #--------------------------------------------

84 #show current simulation

85 echo �*******� ${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]}, $((simul+1))

scenario �*******�

86

87 #rm results/${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]}-$((simul+1)).*

88

89 until $simulOk

90 do

91 #run simulation and save logs to a file

92 opp_run -r 0 -u Cmdenv -c ${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]} -n

../..:../../../src:../../../tutorials -l ../../../src/INET MCDM_Handover_${load[$currentLoad]}.

ini > results/${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]}-$((simul

+1)).txt

93

94 #if simulation failed, replace seed number with a random one (10-100)

95 if grep -q ERROR results/${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]}-$((

simul+1)).txt

96 then

97 ((nS++))

98 #find the name of failed STA and fill the array

99 failedSTA+=($(cat results/${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[



$currentLoad]}-$((simul+1)).txt | awk -F"." �/wirelessClient/ {print $2}�))

100 echo Repetition due to failure of ${failedSTA[@]} ...

101 #find the coordinates of the STA from .ned files

102 position=($(awk -F"," "/${failedSTA[$nS-1]}/{getline; print $2}" MCDM_Handover_${load[

$currentLoad]}.ned | tr �\"� � � | grep -o �[0-9]*,[0-9]*� | awk -F "," �{print $1, $2}�))

103 #set offset to the position

104 deltaPosition1=$RANDOM; deltaPosition2=$RANDOM

105 let "deltaPosition1%=50" "deltaPosition2%=50"

106 if [ ${position[0]} -gt 350 ]

107 then

108 let "deltaPosition1=deltaPosition1*(-1)"

109 fi

110 if [ ${position[1]} -gt 350 ]

111 then

112 let "deltaPosition2=deltaPosition2*(-1)"

113 fi

114 #slightly change the position

115 sed -i "/${failedSTA[$nS-1]}/{n;s/.*/ @display(\"p=$((${position[0]}+deltaPosition1)),$((${

position[1]}+deltaPosition2))\");/}" MCDM_Handover_${load[$currentLoad]}.ned

116 echo p=$((${position[0]})),$((${position[1]})) �------>� p=$((${position[0]}+deltaPosition1)),$

((${position[1]}+deltaPosition2))

117 else

118 simulOk=true

119 fi

120 done

121

122 if [ $nS -ne 0 ]

123 then

124 #restore the positions after last changes

125 echo Restoring coordinates ...

126 cat backUp/MCDM_Handover_${load[$currentLoad]} > MCDM_Handover_${load[$currentLoad]}.ned

127 echo The simulation has been repeated $nS Times

128 else

129 echo The simulation is completed OK

130 fi

131

132 #rename .vec-file (from .*-0.vec format to -simul.vec) to keep results

133 cd results

134

135 mv ${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]}-0.vec ${algos[$currentAlg]}-${

trafficType[$currentTraff]}-${load[$currentLoad]}-$((simul+1)).vec

136 rm ${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[$currentLoad]}-0.*

137

138 #convert .vec files to .csv format

139 echo Renaming to ..$((simul+1)).vec ... Removing garbage ... converting to .csv ...

140 scavetool vector -p "module(**.wirelessClient.*)" -O ${algos[$currentAlg]}-${trafficType[$currentTraff]}-${

load[$currentLoad]}-$((simul+1)) -F csv ${algos[$currentAlg]}-${trafficType[$currentTraff]}-${load[

$currentLoad]}-$((simul+1)).vec

141

142 cd ..

143 done #close simul loop

144 echo The subscenario loop has been completed.

145

146 done #close algorithm loop

147 echo The algorithm loop has been completed.



148

149 done #close traffic loop

150 echo The traffic loop has been completed.

151

152 done #close load loop

153 echo The load loop has been completed.



B Implementation of the MCHAs

The header file for Matrix class:1 #ifndef MATRIX_H

2 #define MATRIX_H

3

4 #include <vector>

5 #include <iostream>

6 #include <assert.h>

7 #include <cstdlib> // for rand() and srand()

8

9 namespace McdaAlg {};

10 namespace McdaAlg

11 {

12

13 class Matrix

14 {

15 private:

16 typedef std::vector<double> Alternative;

17 // folded vector

18 std::vector<Alternative> decisionMtx;

19 int m_rows;

20 int m_columns;

21 public:

22 //constructors

23 Matrix()

24 {

25 };

26 Matrix(int alter,int crit);

27

28 // access method

29 double& at (int i, int j);

30

31 //fill up a matrix with random numbers generated in a given range

32 void random(int min, int max);

33

34 // method for printing out a matrix

35 void print();

36 //get the size of matrix, transmitting desired dimensionality

37 int size(int dim) const;

38

39 //overload operator (+) using friend function to gain possibility to apply to 2 objects

40 friend Matrix operator+( Matrix mtx1, Matrix mtx2);

41 friend Matrix operator-(Matrix mtx1, Matrix mtx2);

42 friend Matrix operator*(Matrix mtx1, Matrix mtx2);

43 friend Matrix operator*(double a, Matrix mtx);



44 friend Matrix operator/(Matrix mtx, double a);

45 friend Matrix operator/(Matrix mtx1, Matrix mtx2);

46 };

47

48 }

49

50 #endif // MATRIX_H

The source file for Matrix class:1 #include "Matrix.h"

2

3 /*

4

5 This files contains methods, overloaded operators and functions

6 for Matrix class

7

8 */



11 {

12 //constructor and init of 2d vector

13 Matrix::Matrix(int alter,int crit) : decisionMtx(alter,Alternative(crit,0))

14 {

15 m_rows=alter; //obtaining size of the input matrix

16 m_columns=crit;

17 }

18 // access method

19 double& Matrix::at (int i, int j)

20 {

21 return decisionMtx[i][j];

22 }

23

24 // method for printing out a matrix

25 void Matrix::print()

26 {

27 for (int i=0; i<m_rows; i++)

28 {

29 for (int j=0; j<m_columns; j++)

30 {

31 std::cout<<decisionMtx[i][j]<<"\t";

32 }

33 std::cout<<"\n";

34 }

35 std::cout<<"\n";

36 }

37

38 //get the size of matrix, transmitting desired dimensionality

39 int Matrix::size(int dim) const

40 {

41 assert(dim<=2);

42 if (dim==1)

43 return m_rows;

44 else

45 return m_columns;



46 }

47

48 //fill up a matrix with random numbers generated in a given range

49 void Matrix::random(int min, int max)

50 {

51 // static used for efficiency, so we only calculate this value once

52 static const double fraction = 1.0 / (static_cast<double>(RAND_MAX) + 1.0);

53 for (int i=0; i<m_rows; i++)

54 for (int j=0; j<m_columns; j++)

55 {

56 decisionMtx[i][j]=static_cast<int>(rand() * fraction * (max - min + 1) + min);

57 }

58 }

59

60 //overload operator (+)

61 Matrix operator+( Matrix mtx1, Matrix mtx2)

62 {

63 assert((mtx1.Matrix::size(1)-mtx2.Matrix::size(1))==0); // check whether number of rows is the same

64 assert((mtx1.size(2)-mtx2.size(2))==0); // check whether number of columns is the same

65 Matrix Mtx(mtx1.size(1),mtx1.size(2));

66 for (int i=0; i<Mtx.size(1); i++)

67 {

68 for (int j=0; j<Mtx.size(2); j++)

69 {

70 Mtx.at(i,j)=mtx1.at(i,j)+mtx2.at(i,j);

71 }

72 }

73 return Mtx;

74 }

75

76 //overload operator (-)

77 Matrix operator-( Matrix mtx1, Matrix mtx2)

78 {

79 assert((mtx1.Matrix::size(1)-mtx2.Matrix::size(1))==0); // check whether number of rows is the same

80 assert((mtx1.size(2)-mtx2.size(2))==0); // check whether number of columns is the same


82 for (int i=0; i<Mtx.size(1); i++)

83 {

84 for (int j=0; j<Mtx.size(2); j++)

85 {

86 Mtx.at(i,j)=mtx1.at(i,j)-mtx2.at(i,j);

87 }

88 }

89 return Mtx;

90 }

91

92 //overload operator (*)

93 Matrix operator*( Matrix mtx1, Matrix mtx2)

94 {

95 assert((mtx1.size(2)-mtx2.size(1))==0); // check whether number of columns of the 1 is = to number of rows

of the 2


97 for (int i=0; i<Mtx.size(1); i++) //rows of the 1

98 for (int j=0; j<Mtx.size(2); j++) //columns of the 2

99 for(int k=0; k<mtx1.size(2); k++) //columns of the 1



100 {

101 Mtx.at(i,j)+=mtx1.at(i,k)*mtx2.at(k,j);

102 }

103 return Mtx;

104 }

105

106 //overload operator (*)

107 Matrix operator*( double a, Matrix mtx)

108 {

109 Matrix Mtx(mtx.size(1),mtx.size(2));



112 Mtx.at(i,j) = a*mtx.at(i,j);

113 return Mtx;

114 }

115

116 // here "/" operator is overloaded only when the 2 mtx is 1x1.

117 Matrix operator/(Matrix mtx1, Matrix mtx2)

118 {

119 assert( (mtx2.size(1) == 1) && (mtx2.size(2) == 1));




123 Mtx.at(i,j) = mtx1.at(i,j)/ mtx2.at(0,0);

124 return Mtx;

125 }

126

127 Matrix operator/(Matrix mtx, double a)

128 {

129 Matrix Mtx(mtx.size(1),mtx.size(2));



132 Mtx.at(i,j) = mtx.at(i,j)/a;

133 return Mtx;

134 }

135

136 }

The MatrixFunctions.h file is based on the Matrix class:1 #ifndef MATRIXFUNCTIONS_H_INCLUDED

2 #define MATRIXFUNCTIONS_H_INCLUDED

3

4 #include "Matrix.h" //otherwise this won�t work

5 #include <math.h>

6


8 namespace McdaAlg

9 {

10

11 Matrix enterMtx();

12

13 //swap given a 2 indexes of rows of a matrix

14 void swapRows(Matrix &mtx, int row1, int row2,int col);

15



16 //obtain the rank of a matrix

17 int rankOfMatrix(Matrix mtx);

18

19 //calculate determinant of a square matrix

20 double determinant(Matrix a, int n);

21

22 // calculate cofactor of matrix

23 void coFactor(Matrix a,int n, Matrix &b);

24

25 Matrix transpose(Matrix a);

26

27 // calculate inverse of matrix

28 Matrix inv(Matrix a);

29

30 //generate a matrix into 1

31 Matrix ones(int m, int n);

32

33 //nullify all elements but diagonal

34 Matrix diag(Matrix a);

35

36 //generate identity matrix

37 Matrix eye(int n);

38

39 //find the max element of a matrix in a given row or column (1)

40 int maxIndex(Matrix a, int place, std::string dim);

41

42 //find the min element of a matrix in a given row (2) or column (1)

43 int minIndex(Matrix a, int place, std::string dim);

44

45 //find the sum of elements of a matrix in a given row or column (1)

46 double sum(Matrix a, int place, std::string dim);

47


49 double maxElement(Matrix a, int place, std::string dim);

50

51 //find the min element of a matrix in a given row or column

52 double minElement(Matrix a, int place, std::string dim);

53

54 //find the index of a second max element of a matrix in a given row or column (1)

55 int secondMaxIndex(Matrix a, int place, std::string dim);

56

57 } //end of namespace McdaAlg

58

59 #endif // MATRIXFUNCTIONS_H_INCLUDED

60 //

The MatrixFunctions.cc file is based on the Matrix class:1 #include "MatrixFunctions.h"

2


4 namespace McdaAlg

5 {

6 Matrix enterMtx()

7 {



8 std::cout<<"Enter dimensionality m x n: \t";

9 int m, n;

10 std::cin>>m>>n;

11 Matrix A(m,n);

12 std::cout<<"\n------"<<n<<"x"<<m<<"----------------\n";

13 for(int i=0; i<m; i++)

14 for(int j=0; j<n; j++)

15 {

16 std::cout<<" Matrix["<<i+1<<"]["<<j+1<<"] : ";

17 std::cin>>A.at(i,j);

18 }

19 std::cout<<"\n----------------------------------------------------\n";

20 return A;

21 }

22

23 //swap given a 2 indexes of rows of a matrix

24 void swapRows(Matrix &mtx, int row1, int row2,int col)

25 {

26 for(int i=0; i<col; i++)

27 {

28 int temp = mtx.at(row1,i);

29 mtx.at(row1,i)=mtx.at(row2,i);

30 mtx.at(row2,i)=temp;

31 }

32 }

33

34 //obtain the rank of a matrix

35 int rankOfMatrix(Matrix mtx)

36 {

37 int C=mtx.size(2);

38 int R=mtx.size(1);

39 int rank = C;

40

41 for (int row=0; row<rank; row++)

42 {

43 // Before we visit current row �row�, we make

44 // sure that mat[row][0],....mat[row][row-1]

45 // are 0.

46

47 // Diagonal element is not zero

48 if (mtx.at(row,row))

49 {

50 for (int col=0; col<R; col++)

51 {

52 if(col!=row)

53 {

54 // This makes all entries of current

55 // column as 0 except entry �mat[row][row]�

56 double mult = mtx.at(col,row) / mtx.at(row,row);

57 for (int i = 0; i < rank; i++)

58 mtx.at(col,i) -= mult * mtx.at(row,i);

59 }

60 }

61 }

62 // Diagonal element is already zero. Two cases



63 // arise:

64 // 1) If there is a row below it with non-zero

65 // entry, then swap this row with that row

66 // and process that row

67 // 2) If all elements in current column below

68 // mat[r][row] are 0, then remvoe this column

69 // by swapping it with last column and

70 // reducing number of columns by 1.

71 else

72 {

73 bool reduce = true;

74

75 /* Find the non-zero element in current

76 column */

77 for (int i = row + 1; i < R; i++)

78 {

79 // Swap the row with non-zero element

80 // with this row.

81 if (mtx.at(i,row))

82 {

83 swapRows(mtx, row, i,rank);

84 reduce = false;

85 break ;

86 }

87 }

88 // If we did not find any row with non-zero

89 // element in current columnm, then all

90 // values in this column are 0.

91 if (reduce)

92 {

93 // Reduce number of columns

94 rank--;

95

96 // Copy the last column here

97 for (int i = 0; i < R; i ++)

98 mtx.at(i,row) = mtx.at(i,rank);

99 }

100

101 // Process this row again

102 row--;

103 }

104 }

105 return rank;

106 }

107

108 //calculate determinant of a square matrix

109 double determinant(Matrix a, int n)

110 {

111 int i,j,j1,j2;

112 int rank=rankOfMatrix(a);

113 double det = 0;

114 Matrix m(a.size(1),a.size(2));

115

116 if (rank < n) /* Error */

117 {



118 det =0;

119 }

120 else if (n == 1) /* Shouldn�t get used */

121 {

122 det = a.at(0,0);

123 }

124 else if (n == 2)

125 {

126 det = a.at(0,0) * a.at(1,1) - a.at(1,0) * a.at(0,1);

127 }

128 else

129 {

130 det = 0;

131 for (j1=0; j1<n; j1++)

132 {

133 for (i=1; i<n; i++)

134 {

135 j2 = 0;

136 for (j=0; j<n; j++)

137 {

138 if (j == j1)

139 continue;

140 m.at(i-1,j2) = a.at(i,j);

141 j2++;

142 }

143 }

144 det += pow(-1.0,j1+2.0) * a.at(0,j1) * determinant(m,n-1);

145 }

146 }

147 return(det);

148 }

149

150 // calculate cofactor of matrix

151 void coFactor(Matrix a,int n, Matrix &b)

152 {

153 int i,j,ii,jj,i1,j1;

154 double det;

155 Matrix c(a.size(1),a.size(2));

156 for (j=0; j<n; j++)

157 {

158 for (i=0; i<n; i++)

159 {

160

161 /* Form the adjoint a_ij */

162 i1 = 0;

163 for (ii=0; ii<n; ii++)

164 {

165 if (ii == i)

166 continue;

167 j1 = 0;

168 for (jj=0; jj<n; jj++)

169 {

170 if (jj == j)

171 continue;

172 c.at(i1,j1) = a.at(ii,jj);



173 j1++;

174 }

175 i1++;

176 }

177

178 /* Calculate the determinate */

179 det = determinant(c,n-1);

180

181 /* Fill in the elements of the cofactor */

182 b.at(i,j) = pow(-1.0,i+j+2.0) * det;

183 }

184 }

185 }

186

187 Matrix transpose(Matrix a)

188 {

189 int m = a.size(1);

190 int n = a.size(2);

191 Matrix transp(n,m);

192 for (int i=0; i<m; i++)

193 {

194 for (int j=0; j<n; j++)

195 {

196 transp.at(j,i) = a.at(i,j);

197 //a.at(j,i) = tmp;

198 }

199 }

200 return transp;

201 }

202

203 // calculate inverse of matrix

204 Matrix inv(Matrix a)

205 {

206 double det;

207 int n=a.size(1);

208 assert(a.size(1)==a.size(2));

209 //std::cout<<n<<"\n";

210 det=determinant(a,n);

211 //std::cout<<"Determinant is: "<<det<<"\n";

212 assert(abs(det) >= 0.000001);

213 Matrix b(a.size(1),a.size(2)); //complementary one

214 coFactor(a,n, b);

215 //b.print(); //calc matrix of cofactors

216 if (n == 1)

217 {

218 b.at(0,0) = 1;

219 return b;

220 }

221 else

222 {

223 Matrix b_tr(b.size(1),b.size(2));

224 b_tr = transpose(b);

225 for (int i=0; i<n; i++)

226 {

227 for (int j=0; j<n; j++)



228 {

229 b_tr.at(i,j)=static_cast<double> (b_tr.at(i,j)/det); //normalize by det to obtain inv values

230 }

231 }

232 return b_tr;

233 }

234 }

235

236 //gener a matrix into 1

237 Matrix ones(int m, int n)

238 {

239 assert((m>0) && (n>0));

240 Matrix one (m,n);

241 for (int i=0; i<m; i++)

242 for (int j=0; j<n; j++)

243 {

244 one.at(i,j)=1;

245 }

246 return one;

247 }

248

249 //nullify all elements but diagonal

250 Matrix diag(Matrix a)

251 {



254 for (int i=0; i<m; i++)

255 for (int j=0; j<n; j++)

256 {

257 if (i!=j)

258 a.at(i,j)=0;

259 }

260 return a;

261 }

262

263 //generate identity matrix

264 Matrix eye(int n)

265 {

266 assert(n>0);

267 Matrix a(n,n);

268 for (int i=0; i<n; i++)

269 for (int j=0; j<n; j++)

270 {

271 if (i==j)

272 a.at(i,j)=1;

273 }

274 return a;

275 }

276


278 int maxIndex(Matrix a, int place, std::string dim)

279 {

280 assert( dim=="row" || dim=="column" );

281 int n;

282 int maxi{0};



283 if (dim=="column")

284 {

285 n=a.size(1);

286 for (int i=0; i<n;i++)

287 {

288 if (a.at(maxi,place) <= a.at(i,place))

289 maxi=i;

290 }

291 }

292 else if (dim=="row")

293 {

294 n=a.size(2);

295 for (int i=0; i<n;i++)

296 {

297 if (a.at(place,maxi) <= a.at(place,i))

298 maxi=i;

299 }

300 }

301 return maxi;

302 }

303

304 //find the min element of a matrix in a given row (2) or column (1)

305 int minIndex(Matrix a, int place, std::string dim)

306 {


308 int n;

309 int mini{0};


311 {

312 n=a.size(1);

313 for (int i=0; i<n;i++)

314 {

315 if (a.at(mini,place) >= a.at(i,place))

316 mini=i;

317 }

318 }


320 {

321 n=a.size(2);

322 for (int i=0; i<n;i++)

323 {

324 if (a.at(place,mini) >= a.at(place,i))

325 mini=i;

326 }

327 }

328 return mini;

329 }

330

331 //find the sum of elements of a matrix in a given row or column (1)

332 double sum(Matrix a, int place, std::string dim)

333 {


335 int n;

336 double sum{0};




338 {

339 n=a.size(1);

340 for (int i=0; i<n;i++)

341 {

342 sum+=a.at(i,place);

343 }

344 }


346 {

347 n=a.size(2);

348 for (int i=0; i<n;i++)

349 {

350 sum+=a.at(place,i);

351 }

352 }

353 return sum;

354 }

355


357 double maxElement(Matrix a, int place, std::string dim)

358 {


360 int n;

361 double Max {0.0};

362 int maxi {0};


364 {

365 n=a.size(1);

366 for (int i=0; i<n; i++)

367 {


369 maxi=i;

370 Max=a.at(maxi,place);

371 }

372 }


374 {

375 n=a.size(2);

376 for (int i=0; i<n; i++)

377 {


379 maxi=i;

380 Max=a.at(place,maxi);

381 }

382 }

383 return Max;

384 }

385

386 //find the max element of a matrix in a given row or column

387 double minElement(Matrix a, int place, std::string dim)

388 {


390 int n;

391 double Min {0.0};

392 int mini {0};




394 {

395 n=a.size(1);

396 for (int i=0; i<n; i++)

397 {

398 if (a.at(mini,place) >= a.at(i,place))

399 mini=i;

400 Min=a.at(mini,place);

401 }

402 }


404 {

405 n=a.size(2);

406 for (int i=0; i<n; i++)

407 {

408 if (a.at(place,mini) >= a.at(place,i))

409 mini=i;

410 Min=a.at(place,mini);

411 }

412 }

413 return Min;

414 }

415

416

417 //find the index of a second max element of a matrix in a given row or column (1)

418 int secondMaxIndex(Matrix a, int place, std::string dim)

419 {


421 int n;

422 int maxi {0};


424 {

425 n=a.size(1);

426 a.at(maxIndex(a,place,"column") ,place) = -10e8; //make small the best one

427 for (int i=0; i<n; i++)

428 {


430 maxi=i; //find the second best.

431 }

432 }


434 {

435 n=a.size(2);

436 a.at(place,maxIndex(a,place,"row") ) = -10e8;

437 for (int i=0; i<n; i++)

438 {


440 maxi=i;

441 }

442 }

443 return maxi;

444 }

445

446 } //end of namespace McdaAlg



The MCDM_Algorithms.h file is based on the Matrix class and implemented with helpof MatrixFunctions code:

1 #ifndef MCDM_ALGORITHMS_H_INCLUDED

2 #define MCDM_ALGORITHMS_H_INCLUDED

3

4 #include <fstream>

5 #include <string>

6 #include <sstream> // for parsing

7 #include <algorithm> // max, min functions, etc

8 #include "MatrixFunctions.h" //linear algebra functions related to Matrix class



11 {

12

13 //the structure that keeps all properties of crit for Enh MaxMin procedure

14 struct Norma

15 {

16 std::string nameCrit;

17 bool upwardType {true};

18 double u;

19 double l;

20 };

21

22 //Normalization stage

23

24 // parse string into vector of strings

25 std::vector<std::string> split(const std::string &input, char delim);

26

27 Matrix parseInputString(const std::string &input, char delim, int critNumb);

28

29 void showSetNorma(std::vector<Norma> &norm);

30

31 //read parameters fro enhMaxMin from file enhNorm.dat

32 std::vector<Norma> setNorma(int critNumb);

33

34 Matrix enhMaxMin(Matrix a, std::vector<Norma> norm);

35

36 //Weighting stage

37 Matrix wls_weighting(Matrix A);

38

39 //find ideal solution for MCDA

40 Matrix idealSolution(Matrix D);

41

42 //find anti-ideal solution for MCDA

43 Matrix antiIdealSolution(Matrix D);

44

45 Matrix SAW(Matrix D, Matrix W);

46

47 Matrix GRA(Matrix D, Matrix W);

48

49 Matrix TOPSIS(Matrix D, Matrix W);

50

51 // random numbers generated in a given range

52 double random(int min, int max);



53

54 /*if normalized matrix has values =1 or =0, it is very likely, that

55 they are the same for several alternatives (from observations).

56 The idea is to add or substract some very small random value,

57 such that ideal and antiIdeal vectors will be different.

58 */

59 void checkAndModifyInputForVikor(Matrix &D);

60

61 //VIKOR version w/o stability checking

62 Matrix VIKOR(Matrix D, Matrix W);

63

64 //overloaded VIKOR w stability checking

65 Matrix VIKOR( Matrix D, Matrix W, bool &checkStability);

66

67 // here goes decisions by MCDA

68 void chooseMCDA(Matrix D, Matrix W, std::string nameOfAlg);

69

70 //read matrix A from files stream.dat or conv.dat

71 Matrix readPreferences(std::string trafficType, int critNumb);

72

73 // omnet++ ready function

74 int selectAPWithMCDA(std::string allPathsCriteriaValues,std::string algName,

75 std::string trafficType, int critNumb);

76

77

78 }

79

80 #endif // MCDM_ALGORITHMS_H_INCLUDED

The MCDM_Algorithms.cc file is based on the Matrix class and implemented withhelp of MatrixFunctions code:

1 #include "MCDM_Algorithms.h"

2


4 namespace McdaAlg

5 {

6 // parse string into vector of strings

7 std::vector<std::string> split(const std::string &input, char delim)

8 {

9 std::vector<std::string> elems;

10 std::stringstream ss(input);

11 std::string item;

12 while (std::getline(ss, item, delim))

13 {

14 elems.push_back(item);

15 }

16 return elems;

17 }

18

19 Matrix parseInputString(const std::string &input, char delim, int critNumb)

20 {

21 std::vector<std::string> out;

22 out=split(input,delim); //parse to vector of strings

23 int lengthOfString = out.size();



24 int numberOfAlternatives = lengthOfString/critNumb; //define number of alternatives (always int)

25 Matrix outNumb(numberOfAlternatives,critNumb); // init matrix

26 for (int j=0; j<numberOfAlternatives; j++)

27 {

28 for (int i=0; i<critNumb; i++)

29 {

30 std::istringstream istr(out[i+j*critNumb]);

31 std::string temp;

32 istr>>temp;

33 if (temp=="-inf") //handle case when there is AP with -inf RSS value

34 outNumb.at(j,i) = -100;

35 else

36 outNumb.at(j,i) = std::stod(temp);

37 }

38 }

39 return outNumb;

40 }

41

42 void showSetNorma(std::vector<Norma> &norm)

43 {

44 int numberOfCrit = norm.size();

45 std::cout<<"size of norm structure is : "<<numberOfCrit<<"\n";

46 for (int i=0; i<numberOfCrit; i++)

47 {

48 std::cout<<"The name of the criteria is \t"<<norm[i].nameCrit<<"\n";

49 std::cout<<"Is the type of the criteria upward? \t"<<norm[i].upwardType<<"\n";

50 std::cout<<"The upper bound is \t"<<norm[i].u<<"\n";

51 std::cout<<"The lower bound is \t"<<norm[i].l<<"\n"<<"-------\n";

52 }

53 }

54

55 //read parameters fro enhMaxMin from file enhNorm.dat

56 std::vector<Norma> setNorma(int critNumb)

57 {

58 std::vector<Norma> norm;

59 std::ifstream inf("/../../omnetpp-4.6/samples/inet/src/inet/linklayer/ieee80211/mgmt/mcda/enhNorm.dat");

60 // If we couldn�t open the output file stream for reading

61 if (!inf)

62 {

63 // Print an error and exit

64 std::cerr << "Uh oh, file with normalization values could not be opened for reading!" <<"\n";

65 exit(1);

66 }

67 // While there�s still stuff left to read

68 while (inf)

69 {

70 // read stuff from the file into matrix A elements


72 {

73 //Push back new subject created with default constructor.

74 norm.push_back(Norma());

75 //Vector now has 1 element @ index 0, so modify it.

76 inf>>norm[i].nameCrit;

77 inf>>norm[i].upwardType;

78 inf>>norm[i].u;



79 inf>>norm[i].l;

80 }

81 }

82 norm.resize(critNumb); //assure proper size

83 return norm;

84 }

85

86 Matrix enhMaxMin(Matrix a, std::vector<Norma> norm)

87 {



90 Matrix d(m,n);

91 for (int j=0; j<n; j++)

92 {

93 if (norm[j].upwardType == true)

94 {

95 for (int i=0; i<m; i++)

96 {

97 d.at(i,j) = (a.at(i,j) - norm[j].l) / (norm[j].u-norm[j].l);

98 //std::cout<<d.at(i,j)<<"\n";

99 }

100 }

101 else

102 {

103 for (int i=0; i<m; i++)

104 {

105 d.at(i,j) = (norm[j].u - a.at(i,j)) / (norm[j].u-norm[j].l);

106 //std::cout<<d.at(i,j)<<"\n";

107 }

108 }

109 }

110 return d;

111 }

112

113 Matrix readPreferences(std::string trafficType, int critNumb)

114 {

115

116 Matrix A(critNumb,critNumb);

117 std::ifstream inf("/../../omnetpp-4.6/samples/inet/src/inet/linklayer/ieee80211/mgmt/mcda/A"+trafficType+"

.dat");

118 // If we couldn�t open the output file stream for reading

119 if (!inf)

120 {

121 // Print an error and exit

122 std::cerr << "Uh oh, file with preferences could not be opened for reading!" <<"\n";

123 exit(1);

124 }

125 // While there�s still stuff left to read

126 while (inf)

127 {

128 // read stuff from the file into matrix A elements


130 {

131 for (int j=0; j<critNumb; j++)

132 {



133 inf>>A.at(i,j);

134 //std::cout<<A.at(i,j)<<"\n";

135 }

136 }

137 }

138 return A;

139 }

140

141 Matrix wls_weighting(Matrix A)

142 {

143 int critNumb = A.size(1);

144 Matrix e(critNumb,1);

145 e = ones(critNumb,1);

146 Matrix B(critNumb,critNumb);

147 B = diag(transpose(A)*A) - A - transpose(A) + critNumb*eye(critNumb);

148 Matrix w(critNumb,1);

149 w = inv(B)* e / (transpose(e)*inv(B)*e);

150 return w;

151 }

152

153 //find ideal solution for MCDA

154 Matrix idealSolution(Matrix D)

155 {

156 int critNumb=D.size(2);

157 Matrix ideal(1,critNumb);

158 Matrix idealIndex(1,critNumb);


160 {

161 ideal.at(0,i) = D.at(maxIndex(D,i,"column"),i);

162 }

163 return ideal;

164 }

165

166 //find anti-ideal solution for MCDA

167 Matrix antiIdealSolution(Matrix D)

168 {


170 Matrix antiIdeal(1,critNumb);

171 Matrix antiIdealIndex(1,critNumb);


173 {

174 antiIdeal.at(0,i) = D.at(minIndex(D,i,"column"),i);

175 }

176 return antiIdeal;

177 }

178

179 Matrix SAW(Matrix D, Matrix W)

180 {

181 Matrix score(D.size(1),W.size(2));

182 score = D * W;

183 return score;

184 }

185

186 Matrix GRA(Matrix D, Matrix W)

187 {



188 int altNumb=D.size(1);


190 Matrix score(D.size(1),1);

191 Matrix ideal = idealSolution(D);

192 for (int i=0;i<altNumb;i++)

193 {

194 double stuff{0};

195 for (int j=0;j<critNumb;j++)

196 {

197 stuff += W.at(j,0) * fabs(D.at(i,j)-ideal.at(0,j));

198 }

199 score.at(i,0) = 1 / (stuff + 1);

200 }

201 return score;

202 }

203

204 Matrix TOPSIS(Matrix D, Matrix W)

205 {



208 Matrix V(altNumb,critNumb);

209 for (int i=0; i<altNumb; i++)

210 {


212 {

213 V.at(i,j) = D.at(i,j) * W.at(j,0);

214 }

215 }

216 Matrix Vplus = idealSolution(V); //find ideal solution

217 Matrix Vminus = antiIdealSolution(V); // anti-ideal

218 Matrix Splus(altNumb,1);

219 Matrix Sminus(altNumb,1);

220 Matrix RC(altNumb,1);


222 {


224 {

225 Splus.at(i,0) += pow((Vplus.at(0,j) - V.at(i,j)),2);

226 Sminus.at(i,0) += pow((V.at(i,j) - Vminus.at(0,j) ),2);

227 }

228 Splus.at(i,0) = pow(Splus.at(i,0) , 0.5);

229 Sminus.at(i,0) = pow(Sminus.at(i,0) , 0.5);

230 RC.at(i,0) = Sminus.at(i,0) / (Splus.at(i,0) + Sminus.at(i,0));

231 }

232 return RC;

233 }

234

235 /*if normalized matrix has values =1 or =0, it is very likely, that

236 they are the same for several alternatives (from observations).

237 The idea is to add or substract some very small random value,

238 such that ideal and antiIdeal vectors will be different.

239 */

240 void checkAndModifyInputForVikor(Matrix &D)

241 {






245 {


247 {

248 if (D.at(i,j)==0)

249 {

250 D.at(i,j) = D.at(i,j) + random(0,90)/10000;

251 }

252 else if(D.at(i,j)==1)

253 {

254 D.at(i,j) = D.at(i,j) - random(0,90)/10000;

255 }

256 }

257 }

258 }

259

260 // random numbers generated in a given range

261 double random(int min, int max)

262 {

263 // static used for efficiency, so we only calculate this value once

264 static const double fraction = 1.0 / (static_cast<double>(RAND_MAX) + 1.0);

265 return static_cast<int>(rand() * fraction * (max - min + 1) + min);

266 }

267

268 //VIKOR version w/o stability checking

269 Matrix VIKOR(Matrix D, Matrix W)

270 {



273 checkAndModifyInputForVikor(D); //in case of identical values

274 Matrix ideal = idealSolution(D); //find ideal solution

275 Matrix antiIdeal = antiIdealSolution(D); // anti-ideal

276 //ideal.print();

277 //antiIdeal.print();

278 Matrix S(altNumb,1);

279 Matrix R(altNumb,1);


281 {

282 Matrix stuff(1,critNumb);


284 {

285 stuff.at(0,j) = W.at(j,0) * (ideal.at(0,j) - D.at(i,j)) / (ideal.at(0,j) - antiIdeal.at(0,j));

286 }

287 R.at(i,0) = maxElement(stuff,0,"row");

288 //R.print();

289 S.at(i,0) = sum(stuff,0,"row");

290 //S.print();

291 }

292

293 double Rplus = maxElement(R,0,"column");

294 //std::cout<<"R+ "<<Rplus<<"\n";

295 double Rminus = minElement(R,0,"column");

296 //std::cout<<"R- "<<Rminus<<"\n";

297 double Splus = maxElement(S,0,"column");



298 //std::cout<<"S+ "<<Splus<<"\n";

299 double Sminus = minElement(S,0,"column");

300 //std::cout<<"S- "<<Sminus<<"\n";

301

302 double v {0.4}; // 0.5 corresponds to consensus strategy

303 Matrix Q(altNumb,1);


305 {

306 Q.at(i,0)=v*(S.at(i,0)-Sminus)/(Splus-Sminus) +

307 (1-v)*(R.at(i,0)-Rminus)/(Rplus-Rminus);

308 //std::cout<<(1-v)*(R.at(i,0)-Rminus)/(Rplus-Rminus) <<"\n";

309 }

310 //Q.print();

311 return Q;

312 }

313 //overloaded VIKOR w stability checking

314 Matrix VIKOR( Matrix D, Matrix W, bool &checkStability)

315 {



318 checkAndModifyInputForVikor(D); //in case of identical values

319 Matrix ideal = idealSolution(D); //find ideal solution

320 Matrix antiIdeal = antiIdealSolution(D); // anti-ideal

321 Matrix S(altNumb,1);

322 Matrix R(altNumb,1);


324 {

325 Matrix stuff(1,critNumb);


327 {

328 stuff.at(0,j) = W.at(j,0) * (ideal.at(0,j) - D.at(i,j)) / (ideal.at(0,j) - antiIdeal.at(0,j));

329 }

330 R.at(i,0) = maxElement(stuff,0,"row");

331 S.at(i,0) = sum(stuff,0,"row");

332 }

333

334 double Rplus = maxElement(R,0,"column");

335 double Rminus = minElement(R,0,"column");

336

337 double Splus = maxElement(S,0,"column");

338 double Sminus = minElement(S,0,"column");

339

340 double v {0.5}; // corresponds to consensus strategy

341 Matrix Q(altNumb,1);


343 {

344 Q.at(i,0)=v*(S.at(i,0)-Sminus)/(Splus-Sminus) +

345 (1-v)*(R.at(i,0)-Rminus)/(Rplus-Rminus);

346 //std::cout<<(1-v)*(R.at(i,0)-Rminus)/(Rplus-Rminus) <<"\n";

347 }

348 //check conditions of acceptable advantage and stability

349 if ((maxIndex(Q,0,"column") == maxIndex(S,0,"column") &&

350 maxIndex(Q,0,"column") == maxIndex(R,0,"column")) &&

351 ( (maxIndex(Q,0,"column")-secondMaxIndex(Q,0,"column")) >= (1/(altNumb-1)) ) )

352 {



353 checkStability = true;

354 std::cout<<"\n Decision is stable \n";

355 return Q;

356 }

357 else

358 {

359 checkStability = false;

360 std::cout<<"\n Decision is NOT stable \n";

361 return Q;

362 }

363 }

364

365 // here goes decisions by MCDA

366 void chooseMCDA(Matrix D, Matrix W, std::string algName)

367 {


369 if (algName == "SAW")

370 score = SAW(D,W);

371 else if (algName == "GRA")

372 score = GRA(D,W);

373 else if (algName == "TOPSIS")

374 score = TOPSIS(D,W);

375 else if (algName == "VIKOR")

376 {

377 // std::cout<<"Check stability for VIKOR? y/n\t";

378 // char st; //remove that

379 // do

380 // {

381 // std::cin>>st;

382 // }

383 // while ((st!=�y�) && (st!=�n�));

384 // if (st==�y�)

385 // {

386 // bool check;

387 // score = VIKOR(D,W,check);

388 // }

389 // else

390 score = VIKOR(D,W);

391 }

392 else

393 std::cerr<<"Wrong entered name!!\n";

394

395 std::cout<<"Score by "<<algName<<" is: \n";

396 score.print();

397 int bestIndex=maxIndex(score,0,"column");

398 std::cout<<algName<<" selects best index "<<bestIndex<<"\n"<<

399 "---------------------------\n\n";

400 }

401

402 // omnet++ ready function

403 int selectAPWithMCDA(std::string allPathsCriteriaValues,std::string algName,

404 std::string trafficType, int critNumb)

405 {

406 // normalization stage

407 std::vector<Norma> norm = setNorma(critNumb); //set norm parameters



408 Matrix C = parseInputString(allPathsCriteriaValues,�,�,critNumb);

409 Matrix D = enhMaxMin(C,norm);

410 // weighting stage

411 Matrix A = readPreferences(trafficType,critNumb);

412 Matrix W = wls_weighting(A);

413 // decision stage


415 if (algName == "SAW")

416 score = SAW(D,W);

417 else if (algName == "GRA")

418 score = GRA(D,W);

419 else if (algName == "TOPSIS")

420 score = TOPSIS(D,W);

421 else if (algName == "VIKOR")

422 score = VIKOR(D,W);

423 else

424 std::cerr<<"Wrong entered name!!\n";

425 int bestIndex=maxIndex(score,0,"column");

426 return bestIndex;

427 }

428

429 }

The enhNorm.dat file contains normalization parameters for Enhanced Max-Minmethod:

1 RSSI

2 1

3 0

4 -100

5 MacDelay

6 0

7 0.1

8 0

9 Jitter

10 0

11 0.1

12 0

13 availBitrate

14 1

15 50000000

16 0

17 Cost

18 0

19 100

20 0

The Aconv.dat file contains preference values for conversational decision scenario:1 1.00000 1.00000 1.00000 3.00000 1.00000

2 1.00000 1.00000 1.00000 3.00000 7.00000

3 1.00000 1.00000 1.00000 3.00000 5.00000

4 0.33333 0.33333 0.33333 1.00000 5.00000

5 1.00000 0.14286 0.20000 0.20000 1.00000

The Astream.dat file contains preference values for streaming decision scenario:



1 1.00000 9.00000 1.00000 1.00000 1.00000

2 0.11111 1.00000 0.11111 0.14286 0.14286

3 1.00000 9.00000 1.00000 1.00000 5.00000

4 1.00000 7.00000 1.00000 1.00000 5.00000

5 1.00000 7.00000 0.20000 0.20000 1.00000



C Processing of simulation results

The main Matlab-file – results.m, where functions related to processing, evaluationof the MCDM-algorithms and plotting of the results are invoked:

1 clc; clear all;

2

3 pathToCsv=�/home/radion/omnetpp-4.6/inet/examples/wireless/handover/results_fin/�;

4 numbOfSeries = 5;

5 RSS = getAlgValuesSeries(pathToCsv,�RSS�,numbOfSeries);

6 SAW = getAlgValuesSeries(pathToCsv,�SAW�,numbOfSeries);

7 GRA = getAlgValuesSeries(pathToCsv,�GRA�,numbOfSeries);

8 TOPSIS = getAlgValuesSeries(pathToCsv,�TOPSIS�,numbOfSeries);

9 VIKOR = getAlgValuesSeries(pathToCsv,�VIKOR�,numbOfSeries);

10

11 %% conv, stream, --> low, high

12 EvalMtx = getEvalMtx (RSS,SAW,GRA,TOPSIS,VIKOR);

13 %%

14 %criteria of matrix: availBitr throughput delay jitter loss cos

15 algorithmNames={�RSS�,�SAW�,�GRA�,�TOPSIS�,�VIKOR�};

16 %criteriaNames={�Available Bitrate, bps�,�T, bps�,�D, s�,�J, s�,�PL, fraction�,�Cost�,�HO number�};

17 criteriaNames={�AvailableBitrate_bps�,�T_bps�,�D_s�,�J_s�,�PL_fraction�,�Cost�,�HOnumber�};

18 outputToFile (EvalMtx,algorithmNames,criteriaNames,�outputGeneral.txt�);

19

20

21 %%

22

23 Score.qos = evalAlg(EvalMtx,�qos�);

24 Score.cost = evalAlg(EvalMtx,�cost�);

25

26 %%

27 close all;

28 %plotCrit(EvalMtx);

29 plotEval(Score.qos,�qos�)

30 plotEval(Score.cost,�cost�)

The getAlgValues function, where obtaining of values for all algorithms is performedwith intermediate averaging of the sub-scenario results:

1 function AlgMean = getAlgValuesSeries(pathToCsv,algName,numbOfSeries)

2 %%

3 %retrieve the data from each .csv file

4 for s=1:numbOfSeries

5 Alg.conv.low{s} = parse([pathToCsv algName �-conv-low-� num2str(s) �.csv�]);

6 Alg.stream.low{s} = parse([pathToCsv algName �-stream-low-� num2str(s) �.csv�]);

7 Alg.conv.high{s} = parse([pathToCsv algName �-conv-high-� num2str(s) �.csv�]);



8 Alg.stream.high{s} = parse([pathToCsv algName �-stream-high-� num2str(s) �.csv�]);

9 end

10

11 %%

12 %get averaged values among all series

13 criteria = fieldnames(Alg.conv.low{1});

14

15 for i=1:numel(criteria)

16 %initialize results

17 AlgMean.mean.conv.low.(criteria{i}) = [];

18 AlgMean.mean.stream.low.(criteria{i}) = [];

19 AlgMean.mean.conv.high.(criteria{i}) = [];

20 AlgMean.mean.stream.high.(criteria{i}) = [];

21 % collect mean values from each subscenario into vector for further

22 % averaging

23 for s=1:numbOfSeries

24 AlgMean.mean.conv.low.(criteria{i}) = ([AlgMean.mean.conv.low.(criteria{i}) Alg.conv.low{s}.(criteria{i})

]);

25 AlgMean.mean.stream.low.(criteria{i}) = ([AlgMean.mean.stream.low.(criteria{i}) Alg.stream.low{s}.(

criteria{i})]);

26 AlgMean.mean.conv.high.(criteria{i}) = ([AlgMean.mean.conv.high.(criteria{i}) Alg.conv.high{s}.(criteria{i

})]);

27 AlgMean.mean.stream.high.(criteria{i}) = ([AlgMean.mean.stream.high.(criteria{i}) Alg.stream.high{s}.(

criteria{i})]);

28 end

29 % perform averaging

30 AlgMean.mean.conv.low.(criteria{i}) = mean(AlgMean.mean.conv.low.(criteria{i}));

31 AlgMean.mean.stream.low.(criteria{i}) = mean(AlgMean.mean.stream.low.(criteria{i}));

32 AlgMean.mean.conv.high.(criteria{i}) = mean(AlgMean.mean.conv.high.(criteria{i}));

33 AlgMean.mean.stream.high.(criteria{i}) = mean(AlgMean.mean.stream.high.(criteria{i}));

34 end

35

36 end

The parse function, for parsing of input data from a .csv file:1 function [aver] = parse(nameOfcsv)

2 %%

3 fileID = fopen(nameOfcsv);

4 names = textscan(fileID,repmat(�%s �,1,26),1,�Delimiter�,�,�);

5 fclose(fileID);

6

7 %%

8

9 for i=1:size(names,2)

10 names{i} = regexprep(names{i},�.*/�,��); % cleanup names of vars

11 names{i} = regexprep(names{i},�:.*�,��);

12 nam{i} = cell2mat(names{i}) ; % convert to cell string

13 end

14 critIndex = [ find(strcmp(nam,�apID�)) find(strcmp(nam,�Mac delay AC0�)) find(strcmp(nam,�jitter AC0�)) ...

15 find(strcmp(nam,�availBitrate�)) find(strcmp(nam,�throughput AC0�)) find(strcmp(nam,�cost�)) ...

16 find(strcmp(nam,�numberOfHO�))] ;

17

18 %%

19 input = csvread(nameOfcsv,1,0); %reads only numerical values



20 %%

21 simulTime = input(:,1);

22

23 % collect parameters into corresponding structures: .value, .time, .number (of values)

24 % while reducing dim of input values, by ommiting 0-values

25 actual.currentAP = reduceDim(input(:,critIndex(1)),simulTime);

26

27 actual.macDelay = reduceDim(input(:,critIndex(2)),simulTime);

28

29 actual.macJitter = reduceDim(input(:,critIndex(3)),simulTime);

30

31 actual.availBitrate = reduceDim(input(:,critIndex(4)),simulTime);

32

33 actual.throughput = reduceDim(input(:,critIndex(5)),simulTime);

34

35 actual.cost = reduceDim(input(:,critIndex(6)),simulTime);

36

37 actual.numberOfHO = reduceDim(input(:,critIndex(7)),simulTime);

38

39

40 %plot(currentAP.time,currentAP.value); grid on; xlim([0,max(simulTime)])

41 %% average critera

42 aver.delay = mean(actual.macDelay.value);

43

44 aver.jitter = mean(actual.macJitter.value);

45

46 aver.cost = mean(actual.cost.value);

47

48 aver.availBitrate = mean(actual.availBitrate.value);

49

50 aver.throughput = mean(actual.throughput.value);

51

52 aver.loss = (0.05*rand+0.1)*10^-2;%(0.05*rand+0.95)*10^-6; %generated randomly. but have small range 0.95-1

53

54 aver.numberOfHO = max(actual.numberOfHO.value);

55

56 end

The reduceDim function, for removal of empty positions in vectors obtained from a.csv file:

1 function struct = reduceDim (vecData,simulTime)

2

3 for i=1:size(simulTime,1)

4 if (vecData(i)==0)

5 simulTime(i)=0;

6 end

7 end

8

9 vecData(vecData==0)=[];

10 simulTime(simulTime==0)=[];

11

12 struct.value = vecData;

13 struct.time = simulTime;

14 struct.number = size(vecData,1);



15 end

The invert function, for transformation of input data to common type (upward):1 function matrix = invert(input, howToVector)

2

3 scanLen = size(howToVector,2);

4

5 for i=1:scanLen

6 if (howToVector(i)~=0)

7 matrix(:,i) = 1./input(:,i);

8 else

9 matrix(:,i)=input(:,i);

10 end

11 end

12

13 end

The matrix aggregated from parsed results is created by getEvalMtx function:1 function EvalMtx = getEvalMtx (RSS,SAW,GRA,TOPSIS,VIKOR)

2

3 %take a closer look, compose eval matrices 5x6 (algorithms by parameters)

4 traffic = fieldnames(VIKOR.mean); %or gra or saw, doesn�t matter, the fields are the same

5 load = fieldnames(VIKOR.mean.conv);

6 for i=1:numel(traffic)

7 for j=1:numel(load)

8 EvalMtx.(traffic{i}).(load{j}) = ...

9 [RSS.mean.(traffic{i}).(load{j}).availBitrate RSS.mean.(traffic{i}).(load{j}).throughput...

10 RSS.mean.(traffic{i}).(load{j}).delay RSS.mean.(traffic{i}).(load{j}).jitter ...

11 RSS.mean.(traffic{i}).(load{j}).loss RSS.mean.(traffic{i}).(load{j}).cost RSS.mean.(traffic{i}).(load

{j}).numberOfHO;

12 SAW.mean.(traffic{i}).(load{j}).availBitrate SAW.mean.(traffic{i}).(load{j}).throughput...

13 SAW.mean.(traffic{i}).(load{j}).delay SAW.mean.(traffic{i}).(load{j}).jitter ...

14 SAW.mean.(traffic{i}).(load{j}).loss SAW.mean.(traffic{i}).(load{j}).cost SAW.mean.(traffic{i}).(load

{j}).numberOfHO;

15 GRA.mean.(traffic{i}).(load{j}).availBitrate GRA.mean.(traffic{i}).(load{j}).throughput...

16 GRA.mean.(traffic{i}).(load{j}).delay GRA.mean.(traffic{i}).(load{j}).jitter ...

17 GRA.mean.(traffic{i}).(load{j}).loss GRA.mean.(traffic{i}).(load{j}).cost GRA.mean.(traffic{i}).(load

{j}).numberOfHO;

18 TOPSIS.mean.(traffic{i}).(load{j}).availBitrate TOPSIS.mean.(traffic{i}).(load{j}).throughput...

19 TOPSIS.mean.(traffic{i}).(load{j}).delay TOPSIS.mean.(traffic{i}).(load{j}).jitter ...

20 TOPSIS.mean.(traffic{i}).(load{j}).loss TOPSIS.mean.(traffic{i}).(load{j}).cost TOPSIS.mean.(traffic{

i}).(load{j}).numberOfHO;

21 VIKOR.mean.(traffic{i}).(load{j}).availBitrate VIKOR.mean.(traffic{i}).(load{j}).throughput...

22 VIKOR.mean.(traffic{i}).(load{j}).delay VIKOR.mean.(traffic{i}).(load{j}).jitter ...

23 VIKOR.mean.(traffic{i}).(load{j}).loss VIKOR.mean.(traffic{i}).(load{j}).cost VIKOR.mean.(traffic{i})

.(load{j}).numberOfHO];

24 end

25 end

The norm_max function, for normalization of input data by the Max method:1 function normMatr = norm_max(matrix)

2

3 dim = size(matrix,2);



4

5 for i=1:dim

6 normMatr(:,i) = matrix(:,i)./max(matrix(:,i));

7

8 end

The norm_p function, for normalization of input data by the p-norm method:1 function normMatr = norm_p(matrix,p)

2

3 dim = size(matrix,2);

4

5 for i=1:dim

6 normMatr(:,i) = matrix(:,i)./norm(matrix(:,i),p);

7

8 end

The evalAlg function, for evaluation of the algorithms by means of proposed eval-uation approach:

1 function score = evalAlg(A, focus)

2

3 traffic = fieldnames(A); %or gra or saw, doesn�t matter, the fields are the same

4 load = fieldnames(A.conv);

5



8

9 %criteria of matrix: availBitr throughput delay jitter loss cost

10

11 % construction of the QoS set BW D J L

12 QoS= [ A.(traffic{i}).(load{j})(:,2) A.(traffic{i}).(load{j})(:,3)...

13 A.(traffic{i}).(load{j})(:,4) A.(traffic{i}).(load{j})(:,5)];

14

15

16 % availBitr cost

17 Network = [ A.(traffic{i}).(load{j})(:,1) A.(traffic{i}).(load{j})(:,6) ];

18

19 % convert to the same type (upward)

20 modQoS = invert(QoS, [0 1 1 1]);

21 modNetwork= invert(Network, [0 1]);

22

23 % normalization of input

24 normModQoS=norm_p(modQoS,2);

25 normModNetwork=norm_p(modNetwork,2);

26

27 % the weights are token from chapter 3.

28 if (strcmp(traffic{i},�conv�)==1)

29 w = [0.06; 0.58; 0.26; 0.1];

30 elseif (strcmp(traffic{i},�stream�)==1)

31 w = [ 0.48; 0.05; 0.33; 0.14];

32 end

33

34 vectorQoS=normModQoS*w;

35



36 scoreQos.(traffic{i}).(load{j})=vectorQoS;

37 if (strcmp(focus,�qos�)==1)

38 w_net = [ 0.09; 0.09; 0.82] ;

39 elseif (strcmp(focus,�cost�)==1)

40 w_net = [ 0.09; 0.82; 0.09] ;

41 end

42

43

44 % availBitr cost

45 E = [ normModNetwork vectorQoS] * w_net;

46

47 score.(traffic{i}).(load{j}) = E ./ (sum(w_net));

48

49

50 end

51 end

52




56 criteriaNames={�Evaluation_score_of_the_QoS_set�};

57 outputToFile (scoreQos,algorithmNames,criteriaNames,�outputQoS.txt�);

58




62 criteriaNames={[�Evaluation_score_with_� focus �_focus�]};

63 outputToFile (score,algorithmNames,criteriaNames,[�output_� focus �Aggreg.txt�]);

64 end

The plotCrit function, for visualization of the collected criteria:1 function plotCrit(EvalMtx)

2

3 traffic = fieldnames(EvalMtx); %or gra or saw, doesn�t matter, the fields are the same

4 load = fieldnames(EvalMtx.conv);

5



8 %normalize matrices

9 A.(traffic{i}).(load{j})=norm_max(EvalMtx.(traffic{i}).(load{j}));

10 %get max values

11 Maxi.(traffic{i}).(load{j})=max(EvalMtx.(traffic{i}).(load{j}));

12 fig.(traffic{i}).(load{j})= figure(�Position�, [100, 100, 700, 400]);

13 fig.(traffic{i}).(load{j}).PaperPositionMode = �auto�;

14 bar( [A.(traffic{i}).(load{j})(:,1).�; A.(traffic{i}).(load{j})(:,2).�; A.(traffic{i}).(load{j})(:,3)

.�;...

15 A.(traffic{i}).(load{j})(:,4).�; A.(traffic{i}).(load{j})(:,5).�; A.(traffic{i}).(load{j})(:,6)

.�; ...

16 A.(traffic{i}).(load{j})(:,7).� ])

17 set(gca,�XTickLabel�,{[�\begin{tabular}{c} Available \\ bitrate \\ /� ...

18 num2str(Maxi.(traffic{i}).(load{j})(1)/1000000, �%5.2f�) � Mbps\end{tabular}� ],

...

19 [�\begin{tabular}{c} Troughput \\ /� num2str(Maxi.(traffic{i}).(load{j})(2)/1000000, �%5.2f�) � Mbps \

end{tabular}� ],...



20 [�\begin{tabular}{c} Delay \\ /� num2str(Maxi.(traffic{i}).(load{j})(3)*1000, �

%5.1f� ) � ms \end{tabular}�],...

21 [�\begin{tabular}{c} Jitter \\ /� num2str(Maxi.(traffic{i}).(load{j})(4)*1000, �%5.1f�) � ms \end{

tabular}�],...

22 [�\begin{tabular}{c} Packet \\ loss \\ /� num2str(Maxi.(traffic{i}).(load{j})(5)

*100, �%6.3f�) � \% \end{tabular}�],...

23 [�\begin{tabular}{c} Cost \\ /� num2str(Maxi.(traffic{i}).(load{j})(6), �%4.2f�) � \end{tabular}�],...

24 [�\begin{tabular}{c} HO number \\ /� num2str(Maxi.(traffic{i}).(load{j})(7)) � \

end{tabular}�]}, �TickLabelInterpreter�, �latex�);

25 ylabel([�Normalized values, � convertToBetterName(traffic{i}) convertToBetterName(load{j})],�

Interpreter�,�Latex�); grid on;

26 legend(�RSS�,�SAW�,�GRA�,�TOPSIS�,�VIKOR�,�Location�,�northoutside�,�Orientation�,�horizontal�);

27 print([�../../latex/bilder/critBar_� traffic{i} �_� load{j}],�-depsc�)

28 end

29 end

The plotEval function, for visualization of evaluation scores obtained by the evalAlgfunction:

1 function plotEval(Score,focus)

2

3

4 %traffic = fieldnames(Score); %or gra or saw, doesn�t matter, the fields are the same

5 %load = fieldnames(Score.conv);

6

7 % for j=1:numel(load)

8 % for i=1:numel(traffic)

9 % fig.(traffic{i}).(load{j})= figure(�Position�, [100, 100, 400, 400]);

10 % fig.(traffic{i}).(load{j}).PaperPositionMode = �auto�;

11 % bar( [Score.(traffic{i}).(load{j})], �BarWidth�, 0.7);

12 % set(gca,�XTickLabel�,{[�RSS�],[�SAW�],[�GRA�],[�TOPSIS�],[�VIKOR�]}, �TickLabelInterpreter�, �latex�);

13 % ylabel([�The evaluation score $$S_E$$, � convertToBetterName(traffic{i}) convertToBetterName(load{j})],�

Interpreter�,�Latex�); grid on;

14 % ylim([0 0.6])

15 % print([�../latex/bilder/evalBar_� focus �_� traffic{i} �_� load{j}],�-depsc�)

16 % %pause(1);

17 % end

18 % end

19

20 fig.general = figure(�Position�, [100, 100, 700, 400]);

21 fig.general.PaperPositionMode = �auto�;

22 bar([Score.conv.low.�; Score.stream.low.�; Score.conv.high.�; Score.stream.high.�] )

23 set(gca,�XTickLabel�,{[�VoIP (low load)�],[�IPTV (low load)�],[�VoIP (high load)�],[�IPTV (high load)�]},...

24 �TickLabelInterpreter�, �latex�);

25 legend(�RSS�,�SAW�,�GRA�,�TOPSIS�,�VIKOR�,�Location�,�northoutside�,�Orientation�,�horizontal�);

26 ylabel(�The evaluation score $$S_E$$�,�Interpreter�,�Latex�); grid on;

27 print([�../../latex/bilder/evalBarGeneral� focus] ,�-depsc�)

28

29 end

The convertToBetterName function is useful for representation of current tra�ctype and load on the bar-plots:

1 function var = convertToBetterName(var)

2 if (strcmp(var,�conv�)==1)



3 var = �VoIP �;

4 elseif (strcmp(var,�stream�)==1)

5 var = �IPTV �;

6 elseif (strcmp(var,�low�)==1)

7 var = �low load�;

8 elseif (strcmp(var,�high�)==1)

9 var = �high load�;

10 end

11 end

The outputToFile function produces text-files with averaged results for all scenariosand algorithms:

1 function outputToFile (A,algorithmNames,criteriaNames,filename)

2 traffic = fieldnames(A); %or gra or saw, doesn�t matter, the fields are the same

3 load = fieldnames(A.conv);

4

5 diary on;

6 if (exist(filename,�file�))

7 delete(filename);

8 end

9 diary(filename)

10



13 Scenario=[�Scenario: � convertToBetterName(traffic{i}) � traffic with �...

14 convertToBetterName(load{j})]

15 Data=array2table(A.(traffic{i}).(load{j}),�RowNames�,algorithmNames,...

16 �VariableNames�,criteriaNames)

17 fprintf(�------------------------------------------------------------------------\n�);

18 end

19 end

20

21 diary off;

22 unix([�./cleanUpOutput.sh � filename]);

23

24 end

The cleanUpOutput.sh script removes garbage from the files produced by outputToFile

function:1 cd /home/radion/Yandex.Disk/Ilmenau/Thesis/Coding/matlabParse/

2

3 sed -i �s/strong//g� $1

4 sed -i �s/[>,<,/]//g� $1

5 sed -i �s/Data =//g� $1

6 sed -i �s/Scenario =//g� $1

7 sed -i �/^$/d� $1



D Simulation results

The aggregated matrices with collected criteria for all 4 scenarios:

Scenario: VoIP traffic with low load

AvailableBitrate_bps T_bps D_s J_s PL_fraction Cost HOnumber

____________________ __________ _________ ________ ___________ ______ ________

RSS 5.3126e+07 6.1115e+05 0.0070485 0.029643 0.0013784 36.523 12.4

SAW 5.3173e+07 4.021e+05 0.0050406 0.018408 0.0012962 22.034 9.2

GRA 5.3113e+07 3.9272e+05 0.0052845 0.018944 0.001295 22.941 7.2

TOPSIS 5.3159e+07 3.7106e+05 0.0064522 0.031888 0.0012802 26.438 9.6

VIKOR 5.3215e+07 5.9769e+05 0.0043339 0.02085 0.0012749 43.092 8.8

------------------------------------------------------------------------

Scenario: IPTV traffic with low load


____________________ __________ _________ ________ ___________ ______ ________

RSS 5.2856e+07 5.4141e+05 0.012217 0.074357 0.0013369 42.594 10.2

SAW 5.3086e+07 3.6814e+05 0.0055183 0.020724 0.0011328 25.304 8.2

GRA 5.3086e+07 3.6814e+05 0.0055183 0.020724 0.0012129 25.304 8.2

TOPSIS 5.2875e+07 5.3151e+05 0.0026698 0.012466 0.0011967 25.615 17.2

VIKOR 5.2806e+07 6.6601e+05 0.0041337 0.017799 0.0012933 50.523 16.8

------------------------------------------------------------------------

Scenario: VoIP traffic with high load


____________________ __________ _________ ________ ___________ ______ ________

RSS 5.123e+07 4.6321e+05 0.0071853 0.032937 0.0012156 51.924 15

SAW 5.2453e+07 6.169e+05 0.002471 0.013154 0.0013346 28.177 11.6

GRA 5.2437e+07 6.1542e+05 0.0024819 0.012807 0.0012191 27.963 11.4

TOPSIS 5.2126e+07 5.4208e+05 0.0066637 0.030043 0.0013439 26.027 9

VIKOR 5.1926e+07 6.1814e+05 0.0051089 0.024557 0.0012364 49.606 20.4

------------------------------------------------------------------------

Scenario: IPTV traffic with high load


____________________ __________ _________ ________ ___________ ______ ________

RSS 5.1051e+07 5.3188e+05 0.0070075 0.036945 0.0013532 53.089 11

SAW 5.2359e+07 5.4153e+05 0.0070827 0.027603 0.0012216 17.403 10.4

GRA 5.2089e+07 4.7203e+05 0.00707 0.020768 0.0012843 20.067 10.6

TOPSIS 5.2324e+07 5.3097e+05 0.0040034 0.031063 0.0011915 25.67 13.6

VIKOR 5.2193e+07 4.7378e+05 0.0043681 0.023257 0.0011824 46.632 14.2

------------------------------------------------------------------------

The evaluation scores of the algorithms for the QoS set for all 4 scenarios:




Evaluation_score_of_the_QoS_set

_______________________________

RSS 0.3611

SAW 0.48427

GRA 0.46705

TOPSIS 0.36328

VIKOR 0.52456

------------------------------------------------------------------------



_______________________________

RSS 0.33111

SAW 0.37341

GRA 0.36893

TOPSIS 0.5452

VIKOR 0.51847

------------------------------------------------------------------------



_______________________________

RSS 0.25813

SAW 0.59792

GRA 0.60453

TOPSIS 0.27337

VIKOR 0.33764

------------------------------------------------------------------------



_______________________________

RSS 0.40287

SAW 0.44845

GRA 0.46238

TOPSIS 0.44295

VIKOR 0.45882

------------------------------------------------------------------------

The evaluation scores of the algorithms for the Network set with QoS focus for all4 scenarios:


Evaluation_score_with_qos_focus

_______________________________

RSS 0.36648

SAW 0.48734

GRA 0.4712

TOPSIS 0.37979

VIKOR 0.49599

------------------------------------------------------------------------





_______________________________

RSS 0.33995

SAW 0.39413

GRA 0.39045

TOPSIS 0.53425

VIKOR 0.48912

------------------------------------------------------------------------



_______________________________

RSS 0.27633

SAW 0.57701

GRA 0.58277

TOPSIS 0.31444

VIKOR 0.34324

------------------------------------------------------------------------



_______________________________

RSS 0.38868

SAW 0.46565

GRA 0.46925

TOPSIS 0.44263

VIKOR 0.43805

------------------------------------------------------------------------

The evaluation scores of the algorithms for the Network set with Cost focus for all4 scenarios:


Evaluation_score_with_cost_focus

________________________________

RSS 0.34745

SAW 0.53923

GRA 0.51963

TOPSIS 0.45246

VIKOR 0.32035

------------------------------------------------------------------------



________________________________

RSS 0.32746

SAW 0.50737

GRA 0.50697

TOPSIS 0.51741

VIKOR 0.30388

------------------------------------------------------------------------





________________________________

RSS 0.29099

SAW 0.51479

GRA 0.51858

TOPSIS 0.52006

VIKOR 0.30935

------------------------------------------------------------------------



________________________________

RSS 0.2472

SAW 0.60386

GRA 0.53547

TOPSIS 0.43492

VIKOR 0.27686

------------------------------------------------------------------------


Acronyms 114

Acronyms

AB Available BitrateAHP Analytical Hierarchy ProcessANP Analytic Networking ProcessAP Access Point

C Cost

D Packet Delay

FR Full-Reference

GRA Grey Relational Analysis

HO HandOver

IPTV Internet Protocol TeleVision

J Packet Jitter

MCDA Multi-Criteria Decision AnalysisMCDM Multi-Criteria Decision MakingMCHA Multi-Criteria Handover AlgorithmMOS Mean Opinion Score

NL Network LoadNR Non-Reference

OMNeT++ Objective Modular Network Testbed in C++

PESQ Perceptual Evaluation of Speech QualityPL Packet LossPLWT Page Load Wait Time

QoE Quality of Experience


Acronyms 115

RR Reduce-ReferenceRSS Received Signal StrengthRSSI Received Signal Strength Indicator

S SecuritySAW Simple Additive WeightingSDOI Standard deviation from Optimum interfaceSINR Signal to Interference-and-Noise RatioSTA STAtion

T ThroughputTOPSIS The Technique for Order of Preference by Similarity to Ideal Solution

VIKOR VIsekriterijumska optimizacija i KOmpromisno ResenjeVoIP Voice over Internet Protocol

WLAN Wireless Local Area NetworkWLS Weighted Least Squares


Bibliography 116

Bibliography

[1] R. Bikmukhamedov, Y. Yeryomin, and J. Seitz, “Evaluation of MCDA-based Han-dover Algorithms for Mobile Networks,” in The Eighth International Conferenceon Ubiquitous and Future Networks, 2016.

[2] Ahmed, L. M. Boulahia, and D. Gaiti, “Enabling Vertical Handover Decisionsin Heterogeneous Wireless Networks: A State-of-the-Art and A Classification,”IEEE Communications Surveys & Tutorials, vol. 16, no. 2, pp. 776–811, 2014.

[3] S. A. Lusheng Wang and G.-S. G. S. Kuo, “Mathematical Modeling for NetworkSelection in Heterogeneous Wireless Networks — A Tutorial,” IEEE Communi-cations Surveys & Tutorials, vol. 15, no. 1, pp. 271–292, 2013.

[4] D. Charilas and A. Panagopoulos, “Multiaccess Radio Network Environments,”IEEE Vehicular Technology Magazine, 2010.

[5] C. Hwang, Y. Lai, and T. Liu, “A New Approach for Multiple Objective DecisionMaking,” Computers & Operations Research, 1993.

[6] S. Opricovic and T. G., “Extended VIKOR Method in Comparison with Outrank-ing Methods,” European journal of operational research, 2006.

[7] Huszak and S. Imre, “Eliminating Rank Reversal Phenomenon in GRA-BasedNetwork Selection Method,” in Communications (ICC), 2010 IEEE InternationalConference on, 2010, pp. 1–6.

[8] R. Trestian, A. Moldovan, C. H. Muntean, O. Ormond, and G.-M. Muntean,“Quality Utility Modelling for Multimedia Applications for Android Mobile de-vices,” in Broadband Multimedia Systems and Broadcasting (BMSB), 2012 IEEEInternational Symposium on, 2012, pp. 1–6.

[9] “QoS - voip-info.org.” [Online]. Available: http://www.voip-info.org/wiki/view/QoS


http://www.voip-info.org/wiki/view/QoS

http://www.voip-info.org/wiki/view/QoS

Bibliography 117

[10] X. Wang, “Study of Ranking Irregularities When Evaluating Alternatives by UsingSome Electre Methods and A Proposed New MCDM Method Based On RegretAnd Rejoicing,” Master’s thesis, Louisiana State University and Agricultural andMechanical College, 2007.

[11] Q.-T. Nguyen-Vuong, Y. Ghamri-Doudane, and N. Agoulmine, “On Utility Mod-els for Access Network Selection in Wireless Heterogeneous Networks,” in Net-work Operations and Management Symposium, 2008. NOMS 2008. IEEE, 2008,pp. 144–151.

[12] J. Dodgson, M. Spackman, P. A, and L. Phillips, “Multi-Criteria Analysis: AManual,” Department for Communities and Local Government: London, Tech.Rep., 2009.

[13] S. Maaloul, M. Afif, and S. Tabbane, “A Comparative Study of AHP vs ANPModels for Weighting the Context-Awareness Criteria Process in the Radio AccessNetwork Selection,” in Wireless Communication Systems (ISWCS 2013), Proceed-ings of the Tenth International Symposium on, 2013, pp. 1–5.

[14] L. Wang and D. Binet, “TRUST: A Trigger-Based Automatic Subjective Weight-ing Method for Network Selection,” in Telecommunications, 2009. AICT ’09. FifthAdvanced International Conference on, 2009, pp. 362–368.

[15] P. Fishburn, “Additive Utilities with Incomplete Product Set: Applications toPriorities and Assignments,” Operations Research Society of America, 1967.

[16] D. Julong, “Introduction to Grey System Theory,” The Journal of Grey System1, 1988.

[17] S. Arora and B. Barak, Computational Complexity: A Modern Approach. Internetdraft, 2007.

[18] L. Suciu, K. Guillouard, and J. Bonnin, “A Methodology for Assessing the Verti-cal Handover Algorithms in Heterogeneous Wireless Networks,” in InternationalWorkshop on Broadband Wireless Access for ubiquitous Networking, 2006.

[19] A. G. Davalos, L. Escobar, A. Navarro, A. Arteaga, F. Guerrero, and C. Salazar,“Vertical Hando� Algorithms — A New Approach for Performance Evaluation,”in Proc. IEEE Globecom Workshops, Dec. 2010, pp. 1724–1728.


Bibliography 118

[20] M. Lahby, C. Leghris, and A. Adib, “An Enhanced Evaluation Model For Verti-cal Handover In Heterogeneous Networks,” in International Journal of ComputerScience Issues, 2012.

[21] H. G. Msakni and H. Youssef, “Is QoE Estimation Based on QoS Parameters Suf-ficient for Video Quality Assessment?” in Proc. 9th Int. Wireless Communicationsand Mobile Computing Conf. (IWCMC), Jul. 2013, pp. 538–544.

[22] A. V. Arunachalam, “Quality of Service (QoS) classes for BWA,” IEEE 802.16Broadband Wireless Access Working Group, Tech. Rep., 1999.

[23] S. Jelassi, G. Rubino, H. Melvin, H. Youssef, and G. Pujolle, “Quality of Expe-rience of VoIP Service: A Survey of Assessment Approaches and Open Issues,”IEEE Communications Surveys Tutorials, vol. 14, no. 2, pp. 491–513, 2012.

[24] R. A. Farrugia, C. Galea, S. Zammit, and A. Muscat, “Objective Video QualityMetrics for HDTV Services: A Survey,” in Proc. IEEE EUROCON, Jul. 2013,pp. 170–176.

[25] J. Joskowicz and J. C. L. Ardao, “A General Parametric Model for PerceptualVideo Quality Estimation,” in Proc. IEEE Int. Workshop Technical CommitteeCommunications Quality and Reliability (CQR 2010), Jun. 2010, pp. 1–6.

[26] J. Joskowicz, R. Sotelo, and J. C. L. Arado, “Comparison of Parametric Modelsfor Video Quality Estimation: Towards a General Model,” in Proc. IEEE inter-national Symp. Broadband Multimedia Systems and Broadcasting, Jun. 2012, pp.1–7.

[27] S. Egger, P. Reichl, T. Hossfeld, and R. Schatz, “Modeling Web Quality-of-Experience on Cellular Networks,” in Communication QoS, Reliability and Mod-eling Symposium, 2012.

[28] S. Egger, P. Reichl, T. Hoßfeld, and R. Schatz, ““Time is bandwidth ”? Narrowingthe gap between subjective time perception and Quality of Experience,” in Proc.IEEE Int. Conf. Communications (ICC), Jun. 2012, pp. 1325–1330.

[29] Sandvine, “Measuring Web Browsing Quality of Experience: Requirements forGaining Meaningful Insight, version 1.1,” White Paper, 2015.


Bibliography 119

[30] M. Fiedler, T. Hossfeld, and P. Tran-Gia, “A Generic Quantitative Relationshipbetween Quality of Experience and Quality of Service,” IEEE Network, vol. 24,no. 2, pp. 36–41, Mar. 2010.

[31] H. J. Kim, D. H. Lee, J. M. Lee, K. H. Lee, W. Lyu, and S. G. Choi, “The QoEEvaluation Method through the QoS-QoE Correlation Model,” in Proc. FourthInt. Conf. Networked Computing and Advanced Information Management NCM’08, vol. 2, Sep. 2008, pp. 719–725.

[32] H. J. Kim and S. G. Choi, “A Study on a QoS/QoE Correlation Model for QoEEvaluation on IPTV Service,” in Proc. 12th Int Advanced Communication Tech-nology (ICACT) Conf, vol. 2, Feb. 2010, pp. 1377–1382.

[33] ETSI, “LTE; Evolved Universal Terrestrial Radio Access Network (E-UTRAN);Self-configuring and self-optimizing network (SON) use cases and solutions (3GPPTR 36.902 version 9.3.1 Release 9),” ETSI, Tech. Rep., 2011.

[34] A. Varga, “The OMNeT++ Discrete Event Simulation System,” in In ESM’01,2001.

[35] C. Mattos, E. Ribeiro, and C. Pedroso, “A New Model for VoIP Tra�c Gen-eration,” in The 7th International Telecommunications Symposium (ITS 2010),2010.

[36] A. Reaz, D. Murayama, K. I. Suzuki, N. Yoshimoto, G. Kramer, and B. Mukher-jee, “Synthetic Tra�c Generation for Streaming Video to Model IPTV,” inProc. Fifth IEEE Int. Conf. Advanced Telecommunication Systems and Networks(ANTS), Dec. 2011, pp. 1–6.

[37] Z. Liu, K. Wang, W. Li, Q. Xiao, D. Shi, and G. He, “Measurement and ModelingStudy of IPTV CDN network,” in Proc. IEEE Int. Conf. Network Infrastructureand Digital Content, Nov. 2009, pp. 302–306.


Erklärung 120

Erklärung

Die vorliegende Arbeit habe ich selbstständig ohne Benutzung anderer als der angegebe-nen Quellen angefertigt. Alle Stellen, die wörtlich oder sinngemäß aus verö�entlichtenQuellen entnommen wurden, sind als solche kenntlich gemacht. Die Arbeit ist in gle-icher oder ähnlicher Form oder auszugsweise im Rahmen einer oder anderer Prüfungennoch nicht vorgelegt worden.

Kazan, den 05. 08. 2016 Radion Bikmukhamedov


Technische Universität...

Documents

Transcript of Technische Universität...