1Georg-August-Universität Göttingen Professur für Produktion und Logistik Platz der Göttinger Sieben 3 (Raum 1.223) D-37073 Göttingen 2Universität Karlsruhe (TH) Institut für Industriebetriebslehre und Industrielle Produktion (IIP) Hertzstr. 16 D-76187 Karlsruhe

Quantifying Eco-Efficiency with Multi-Criteria Analysis

Prof. Dr. Jutta Geldermann1

Dr. Martin Treitz2

Research Paper Nr. 5

Göttingen, Oktober 2008

ISSN 1867-0121 Kontakt:

Herausgeber: Prof. Dr. Jutta Geldermann Schwerpunkt Unternehmensführung Professur für Produktion und Logistik Wirtschaftswissenschaftliche Fakultät Wirtschaftswissenschaftliche Fakultät Georg-August-Universität Göttingen Georg-August-Universität Göttingen Prof. Dr. Jutta Geldermann (Produktion und Logistik) Platz der Göttinger Sieben 3 Prof. Dr. Lutz M. Kolbe (Informationsmanagement) D-37073 Göttingen Prof. Dr. Klaus Möller (Controlling) Tel. +49 551 39 7257 Prof. Dr. Olaf N. Rank (Organisation) Fax +49 551 39 9343 Platz der Göttinger Sieben 3, D-37073 Göttingen Mail produktion@wiwi.uni-goettingen.de Web www.wiwi.uni-goettingen.de/man Web www.produktion.uni-goettingen.de

- 1 -

Quantifying Eco-Efficiency with Multi-Criteria Analysis

Abstract

Based on the efficiency definition by (Koopmans, 1951) a case study is presented in this paper comparing the results of a multi-criteria method and an eco-efficiency analysis for emerging technologies for surface coating. Multi-criteria analysis aims at resolving incomparabilities by incorporating preferential information in the relative measurement of efficiency during the course of an ex-ante decision support process. The outranking approach PROMETHEE is employed in this paper for the case study of refinish primer application with data from an eco-efficiency analysis presented by (Wall et al., 2004; Richards and Wall, 2005). Comprehensive sensitivity and uncertainty analyses (including the first implementation of the PROMETHEE VI sensitivity tool) elucidate the variability in the underlying data and the value judgements of the decision makers. These advanced analyses are considered as the distinct advantage of MCA in comparison to the eco-efficiency analysis (Saling et al., 2002), which just comprises various types of normalisation of different criteria.

1 Introduction

The term eco-efficiency was coined by the World Business Council for Sustainable Development (WBCSD), comprising almost 200 international companies in a shared commitment to sustainable development through economic growth, ecological balance, and social progress (WBCSD, 2006). The concept of eco-efficiency has emerged as one of the crucial themes linking the economy and environment and presenting opportunities for joint improvement in economic and environmental performance. However, methods for quantified eco-efficiency analyses for the comparison of the sustainability of different alternatives are in their early stages of development although the need for comprehensive evaluations of different technological options is well acknowledged. Such analyses require the simultaneous consideration of different mass and energy flows and economic performance, leading to a multi-criteria problem cause by various units of measurements and goal conflicts. This became most obvious in the European Union when the Directive on the Integrated Pollution Prevention and Control (IPPC 96/61/EG) was adopted, aiming “to achieve a high level of protection for the environment taken as a whole” (art. 1). The assessment and comparison of effects of industrial installations call for suitable approaches to gauging the effectiveness of these measures. A special technical challenge is to avoid the shift of environmental problems from one medium to another. Thus, an information exchange on Best Available Techniques (BAT) was organised by the European Commission for all industrial activities with a significant contribution to environmental pollution as listed in Annex I of the Directive. BAT covers all aspects of the technology used in production and in the way that installations are

- 2 -

designed, built, maintained, and decommissioned. BAT means using the most effective economically and technically viable means to achieve a high level of protection for the environment and for human health and safety. In this way, BAT delivers a comprehensive description of aspects relevant for eco-efficiency.

The first round of 31 “BAT reference documents” (BREFs for short) has been completed by end of 2006, with the last three BREFs related to ceramic manufacturing, large volume inorganic chemicals and surface treatment using organic solvents. Altogether, around 55.000 installations are covered by the IPPC Directive, encompassing an immense economic dimension. It is a significant challenge both in terms of environmental protection and competitiveness to regulate and operate all these industrial installations in a successful way. In spite of some criticism, (Hitchens et al., 2001) come to the conclusion that the IPPC Directive did not hamper the competitiveness of the European Industry but rather promoted innovation and deployment of environmentally friendly technologies. Especially linkages between environment and energy savings are important aspects given the current developments on energy markets.

While the definition of BAT is focused on “available” techniques1, the information exchange also includes sections on so-called “Emerging Technologies”, being developed either by companies or institutes and having the potential to become available in the near future. As companies keep their innovations confidential for competition reasons, institutes provide an open policy in publication but might lack practical experience in scale-ups. Thus, if companies want to introduce innovative technologies in order to improve the eco-efficiency of their production processes, it is important to have credible and reliable information for prospective analyses. Since innovative or emerging technologies have to compete with existing technologies on economic, technical, ecological, and social aspects, the effectiveness in all these dimensions needs consideration.

This paper describes and discusses the quantification of eco-efficiency by Multi-Criteria Analysis (MCA), especially for emerging technologies. Section 2 gives an introduction to the problem of defining eco-efficiency. The application of MCA is illustrated in Section 4 with a case study about emerging technologies for surface treatment. Six types of primers and their application techniques in vehicle refinishing are being compared on the basis of data delivered by (Wall et al., 2004; Richards and Wall, 2005) with an eco-efficiency analysis. Preferential information is modelled by weighting factors and preference functions based on paired comparisons within the Outranking approach PROMETHEE. Special emphasis is put on comprehensive sensitivity analyses. Finally, Section 5 summarises the findings.

1 Article 2, para. 11 gives the following definition: “'available' techniques shall mean those developed on a scale which allows implementation in the relevant industrial sector under economically and technically viable conditions, taking into consideration the costs and advantages, whether or not the techniques are used or produced inside the Member State in question, as long as they are reasonably accessible to the operator”.

- 3 -

2 The Problem of Defining Eco-Efficiency

The optimal allocation of resources to maximise the desired output for the given input is the core question in business economics (Koopmans, 1951; Koopmans, 1975). In the context of thermodynamic processes (which underlie many environmentally relevant production processes) the input and output parameters are often limited to energy quantities, such as the transferred or converted energy compared to the employed energy (for example in a power station, wind turbine, etc.). However, this definition considers only the heat quantity and not the quality, e.g. temperature, which is relevant for defining its convertibility in, for example, refrigeration systems and thus for its economic value (for a discussion see (Grassmann, 1950)). If it is not possible to define a single common denominator, such as the heat content, the definition of the degree of efficiency is quite difficult and can only be based on relative comparisons. Consequently, the allocation of resources is efficient if no improvement (i.e., an addition to the output of one or more goods at no cost to the others) is possible. This relative efficiency definition is called ‘Pareto efficiency‘ and a possible improvement is referred to as a ‘Pareto improvement’ or ‘Pareto optimisation’ (cf. e.g., (Moffat, 1976)). Mathematically, every Pareto efficient point in the commodity space is equally acceptable. Trade-offs and compromises are to be made when moving from one efficient point to another.

The definition of eco-efficiency in the context of technique assessment is complex since ecological, economic, technical and social parameters must be considered and representative ones selected. As the discussion for public goods and external costs shows, no competitive markets exist which could guide resources to their maximised utility (cf. e.g. (Rabl and Eyre, 1998; Schleisner, 2000)). Hence, no common denominator for eco-efficiency exists, and only relative comparisons can lead to value judgements. Relative efficiency measurements are the starting point for the Data Envelopment Analysis (DEA) and Multi-Criteria Analyses (MCA), two different approaches to resolving incomparabilities in a technique assessment, which are briefly introduced in the following. After that, approaches for Life Cycle Assessment (LCA) and particularly the so-called eco-efficiency methodology are being compared to the more formal MCA approach.

2.1 Data Envelopment Analysis (DEA)

The Data Envelopment Analysis (DEA) is an approach to comparing the relative efficiency of so-called ‘decision making units’ (DMUs) in general. The decision making units are characterized by their vector of external inputs and outputs. By using scalarizing functions, the inputs and outputs are aggregated to an efficiency measure for each unit (Charnes et al., 1978; Belton and Stewart, 1999; Kleine, 2001; Cooper et al., 2004; Kleine, 2004). DEA has been developed for the evaluation of non-profit organisations, whose inputs and outputs can hardly be monetarily valued with market prices and are therefore more difficult to compare.

- 4 -

In general, DEA assumes that inputs and outputs are ‘goods’, but from an ecological perspective also pollutants with negative properties have to be considered. Thus, „ecologically extended DEA models“ have been derived by incorporating a multi-dimensional value function (Dyckhoff and Allen, 2001). The fact that no explicit weights are needed to aggregate the indicators is seen as an advantage. Nevertheless, it is possible to integrate preferential information into DEA (Korhonen et al., 2002; Mavrotas and Trifillis, 2006).

Recently, there are more and more applications of DEA for technique assessment or technology selection and environmental performance measuring (Sarkis and Weinrach, 2001; Keh and Chu, 2002; Zaim, 2004; Zhou et al., 2006; Kuosmanen and Kortelainen, 2007). Especially in the context of the regulation of the energy market and particularly the electricity distribution, DEA benchmarking has been tested for various large samples (Korhonen and Luptacik, 2004; Estellita Lins et al., 2007).

It can be concluded that the application of DEA is useful for the ex-post evaluation of many similar organisations but is less suitable for the comparison of few emerging technologies. A crucial point of the DEA is the determination of the efficiency frontier, and thus the virtual efficient production process to which the real existing organisations are compared. Such a virtual efficient technology cannot be constructed by any combination of existing technologies and will never exist in reality.

2.2 Multi Criteria Analysis (MCA)

The efficiency definition is also the starting point for Multi Criteria Analysis (MCA). In a decision problem, all non-dominated alternatives are called efficient.2 Through special focus on the dominance relation multi-criteria methods seek to reduce incomparabilities by explicitly incorporating preferential information of the decision maker (Brans and Mareschal, 2005). The research field of Multi Criteria Analysis comprises methods for Multi Attribute Decision Making (MADM), covering the assessment of a finite set of alternatives (discrete solution space), and Multi Objective Decision Making (MODM) focussing on alternatives restricted by constraints (continuous solution space). The comparison of emerging technologies calls for MADM, for which two main streams exist (Belton and Stewart, 2002; Figueira et al., 2005):

• the ‘classical’ approaches, which are based on the assumption that clear judgements exist about utility values of the attributes and their weightings, which can be formalised within the multi-criteria technique. Examples are the Multi Attribute Value/Utility Theory (MAVT/MAUT) or the Analytical Hierarchy Process (AHP).

2 Alternatives are dominated if there is another alternative that is not worse in any attribute and better in at least one.

- 5 -

• the Outranking approaches, which suppose that the preferences are not apparent to the decision maker, and therefore the decision support aims at giving insights into the consequences of different weightings. The main difference to the ‘classical’ MCA methods lies in the consideration of weak preferences and incomparable criteria. The most prominent Outranking models are ELECTRE and PROMETHEE.

Both ‘classical’ and Outranking approaches structure the decision making process and thus support the understanding of preferences. During the last decades, behavioural aspects of decision making became more important (French et al., 1998; Pöyhönen et al., 2001; Hodgkin et al., 2005), while comparisons of different algorithms are no longer in the focus of the scientific debate (Lootsma, 1996; Simpson, 1996). Thus, MCA can be considered as mature, which explains its wide use in environmental contexts (Miettinen and Hämäläinen, 1997; Seppälä et al., 2002), in technique assessment (Geldermann et al., 2000; Geldermann and Rentz, 2001), and in technology foresighting (Gustafsson et al., 2003). It can provide support for the decision maker in his/her quest for better understanding of the interdependencies in the weighting of environmental criteria. However, this discussion is highly controversial and it is important to note that some authors favour a more technical approach, whilst others stress the importance of detailed stakeholder involvement because of context sensitivity and the significant influence on the overall results (Joubert et al., 1997; Prato, 1999; Brouwer and van Ek, 2004; Munda, 2004b).

In contrast to DEA, which aims at an ex-post evaluation of many similar units for the purpose of monitoring and control, the objective of MCA is the ex-ante assessment of a few individual options by explicitly considering the subjective preferences of a decision maker for the purpose of decision support, planning and choice, (Belton and Stewart, 1999). The selection of one final solution out of the mathematically equivalent set of Pareto optimal solutions is not a purely mathematical question which can be addressed objectively. Thus, the explicit acknowledgement of the remaining subjectivity within decision processes is the goal of multi-criteria methods (see also (Munda et al., 1995; Martinez-Alier et al., 1998; Munda, 2004a).

Concerning the efficiency definition it can be said that MCA provides a partial ordering of the alternatives. Multi-criteria methods try to reduce these incomparabilities by incorporating preference models. A wide range of literature about elicitation of value judgements, preference modelling, and the ability of the decision maker to provide this information exists (see (Weber et al., 1988; Weber and Borcherding, 1993; Belton and Stewart, 1999; Mustajoki and Hämäläinen, 2000) for more information). It is also acknowledged that the weighting within MCA is context specific since no objective values exist. Apart from the classical weighting techniques (e.g. direct ratio, SWING, SMART (Edwards, 1977; Winterfeld and Edwards, 1986), SMARTER (Edwards, 1977; Barron and Barret, 1996), eigenvector method (Saaty, 1980) etc.), specialised methods are discussed in the case of environmental impacts. The normalisation of attribute weights in these various approaches has been identified as a procedural source of biases (Salo and Hämäläinen, 1997; Pöyhönen and Hämäläinen, 2001).

- 6 -

Another classical problem in MCA is the phenomenon of rank reversals, in particular, when a new decision alternative, which is added to an otherwise unchanged decision problem, causes a reordering of the previous rank order (Keyser and Peeters, 1996; Simpson, 1996). But (Leskinen and Kangas, 2005) show that rank reversals caused by inconsistency are natural and acceptable and that geometric-mean aggregation (instead of the traditional arithmetic-mean aggregation rule) does not cause undesired rank reversals.

2.3 Life Cycle Assessment (LCA)

Over the last 35 years, life cycle assessment (LCA) has been developed in varying forms to evaluate the environmental impacts of products, services, or processes ‘from cradle-to-grave’. The International Standards Organisation (ISO) has established guidelines and principles for LCA in order to provide information for decisions regarding product development and eco-design, production system improvements, and product choice at the consumer level (cf. (Wrisberg et al., 2002; Ness et al., 2007). LCA may provide qualitative or quantitative results. The latter makes it easier to identify problematic parts of the life-cycle and to specify what gains can be made with alternative ways of fulfilling the function. Choosing between ecological profiles involves balancing different types of impact and is thus typical of a multi-criteria decision problem when explicit or implicit trade-offs and pair-wise comparisons are needed to construct an overall judgment. Consequently, the combination of LCA with a subsequent Multi Criteria Analysis has been proposed by various authors (Maystre et al., 1994; Geldermann and Rentz, 2001; Belton and Stewart, 2002; Seppälä et al., 2002; Keefer et al., 2004; Hämäläinen, 2004).

Within LCA four basic steps are identified in ISO 14044:2006 - Environmental management - Life cycle assessment - Requirements and guidelines. It resembles the ubiquitous decision analytic cycle with the main phases: problem formulation, evaluation of options, and review of the decision models (Geldermann and Rentz, 2004b; French and Geldermann, 2005). Table 1 summarises the modules for an integrated technique assessment, comprising ecological and techno-economic aspects.

Decision support is particularly necessary for the simultaneous consideration of ecological, economic, technical, and social criteria when quantifying the eco-efficiency of emerging technologies. In the majority of practical decision problems no alternative exists that is the best in all criteria. In fact, each alternative offers both strengths and weaknesses, which must be counterbalanced. Therefore, the use of multi-criteria analysis (MCA) is suggested in order to support structuring the problem, formalising the trade-offs between the alternatives and fostering the transparency of the decision.

- 7 -

Table 1: Modules of an integrated technique assessment

Phases of Life Cycle

Assessment according to

ISO 14044

Modules of an integrated

technique assessment

Tasks

Goal and scope definition

Goal definition Selection of scenarios

Setting of system boundaries

Definition of functional units

Inventory analysis

Mass and Energy Balance Data collection

Modelling of relevant mass and

energy flows

Impact assessment

Calculation of Characteristic

Figures

Calculation of technical and

economic characteristic figures

Calculation of impact potentials and

critical volumes for ecological

evaluation

Interpretation

Decision Support Data editing and Normalisation

Weighting of the criteria

Multi Criteria Analysis

Sensitivity Analysis

2.4 The Eco-Efficiency Analysis

LCA has been further developed into the “eco-efficiency” analysis by BASF and has been applied in more than 220 analyses of products ranging from vitamins to basic chemicals (Saling et al., 2002; Wall et al., 2004; Saling et al., 2005). Based on the ISO14044 standards, some additional enhancements like an economic assessment allow for expedient review and decision-making at all business levels. A modular design aims at focusing on decision-relevant aspects by concentrating on specific events in a life cycle where the alternatives

- 8 -

under consideration differ. The authors state that “the representation of a multiplicity of individual results from the actual life cycle assessment is frequently opaque, difficult to interpret and thus not very meaningful.” (Saling et al., 2002), p.11) In order to overcome these difficulties, schemes of weighting factors have been developed, such as those based on relevance factors indicating how important the individual environmental compartment is for a particular eco-efficiency analysis. Societal views of the individual ecological impact categories were reflected through surveys, public opinion polling and expert interviews. Much emphasis is put on catchy presentations of the results as done by the Boston Consulting Matrix or Spider Diagrams.

2.5 Normalisation as a common difficulty in quantifying eco-efficiency

Well known approaches addressing the issue of valuation of potential environmental impacts are the Cumulative Energy Demand, Eco Indicator method, environmental pressure indicators, MIPS (Material Input per Unit Service), the calculation of shadow prices based on abatement costs, determination of weights based on eco-taxes, panel methods, or the total relevance using verbal predicates based on ecological relevance and specific contribution (Berkel and Lafleur, 1997; Soest et al., 1998; Wrisberg et al., 2002; Ness et al., 2007). All these approaches have the general quest for normalisation in common. During the normalisation procedure the measure of the performance of alternatives is modified to be comparable, thus ensuring the applicability of preference or utility aggregation while still considering all decision-relevant criteria.

In LCA software several data conversion techniques are applied to transform the information supplied by the various data sources like the “person equivalent”, which expresses the environmental impact or resource consumption that an average person contributes per year worldwide, in Europe, or in a specific country (van Oers and Huppes, 2001; Huijbregts et al., 2003). Similarly, the specific contribution is defined as the share of the collected results of the average impact potential in relation to the total impact potential for the EU in the respective impact category for a year. Data availability, however, limits the applicability of such normalisation approaches in many case studies.

In MCA normalisation is achieved by scale transformations through which measures of preference, value, or utility are derived. MCA can be applied to interpret the impact category weights within an LCA and can offer rules for aggregation and normalisation (Seppälä and Hämäläinen, 2001). Though weighting of criteria is unavoidable (since even ‘no weighting’ would imply an equal weighting of all considered criteria), it causes problems as case specific normalisation may not be compatible with generally established weighting factors.

In the context of MCA, effects like rank reversals (Keyser and Peeters, 1996; Simpson, 1996; Leskinen and Kangas, 2005), independence of irrelevant alternatives (Sen, 1970; Neumann, 2007), and splitting effects of weights (Weber and Borcherding, 1993) have been investigated for a long time. Some of these effects can be explained by the fact that the consideration of further alternatives in the current decision problem enhances the knowledge

- 9 -

base, leading to a different view of the decision maker on the decision problem (Keyser and Peeters, 1996). Therefore, it is always emphasized that MCA can only support decision making by delivering a structured image of the often contradictory and inconsistent reality, but can never replace the decision maker.

3 Case Study

Though quantifying eco-efficiency still lacks a consistent theory and ranked preferences may be prone to undesired rank reversals,3 decisions about emerging technologies must be made. The aim of this case study is the critical reflection of the eco-efficiency analysis as developed and applied by BASF. Although a company can base its decisions on whatever grounds it likes, so long as its shareholders and stakeholders do not contradict, this special approach is much being promoted (Saling 2002; Saling et al., 2005). Thus, a genuine case study performed by (Wall et al., 2004; Richards and Wall, 2005) is re-analysed with MCA. The starting point for the recalculation of the case study was an initiative by an informal working group of various European coating producers on environmental assessment. They were uncertain if all coating producers should apply the BASF eco-efficiency analysis or if other approaches would be beneficial or even more meaningful. Though these practitioners had been aware of MCA in general, they were sceptical about too scientific and sophistic approaches.

Many manufactured items receive surface coatings for decoration and/or protection against damage. In a number of places along the production line emissions of VOC (Volatile Organic Compounds) can occur. Because VOC are a major contributor to photochemical smog, control of VOC emissions is a major concern for the industry’s commitment to the environment. That is why a special BREF is dedicated to surface treatment using solvents. Not only large installations, such as serial coating of automobiles, are under consideration but also small and medium sized companies like vehicle refinishing, which refers to the commercial application of paint coating materials to automobiles, trucks, motorcycles, and other like equipment, often as a repair after accidents (Geldermann and Rentz, 2004a). Thus, coating manufacturers, such as BASF (Wall et al., 2004), have to supply appropriate coating materials and introduce innovative technologies. One example are UV-cured undercoats, which promise enhanced performance, higher productivity and an improved eco-efficiency compared to traditional thermally cured coatings. Ultraviolet (UV) curing is a process in which UV energy produced by a mercury discharge lamp is absorbed by a sensitizer, causing a reaction in the monomer which makes it hard and dry (Koleske, 2002). These combinations of new coating products and matched application technologies are

3 Various Nobel Price winners like K. Arrow, G. Debreu or A. Sen pointed out these difficulties in their famous works, and no satisfying solutions are ready at hand up until now. Thus, mankind will still have to live with some imperfections.

- 10 -

currently considered as emerging technologies in the surface coating sector (Geldermann et al., 2007).

Although other coating layers (e.g. base-coat, clear-coat) also have to be applied during the refinishing process, only the primer application is considered in this case study. Primer means any coating that is designed for application to bare metal or existing finishes to provide corrosion protection prior to the application of a surfacer, which is applied later for the purpose of corrosion resistance, to ensure adhesion of the topcoat, and to promote the formation of a uniform surface finish by filling in minor surface imperfections. It is assumed that these subsequent process steps and products to be used are independent from the type of the primer applied. After its application the primer must be dried and cured, changing its state from liquid to a continuous solid film. Therefore, heat is applied either directly (by thermal drying) or indirectly (by UV or infra red drying), which results in the evaporation of the solvents and the film building of the primer. If the product contains no solvents, the coating layer is formed by chemical reactions between the components. In addition to the application step itself, related activities like, for example, cleaning of the surface and the equipment are taken into account. Furthermore, the extra fuel consumption of the automobile over its lifetime resulting from the weight of the dried primer is considered. Table 2 summarises the used data for the six alternatives (Wall et al., 2004; Richards and Wall, 2005),:

(i) One-Component UV Curing Primer (1K UV),

(ii) One-Component UV Curing Primer, using an aerosol can (1K UV-A).

(iii) Conventional Two-Component Urethane Primer with thermal drying (2K U-T),

(iv) Two-Component Urethane with infra red drying (2K U-IR),

(v) Epoxy with thermal drying (E-T),

(vi) Epoxy with infra red drying (E-IR)

The weights of the environmental criteria have been taken form the underlying case study (Wall et al., 2004; Richards and Wall, 2005), while the weighting of costs has been assumed to be 50%.

Table 2: General Data of the Case Study (Wall et al., 2004; Richards and Wall, 2005)

Criterion Unit Weight 1K UV 1K UV-A 2K U-T 2K U-IR E-T E-IR

Total Costs [$/job] 50 % 14.58 22.00 27.65 19.05 24.94 18.37Energy [MJ/CB] 12.5% 30 25 800 35 690 35

GWP [g CO2-Eq. /CB] 5% 1 700 1 700 52 000 2 000 45 500 1 700

POCP [g Ethene-Eq./CB] 5% 5.5 20 32 26 26 24

Health Effects norm. scores 10% 0.94 0.96 0.94 0.92 0.87 0.85

- 11 -

Risk norm. scores 5% 0.41 0.71 0.91 0.99 0.85 0.93

Resources scores 12.5% 100 105 375 120 330 105

GWP = Global Warming Potential, POCP = Photochemical Oxidant Creation Potential, CB = Coated body

3.1 Application of the Eco-Efficiency Analysis

As outlined in Section 2.4, the eco-efficiency analysis mainly follows the ISO 14044 phases (Wall et al., 2004; Richards and Wall, 2005). Following a company specific normalisation and weighting, the corresponding arithmetic values are summarized in a special plot, called “environmental fingerprint”, which illustrates the relative ecological pros and cons of the analysed alternatives. The least favourable alternative is assigned a value of 1, and the further inward an alternative is located, the better it is. Thus, Figure 1 presents alternative 2K U-T and E-T as the two worst alternatives, since their profiles cover the largest areas, while 1K UV comes out best with the smallest area. The alternatives are, however, hardly distinguishable in such a graphical representation. The environmental fingerprint highlights the environmental impact drivers and points out improvement potentials. Finally, an Eco-Efficiency portfolio (in analogy to the growth-share matrix developed by Boston Consulting in 1970) consolidates all of the individual environmental and economic results into one representation (Wall et al., 2004).

Figure 3 shows that the alternatives E-T and 2k U-T are the worst alternatives when taking the environmental impact and costs into account.

Section 2.5 already commented on the difficulties inherent in normalisation in general. Since this eco-efficiency analysis employs BASF-specific normalisation and weighting factors, the transfer of this approach to quantifying the eco-efficiency of emerging technologies of all European coating producers is problematic. Another methodological question touches upon the interpretability of the graphical representations: Does the size of the “ecological fingerprint” really relate to the area marked by the connected points in Figure 1? This is certainly not the case because neither the angle between the axes nor their length reflects the weighting of the depicted alternatives.

- 12 -

- 13 -

Figure 1: Ecological Fingerprint of six automotive refinish primers (Wall et al., 2004)

Figure 2: Ecological fingerprint of six automotive refinish primers (Wall et al., 2004)

Figure 3: Results of the Eco-Efficiency Analysis (Wall et al., 2004)

3.2 Application of an Multi Criteria Analysis (MCA)

From the numerous approaches to MCA, the Outranking approach “PROMETHEE” has been chosen because of its flexibility and understandability. Outranking can be defined in the following way: alternative at outranks at', if there is a ”sufficiently strong argument in favour of the assertion that at is at least as good as at' from the decision maker’s point of view” (Brans et al., 1986). Accordingly, the outranking relation is the result of pair-wise comparisons between the alternatives with regard to each criterion (Stewart, 1992; Roy and Bouyssou, 1993). The outranking method PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluation) (Brans et al., 1986; Brans and Mareschal, 2005) derives the preference values by generalised criteria, which can be defined by the decision maker specifically for each considered criterion. Box 1 summarizes the PROMETHEE algorithm,

- 14 -

and Figure 4 illustrates the six proposed types of generalised preference functions, as they have been implemented in Matlab (Schrader, 2005; Treitz, 2006). Although “Decision Lab 2000“ is a commercial software package for the implementation of PROMETHEE, the academic implementation in Matlab allows for further sensitivity and uncertainty analyses, as will be shown later.

First of all, the basic PROMETHEE algorithm (see Box 1) has been applied to the decision table (as shown in Table 2). The preference function Type VI (Gaussian Criterion) has been chosen for all criteria, using half of the difference between the maximal and the minimal attribute value of each criterion as the inflection point s of the respective Gaussian function (cf. Figure 4)

- 15 -

Box 1: Outline of the PROMETHEE algorithm

As with most MADM methods, PROMETHEE is built on the basic notation, with a set A of T alternatives that must be ranked, and K criteria that must be optimised:

A := {a1,...,aT} : Set of discrete alternatives, scenarios or techniques at (t = 1...T) F := {f1,...,fK} : Set of criteria relevant for the decision fk (k = 1...K)

The resulting multiple criteria decision problem can then be concisely expressed in a matrix format. The goal achievement matrix or decision matrix

,...,Kk,...,TttkxD

11=== )(: is a (T × K) matrix

whose elements xtk = fk(at) indicate the evaluation or value of alternative at , with respect to criterion fk :

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

)()()(

)()(:

1

111

1

111

TKt

tk

K

TKT

tk

K

afafaf

afaf

xxx

xxD

L

MM

L

L

MM

L

(1)

Thus, the algorithm for PROMETHEE can be summarized as follows (Brans et al., 1986):

(1) Specify for each criterion kf a generalised preference function )(dpk

(see also Figure 4).

(2) Define a vector containing the weights, which are a measure for the relative importance of each criterion, wT = [w1, ..., wk].

(3) Define the Outranking-Relation π for all the alternatives Aaa tt ∈', :

⎪⎩

⎪⎨⎧

−⋅=

→×

∑=

K

ktktkkktt afafpwaa

AA

1'' ))()((),(

]1,0[: ππ (2)

(4) As a measure of the strength of the alternatives Aat ∈ , the leaving flow (Phi plus) is

calculated: Φ+ ∑≠=

⋅=T

ttt

ttt aaT

a

'1'

' ),(1)( π

(5) As a measure of the weakness of the alternatives Aat ∈ , the entering flow (Phi

minus) is calculated: Φ- ∑≠=

⋅=T

ttt

ttt aaT

a

'1'

' ),(1)( π (4)

(6) Graphical evaluation of the outranking relation.

- 16 -

Based on Phi net (=Phi plus – Phi minus), a total preorder can be derived.

Figure 4: Six types of generalised preference functions within PROMETHEE as a MatLab Screenshot

On the basis of these parameters the One-Component UV Primer (1K UV) has the highest Net flow compared to the other alternatives and is ranked first. Figure 5 shows the aggregated outranking flows of Phi Plus (the relative strength of an alternative), Phi Minus (the relative weakness of an alternative) and the Phi Net (overall rating). The small Phi Minus

- 17 -

flow for the alternative 1K UV indicates that it has a strong performance in most criteria, whereas the small Phi Plus flow of alternative 2K U-T is a sign that this alternative is weak in most attribute values.

Figure 5: Aggregated Outranking Flows of the Alternatives

This result can also be displayed as a spider diagram of the net flows of each criterion (cf. Figure 6). With the advent of more powerful computer aid, notably in the 1990s, various visualisation techniques for MCA have been investigated (Vetschera, 1994). Quite a few decision makers prefer spider diagrams. However, their readability decreases with the number of considered alternatives and criteria. Different from the ecological fingerprint in Figure 1, the outermost alternative is the better one while the dimension of the axes depicts the preference values of the alternatives (and not only a normalisation to the highest parameter value). The spider diagram shows that the 1K UV is the dominating alternative with respect to costs, risk, and POCP, that it shares this position with other alternatives for the criteria Energy, GWP and Resources (Res), and that it is outranked only with respect to the health criterion. Hence, also a change of the weight of the different criteria will show the 1K UV primer as the outstanding alternative.

A serious drawback of spider diagrams is their potential misinterpretation if the surface marked by the connecting lines is considered as a value judgement about the “size of the ecological damage”. The screenshot in Figure 6 should certainly not evoke such an association since the axes show the PROMETHEE net flows, with the largest value as the best one (whereas the eco-efficiency analysis depicts the potential damage: the smaller, the better).

- 18 -

Alternatives:

Figure 6: Spider Diagram of the Phi Net Flows

3.3 Uncertainty Analysis

The majority of environmental decision problems involves uncertainty and risk. By their very nature, estimates and long-term forecasts, as required in LCA and for the evaluation of emerging technologies, are obviously uncertain; and a technology considered optimal on the basis of particular assumptions made today is highly unlikely to turn out optimal in the actual situation in a decade. For reviews discussing different types of uncertainty, variability, and risk see, for example, (French, 2003; Huijbregts et al., 2003; Bertsch, 2008).

Considering the above mentioned difficulties in modelling value judgements and normalisation, it is important to carry out sensitivity analyses as proposed in the ISO 14044. Sensitivity analyses investigate how the results of a model change with variations in the input variables and are therefore essential in any multi-criteria decision analysis problem. Using PROMETHEE several local and global (Saltelli et al., 2000) sensitivity analyses can be carried out (Treitz, 2006).

Especially the use of Monte Carlo Techniques and the Principal Component Analysis (PCA) (Timm, 2002) allows a simultaneous consideration of the uncertainty of the process data and the value judgements of the decision maker. Besides investigating the robustness of the decision, the strengths and weaknesses of each alternative can be addressed and so, the distinguishability of all alternatives can be evaluated (Basson, 2004). Consequently, sensitivity analyses are important to iteratively re-model the decision problem and to facilitate learning about the given problem.

The Principal Component Analysis (PCA) is based on the matrix M of the single criterion net flows in which the strengths and weaknesses of an alternative can be analysed for each criterion individually. By calculating the eigenvectors of the covariance matrix M’M and building a matrix of all eigenvectors sorted by the magnitude of the eigenvalues, the axes can be transformed (Timm, 2002). The transformation represents a rigid rotation of the old

- 19 -

axes into the new principal axes based on the matrix of the sorted eigenvectors. The component scores in the new coordinate system are uncorrelated and have maximum variance. By projecting the cloud of alternatives from the Rn onto the plane of the two eigenvectors with the largest eigenvalues (henceforth called 1st and 2nd

principal component), the so-called GAIA plane (Brans et al., 1986), as much information as possible is preserved. Apart from alternatives and criteria axes, the weighting vector (the PROMETHEE ‘decision stick’ or weighting vector π) can be projected on the GAIA plane. By defining upper and lower bounds for each weight, the convex hull of all valid weighting combinations can be projected on the GAIA plane. (Brans and Mareschal, 2005), p. 181) call the projection of the set of all extreme points of the unit vectors associated to all allowable weights the “Human Brain” or the PROMETHEE VI sensitivity tool. While this idea has been roughly sketched by (Brans and Mareschal, 2005), the implementation in Matlab is its first software realisation (Treitz, 2006).

The GAIA plane in Figure 7 illustrates the projection of all alternatives, criteria and the weighting vector. It can easily be seen that the criteria are not mutually independent (as claimed by (Saling et al., 2002) and (Wall et al., 2004)) since the projections of the criteria do not span the decision space equally, but rather point into one direction.

Figure 7 also shows the variation in value judgements with the grey area around the weighting vector π (as the PROMETHEE VI sensitivity tool) and the uncertainty in the attribute data with the scatter plot of 1,000 points created by changing all attribute values in a Monte Carlo Simulation using a standard deviation of 5 % and a normal distribution. This plot makes clear that a differentiation between the 1K UV-A alternative and the two IR alternatives (E-IR and 2K U-IR) might be difficult if the uncertainty in the underlying data is taken into account. This also becomes obvious when the small differences of their projections on the weighting vector π are compared. But even under these uncertain conditions, the 1K UV curing coat is always the best ranked alternative.

Furthermore, the selection of the preference function does not influence the ranking in its first position. This can be confirmed with a sensitivity analysis of all possible combinations of preference functions within PROMETHEE (see Figure 8).

- 20 -

Alternatives:1K UV1K UV-A2K U-T2K U-IRE-TE-IR

Criteria:

PROMETHEE VI sensitivity area

Figure 7: Illustration of Alternatives using PCA and Monte Carlo Simulation

Figure 8: Influence of the selected type of preference function (279.936 combinations)

In summary, the analysis identifies the One-Component UV Primer (1K UV) as the best alternative even if uncertainties that are possibly caused by measurement errors or estimation methods and the difficulty to formalize value judgments are taken into account. It has the least total cost thanks to simpler coating preparation, less cleaning of equipment is required and shorter cure times are possible which results in labour and energy savings. The aerosol UV-primer has similar advantages but at higher material prices. With regard to their market penetration, additional criteria should be considered, such as extra training of the coaters or initial investment (which might be necessary to change to UV curable products since most of the body fillers used in refinishing at the moment are dried using infra red).

The PROMETHEE multi-criteria analysis helped to gain further insights into the given selection or ranking problem concerning different techniques, and enabled the decision

- 21 -

maker to understand his own preferences and transparently model and communicate them. The sensitivity and uncertainty analyses in particular elucidate the variability in the underlying data and the subjective value judgments.

4 Conclusions

In the majority of practical decision problems – such as technique assessment - no alternative exists that is the best in all criteria, i.e., (almost) all alternatives are efficient. Multi-criteria analysis is one approach to resolving incomparability by incorporating preferential information in the relative measurement of efficiency during the course of an ex-ante decision support process. Various MCA approaches have been developed to support structuring the problem, formalising the trade-offs between the alternatives, and fostering the transparency of the decision since the final choice of one of the efficient alternatives is by its very nature subjective. The outranking approach PROMETHEE is employed for the case study of refinish primer application with data from an eco-efficiency analysis presented by (Wall et al., 2004; Richards and Wall, 2005). Using a multi-criteria approach supports transparent and traceable decisions. With the help of sensitivity and uncertainty analyses the variability in the underlying data and the value judgements of the decision makers can be analysed and help re-model the decision problem. These advanced analyses are considered as the distinct advantage of MCA in comparison to the eco-efficiency analysis (Saling et al., 2002), which just comprises various types of normalisation of the different criteria. MCA offers mathematically sound approaches for the quantification of eco-efficiency and for the consideration of unavoidable subjective aspects in preference modelling. Extended distinguishability analyses can help to overcome the limitations of graphical illustrations, and suitable software tools (such as the proposed implementation in MatLab) allow for various modes of reporting.

- 22 -

5 References

Barron, F.H., Barret, B.E., 1996. Decision Quality Used Ranked Attribute Weights Management Science, 42, 1515-1523.

Basson, L., 2004. Context, Compensation and Uncertainty in Environmental Decision Making; Ph. D Thesis, Department of Chemical Engineering, University of Sydney, Australia.

Belton, V., Stewart, T., 1999. DEA and MCDA: Competing or Complementary Approaches. Meskens, N, Roubens, M (eds.) Mathematical Modelling: Theory and Applications Advances in Decision Analysis. (6), 87-104; Kluwer Academic Publishers: Boston.

Belton, V., Stewart, T., 2002. Multiple Criteria Decision Analysis - An integrated approach; Kluwer Academic Press: Boston.

Berkel, R.v., Lafleur, M., 1997. Application of an industrial ecology toolbox for the introduction of industrial ecology in enterprises - II Journal of Cleaner Production, 5 (1-2), 27-37.

Bertsch, V., 2008. Uncertainty handling in multi-attribute decision support for industrial risk management; Universitätsverlag: Karlsruhe.

Brans, J.-P., Mareschal, B., 2005. PROMETHEE Methods. Figueira, J, Greco, S, Ehrgott, M (eds.) Multiple Criteria Decision Analysis - State of the Art Surveys., 163-195; Springer: New York.

Brans, J.-P., Vincke, Ph., Mareschal, B., 1986. How to select and how to rank projects: The PROMETHEE method European Journal of Operational Research, 24, 228-238.

Brouwer, R., van Ek, R., 2004. Integrated ecological, economic and social impact assessment of alternative flood control policies in the Netherlands Ecological Economics, 50 (1-2), 1-21.

Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring the efficiency of decision making units European Journal of Operational Research, 2 (6), 429-444.

Cooper, W.W., Seiford, L.M., Zhu, J., 2004. Data Envelopment Analysis: History, Models and Interpretations. Cooper, W W, Seiford, L M, Zhu, J (eds.) International Series in Operations Research and Management Science Handbook on Data Envelopment Analysis. (1), 1-40; Kluwer: Boston.

Dyckhoff, H., Allen, K., 2001. Measuring the ecological efficiency with data envelopment analyis (DEA) European Journal of Operational Research, 132, 312-325.

Edwards, W., 1977. How to use Multiattribute Utility Measurement for Social Decision Making IEEE Transactions on Systems, Man, and Cybernetics, SMC-7, 326-340.

- 23 -

Estellita Lins, M.P., Vervloet Sollero, M.K., Calôba, G.M., Moreira da Silva, A.C., 2007. Integrating the regulatory and utility firm perspectives, when measuring the efficiency of electricity distribution European Journal of Operational Research, 181 (3), 1413-1424.

Figueira, J., Greco, S., Ehrgott, M., 2005. Multiple Criteria Decision Analysis - Stae of the Art Surveys; Springer: New York.

French, S., Simpson, L., Atherton, E., Belton, V., Dawes, R., Edwards, W., Hämäläinen, R.P., Larichev, O., Lootsma, F., Pearman, A., Vlek, C., 1998. Problem formulation for multi-criteria decision analysis: Report of a workshop Journal of Multi-Criteria Decision Analysis, 7, 242-262.

French, S., 2003. Modelling, making inferences and making decisions: the roles of sensitivity analysis TOP, 11 (2), 229-252.

French, S., Geldermann, J., 2005. The varied contexts of environmental decision problems and their implications for decision support Environmental Policy and Science, 8 (4), 378-391.

Geldermann, J., Schollenberger, H., Rentz, O., Huppes, G., van Oers, L., France, C., Nebel, B., Clift, R., Lipkova, A., Saetta, S., Desideri, U., May, T., 2007. An integrated scenario analysis for the metal coating sector in Europe Technological Forecasting and Social Change, 74 (8), 1482-1507.

Geldermann, J., Rentz, O., 2001. Integrated technique assessment with imprecise information as a support for the identification of Best Available Techniques (BAT) OR Spectrum, 23, 137-157.

Geldermann, J., Rentz, O., 2004a. Decision support through mass and energy flow management in the sector of surface treatment Journal of Industrial Ecology, 8 (4), 173-187.

Geldermann, J., Rentz, O., 2004b. Environmental Decisions and Electronic Democracy Journal of Multi-criteria Analysis, 12 (2-3), 77-92.

Geldermann, J., Spengler, T., Rentz, O., 2000. Fuzzy Outranking for Environmental Assessment, Case Study: Iron and Steel Making Industry Fuzzy Sets and Systems, 115 (1), 45-65.

Grassmann, P., 1950. Zur allgemeinen Definition des Wirkungsgrades Chemie Ingenieur Technik, 22 (4), 77-96.

Gustafsson, T., Salo, A., Ramanathan, R., 2003. Multicriteria methods for technology foresight Journal of Forecasting, 22 (2-3), 235-256.

Hämäläinen, R.P., 2004. Reversing the perspective on the applications of decision analysis Decision Analysis, 1 (1), 26-31.

Hitchens, D., Farrell, F., Lindblom, J., Triebswetter, U., 2001. The Impact of Best Available Techniques (BAT) on the Competitiveness of European Industry, Report EUR 20133 EN;

- 24 -

Joint Research Centre (DG JRC) Institute for Prospective Technological Studies (IPTS): Brussels.

Hodgkin, J., Belton, V., Koulouri, A., 2005. Supporting the intelligent MCDA user: A case study in multi-person multi-criteria decision support European Journal of Operational Research, 160 (1), 172-189.

Huijbregts, M.A.J., Breedveld, L., Huppes, G., de Koning, A., Suh, S., 2003. Normalisation figures for environmental life-cycle assessment - The Netherlands (1997/1998), Western Europe (1995) and the world (1990 and 1995) Journal of Cleaner Production, 11 (7), 737-748.

Joubert, A.R., Leiman, A., de Klerk, H.M., Katua, S., Aggenbach, J.C., 1997. Fynbos (fine bush) vegetation and the supply of water: a comparison of multi-criteria decision analysis and cost-benefit analysis Ecological Economics, 22 (2), 123-140.

Keefer, D.L., Kirkwood, C.W., Corner, J.L., 2004. Perspective on Decision Analysis Applications, 1990 - 2001 (with discussion by R.P. Hamäläinen and S. Cantor; in press) Decision Analysis, 1 (1), 5-24.

Keh, H.T., Chu, S., 2002. Retail productivity and scale economics at the firm level: a DEA approach Omega, 31, 75-82.

Keyser, W.d., Peeters, P., 1996. A note on the use of PROMETHEE multicriteria methods European Journal of Operational Research, 89, 457-461.

Kleine, A., 2001. Data Envelopment Analysis aus entscheidungstheoretischer Sicht OR Spectrum, 23 (2), 223-242.

Kleine, A., 2004. A general model framework for DEA Omega, 32 (1), 17-23.

Koleske, J.V., 2002. Radiation curing of coatings; ASTM manual series: West Conshohocken.

Koopmans, T.C., 1951. Analysis of Production as an Efficient Combination of Activities. Koopmans, T C (eds.) Activity Analysis of Production and Allocation., 33-97; John Wiley and Sons: New York.

Koopmans, T.C., 1975. Concepts of optimality and their uses Nobel Memorial Lecture., 239-256; Economic Sciences.

Korhonen, P., Luptacik, M., 2004. Eco-efficiency analysis of power plants: An extension of data envelopment analysis European Journal of Operational Research, 154 (2), 437-446.

Korhonen, P., Siljamäki, A., Soismaa, M., 2002. Use of Value Efficiency Analysis in Practical Applications Journal of Productivity Analysis, 17, 49-64.

Kuosmanen, T., Kortelainen, M., 2007. Valuing environmental factors in cost-benefit analysis using data envelopment analysis Ecological Economics, In Press, Corrected Proof.

- 25 -

Leskinen, P., Kangas, J., 2005. Rank reversals in multi-criteria decision analysis with statistical modelling of ratio-scale pairwise comparisons Journal of the Operational Research Society, 56, 855-861.

Lootsma, F., 1996. Comments on Roy, B.; Vanderpooten, D.: The European School of MCDA: Emergence, Basic Features and Current Work. Journal of Multi Criteria Decision Analysis, 5, 37-38.

Martinez-Alier, J., Munda, G., O'Neill, J., 1998. Weak comparability of values as a foundation for ecological economics Ecological Economics, 26 (3), 277-286.

Mavrotas, G., Trifillis, P., 2006. Multicriteria decision analysis with minimum information: combining DEA with MAVT Computers and Operations Research, 33, 2083-2098.

Maystre, J., Pictet, J., Simos, J., 1994. Méthodes multicritères ELECTRE - Description, conseils pratiques et cas d'application à la gestion environnementale; Presse Polytechniques et Universitaires Romandes: Lausanne.

Miettinen, P., Hämäläinen, R.P., 1997. How to benefit from decision analysis in environmental life cycle assessment (LCA) European Journal of Operational Research, 102, 279-294.

Moffat, D.W., 1976. Economics Dictionary; Elsevier Scientific Publishing: New York.

[49] Munda, G., Nijkamp, P., Rietveld, P., 1995. Qualitative multicriteria methods for fuzzy evaluation problems: An illustration of economic-ecological evaluation European Journal of Operational Research, 82 (1), 79-97.

Munda, G., 2004a. Social multi-criteria evaluation: Methodological foundations and operational consequences European Journal of Operational Research, 158 (3), 662-677.

Munda, G., 2004b. Social multi-criteria evaluation: Methodological foundations and operational consequences European Journal of Operational Research, 158 (3), 662-677.

Mustajoki, J., Hämäläinen, R.P., 2000. Web-HIPRE: Global Decision Support by Value Tree and AHP Analysis INFOR, 38 (3), 208-220.

Ness, B., Urbel-Piirsalu, E., Anderberg, S., Olsson, L., 2007. Categorising tools for sustainability assessment Ecological Economics, 60, 498-508.

Neumann, M., 2007. Choosing and Describing: Sen and the Irrelevance of Independence Alternatives Theory and Decision, 63, 79-94.

Pöyhönen, M., Vrolijk, H., Hämäläinen, R.P., 2001. Behavioral and procedural consequences of structural variation in value trees European Journal of Operational Research, 134, 216-227.

Pöyhönen, M., Hämäläinen, R., 2001. On the Convergence of Multiattribute Weighting Methods European Journal of Operational Research, 129 (3), 106-122.

- 26 -

Prato, T., 1999. Multiple attribute decision analysis for ecosystem management Ecological Economics, 30 (2), 207-222.

Rabl, A., Eyre, N., 1998. An estimate of regional and global O3 damage from precursor NOx and VOC emissions Environment International, 24 (8), 835-850.

Richards, B., Wall, C., 2005. Automotive Refinish Primers for Small Surface Damage Repair http://corporate.basf.com/file/15376.file5.

Roy, B., Bouyssou, D., 1993. Aide multicritère à la décision; Economica: Paris.

Saaty, T.L., 1980. The Analytic Hierarchy Process; McGraw Hill: New York.

Saling, P., Maisch, R., Silvani, M., König, N., 2005. Assessing the Environmental-Hazard Potential for Life Cycle Assessment, Eco-Efficiency and SEEbalance International Journal of Life Cycle Assessment, 10 (5), 364-371.

Saling, P., Kircherer, A., Dittrich-Krämer, B., Wittlinger, R., Zombik, W., Schmidt, I., Schrott, W., Schmidt, S., 2002. Eco-Efficiency Analysis by BASF: The Method International Journal of Life Cycle Assessment, 7 (4), 203-218.

Salo, A., Hämäläinen, R.P., 1997. On the measurement of preferences in the analytical hierarchly process (with discussion) Journal of Multi Criteria Decision Analysis.

Saltelli, A., Chan, K., Scott, E.M., 2000. Sensitivity Analysis; John Wiley and Sons: Chichester.

Sarkis, J., Weinrach, J., 2001. Using data envelopment analysis to evaluate environmentally consious waste treatment technology Journal of Cleaner Production, 9, 417-427.

Schleisner, L., 2000. Comparison of methodologies for externality assessment Energy Policy, 28, 1127-1136.

Schrader, B., 2005. Strategische Entscheidungsunterstützung: Implementierung und beispielhafte Anwendung eines Software Tools zur Evaluierung von Mehrzielentscheidungsproblemen.

Sen, A.K., 1970. Collective Choice And Social Welfare; Holden-Day: San Francisco.

Seppälä, J., Basson, L., Norris, G.A., 2002. Decision analysis frameworks for life-cycle impact assessment Journal of Industrial Ecology, 5 (4), 45-68.

Seppälä, J., Hämäläinen, R.P., 2001. On the meaning of the distance-to-target weighting method and normalisation in life cycle impact assessment International Journal of Life Cycle Assessment, 6 (4), 211-218.

Simpson, L., 1996. Do decision makers know what they prefer?: MAVT and ELECTRE Journal of the Operational Research Society, 47, 919-929.

Soest, J.P.V., Sas, H., De Witt, G., 1998. Apples, Oranges and the Environment: Prioritising Environmental Measures on the Basis of their Cost-effectiveness; Centre for Energy Conservation and Environmental Technology: Delft.

- 27 -

Stewart, T.J., 1992. A Critical Survey on the Status of Multiple Criteria Decision Making Theory and Praxis OMEGA - International Journal of Management Science, 20 (5/6), 569-586.

Timm, N.H., 2002. Applied multivariate analysis; Springer: New York.

Treitz, M., 2006. Production Process Design Using Multi-Criteria Analysis Dissertation, University of Karlsruhe (available online at: http://www.uvka.de/univerlag/volltexte/2006/178/); Karlsruhe University Press: Karlsruhe.

van Oers, L., Huppes, G., 2001. LCA Normalisation Factors for the Netherlands, Western Europe and the World International Journal of Life Cycle Assessment, 6 (5), 256.

Vetschera, R., 1994. Visualisierungstechniken in Entscheidungsproblemen bei mehrfacher Zielsetzung / Visualization techniques in multicriteria decision-making OR Spektrum, 16, 227-241.

Wall, C., Richards, B., Bradlee, C., 2004. The Ecological and Economic Benefits of UV Curing Technology RadTech Report 2004.

WBCSD, 2006. Eco-efficiency module; World Business Council for Sustainable Development (WBCSD): Geneva.

Weber, M., Borcherding, K., 1993. Behavioural problems in weight judgements European Journal of Operational Research, 67, 1-12.

Weber, M., Eisenführ, F., Von Winterfeldt, D., 1988. The effects of splitting attributes on weights in multiattribute utility measurement Management Science, 34, 431-445.

Winterfeld, D.v., Edwards, D., 1986. Decision Analysis and Behavorial Research; Cambridge University Press: Cambridge.

Wrisberg, N., Udo de Haes, H.A., Triebswetter, U., Eder, P., Clift, R., 2002. Analytical Tools for Environmental Design and Management in a Systems Perspective; Kluwer Academic Publishers: Dordrecht.

Zaim, O., 2004. Measuring environmental performance of state manufacturing through changes in pollution intensities: a DEA framework Ecological Economics, 48 (1), 37-47.

Zhou, P., Ang, B.W., Poh, K.L., 2006. Slacks-based efficiency measures for modeling environmental performance Ecological Economics, 60 (1), 111-118.

- 28 -

Bisherige Publikationen der Research Paper Reihe: Nr. 1: Isbruch, F. (2008) Anwendbarkeit und Ausgestaltung übergreifender

Controllinginstrumente in Unternehmenskooperationen, Research Paper der Georg-August-Universität Göttingen, Wirtschaftswissenschaftliche Fakultät, Schwerpunkt Unternehmensführung, Professur für Unternehmensrechnung und Controlling, Göttingen, Juli 2008

Nr. 2: Möller, K., Schmälzle, H. (2008) Messung und Steuerung im Innovationsprozess, Research Paper der Georg-August-Universität Göttingen, Wirtschaftswissenschaftliche Fakultät, Schwerpunkt Unternehmensführung, Professur für Unternehmensrechnung und Controlling, Göttingen, Juli 2008

Nr. 3: Geldermann, J., Bertsch, V., Gering, F. (2008) Risikomanagement und Mehrzielentscheidungsunterstützung unter Unsicherheit am Beispiel der Planung effizienter Notfallmaßnahmen nach kerntechnischen Störfällen, Research Paper der Georg-August-Universität Göttingen, Wirtschaftswissenschaftliche Fakultät, Schwerpunkt Unternehmensführung, Professur für Produktion und Logistik, Göttingen, Oktober 2008

Nr. 4: Möller, K., Krull, T. (2008) Reporting immaterieller Vermögenswerte in den Geschäftsberichten der DAX-30-Unternehmen, Research Paper der Georg-August-Universität Göttingen, Wirtschaftswissenschaftliche Fakultät, Schwerpunkt Unternehmensführung, Professur für Unternehmensrechnung und Controlling, Göttingen, Oktober 2008