Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy...

65
1 Eurochild Evaluation 2010 Strategic membership review Impact analysis Delivered on 28 February 2011 Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL 6255 AN Noorbeek Chamber of Commerce Maastricht: 14083101 VAT Number: NL2041.81.951.B01

Transcript of Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy...

Page 1: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

1

Eurochild

Evaluation 2010

SSttrraatteeggiicc mmeemmbbeerrsshhiipp rreevviieeww

IImmppaacctt aannaallyyssiiss

Delivered on

28 February 2011

Research, Evaluation and Consultancy

Dominique Danau SAGO Research

Dorpstraat 22

NL – 6255 AN Noorbeek

Chamber of Commerce Maastricht: 14083101

VAT Number: NL2041.81.951.B01

Page 2: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

2

Abbreviations .......................................................................................................................... 3

1. Introduction ........................................................................................................... 4

2. Strategic membership review ............................................................................ 4

2.1 Objective .......................................................................................................................... 4

2.2 Approach ......................................................................................................................... 4

2.3. Findings ........................................................................................................................... 5

2.3.1. Familiarity with the work of Eurochild .............................................................. 5

2.3.2. Appreciation of the work of Eurochild ............................................................... 5

2.3.3. Engagements of organisations at EU level ....................................................... 6

2.3.4. Considerations of organisations to become member of Eurochild ........... 6

2.4. Conclusions and recommendations ............................................................................ 7

3. Analysis of (potential) outcomes and impacts .............................................. 10

3.1. Objective and limits ...................................................................................... 10

3.2. Approach ...................................................................................................... 11

3.2.1. Documentation analysis ................................................................... 11

3.2.2. Telephone interviews ........................................................................ 12

3.2.3. E-mail query .................................................................................... 13

3.2.4. Analysis of peer reviews .................................................................... 13

3.3. Framework for impact analysis ...................................................................... 14

3.3.1. Social impact assessment ................................................................. 16

3.3.2. Logical Framework ........................................................................... 17

3.3.3. Presentation of the framework .......................................................... 23

3.4. Implementation of the framework: findings .................................................. 29

3.4.1. Documentation analysis ................................................................ 30

3.4.2. Analysis of telephone interviews .................................................... 35

a. Intervention strategies................................................................ 35

b. Outcomes and impact at national level ........................................ 41

c. Outcomes and impact at EU level ................................................ 42

d. External factors ......................................................................... 44

3.4.3. Analysis of e-mail query ................................................................ 45

3.4.4. Analysis of peer reviews ................................................................. 46

Conclusions and recommendations in relation to impact analysis .............. 48

4.1. Inputs ............................................................................................................................. 48

4.2. Activities and outputs.................................................................................................. 48

4.3. Outcomes and impact ................................................................................................. 49

4.3.1. Shifts in social norms ........................................................................................... 49

4.3.2. Strengthened organisation capacity ................................................................. 50

4.3.3. Strengthened alliances ......................................................................................... 52

4.3.4. Strengthened base of support ............................................................................ 52

4.3.5. Improved policies .................................................................................................... 53

4.4. External factors ............................................................................................................ 55

4.5. Recommendations on the basis of the impact analysis ......................................... 56

Page 3: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

3

5. Conclusions and recommendations in relation to the implementation of the impact analysis framework ............................................................................. 58

5.1. Conclusions ................................................................................................................... 58

5.2. Recommendations on the basis of the implementation of the impact analysis

framework .............................................................................................................................. 59

Bibliography ............................................................................................................... 60

Annex 1 ....................................................................................................................... 62

Annex 2 ....................................................................................................................... 63

Abbreviations

CRAG Child Rights Action Group

DG Directorate General

EAPN European Anti Poverty Network

EC European Commission

EESC European Economic and Social Committee

EP European Parliament

EPAP European Platform Against Poverty and Social Exclusion

EPSCO Employment, Social Policy, Health and Consumer Affairs Council

EU European Union

EY European Year

FRA Fundamental Rights Agency

HR Human Resources

MEP Member of the European Parliament

NAP National Action Plan

NGO Non-governmental organisation

TWG Thematic Working Group

UK United Kingdom

UNCRC United Nations Convention on the Rights of the Child

Page 4: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

4

1. Introduction

The evaluation of Eurochild for 2010 is composed of two components:

A strategic membership review and

An analysis of (potential) outcomes and impacts of Eurochild in the domains of child well-being/poverty and children‘s rights.

Both elements of the evaluation fit very well in the further strategic development of the network.

2. Strategic membership review

2.1 Objective

This first component of the evaluation process 2010 has to be considered within the framework of the further strategic development of Eurochild‘s network. The main objective of this strategic membership review is to analyse whether a (limited) number of organisations that are important actors at national level in the fields of child well-being and/or children‘s rights, have an interest in becoming a member of the network.

2.2 Approach

In order to do a strategic membership review at EU level, a mapping per Member State of strategic national actors in the domain of child well-being would have to be done. However, within the framework of this evaluation a complete mapping was not possible due to time constraints. Therefore, it was

decided with the Eurochild team to select a number of Member States and a number of organisations:

Belgium: 2 organisations.

Spain: 2 organisations.

Romania: 2 organisations.

Czech Republic: 1 organisation.

The selected organisations had showed an interest in Eurochild in the past, but did not become a member yet.

Page 5: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

5

These organisations were contacted and 6/7 organisations were willing to participate. 1 Romanian organisation did not reply after an initial contact mail and two follow-up mails.

With these six organisations telephone interviews were held on the basis of a semi-structured questionnaire (annex 1).

Furthermore, in the analysis, also the input is used of the interviews with Eurochild members and Management Board members in the framework of the impact analysis. One of the additional questions asked during these interviews was about the further development of the network in terms of growth. In total 14 telephone interviews were held with members, including members of the Management Board of Eurochild.

2.3. Findings

On the basis of the interviews with the members, it is clear that growth of the network just because of growth is not the issue. Further growth should be considered in terms of the strategic development of the network. The current approach whereby organisations apply for membership should be continued, but besides that, organisations have to be identified that are important national players in terms of policy influencing.

2.3.1. Familiarity with the work of Eurochild

All six organisations participating in this part of the evaluation know Eurochild to some extent, ranging from ―having contacts with Eurochild for more than 7 years‖ to ―knowing Eurochild since 2010‖ and different possibilities in between. None of them stated that they were directly contacted by Eurochild in the framework of network development; some of them met the Secretary General during a conference, others came into contact via other organisations, like e.g. Euronet. All organisations know the work of Eurochild in the sense that they have read

documents that they received directly from the Secretariat or that they retrieved from the website.

2.3.2. Appreciation of the work of Eurochild

The work of Eurochild is by the organisations assessed as useful and relevant for them to a varying degree, depending on the focus of the organisation. Organisations that used to be member of Euronet are more interested in children‘s‘ rights issues, and less in child well-being subjects. Various aspects are brought forward that are related to the content of the information provided by Eurochild:

Page 6: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

6

The primarily focus of all the contacted organisations is the national/regional level, but at the same time they realise that the EU level is becoming increasingly important and that the EU impact on this policy domain is increasing.

While for some organisations their first interest is children‘s rights, with the EY2010 on combating poverty in their work they paid also more attention to child poverty and for this matter, they‘ve used the information from the Eurochild website on this issue actively (in the own Newsletter for example).

It appears that for organisations focusing on children‘s rights, Eurochild is not yet recognised as the network (also) active in the domain of children‘s rights.

2.3.3. Engagements of organisations at EU level

All organisations are or have been involved in European or international activities. For two organisations their involvement in European activities is currently very limited and almost stopped with the ending of Euronet in 2009. A short overview for the four remaining organisations:

3/6 organisations are still involved in EU level activities. One organisation is part of another European network (European Network of Single Parent Families); one organisation is involved in a European project (on HR and regional development) and one organisation is part of a European partnership (Alliance 2015) in the field of development cooperation.

1/6 organisations has prioritised its international work and has focused on UNCRC work and Latin-America/Caribbean countries activities.

The two organisations that are currently only to a limited extent involved in EU level activities, state that for the moment, the regional/national level is the focus of their work and that EU level activities are not a priority for them.

2.3.4. Considerations of organisations to become member of

Eurochild

Networking is considered to important by the organisations as a way to gain knowledge about:

EU level developments and trends. Developments in other Member States. Particular themes that are not well developed yet in the own country.

Furthermore, it is argued that if the network of which the organisation would become member is well-known, it makes the own work more valuable at national level. One organisation argues that by being a member of a well-known and qualitative EU network, the national organisation could put pressure on the own government to pursue a more child-centred approach. Not only

Page 7: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

7

because examples and arguments from other EU Member States could be used within the own country, but also just because the fact of being a member of a well-known EU network in that particular field (status).

Also the fact that the network represents a larger group of NGOs in the field of child well-being and children‘s rights is seen as important to lobby at EU level. A group can accomplish more than individual NGOs on their own.

At the same time, interviewees are well aware of the fact that investments are necessary to be part of a EU network. While financial investments are an issue, especially for organisations where budgetary cuts have been or will be pursued, the necessary time investment is also considered to be an important factor. The organisations considering to join the network express that if they become a member, they want to be an active member. Therefore, it would be useful to have an estimate of the time needed.

5/6 organisations are considering becoming a member. 1/6 organisations has prioritised its international activities, and has chosen not to concentrate on the EU level within this framework. 1 out of the 5 organisations wants to become a member on the short term, while the remaining four have still some questions, like e.g. what is expected from members? What is the difference in services delivered to a member and a non-member? Why is it necessary to follow trends at EU level via the Eurochild membership?

2.4. Conclusions and recommendations

On the basis of the interviews with the organisations that are not (yet) a member of Eurochild and on the basis of the interviews with members that have been taken place in the context of the impact analysis of the 2010 evaluation, the following conclusions and recommendations () are drawn:

a. The Eurochild network is in a stage in which growth in terms of adding numbers to the network is not an issue anymore. There might still be a concern of geographical coverage, but 135 members in 35 countries is a sound basis.

Page 8: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

8

Figure 1: Membership of Eurochild as of 31.01.2011 – number of members and associated members per country

Further growth should be rather considered from a strategic viewpoint: what organisations (per country/Member State) are useful/necessary for the further strategic development of the network? It should also be taken into account that the nature of members will also influence the activities of the network (advocacy organisations, service providers, etc.).

b. The organisations interviewed in the framework of this evaluation 2010

said that they were not contacted as such by Eurochild, but met a Eurochild team member or board member during a (non-Eurochild) event. A more pro-active approach in identifying potential members and

contacting them is recommended. It would be useful to do a mapping per Member State in order to see what organisations play what kind of role in their own context and what organisations might have an added value for Eurochild.

Follow-up of previous contacts is relevant. It might be that an

organisation was a year ago not interested in becoming a member, while the situation might have changed. Finding (new) entry points

is to be considered. E.g. one of the organisations is currently not considering to become a member but started recently with activities in relation to child participation. This could be a new entrance point to keep the attention of the organisation.

c. Time and money are factors playing a role in the decision of

organisations to become a member of Eurochild. The organisations interested in becoming a member ask for transparent and to the point information what membership involves. It is recommended to present to potential members a clear overview

of what the necessary investments are in terms of:

0

5

10

15

20

25A

lban

ia

Au

stri

a

Aze

rbai

jan

Be

lgiu

m

Bu

lgar

ia

Cro

atia

Cyp

rus

Cze

ch R

ep

ub

lic

De

nm

ark

Esto

nia

Fin

lan

d

Fran

ce

FYR

Mac

ed

on

ia

Ge

rman

y

Gre

ece

Hu

nga

ry

Ice

lan

d

Ire

lan

d

Ital

y

Latv

ia

Lith

uan

ia

Luxe

mb

ou

rg

Mal

ta

Mo

ldo

va

Ne

the

rlan

ds

Po

lan

d

Po

rtu

gal

Ro

man

ia

Serb

ia

Slo

vaki

a

Slo

ven

ia

Spai

n

Swe

de

n

Ukr

ain

e

Un

ite

d K

ingd

om

Page 9: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

9

Membership fee.

Estimation of time needed as a member to take actively part in the network, as well as a year plan, in order for organisations to make choices in which activities to participate.

Furthermore, it is very useful for potential members to know what the extra services are they will have access to when they become a member, compared to the information they can already consult and use as a user of the Eurochild website.

d. Organisations that know Eurochild from a distance, do not consider Eurochild in the first place as a children‘s rights organisation. Eurochild is still much appreciated in the first place as an organisation active in

the area of ‗child poverty‘. This is amongst others related to the main funding source (DG Social Affairs and Employment, and not DG Justice where children‘s rights issues are located), and its history. For potential members it should be apparent that Eurochild is both

about child well-being/child poverty as well as about children‘s rights.

Page 10: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

10

3. Analysis of (potential) outcomes and impacts

3.1. Objective and limits

SAGO Research received the request to develop and implement a framework with criteria and indicators for assessing the impact of Eurochild, as part of the 2010 evaluation of Eurochild.

This impact analysis will feed into the work of the Policy Steering Group

(previously Policy Working Group) that can be considered as the Think Tank of Eurochild. The members provide expertise to Eurochild on strategic aspects of its work1.

The main objective of this impact analysis is the:

Definition of a framework with criteria and indicators assessing the impact of Eurochild, to help the work of the Policy Steering Group. This framework will be tested in two domains: child well-being and children’s rights.

This impact evaluation has to be considered in its context:

The limited time frame (November 2010 – February 2011), meaning that not an extensive impact evaluation can take place.

Within the same time frame, it was agreed that for the evaluation of Eurochild 2010 also another component will be looked at, i.e. a strategic membership review (see paragraph 2 of this report).

The financial resources that are fixed.

Nevertheless, within this context it is possible to develop a framework that can be used for future impact analyses and to implement it a first time.

1 Eurochild, (2010), Terms of Reference of the Policy Steering Group, May 2010, Brussels

Page 11: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

11

3.2. Approach

For this impact analysis two domains were selected in concertation with the Eurochild Secretary General, i.e. child well-being/poverty on the one hand and children‘s rights on the other. These two domains are further explained in paragraph 3.4.

The aim is that the framework can be used by the Policy Steering Group for further impact analyses. Therefore, the framework should be pertinent, but also user-friendly. Part of this framework is the definition of criteria and indicators,

as well as research questions linked to these.

Different methods are used for data-gathering:

3.2.1. Documentation analysis

An analysis was made of different documents produced at EU level, as well as documents produced by Eurochild with the aim of identifying links between both.

The EU level documents used for this analysis are:

1. European Council, (2010), The Stockholm Programme – an open and secure Europe serving and protecting citizens, (2010/C 115/01), in: Official Journal of the European Union, 4.5.2010.

2. Consultation document: European Commission‘s Communication on the Rights of the Child (2011-2014) (June – August 2010).

3. Opinion of the European Economic and Social Committee on Child poverty and child well-being. Exploratory opinion. Brussels, 14 July 2010. Rapporteur-General, Ms Brenda King.

4. Belgian Presidency of the EU, (2010), Call for an EU Recommendation on Child Poverty and Child Well-Being, June 2010. Background paper to the EU Presidency Conference: Child Poverty and Child Well-Being, 2-3 September 2010.

5. Speech of Roberta Angelili, Vice President of the EP, during the 5th European Forum on the rights of the child. Moving forward on the EU strategy on the Rights of the Child, October 2010.

6. Speech of Viviane Reding, Vice President of the EC, during the 5th European Forum on the Rights of the Child, A renewed commitment to children’s rights, October 2010.

7. European Commission, (2010), The European Platform against Poverty and Social Exclusion: A European framework for social and territorial cohesion, SEC(2010) 1564 final, Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, Brussels (December 2010).

8. European Commission, (2010), Commission Staff Working Paper. List of Key initiatives. Accompanying document to The European Platform against Poverty and Social Exclusion: A European framework for social

Page 12: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

12

and territorial cohesion, COM(2010) 758 final, Brussels (December 2010).

The Eurochild documents used for this analysis are:

1. Eurochild policy position: Call for EU action on child poverty and well-being. October 2009.

2. Eurochild policy briefing: Indicators: an important tool for advancing child well-being. Supporting document to Eurochild‘s 6th Annual Conference in Cyprus. November 2009.

3. Eurochild response to the European Commission Consultation on the future “EU2020 Strategy‖. January 2010.

4. Eurochild key messages. 2010 European year to combat poverty and social exclusion. February 2010.

5. Eurochild policy position: Eurochild’s proposals for the development of the EU’s strategy on the rights of the child. May 2010.

6. Eurochild’s response to the Consultation on the European Commission’s Communication on the Rights of the Child (2011-2012). August 2010.

7. Valuing Children’s potential. How children’s participation contributes to fighting poverty and social exclusion, edited by Mieke Schuurman. September 2010.

8. Eurochild contribution to the BE Presidency conference ―EU coordination in the social field in the context of Europe 2020: Looking back and building the future‖. Where will action against child poverty and social exclusion fit within the next EU 10-year strategy? September 2010.

9. Eurochild‘s contribution to the EAPN/BAPN conference ―Laying the foundations for a fairer Europe – Ensuring an adequate minimum income for all‖. Ensure adequate resources throughout the life cycle. A children‘s perspective. September 2010.

10. Child participation – how to involve children. Speech for the 5th European Forum on the Rights of the Child, Brussels, 14 October 2010.

3.2.2. Telephone interviews

In the period January-February 2011 21 telephone interviews were organised

with a series of stakeholders:

Table 1: Overview of telephone interviews

Management

Board

Policy

Steering

Group

Network

members

European

Commission

European

Parliament

Other

organisa-

tions

Secretariat

6 4 5 3 1 1 1

Page 13: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

13

For the telephone interviews a semi-structured questionnaire was used (annex 2). The main objective of the telephone interviews was to gather opinions and perceptions of different actors on the issue of impact of Eurochild.

3.2.3. E-mail query

When it became clear that members of the European Parliament were not easy to be contacted and were not very keen on participation via a telephone interview because of their busy schedule, the evaluator decided to find out more information via a short e-mail query to members of the European Parliament. A search was done to find out which questions were posed in the European Parliament in the period October 2009 – December 2010 related to child well-being, child poverty and/or children‘s rights. The eight MEPs who posed a question on one of these issues in the given time framework, were sent a short e-mail. In this e-mail they were asked:

Whether they knew Eurochild. If so, whether they had used material from Eurochild in preparing their

EP interventions and questions.

3.2.4. Analysis of peer reviews

The Peer Review in Social Inclusion and Social Protection and the Assessment in Social Inclusion are both sub-programmes of the Progress programme – the Community Programme for Employment and Social Solidarity, established in October 2006. Before being part of Progress, peer review were organised in the framework of the European Action Programme on combating social exclusion and poverty (2002-2006). Peer reviews are a key instrument of the Social 'Open Method of Coordination' They enable an open discussion on social protection and social inclusion policies in the different EU Member States and facilitate the mutual learning process among them. (for further information see: http://www.peer-review-social-inclusion.eu/general-information) . The main aim of this analysis is to identify what role Eurochild plays in this mutual learning process. Since peer reviews are considered to be one of the key instruments of the social OMC, having Eurochild involved is one of the indicators of showing recognition of the network. In the period 2004-2010 in total 58 peer reviews took place related to various themes:

Integration of ethnic minorities and immigrants. Quality and accessibility of social services. Homelessness and housing exclusion. Children and families. Promoting active inclusion. Over-indebtedness and financial exclusion. Ageing and providing adequate and sustainable pensions. Health and long-term care. Interaction of social, economic and employment policies. Governance.

Page 14: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

14

8/58 peer reviews took place related to child well-being and child poverty. Furthermore, one of the policy assessment by the network of independent experts was on tackling child poverty and promoting the social inclusion of children in the EU (April 2007).

3.3. Framework for impact analysis

The impact analysis that is conducted in the framework of the 2010 evaluation of Eurochild is based on a framework. One of the main starting points of this framework is that it is user-friendly and ready to use, also for people not daily

involved in evaluation activities.

It is natural and useful that Eurochild as an advocacy network wants to know how successful it has been, how much impact it has had with its work. In literature, several contributions2 can be found about the complexity of measuring impact. Measuring social impacts of networks is yet a different level and this because of various reasons3:

First of all because a network is composed of multiple (national) members, pursuing their own mission and objectives. The missions and objectives of individual member organisations might be not completely in line with the mission and objectives of the network.

Eurochild can be considered as a network aiming at social change. In literature several functions can be found of international social change networks4:

Analyse global problems from local, national and regional perspectives and knowledge.

Filter, process and manage knowledge for the members.

Promote dialogue, exchange and learning amongst members.

Shape the agenda by amplifying little known or understood ideas for the public.

2 See e.g. Wilson-Grau, R., (2007), Evaluating the effects of international advocacy networks,

paper presented at the Advocacy Impact Evaluation Workshop at the Evans School for Public

Affairs, University of Washington, 4-6 December 2007, Seattle or Church, M., et al, (2002),

Participation, Relationships and Dynamic Change: new thinking on evaluating the work of international networks, Working Paper No 121, Development Planning Unit, University College London, London

3 Nuňez, M., and R., Wilson-Grau, (2003), Towards a conceptual framework for evaluating

international social change networks; Chapman, J., (2002), Monitoring and evaluating advocacy, PLA notes, 43:p48-52

4 Based on Wilson-Grau, R., (2007), Evaluating the effects of international advocacy networks, paper presented at the Advocacy Impact Evaluation Workshop at the Evans School for Public Affairs, University of Washington, 4-6 December 2007, Seattle, Yeo, S., (2004),Creating, Managing and Sustaining Policy Research advocacy networks, preliminary paper, Yeo, S.,

(2004), Evaluation of the SISERA network for IDRC, Preliminary report, Portes, R., and S. Yeo, (2001), Think-net: the CPR model of a research network CEPR, Prepared for the workshop on local to global connectivity for voices of the poor, 11-13 December 2000, World Bank, Washington, Reinicke, W., et al, (2000), Critical choices; The UN, advocacy networks and the future of global governance, Global Public Policy Institute

Page 15: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

15

Convene organisations or people.

Facilitate action by members.

Build community and often a movement by promoting and sustaining the values and standards of the group of individuals or organisations within it.

Mobilise and rationalise the use of resources for members to carry out their activities.

Strengthen international consciousness, commitment and solidarity. In reality a network will perform a combination of two or more of these functions, just like it is the case with Eurochild (e.g. combination of promoting dialogue, exchange and learning amongst members, shaping the agenda, mobilising the use of resources of members).

A network like Eurochild is not operating in a vacuum, nor in isolation.

Various other actors are operating in the same domain. Claiming or measuring individual attribution is very difficult and may even harm cooperation with other stakeholders. Who can assume credit for the change that happened?

The influence of external factors is also very unpredictable, i.e. like the political situation may influence outcomes more than anything within the control of the advocate.

Furthermore, cause and effect tend to get separated by time and distance as the impact of an action provokes a ‗wave‘ effect of different changes and responses over time. Policy reforms tend to be slow and incremental. Some effects might be visible on the short term, while other effects become only apparent after a longer time. Depending on when the impact is assessed, the results of the impact assessment will be different.

As a result of successful policy influencing, goals may shift, but also

actions may shift between international, national and local levels during the course of an intervention. This will make it unclear where policy success should be sought5.

Furthermore, advocacy can mean various things and includes various tools, such as influencing, lobbying, campaigning, etc. It may be difficult to assess which approach makes the difference.

Despite the thresholds for measuring social impact of a network of social change (like Eurochild) and the limits of this impact assessment, it is possible to come up with a framework and an initial implementation of this framework in the context of the evaluation of Eurochild 2010.

The aim of this framework is not to give an extensive, academic background and overview on impact assessment, but rather to deliver an instrument that can be used by the Policy Steering Group of the Eurochild network for further

5 Chapman, J., and A., Wameyo, (2001), Monitoring and Evaluating Advocacy: A Scoping

Study

Page 16: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

16

work on this issue. Furthermore, this framework will be implemented for the first time on a limited scale within the Eurochild evaluation 2010.

For the framework, two entry points are used: Social impact assessment. The Logic Model or the Theory of social change.

3.3.1. Social impact assessment

Social impact assessment is one of the forms of impact assessment, besides environmental impact assessment and health impact assessment. There are various definitions on what social impact assessment exactly is. In this framework the definition of Vanclay (19996) is used:

―Social impact assessment is the process of analysing (...) and managing the intended and unintended consequences on the human environment of interventions (policies, plans, programmes, projects and other social activities) and social change processes so as to create a more sustainable biophysical and human environment‖.

The types of social impact can be diverse7, i.e. impact on: People‘s way of life: behaviour and relationships. Cultural: customs, values and beliefs. Community: infrastructure, services, networks, cohesion. Quality of life: quality of the air, the water, security, liveability. Health: mental and physical well-being.

Social impact assessment is an overarching framework embodying all kinds of human impacts, going from demographic processes (like e.g. changes in population size), to socio-economic processes like impoverishment, socio-cultural processes (.e.g. exclusion, empowerment), political processes (e.g. democratisation), geographical processes (e.g. urbanisation) and institutional processes (e.g. globalisation). Therefore, a selection needs to be made of the domains in which impact is looked for.

6 Vanclay, F. 1999b. ―Summary of workshop on International Guidelines and Principles for

Social Impact Assessment‖, report to the closing session of the meeting of the International Association of Impact Assessment, Glasgow.

7 Based on The Rockefeller Foundation – The Goldman Sachs Foundation, (2003), Social Impact Assessment. A discussion among grantmakers, New York and Vanclay, F., (2001), Social Impact Assessment, one of the contributing papers to the World Commission on Dams, Cape Town, South Africa

Page 17: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

17

3.3.2. Logical Framework

Eurochild (as an intervention network) is based on a theory of change, i.e. a model of how and why the change Eurochild wants to see in society will occur. It is a ‗map‘ showing how to get from ‗here‘ (start of an intervention) to ‗there‘ (point of measurement). This model is often implicit present in the way of working, in work plans, etc. Part of this impact assessment framework is to make explicit what is happening during an intervention.

A useful tool in doing this, is the Logical Framework. It describes logical linkages between inputs, activities, outputs and outcomes. It articulates the steps by which an organisation, a network, etc. (e.g. Eurochild) is intended to achieve its objectives8:

Inputs: what is invested, like knowledge, skills, expertise, but also financial resources, materials, etc.

Activities: ‗things‘ that are done and that are leading to outputs and outcomes (e.g. implementation of communication strategy).

Outputs: immediate results of the activities (e.g. publications) and the people that are reached through the activities (e.g. informed policy makers, Eurochild members involved in the topics selected).

Outcomes:

Short term: changes in awareness, knowledge, opinions, attitudes, skills.

Long term changes: changes in practices, policies and procedures. The network influences outcomes.

Impact: long-term changes in organisational, institutional, regulatory frameworks. The network contributes to impact.

8 Based on Wilson-Grau, (2007) and Mc Cawley, F., (2002), The Logic Model for program

planning and evaluation, University of Idaho Extension

Page 18: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

18

Figure 2: Logic Framework

Outcomes: - short term outcomes: e.g.

changes in awareness,

knowledge, opinions, attitudes,

skills

- long term outcomes: e.g. changes in practices, policies,

procedures

Indicators

Process

indicators

e.g.

Were inputs made

as planned, in terms of timing,

quality, etc.?

Were activities

conducted as

planned, in terms

of format, quality, etc?

Outcome

indicators

e.g.

Did participants demonstrate the

desired level of

awareness

increase?

Did procedures change to the

extent expected?

Factors beyond

control but crucial for

the realisation of the

activities, the results,

the outputs, outcomes and finally

impact.

External factors

Intervention logic Inputs: - time - money

- human resources

- knowledge base (e.g. info,

material)

Outputs: - things: e.g. policy papers

- people: e.g. number and nature

of participants in working groups

Activities: e.g. review of governance

procedures, develop communication strategy,

etc.

Impact

Page 19: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

19

The indicators in the left column are used for assessment. Examples are: Process indicators:

The extent to which inputs are made as planned (amount of input, quality and timing)

The extent to which activities are conducted as planned (content, quality, format, timing).

The extent to which planned outputs are produced (amount, quality and timing)

The extent to which the desired level of participation is achieved (number and nature of participants, degree of satisfaction of participants).

Outcome indicators: The extent to which improved awareness of policy makers is

demonstrated. The extent to which policies are altered to the extent that is was

expected.

While impact assessment is about analysing the process, the logical framework allows to structure this process and to define the relations within the process.

In literature, it is argued that this linear, causal chain should not be used for the evaluation of advocacy networks, because they do not follow linear patterns. Their advocacy activities take place in complex situations without predetermined results as also argued earlier in paragraph 3.3.9. However, for the building of this impact analysis framework, the elements of the logical framework are used to structure the process of impact analysis, and not so much to follow their linear chain. The evaluator is very much aware of the complex environment in which advocacy networks operate. The elements of the logical framework are useful to depict the complex process of impact analysis. For the assessment an interactive approach is used, involving many different stakeholders. This approach will certainly enhance learning about success and failure and will identify achievements more comprehensively.

9 See e.g. Wilson-Grau, R., (2007)

Page 20: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

20

In order to assess impact, it is important to identify the outcomes and outcome categories. One way of categorising outcomes is based on Reisman, et al (2007)10: Scheme 1: Categorising outcomes/impact

Shift in social norms

Description: knowledge, attitudes, values and behaviour that compose the normative structure of culture and society.

Examples of outcomes

Changes in awareness.

Changes in attitudes.

Increased agreement on a definition of an issue (e.g. use of common language).

Changes in beliefs.

Examples of strategies Media campaign.

Message development (e.g. defining the problem).

Strengthened organisational capacity

Description: staffing and leadership, organisational structures and systems, finances and strategic planning. The (further) development of these core capacities are critical organisational conditions of advocacy and policy change efforts.

Examples of outcomes

Improved management of organisational capacity of organisations involved with advocacy and policy work.

Improved strategic abilities of organisations involved with advocacy and policy work.

Improved capacity to communicate and to promote advocacy messages of organisations involved with advocacy and policy work.

Improved stability of organisations involved with advocacy and policy work.

Examples of strategies Leadership development.

Organisational capacity building.

Strategic planning.

10

Reisman, J., Gienapp, A., and S., Stachowiak, (2007), A guide to measuring advocacy and

policy, funded and prepared for Annie E. Casey Foundation

Page 21: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

21

Strengthened alliances

Description: level of coordination, collaboration and mission alignment among partners. These structural changes in institutional relationships and alliances are important in presenting common messages, pursuing common goals, enforcing policy changes, etc.

Examples of outcomes

Increased number of partners supporting an issue

Increased level of collaboration.

Improved alignment of partnership efforts (e.g.

shared priorities, shared goals).

Strategic alliances with important partners.

Increased ability of coalitions working towards policy change.

Examples of strategies Partnership development.

Coalition development among unlikely allies.

Strengthened base of support

Description: grassroots, leadership and institutional support for particular policy changes. This category encompasses many layers of societal engagement, like e.g. increases in civic participation, positive media attention, etc.

Examples of outcomes

Increased public involvement.

Increased breadth of actors supporting an issue.

Increased media coverage.

Increased awareness of campaign messages among selected groups (e.g. policy makers, opinion leaders).

Increased visibility of intervention message (e.g. in media).

Examples of strategies Media campaign.

Policy analysis.

Page 22: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

22

Improved policies

Description: stages in the policy arena, i.e. policy development, proposal, demonstration of support, adoption, funding and implementation. This outcome area is frequently the focus of measuring the success of advocacy and policy work. However, it is rarely achieved without changes in the other outcome categories. In literature, various stages of policy influencing are described by various authors. One example is of the Oxfam Policy Department11 distinguishing the following stages:

Increased awareness about an issue Contribution to debate Changed opinions Changed policy Policy change is implemented Positive change in people‘s lives.

Examples of outcomes Policy development.

Policy adoption (e.g. legislation).

Policy implementation (e.g. funding).

Examples of strategies Development of policy proposals, ‗white‘ papers.

Pilot programmes.

Changes in impact

Description: the ultimate changes in social and physical lives and conditions. These are long-term outcomes and are influenced by changes in policies.

Examples of outcomes Improved social and physical conditions like e.g. poverty, equality, health.

Examples of strategies Combination of direct interventions, personal behaviours, etc.

11 Roche, Chris (1999), Impact Assessment for Development Agencies: Learning to Value Change,

Oxfam

Page 23: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

23

3.3.3. Presentation of the framework

The Eurochild team composed of a representative of the Secretariat and of the Policy Steering Group prepared an overview of expected outcomes and impacts. These were by the evaluator categorised under the headings defined in paragraph 3.3.2. This first result was presented to the PSG on the 12th of January 2011. On the basis of the discussions and work done in this PSG, the evaluator adapted the framework. The final result is presented here below in paragraph 3.3.3.. Hereafter, the framework to be used for the impact analysis, is presented. Each of the elements is described: what is the content and how each of the elements

can be used. a. Input: these are the investments made to make the intervention possible:

personnel (number and competencies), financial means and time investments. Example: 2 policy officers during 1 year, for each 10 hours per week as well as an investment of € 5.000 for campaigning material. Questions/indicators:

What have been the investments? What have been the objectives of these investments? How would you assess these investments in terms of quantity, quality

and timing? b. Activities; the activities that are necessary to achieve the desired results.

These activities determine the intervention. Examples: workshops, meetings, training sessions, policy statements, position papers, responses to consultations, campaigns, people reached. Questions/indicators:

Which activities have been organized? Were activities conducted as planned (quantity, content, quality,

format, timing? c. Outputs: immediate results of the activities. Furthermore, within this

element of the framework also the number and nature of people reached through the activities are identified.

Questions/indicators:

What are the expected outputs? What are the outputs produced? How many people were reached? Who are the people reached? Were outputs produced as planned (quantity, content, quality,

format, timing). To what extent was the desired level of participation achieved?

Page 24: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

24

d. Expected outcomes: outcomes occur along a path from shorter-term achievements to longer-term achievements (impact). The expected outcomes are formulated in relation to national and European level and short term and longer term.

Questions/indicators:

What are the expected outcomes? What are the outcomes produced? Were outcomes produced as planned (quantity, content, quality,

format, timing).

A distinction is proposed between European/national level and short/long term. Expected outcomes/impact at national level might be more difficult to document, since national members are not required to keep track of policy impacts within the framework of their involvement in Eurochild.

Page 25: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

25

Scheme 2: Expected outcomes/impact and indicators

Expected outcomes on children’s rights Category Indicators (short term – can be assessed in

2011)

National level European level

Children‘s rights have a

significant position

within the Europe 2020

strategy.

Improved policies Actions on children‘s rights are announced/

taken within the EPAP.

The EU Strategy on

children‘s rights

reflects inputs from NGOs and other

relevant stakeholders.

Improved policies A consultation process on the future EU strategy

on the rights of the child is implemented in a satisfactory way (taking into sufficient time for consultation, dissemination of information about consultation process, wording of consultation, etc.).

Eurochild its members respond to the consultation.

Children and young people are involved in the consultation process.

Input from Eurochild is taken into account in

the EU strategy on children‘s rights.

Protection of children‘s

rights is recognised at

national level. National governments are

supportive of EU

cooperation on child

rights.

Protection of children‘s

rights is recognised at

EU level.

Improved policies Support to the EU strategy is demonstrated by

evidence (e.g. reports to EC on different policy

areas, legislation, funding of interventions, links to CRC reporting process)

A majority of members influencing policy at regional/national level in the domain of children’s rights is using Eurochild material for this purpose.

Page 26: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

26

Expected outcomes on children’s rights Category Indicators (short term – can be assessed in

2011)

National level European level

Children‘s rights are

mainstreamed within

EC work & policy

development.

Strengthened base of

support.

Improved policies.

UNCRC is mentioned in key documents across the DGs.

Training on child rights is delivered.

Eurochild is seen as an influential actor in the

promotion of children‘s rights.

Strengthened alliances

Strengthened

organisational capacity.

Eurochild cooperates with other NGOs on children’s rights issues.

Eurochild is invited by national governments to participate in (high-level) events on children’s rights.

Eurochild is invited by the EC to participate in (high-level) events on children’s rights.

The language used by Eurochild in relation to children’s rights is taken up at national level.

The language used by Eurochild in relation to children’s rights is taken up at EU level.

Page 27: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

27

Expected outcomes on child poverty and child

well-being

Category Indicators (short term – can be assessed in

2011)

National level European level

Member States support

a road map towards a

Commission Recommendation on

child poverty & well-

being.

The EC develops a road

map towards a

Commission Recommendation on

child poverty & well-

being.

Strengthened base of

support

Support is demonstrated by evidence (e.g.,

policy documents/legislation (funding of

appropriate interventions).

Child poverty & well-

being acknowledged as

a thematic priority

within the European Platform Against

Poverty.

Improved policies Actions on child well being and combating child

poverty are announced/taken within the EPAP.

Members gather at

national/regional level

commitment to EU

cooperation on child

poverty & well-being.

Strengthened

organisational capacity

Strengthened base of

support

A majority of members influencing policy at regional/national level in the domain of child well being and combating child poverty is using Eurochild material for this purpose.

The discourse of members in the field of child well being and combating child poverty is used at national/regional policy level(s). Evidence of this use can be demonstrated.

Page 28: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

28

Expected outcomes on children’s rights Category Indicators (short term – can be assessed in

2011)

National level European level

Child well being and combating child poverty is

recognised as a political priority.

Improved policies

Shifts in social norms

Child poverty targets at EU level are set within the Europe 2020 strategy.

Child poverty targets at national level are set within the NRPs.

Actions to fight child poverty are set within the NRPs

An increased understanding on the need to fight child poverty amongst policy makers at national and European level is demonstrated (e.g. in policy texts and discourse).

Eurochild contributes to increased awareness for

involving children in policy development.

Improved policies

An increased understanding on the need to

fight child poverty amongst policy makers at EU level is demonstrated (e.g. in policy texts and

discourse).

Through the membership, children and young

people are engaged in efforts to fight child

poverty.

Eurochild is recognised as an influential actor in

the domain of child well being and combating child poverty.

Strengthened alliances

Strengthened organisational capacity.

Eurochild is invited by other NGOs to cooperate on child well-being issues.

Eurochild is invited by national governance to participate in (high-level) events on child well- being.

Eurochild is invited by the EC to participate in (high-level) events on child well-being.

The language used by Eurochild in relation to child well-being is taken up at national level.

The language used by Eurochild in relation to child well-being is taken up at EU level.

Page 29: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

29

e. External factors: these are factors, activities but also actors that interfere with the work of the advocate. In this framework it is important to do a stakeholders analysis and perform an environmental scan.

Stakeholder analysis: the main steps include:

Identifying the principal stakeholders at various levels (local, regional, national, EU, international).

Investigating their roles, interests and capacity to act. Identifying the extent of cooperation or conflict in the relationships

between stakeholders.

Environmental scan: Examples: politics, economy, climate, cultural milieu, history, biophysical environment, price structure, global markets, demographic patterns, resources. It is necessary to assess what external factors are likely to influence the program's ability to achieve expected results.

3.4. Implementation of the framework: findings

Two subjects are selected:

children‘s rights. The work of Eurochild is underpinned by the principles enshrined in the UN Convention on the Rights of the Child. Therefore, this domain is not the most straightforward one for the implementation of the framework. Limiting the object of evaluation is very important in this case. Within the domain of children‘s rights, the impact assessment will look at the contribution of Eurochild to impact in the domain of preparation of the EU Strategy on the Rights of the Child12 and implementation of the EC 2006 Communication in 2010.

child well-being/child poverty. The aim of the impact assessment of Eurochild in relation to child poverty is to look at the contribution of Eurochild to the impact on EU level policies and policy documents related to child poverty and child well-being in 2010 (like e.g. documents produced in the framework of the EY2010).

12 In the framework of the Communication from the Commission – Towards an EU Strategy on

the rights of the child – COM (2006) 367 final

Page 30: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

30

3.4.1. Documentation analysis

The purpose of this analysis of documents is to verify whether links can be identified between language used in documents produced by Eurochild in the fields of child well-being/child poverty and children‘s rights and documents produced by the EC or at EU level. For this a limited time period has been chosen, i.e. last months of 2009 and the whole of 2010 and a selection of documents. In the time line here below an overview is given of these documents. The dotted arrows illustrate links that can be identified between documents.

Figure 3: Documentation analysis

EU

-leve

l doc

umen

ts

Doc

umen

ts p

repa

red

by E

uroc

hild

Oct. 09 Nov. 09 Jan. 10 Feb. 10 May 10 Aug. 10 Sept. 10 09

Dec. 10 Jan. 11

The

Sto

ckho

lm

Pro

gram

me

Con

sulta

tion

on th

e E

C

Com

mun

icat

ion

on th

e

Rig

hts

of th

e C

hild

(20

11-

2014

)

Cal

l for

an

EU

Rec

omm

enda

tion

on

Chi

ld P

over

ty a

nd C

hild

W

ell-b

eing

.

EP

AP

+ L

ist o

f

initi

ativ

es.

Pol

icy

posi

tion:

Cal

l for

EU

actio

n on

chi

ld p

over

ty a

nd

wel

l-bei

ng

Pol

icy

brie

fing:

Indi

cato

rs: a

n

impo

rtan

t too

l for

adv

anci

ng

child

wel

l-bei

ng.

Res

pons

e to

the

EC

Con

sulta

tion

on th

e fu

ture

EU

2020

str

ateg

y

Key

mes

sage

s: E

Y20

10 to

com

bat p

over

ty a

nd s

ocia

l

excl

usio

n.

Pol

icy

posi

tion:

pro

posa

ls fo

r

the

deve

lopm

ent o

f the

EU

stra

tegy

on

the

right

s of

the

child

.

Res

pons

e to

the

EC

Con

sulta

tion

on th

e R

ight

s of

the

Chi

ld .

Valuing children’s potential. How children’s participation contributes to

fighting poverty and SE.

Contribution to the BE presidency conference: Where will action against child poverty and SE

within the next EU 10-year strategy?

Contribution to the EAPN/BAPN conference. Ensure adequate resources throughout the life

cycle. A children’s perspective.

Oct. 10 09

Spe

eche

s by

EC

and

EP

in th

e 5t

h F

orum

on

the

right

s of

the

chld

Child participation – how to involve children. 5th

European Forum on Rights of the Child.

Opi

nion

of t

he E

ES

C o

n

Chi

ld p

over

ty a

nd

child

ren’

s w

ell-b

eing

.

Page 31: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

31

This list of documents used for this analysis is not an exhaustive catalogue of all documents produced by Eurochild, nor at EU level in this time span, which might be interesting for this analysis. It is a good selection of documents that provides a solid basis for analysis.

The Stockholm programme (July 2010)

The draft Stockholm programme was presented by the Presidency on 16 October 2009. Various exchanges of views on the Programme took place by COREPER (October, November 2009), Justice Home Affairs Counsellors (October, November 2009), the Justice and Home Affairs Council (November, December 2009). The results of these discussions were presented to the General Affairs Council and to the European Council for approval. To enable the Union

and its member states to build on the achievements and to meet future challenges in the area of freedom, security and justice, the multi-annual programme – the Stockholm programme – has been adopted by the European Council for the period 2010-2014. The programme defines strategic guidelines for legislative and operational planning within the area of freedom, security and justice in accordance with Article 68 of the Treaty of the Functioning of the European Union (TFEU).

Within this programme the rights of the child are recognised, namely: ―the principle of the best interest of the child being the child’s right to life, survival and development, non-discrimination and respect for the children’s right to express their opinion and be genuinely heard in all matters concerning them according to their age and level of development as proclaimed in the Charter of Fundamental Rights of the European Union and the United Nations Convention on the Rights of the Child, concern all Union policies‖ (p 115/9). Furthermore, it is argued that an ambitious Union strategy on the rights of the child should be developed.

It is expressed that children in particularly vulnerable situations should receive special attention.

In its policy position of October 2009 Eurochild calls for a comprehensive EU strategy on the rights of the child.

Consultation document: European Commission‘s Communication on the Rights of the Child (2011-2014) (Summer 2010)

The EC intended to adopt a Communication on Children‘s Rights (2011-2014) at the end of 2010 to further advance the EU Strategy on the Rights of the Child. In view to do this, stakeholders were given the opportunity to present their views to the Commission about which concrete actions could be developed at EU level that would bring real added value. This consultation took place in the period June-August 2010. A total of 31 questions was proposed to individuals and organisations dealing with the protection and promotion of children‘s rights at local, regional, European or international level.

Page 32: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

32

Eurochild responded in August 2010 to this consultation. The timing of this consultation was an issue, since it was just launched before the summer break, making it very difficult to consult with members. Consultation with children was almost not possible because of the limited time scale (8 weeks). Besides the answers to the questions, Eurochild pinpointed also that the comprehensiveness of the strategy and the believe that the forthcoming Communication should first and foremost set out the principles underpinning EU action in children‘s rights. In particular attention should be given to the political leadership of the European Union on children‘s rights, the methods of cooperation between the different Commission services, and the engagement with EU member states and other key stakeholders.

Opinion of the European Economic and Social Committee on Child poverty and children‘s well-being – Exploratory opinion (July 2010)

The EESC was asked in April 2010 to draw up an exploratory opinion on child poverty and children‘s well-being on behalf of the future Belgian presidency.

In this exploratory opinion, child poverty and well-being are recognised as being multi-dimensional. Furthermore, in this note, the EESC supports Eurochild's recommendation that childcare services need to be broadened to adopt an inclusive concept of services from pre-natal to preschool that is open to all children and families. Eurochild argues that the Barcelona targets ignore many of the good practice around early childhood policy. The EESC recommends that with regard to the Barcelona targets, there is a need for the development of common EU "quality standards" for early years‘ services, including early years‘ care and education, as identified by the European Commission‘s Childcare Network, which should influence the development of national policies and practices, including the use of the Structural Funds.

Prevention and early intervention is one of the key-messages put forward by Eurochild in its publication of February 2010.

The emphasis on early years‘ education and care is also expressed in Eurochild‘s response to the European Commission Consultation on the

future EU2020 Strategy. While many EU policy documents make reference to the importance of investment in early years education and care, Eurochild regrets that this is not mentioned in the EU2020 strategy and considers it a missed opportunity.

Call for an EU Recommendation on Child Poverty and Child Well-being. Background paper to the EU Presidency conference: Child poverty and child well-being (September 2010).

This paper was requested by the Belgian Presidency of the European Union, in collaboration with UNICEF, Eurochild and the EC, and is based on an original draft written by an independent expert on children‘s rights. The authors have

Page 33: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

33

used the work of H. Frazer, E. Marlier and I. Nicaise, i.e. the study on ‗A social inclusion roadmap for Europe 2020‘, that they had prepared for the Belgian EU Presidency. The aim of the paper was amongst other to feed into the EPAP.

Some of the key messages expressed by Eurochild as input in the EY2010, are reflected in this document, while formally only once an explicit reference is made to Eurochild material. However, with Eurochild as one of the major sources of input for this paper, this is not surprisingly.

Examples of key messages reflected in this document are messages related to resource allocation, early childhood care and children‘s participation.

Speech by Viviane Reding, on A renewed commitment to children‘s rights, 5th

European Forum on the Rights of the Child and speech by Roberta Angelili, on Moving forward on the EU strategy on the rights of the child, October 2010

The European Forum on the Rights of the Child is a permanent group for the promotion of children‘s rights and should meet in plenary at least twice a year. The Forum was launched following the adoption by the European Commission on 4 July 2006 of the Communication entitled ‗Towards an EU strategy on the Rights of the Child‘. The role of the Forum is to advise and assist the Commission and other European Institutions, in particular as regards mainstreaming of children's rights, and exchange information and good practice. The Forum aims to be a place allowing for children‘s opinions to be heard and taken into account.

Members of the Forum are Member States representatives, Ombudspersons for children, representatives of the European Parliament, the Committee of the Regions, the European Economic and Social Committee, the Council of Europe, UNICEF and NGOs. The Forum is chaired by the Commission. On the 14th of October 2010 the 5th meeting took place providing the opportunity to discuss how to move forward on the EU strategy on the Rights of the Child. Members of the Forum also discussed two important issues, namely, how to make justice more child friendly and child participation. Two speeches of EP and EC officials are selected for this documentary analysis, as well as the presentation by the Secretary General of Eurochild.

In her speech Viviane Reding, the Vice President of the EC responsible for Justice, Fundamental Rights and Citizenship called for the need to have data in the field of child protection and children‘s rights. This call was clearly expressed by Eurochild, amongst others in its Annual Conference in Cyprus, November 2009, which was all about monitoring child well-being and about the importance of indicators to advance child well-being. A plea was made for data covering all stages of child development. Also in her speech, Viviane Reding acknowledges the necessity of a comprehensive approach and of addressing horizontal issues like child participation and mainstreaming. Amongst others in key messages presented by Eurochild in the framework of the EY2010, a child mainstreaming approach is put forward as well as the necessity of multi-dimensional policies and child participation. But also in various other papers child participation is put forward as a way of fighting poverty and social exclusion (e.g. speech given

Page 34: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

34

by the Eurochild Secretary General in the 5th Forum on the Rights of the Child).

Roberta Angelilli, Vice President of the EP also presented in her speech the necessity of having indicators. She announces a pilot project about a European-wide methodology for developing evidence based policies for children‘s rights, aiming to develop indicators and benchmarks, in order to improve comparability, objectivity and reliability of data on minors at EU level in relation to various aspects. A second part of this project is on child participation and consultation, which is also to be found all over Eurochild‘s work. Both child participation and developing indicators, also with the aim of increasing accountability, have been put forward by Eurochild, amongst others in its key messages for EY2010.

The European Platform against Poverty and Social Exclusion (EPAP): A European framework for social and territorial cohesion (COM(2010) 758 final) and Commission Staff working paper with a list of key initiatives related to the EPAP (SEC(2010) 1564 final) (December 2010)

The EC proposed to establish, as one of its seven flagship initiatives within its EU2020 Strategy, a European Platform against Poverty and Social Exclusion (EPAP). The Platform aims to create a joint commitment among the Member States, EU Institutions and key stakeholders to fight poverty and social exclusion. As part of this EPAP a Recommendation is proposed on child poverty outlining common principles and effective monitoring tools to combat and prevent poverty in early age. Eurochild has made a first analysis of the EPAP (positive points and concerns) on the 17th of December 2010.

Such a Recommendation was already proposed by Eurochild in its policy position in its Call for EU action on child poverty and well being of October 2009.

Also in its Response to the Consultation on the EC‘s Communication on the Rights of the Child (2011-2014) of August 2010 the adoption of a Recommendation on child poverty and well-being is put forward as part of the EU child rights strategy and to be implemented through the Social OMC.

The links identified are not proven to be causal relations, but identify influence that has been exercised. Furthermore, this kind of policy influencing did not happen in isolation, but in combination with the use of other tools.

Page 35: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

35

3.4.2. Analysis of telephone interviews

a. Intervention strategies

Definition of intervention strategies An intervention strategy is a systematic plan followed to bring about changes with a specific objective. Eurochild has always used a mix of tools within its intervention strategies, ranging from:

the writing of policy responses and policy positions: during 2009-2010 a whole list of policy responses and positions have been written in relation to children‘s rights and child poverty/child well-being;

meetings with policy makers at different levels of governance: bilateral and group meetings with MEPs, like events organised at the EP,

being present in high-level meetings, like meetings in the framework of the EU Belgian presidency,

campaigning, like the End Child Poverty campaign in the framework of the EY2010 whereby 14,000 signatures were collected and handed over to policy makers.

using new social media like Twitter and Facebook. the surveys (e.g. on alternative care) have been very useful tools as well

to provide information that can easily be used during interventions at EU and national level.

being present in the media, etc.

In using these different tools, various roles are played13:

Expert informant, providing expert information and advice to policy makers.

Capacity building, providing support to various stakeholders so that they on their turn can enhance their capacity to influence policy decisions.

Lobbyist, thereby entering into the policy formulation process making direct approaches to influence policy.

Empowering communities working with children and children themselves.

On the basis of interviews, it can be concluded that intervention strategies as such are not discussed with members. Individual parts of the strategies (tools) are part of debates, like e.g. how to approach a campaign, members are invited to provide input for policy papers, etc. However, so far discussions do not follow an intervention logic:

What is the expected output/outcome? What activities will be used to achieve this output? What indicators will be used to assess the output? What is the timeframe?

Example: The expected outcome is the increased presence of Eurochild in European and national fora. Therefore, we will start

13 Subedi, Nani Ram, (2008), Advocacy strategies and approaches. A training of trainers

manual. Second Edition, International Centre for Integrated Mountain Development, Kathmandu, Nepal

Page 36: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

36

up a media campaign and will make a mapping of events/activities in which we want to be involved in 2011. The indicators that we will use is numbers of European and national events/activities in which we are involved, the capacity in which we are involved in these events and activities and compare the figures with the figures of 2010.

For interviewees it is not always clear to see what intervention tools have been used to achieve a specific objective and therefore, it is not always apparent to see the differences in intervention strategies used in the domains of children‘s rights on the one hand and child well-being/child poverty on the other hand. So far as members are concerned there is no explicit intervention strategy (which mix of tools for what purposes) in relation to both domains. Members agree that it is important that a mixture of tools is used. Effectiveness of intervention strategies are not much discussed with members. It is useful to review tools that are used in terms of strengths and weaknesses. E.g. some members argue that the campaign did not turn out to what was expected. The campaign was by some considered as a failure in terms of number of signatures, however not in terms of profiling. Members assess it as it was probably underestimated how much time it would take to run such a campaign. Furthermore, it is also stated during interviews that the running of such a campaign would need a bottom-up approach: such an initiative would have to come from the members in order to feel ownership and to support fully the implementation of it. Members express that Eurochild does not need to react to everything that the EC is communicating; the intervention strategies should be focused. It is also argued that in the first place it should be clear what Eurochild is aiming for and developing intervention strategies accordingly (including a sustainable financial strategy). History and approach During the interviews, it is argued that the interventions in relation to child poverty have been in place for a longer time compared to interventions by Eurochild in relation to children‘s rights. Euronet used to be THE organisation

in relation to children‘s rights, while Eurochild was THE organisation for child well-being/poverty. Since Euronet has stopped its activities (2009), Eurochild had new opportunities to develop more interventions in the domain of children‘s rights. The opinions of members differ in relation to the extent that Eurochild has succeeded in filling in the gap that was created with the ending of Euronet. Some members define the intervention strategy of Eurochild in relation to children‘s rights as ‗a bit shy‘ and perhaps not that combative. Eurochild could have been more outspoken and prompt in developing activities in this domain and in presenting itself as an organisation active in this field. Eurochild is not yet considered to be THE organisation at EU level voicing children‘s rights.

Page 37: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

37

During the interviews it is also expressed that national organisations see Eurochild perhaps in the first place as an organisation voicing child well-being messages, while European and international organisations know better the children‘s rights activities of Eurochild. In the domain of child well-being/poverty, Eurochild is rather developing own material, according to members, while in relation to children‘s rights, Eurochild is more cooperating with other actors and is more relying on material of others. The estimation is done that in relation to children‘s rights, proportionally less investments were made, mainly by the Secretariat, and some members because of their history in children‘s rights. It should not be forgotten hat 2010 has been a specific year, since Eurochild had to focus on child poverty, taking into account the EY2010 on combating poverty and social exclusion. In the future, eventual new priorities of Eurochild, like children‘s rights need to be reflected in the membership as well. At the same time, it is argued that the distinction between child well-being/child poverty and children‘s rights is perhaps mainly semantic. Members as crucial actors in policy influencing One of the strengths of the intervention strategies is the involvement of network members in the building of expertise used for interventions, which is the strength of a membership organisation. The expertise to build up interventions is available in the network. It is argued that if a network does not involve its members, the impact is not so big. By involving members, their expertise is used, in some cases the capacity of members is further developed to influence policies at their own levels of governance, being it local, regional or national. Eurochild is considered to be very good to keep contacts with its members. Furthermore, it is considered as an important asset by policy makers that Eurochild succeeds in connecting EU level knowledge with grassroots level practice of members. Role of Thematic Working Groups (TWG) The TWG are viewed as laboratories where learning and exchange take place. The role of the TWG in relation to feeding into the intervention strategies is considered to be crucial, but not yet optimal. In practice it depends much on what input members/participants deliver. It is crucial that the chairs of the TWG are part of the PSG to keep a kind of cross reference. An interesting activity organised in the framework of the TWG on family and parenting support was the study visit that took place in April 2010. Members from different national/regional delegations (Bulgaria, Finland, Ireland, Netherlands/Flanders and Wales) visited Sweden and Denmark during 5 days. Round Table debates were organised as well as field visits. This study visit project had a two-stage approach to look at:

What family support policies and practices give the best outcomes for children and

Page 38: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

38

How lessons learnt could be used to influence policy and practice in participating countries.

One of the suggestions was to have an additional TWG on the monitoring of the CRC to make a more explicit link with children‘s rights. Required skills A further element are the skills and capabilities necessary to liaise at the appropriate level with policy makers and politicians. Specific skills are necessary for advocacy, e.g. building up individual relationships with MEPs,... It is said that these skills could be further developed within the Secretariat. In general, it is expressed that more face-to-face contacts with policy makers and politicians would certainly help making Eurochild‘s intervention strategies more fruitful. It is suggested to organise training sessions on advocacy for Secretariat and members. Furthermore, it is considered important to have a common framework for advocacy purposes. It is clear that the advocacy work that Eurochild has accomplished in 2010 towards MEPs has increased (e.g. various meetings and contacts with individual MEPs are reported on).

Structure of domains at EU and national levels Another element to be taken into account is related to the structure of the work at EU level. Eurochild is funded through DG Social Affairs and Employment, enabling the network to work on child well-being/poverty. Some members argue that this is an important determining factor in the intervention strategies of Eurochild. Children‘s rights is the responsibility of DG Justice. There is a clear division in responsibilities, without real horizontal linkages, while child well-being should be part of a holistic children‘s rights based approach. Some members state that the mainstreaming approach should be more used as an argument in linking both aspects in Eurochild‘s work. Also at national level the targets of Eurochild are compartmentalised across ministries. Part of the rationale not getting too close to children‘s rights issues, might be related to this situation.

Cooperation with other stakeholders There is also an issue of cooperation amongst the different stakeholders active in the area of child well-being/poverty and children‘s rights. Stakeholders want to profile themselves in their specific area of expertise and make clear what their influence is and has been. This is weakening the overall objective and is not really helping progress in the work. There are various elements that constitute the overall picture and the different stakeholders can all contribute to this same picture. It is said that much more cooperation instead of competition is necessary. Having said this, the issue of impact assessment related to individual organisations becomes in this way perhaps trivial. However, success of Eurochild or any other organisation for that matter in the domain of child well-

Page 39: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

39

being and child poverty cannot be measured by assessing progress made in this policy domain. There are also other – external factors – like the political will that play an important role (see paragraph d). While individual policy makers might be very committed to the cause of advancing children‘s rights, they on their own are often not in a position to prioritise one theme over the other.

In general, the cooperation between Eurochild and other organisations like FRA and cooperations within grouping like e.g. CRAG are considered to be a strength in intervention strategies ―uniting forces”).

Timing Important is also the timing of the implementation of the intervention strategy. The intervention strategy used in the framework of the Belgian EU strategy has worked well. The interventions started sufficiently early in order to influence the thinking of the policy makers, when shaping the programme.

At national level, it should become common to work closely with the presidencies and members at national level. Another proposal is to work more closely with regional representation at the EU; they have working groups on specific themes that might be useful.

Events in the framework of the Belgian EU Presidency: involvement of Eurochild

2nd and 3rd September 2010, Marche-en-Famenne, Conference on child poverty, organised by the Belgian Presidency of the EU, the EC, King Baudoin Foundation, UNICEF and Eurochild.

Eurochild prepared a background paper for the conference together with UNICEF.

8th – 10th of September 2010, Antwerp, Expert conference ‗Europe de l‘Enfance‘. On the initiative of the EU initiative of the EU Presidency, the ministers and officials competent for childhood policy will meet biannually under the name ‗Europe de l‘Enfance‘.

Eurochild attended the meeting; a staff member, the President and the Secretary General participated in panels.

28th – 29th of September 2010, Ghent, conference on ‗Breaking the cycle of disadvantage – Social Inclusion in and through education‘.

Eurochild was invited to moderate one of the working groups

.

Page 40: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

40

19th of October 2010, Brussels, Meeting for European Ministers for Combating Poverty organised as extension of the Round Table on Poverty and Social Exclusion. In order to meet the poverty reduction target, the Belgian Presidency requested the Member States define clear and quantified sub goals to reduce child poverty and include the fight against child poverty in the priorities of the European Platform against Poverty.

Eurochild participated in preparatory meetings. The President of Eurochild presented a study in one of the workshops; the Secretary General was rapporteur.

2nd – 21st of October 2010, Brussels, as part of the EY2010, an exhibition of photographs is presented ―draw me a future‖ in praise of social workers in Brussels, who lead the way in combating child poverty.

22nd of October 2010, Brussels, Child in Flanders conference.

17th – 18th of November 2010, Brussels, Expert conference ‗Vulnerable unaccompanied minors, a European challenge‘.

6th of December 2010, Brussels, EPSCO council of Ministers (Employment, Social Policy, Health and Consumer Affairs). The EPSCO adopted the Declaration on the European Year for Combating Poverty and Social Exclusion. Child poverty is recognised as an absolute priority.

7th and 8th of December 2010, Brussels, Fundamental Rights Conference, Challenges and strategies to protect particularly vulnerable children and deliver child-friendly justice in the EU. Ensuring justice and protection for all children.

Eurochild was invited by the FRA to organise a pre-meeting with children prior to the conference. The children delivered their messages at the opening session.

17th of December 2010, Brussels, Closing session of the European Year of Combating Poverty in which the State Secretary for Poverty reduction and Social Integration stated that major steps forward have been made in the fight against child poverty. ―A European Recommendation is underway‖.

Eurochild attended the meeting.

Page 41: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

41

b. Outcomes and impact at national level

For member organisations, the EU level is often not a priority in their work. The work of Eurochild is at national level used to give weight to arguments, to give examples. At national/regional level members use arguments produced by Eurochild to influence their level of governance ―We have used the EU policy debate for our own national level influence‖.

Strengthening organisational capacity

According to some members, Eurochild had certainly impact on their organisation. It has improved the understanding of some issues, like child

poverty from a multi-perspective view. Furthermore, through the involvement in Eurochild, NGOs became aware of the necessity of interaction with other stakeholders, like e.g. local authorities. Eurochild has enabled some of its member organisations to build partnerships with other organisations in Europe.

Member organisations also express that their capacity as an organisation has been strengthened through their involvement in Eurochild. Some organisations took over elements of the network model of Eurochild, e.g. the Thematic Working Groups. For some members, the impact of Eurochild at national level should primarily be looked for at the level of capacity building of member organisations and through this, also of other organisations in the country. This can be considered as ‗horizontal capacity building’, or member organisations helping other organisations at regional/national level to build their capacity with the help of the work of Eurochild and this to strengthen their quality.

What the exact use is of Eurochild documents at national level is not delivered. Some members argue that if documents would be available in their own language, impact would be bigger. In the framework of Eurochild, a translation fund is available which is currently not really used (only for 10%).

Policy influencing

The national structure that is used in pursuing advocacy goals, is different for each Member State. The structure of one umbrella organisation would not work for all Member States, taking into account the role of NGOs in a certain Member States as well as the role of other stakeholders, like e.g. regional authorities in

the domain of child well-being and children‘s rights.

It is clear that the level of influence that can be achieved at national level, is varying across Member States and is depending on the framework in that particular Member State. In some countries, the need for capacity building of NGOs might not be a priority anymore, while in others this capacity building is probably the highest level of impact to be achieved currently.

At Member State level examples are given of visible progress made like e.g. child poverty targets that have been defined in Bulgaria. However, for members it is difficult to give even an estimation of what their impact has been at national level through their involvement in Eurochild.

Page 42: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

42

Another way of involving the national/regional level is inviting local actors to Eurochild events, as happened during the past years. For EU level participants this is interesting, since it gives concrete examples. For local participants this is also useful, since it enables them to make their voice heart at EU level.

Diverse membership

It should also taken into consideration that the membership of Eurochild is diverse and that not all members are into advocacy and policy influencing. Some members focus on service delivery or some members that are networks themselves emphasise capacity building of their members to improve the quality of services. This is one the reasons why for members it was difficult during the telephone interviews to give concrete examples of outcomes and impact on policies at national level.

c. Outcomes and impact at EU level

Assessing the outcomes and impact has also to do with expectations. It is clear that the expectations vary among members in relation to what could have been achieved. E.g. the fact that the Recommendation is announced for 2010 in the EPAP is a success for some, but is not good enough for others.

Increased visibility and profile

All agree that Eurochild has become much more visible at EU level; it has a very good profile, in the EC as well as in the EP. Much work has been done during 2010 to further develop this profile. Especially more work was invested at the level of the EP (contacts with various MEPs on children‘s rights e.g. advocacy the create an informal group of MEPs on children‘s rights, meeting to follow-up the children‘s rights strategy, meetings on specific issues like maternity leave).

Trustful partner

Eurochild is by external stakeholders considered as a trustful partner, e.g. in the framework of the Belgian EU presidency. Their work and recommendations made in the framework of the Belgian presidency is considered to be very successful. ―It will help work on the Recommendation as well‖. Unfortunately this Recommendation has been postponed to 2011 because of changes in the policy framework.

Eurochild is consulted by different instances like the FRA to add a European perspective and they establish partnerships with other international organisations, like UNICEF and more recently with the Council of Europe.

Qualitative content

External stakeholders are of the opinion that the best way to influence is to come up with subsistence, to have profound knowledge of the domain and to have added value. It is assessed by external stakeholders (including policy makers) that Eurochild is doing that; that Eurochild is exploiting existing data and is at the same time coming up with new data. Policy makers state that Eurochild does not only have a wish-list, but that they are very good in delivering trustful input. ―They know how to prepare a dossier‖.

Page 43: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

43

It is also said by external stakeholders that Eurochild is trying to link children‘s rights and child well-being/child poverty in a pragmatic way, building on activities that have most potential at EU level.

Policy makers also are of the opinion that Eurochild has fed successfully into the EU2020 Strategy. ―They are working on the spot, using material from grass-roots organisations‖.

Child participation

Eurochild is at a point of further professionalising its activities in relation to child participation. There are professional organisations knowing how to empower children. Existing methodologies should be reviewed. Child participation may be an issue for various reasons, like fashion, political

arguments and personal arguments.

At the same time, it is the area of child participation that is mentioned by various interviewees as one of the domains where Eurochild made a different the last year. By voicing child participation in various papers and statements, this message has been taken up in EU level discourse. Furthermore, in the framework of the Belgian Presidency, Eurochild was invited by the FRA to organise a pre-meeting to the Fundamental Rights Conference in Brussels (December 2010) with children.

Indicators

Progress in relation to the development of indicators on child well-being, as part of the indicators on social inclusion, is considered to be significant.

More focus

It is important to focus the work at EU level. Intervention strategies should always be built on knowledge that is generated in a triangle between knowledge, practice and policy. There is the danger that because Eurochild is hitting everything, the spread is becoming thinner as well as the impact dispersed. There is an issue of prioritising.

The relation between input and output at EU level is in general assessed as being very effective. The active engagement of members has been crucial.

However, only members with the capacity and a strong infrastructure can play an active role in advocacy at EU level.

Page 44: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

44

d. External factors

Various external stakeholders play a role in shaping the agenda and reacting to the agenda of child well-being and children‘s rights at EU and national level, i.e. UNICEF, UN, CRAG, FRA, Council of Europe and numerous national organisations.

At EU level Eurochild has succeeded in cooperating with these stakeholders in a successful way. These cooperations are considered to be important to strengthen the voice and to progress the work.

The EU2020 Strategy build on its predecessor, the Lisbon Strategy. The immediate focus of the new strategy is on recovering from the economic crisis. The whole strategy is about growth: smart, sustainable and inclusive, whereby the emphasis is on reinforcing education and life-long learning, research and innovation and modernising industry. Inclusive growth means more and better jobs, investments in skills and training, modernising the labour market and ensuring that the benefits of growth reach all parts of the EU. Within this context a poverty target has been defined and an EPAP was launched, ensuring economic, social and territorial cohesion.

This new EU agenda will certainly set the boundaries of the work of social NGOs at EU level. Some interviewees see it as a challenge, while others consider it as a major drawback. Cooperation between international and national actors is considered to play even a more important role in this new context. It argued by some that there should be one voice to strengthen the impact.

At the same time, the work accomplished in the context of the EY2010 and in the framework of the EU Belgian presidency is considered to have been fruitful opportunities to make progress. EU2010 did not only create a favourable climate, it has been a purposeful strategy of Eurochild to work towards this year.

The financial crisis will have effects on the work of the members at national level, and eventually also on the work of Eurochild.

Furthermore, as already said, the compartmentalisation of the domains of child well-being and children‘s rights at EU and national level, is a hindering factor.

e. Future outcomes and impact

In general, it is argued that Eurochild is becoming more attractive as a partner. It is viewed to be important for Eurochild to set its own priorities; to define its own agenda. Some members argue that Eurochild is moving closer to the EC, also because of its own success (―the more you are visible and successful, the more you are asked”). It is claimed that this might call for a revision of the current frame of thinking.

Towards the future it is recommended to not only focus on social aspects, but also on economic arguments, since the current EU framework is one of economic growth and factors leading to this. This could be approached from the angle of what costs are involved not taking care of children.

Page 45: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

45

The future of Eurochild is considered not to be so much about expanding the network (unless in the view of strategic development for policy development), but rather to deepen the activities; the critical mass currently available is enough.

3.4.3. Analysis of e-mail query

11 MEPs were contacted. 3/11 MEPs were contacted for a telephone interview; 8/11 were sent written questions to be answered. These eight MEPs posed a question (written or oral) about child being or child poverty in the period October 2009/December 2010 in the EP. The questions were about various aspects:

How does the Commission intend to ensure compliance with standards set [like e.g. Article 24 of the Charter of Fundamental Rights, Stockholm Programme] and what step does it intend to take, particularly in connection with the drafting of the action plan provided for in the Stockholm programme, with a view to ensuring that comprehensive strategic action is taken in defence of children? (February 2010).

Can the Commission clarify what actions the new Commissioners and the High Representative of the Union for Foreign Affairs and Security Policy intend to take to honour their pledges highlighted in Article 3 of the Treaty of Lisbon and in other provisions with respect to implementing the CRC in their respective policy areas? (March 2010).

Is the Commission prepared to send a clear message to the MS in order to support the development of a European strategy on the rights of the child as a priority in implementing the Stockholm programme? (March 2010).

Can the Commission provide information what European funding is being earmarked for measures to combat child poverty? What is the Commission‘s assessment of the campaign which has been launched in the UK and can such an approach be made to serve as a model? Are measures being considered to give more active support to assist the MS and encourage them to introduce comprehensive campaigns to combat child poverty? In the context of the EY2010 are funds available which could be specifically earmarked for combating child poverty? If so, how much funding is available and how will it be allocated? (March 2010).

With regard to the EU‘s commitment to its Strategy on the Rights of the Child, is the Commission willing to move away from an issue-based approach to providing strategic direction and supporting implementation of a children‘s rights perspective across EU policy, legislation and programming? (February/April 2010).

Shouldn‘t there be monitoring at Community level of how the protection of children‘s rights is implemented, as well as stricter supervision as regards how the MS ensure that children‘s rights are protected and whether the institutions responsible for doing so are carrying out their work properly with a view to protecting the most vulnerable section of society – children? (April 2010).

Page 46: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

46

What targets have so far been set for reducing child poverty? Is the Commission considering the possibility of setting up an annual assessment panel on child welfare in the EU? What measures will it take improve the collection of data on child poverty? What measures have been taken to promote the exchange of experience and mutual learning among MS in relation to policies and practices to combat child poverty? How will it integrate child welfare into the various structural policies (July 2010).

Will the Commission take specific measures in the context of the EU2020 Strategy to protect children? Does it have comparative data for the MS indicating the number of children living below the poverty line? What is the situation with regard to Greece? (September 2010).

Can the Commission state how it intends to resolve the sad paradox of a rich continent which has children in poverty, bearing in mind that all the facets of life in our future societies – be these demographic, economic, social or cultural, etc. – depend on the children of today, an increasing number of whom are innocent victims of the terrible and traumatising malaise of poverty? (October 2009).

Only 1/3 MEPs accepted a telephone interview. A majority of MEPs responding to the written questions state that they know Eurochild and confirm that they have used information of Eurochild in preparing questions, besides other information sources. One MEP does not know Eurochild and gets the necessary information from other sources (e.g. Plan International and S.O.S. Children Villages). In July 2010 an Intergroup has been created at the EP on Family and the Right of the Child & Bioethics, counting 106 MEPs.

3.4.4. Analysis of peer reviews

In the period 2004-2010 eight peer reviews took place in the framework of the Peer Reviews in Social Protection and Social Inclusion related to child well-being and child poverty. Furthermore, one of the policy assessment by the network of independent experts was on tackling child poverty and promoting the social inclusion of children in the EU (April 2007). In 3/8 peer reviews Eurochild was one of the invited stakeholders (May 2006, January 2010 and May 2010). These peer reviews were on:

Promoting social inclusion of children in a disadvantaged rural environment (Hungary).

Federal Foundation Mother and Child for pregnant women in emergency situations (Germany).

Sure Start, a UK government‘s programme helping children from disadvantaged backgrounds and their families.

In one other peer review (City Strategy for tackling unemployment and child poverty UK – July 2009), Eurochild was referred to in the discussion paper.

Page 47: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

47

In 10/27 national experts‘ reports on ‗tackling child poverty and promoting the social inclusion of children in the EU‘ Eurochild is referred to.

Page 48: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

48

Conclusions and recommendations in relation to

impact analysis

4.1. Inputs

Investments have been made by Eurochild in relation to child well-being/child poverty and children‘s rights to a varying degree. These investments are ex-post difficult to identify, especially investments made by members in these areas. There is a difference in history in interventions of Eurochild in relation to children‘s rights (used to be THE domain of Euronet) and child well-being/child

poverty (used to be THE domain of Eurochild). Therefore the current assessment is that proportionally less investments were made in relation to child poverty, but this might change in the future. According to some members Eurochild could be more outspoken and prompt in filling in the gap created by the ending of Euronet. Eurochild is for the moment not yet considered as the organisation at EU level voicing children‘s rights. A current challenge is upgrading the skills available at the Secretariat and at the level of the network as a whole, to liaise with the appropriate level of policy makers and politicians. It needs time and capacity to build up relationships with policy makers and politicians at different levels of governance, the EU level to begin with. First of all a common framework should be developed on advocacy and training sessions are recommended to be organised.

4.2. Activities and outputs

Eurochild has always used a mixture of tools within its interventions strategies. It is however not clear for the members what specific mix has been used for specific objectives. Discussions with member are mainly taking place around the use of particular instruments, like e.g. the approach of a campaign. Debates about intervention strategies do currently not follow a clear intervention logic (what are expected outputs/outcomes/impact, what activities are needed for this, what indicators are used, what is the timeframe?). During the timeframe that has been considered for this impact assessment, various papers have been produced in relation to child well-being/poverty and children‘s rights (see list in paragraph 3.2.1.). Furthermore, various events/activities have been organised (see for an overview the Info Flash announcements) and Eurochild has been participating in a series of events (more than 90 in 2010 and Eurochild has delivered a speech in about 1/3rd of these events). A particular interesting activity that was organised under the form of a project were the study visits to Denmark and Sweden. In terms of impact analysis it would be useful to plan a further tracking of how lessons learnt could be used for policy development.

Page 49: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

49

The strength of Eurochild as a membership organisation is that the expertise of the members can be used in building up interventions. In this way EU level knowledge is integrated with grass-roots level expertise, which is much appreciated by relevant policy makers. The fact that the targets of Eurochild are compartmentalized across DGs at EU level and ministries at national level, do hinder a holistic approach. However, the mainstreaming approach could be used more prominently. Networking and cooperation with relevant stakeholders is considered to be crucial in order to strengthen the voice of child well-being/poverty and children‘s rights at EU level. It is assessed that Eurochild is doing this in an effective way (cooperation with UNICEF, CRAG, UN, FRA, CoE). Timing is considered to be another important factor in the success of an intervention strategy. The example of the performance of Eurochild during the Belgian EU Presidency is put forward as good practice. The intervention started sufficiently early and at different ‗fronts‘ at the same time.

4.3. Outcomes and impact

This paragraph is structured on the basis of scheme 1 and scheme 2. The categories of expected and potential outcomes and impact are the starting point. After this, the findings are structured on the basis of the expected outcomes and impact at national and EU level, which are linked to indicators that can be assessed in 2011. As a consequence only the short-term indicators are taken into account for this report.

4.3.1. Shifts in social norms

Outcomes Indicators

Child well being and combating

child poverty is recognised as a

political priority.

An increased understanding on the need to fight child poverty amongst policy makers at national

and EU level is demonstrated (e.g. in policy texts and discourse).

At EU level, the increased awareness is demonstrated to some extent, e.g. by the recognition of the need for a Recommendation on child poverty, the need for having a Strategy on the rights of the child. However, at the same time, the question is how much progress has been made: the Recommendation is postponed to 2012 and the Strategy is not yet concretised.

Page 50: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

50

The review of the 2008-2010 National Strategy Reports on social protection and social inclusion14, shows that there is little overall improvement in child poverty levels between 2005 and 2007, however in various Member States the need to fight child poverty is demonstrated. 19 Member States had made tackling child poverty and social exclusion as one of their key priorities. Furthermore, the majority of Member States include quantified targets for the reduction of child poverty in their NAPs/inclusion; however in some cases targets were not very ambitious. It is encouraging, though, that a majority of Member States adopt a multi-dimensional approach involving a mix of different policies areas.

During the telephone interviews, examples are given of some Member States were specific initiatives were taken, like e.g. in Bulgaria where the government approved in October 2010 social inclusion targets, with a general poverty reduction target and with specific aims in relation to children as well.

Some members did translate some of the documents produced by Eurochild to send it to their members. However, in the short time frame of this evaluation, it was not possible to give a complete overview of shifts in thinking at national level as a result of the intervention strategies of Eurochild. For members it is difficult to come up with concrete examples of outcomes/impact.

4.3.2. Strengthened organisation capacity

Outcomes Indicators

Eurochild is seen as an

influential actor in the promotion of children‘s rights.

Eurochild cooperates with other NGOs on children’s rights issues.

Eurochild is invited by national governments to participate in (high-level) events on children’s rights.

Eurochild is invited by the EC to participate in (high-level) events on children’s rights.

The language used by Eurochild in relation to children’s rights is taken up at national level.

The language used by Eurochild in relation to children’s rights is taken up at EU level.

Members gather at

national/regional level

commitment to EU cooperation

on child poverty & well-being.

A majority of members influencing policy at regional/national level in the domain of child well being and combating child poverty is using Eurochild material for this purpose.

14 Eurochild, (2009), Ending child poverty within the EU? A review of the 2008-2010 National

Strategy reports on social protection and social inclusion, Brussels

Page 51: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

51

Eurochild is recognised as an

influential actor in the domain

of child well being and

combating child poverty.

Eurochild is invited by other NGOs to cooperate on child well-being issues.

Eurochild is invited by national governance to participate in (high-level) events on child well-being.

Eurochild is invited by the EC to participate in (high-level) events on child well-being.

The language used by Eurochild in relation to child well-being is taken up at national level.

The language used by Eurochild in relation to child well-being is taken up at EU level.

The profile and visibility of Eurochild at EU level has increased.

Eurochild is considered to be a trustful partner by policy makers at EU level.

Some members interviewed are of the opinion that Eurochild had certainly an impact on their organisation, in the way they approach child well-being from an holistic perspective and in their understanding of some issues in relation to child well-being. This has strengthened their own capacity to act at a national level.

A few members argue that in their Member State a strengthened organisational capacity is the maximum level of influence possible for the time being. A stronger NGO sector is needed to enable policy influencing.

Eurochild has been invited e.g. in the framework of the EU Belgian Presidency to various high-level meetings in relation to child well-being (see also in paragraph 3.4.2.a). Also at EU level Eurochild has been invited to about 20 meetings. The latest years, Eurochild has been more invited to peer reviews as an external stakeholder.

The language and messages used and produced by Eurochild can be traced in policy documents and discourse at EU level. Various examples were listed in paragraph 3.4.1. Although a causal relation is not proven, there is certainly influence, which is also confirmed by interviewees.

So far, tracking and monitoring policy changes and outcomes/impact at national level via the membership has not been done in a structural way. For members it is difficult to say what the influence of Eurochild has been at national level. They argue that they use material from Eurochild to do their own policy development and advocacy at national level, but in the remit of this evaluation it is difficult to say to what extent.

Page 52: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

52

4.3.3. Strengthened alliances

Outcomes Indicators

Eurochild is seen as an

influential actor in the

promotion of children‘s rights.

Eurochild cooperates with other NGOs on children’s rights issues.

Eurochild is recognised as an

influential actor in the domain of child well being and

combating child poverty.

Eurochild is invited by other NGOs to cooperate on child well-being issues.

Eurochild cooperates with other NGOs and stakeholders on children‘s rights issues. These cooperations (like e.g. the informal CRAG grouping of NGOs) are assessed as being fruitful by the different actors involved and by external stakeholders.

The Eurochild network is still growing, which shows an interest of NGOs to cooperate with Eurochild. 15 new members were welcomed in the network in 2010.

Furthermore, Eurochild has enabled some of its members to build relationships with others in the EU (e.g. Bulgaria and the Netherlands)

4.3.4. Strengthened base of support

Outcomes Indicators

National level European level

Children‘s

rights are

mainstreamed

within EC

work & policy development.

Training on children’s rights is delivered.

Members gather at

national/regional

level commitment

to EU

cooperation on

child poverty & well-being.

The discourse of members in the field of child

well being and combating child poverty is used at national/regional policy level(s). Evidence of this use can be demonstrated.

Page 53: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

53

4.3.5. Improved policies

On the basis of this evaluation, examples can be given of Member States where member‘s discourse is used at regional/national level, like e.g. in Örebro (Sweden) where children‘s issues are part now of the local strategy or the child poverty targets introduced in Bulgaria.

There is evidence of increased visibility of Eurochild at EU level. No data are found on training on children‘s rights.

Protection of

children‘s rights is recognised at

national level.

National

governments

are supportive of EU

cooperation on

child rights.

Protection of

children‘s rights is recognised at

EU level.

A majority of members influencing policy at regional/national level in the domain of children’s rights is using Eurochild material for this purpose.

Children‘s rights

are

mainstreamed

within EC work

& policy development.

UNCRC is mentioned in key documents across the DGs.

Child poverty &

well-being

acknowledged as

a thematic

Actions on child well being and combating child poverty are announced/taken within the

EPAP.

Outcomes Indicators

European level National level

Children‘s rights have a significant

position within

the Europe 2020

strategy.

Actions on children‘s rights are announced/

taken within the EPAP.

The EU Strategy

on children‘s

rights reflects inputs from

NGOs and other

relevant

stakeholders.

A consultation process on the future EU strategy on the rights of the child is implemented in a satisfactory way (taking into sufficient time for consultation, dissemination of information about consultation process,

wording of consultation, etc.). Eurochild its members respond to the

consultation. Children and young people are involved in the

consultation process.

Page 54: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

54

priority within

the European

Platform Against

Poverty.

Child well being and combating

child poverty is recognised as a

political priority.

Child poverty targets at EU level are set within the Europe 2020 strategy.

Child poverty targets at national level are set within the NRPs.

Actions to fight child poverty are set within the NRPs

An increased understanding on the need to fight child poverty amongst policy makers at national level is demonstrated (e.g. in policy texts and discourse).

At EU level improvements are announced in policy documents, but actual evidence cannot yet be ascertained:

In the EPAP it is pronounced that in 2012 the EC will present a Recommendation on child poverty outlining common principles and effective monitoring tools to combat and prevent poverty in early age. Such a Recommendation has been proposed by Eurochild e.g. in its policy position on the Call for EU action on child poverty and well-being of October 2009. Furthermore, the Commission will support through the Progress programme regular exchanges and partnerships between a wider set of stakeholders in specific priority areas, such as active inclusion, child poverty, Roma inclusion, homelessness and financial inclusion.

In the summer of 2010 a consultation was used on the future EU strategy on the rights of the child. Eurochild responded to this consultation in August. The timing was really an issue, considered the summer break, making it very difficult to consult with members. Consultation with children was almost not possible, because of the timing and the time frame (8 weeks).

At least five members of Eurochild have sent individual replies to the consultation. Nine members sent contributions to Eurochild.

The UN Convention on the Rights of the Child is the first legally binding international instrument to incorporate the full range of human rights—civil, cultural, economic, political and social rights and is mentioned in various documents at EU level, like e.g. the EC Consultation on Children‘s rights of 2010, the document prepared in the framework of the EU Belgian Presidency on the Call for an EU Recommendation, in the Stockholm Programme and in the speeches of the EC and the EP during the 5th Forum on the Rights of the Child. However, it is e.g. not mentioned in the 2008 report of the Social Protection Committee on Child poverty and Well-being in the EU15.

15 European Commission, (2008), Child Poverty and Well-Being in the EU. Current status and

way forward, report by the Social Protection Committee, Brussels

Page 55: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

55

At EU level the EU2020 Strategy only refers to a general poverty reduction target (at least 20 million fewer people in or at risk of poverty and social exclusion), not to child poverty targets.

At national level, the majority of Member States included quantified targets for the reduction of child poverty in their NAPs/inclusion (2008-2010). While some in some Member States the targets were not very ambitious, good examples of quantified target setting could be identified16.

In the 2009 Eurochild report on the NAPs/Inclusion 2008-2010 a good analysis is made on how messages about child well-being/combating child poverty and children‘s rights are reflected in national plans. For a follow-up, the reports to be produced in the framework of the EU2020 Strategy will have to be waited for. Member States will have to produce two reports each year explaining their plans in relation to moving closer to the EU2020 Strategy targets. Stability/convergence programmes about public finances and fiscal policy will have to be submitted before governments adopt their national budgets for the following year. National reform programmes will have to be delivered containing elements necessary for monitoring progress towards the EU2020 targets.

4.4. External factors

Different external factors intervene with the policy influencing work and impact of Eurochild. These are first of all various other organisations and stakeholders that operate in this field. Measuring ‗individual‘ contribution to change is very difficult. Secondly, various factors and processes in the environment of Eurochild interact with the interventions that Eurochild is planning and implementing, like e.g. political willingness, but also broader developments that are not to be influenced, like e.g. the past and current financial crisis. Specific external factors that are identified are:

Stimulating:

Fruitful cooperation with various stakeholders (cooperation instead of competition).

EU Belgian Presidency that created opportunities.

EY2010 that has put focus on combating poverty and social exclusion.

Hindering:

The current EU-policy framework emphasizing economic growth.

The financial crisis.

Compartmentalization of policy domains of child well-being/poverty and children‘s rights across DGs and ministries without real horizontal linkages.

16

Eurochild, (2009), Ending child poverty within the EU? A review of the 2008-2010 National

Strategy reports on social protection and social inclusion, Brussels

Page 56: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

56

4.5. Recommendations on the basis of the impact analysis

On the basis of the findings the following recommendations are made:

a. It is recommended to discuss intervention strategies explicitly with members thereby following an intervention logic:

What are the objectives? What are the activities developed to achieved these objectives? What investments will be made? What is the expected output/outcome? What activities will be used to achieve this output? What indicators will be used to assess the output? What is the timeframe? What is the effectiveness of the strategy followed? Thereby it is important to have a clear vision on the overall strategic objectives to be achieved.

b. It is advised to continue to involve members in policy development, e.g. through the Thematic Working Groups. The integration of EU level knowledge and grass-roots level evidence is very much appreciated by various internal and external stakeholders.

c. It is advised to follow-up initiatives that are launched in the domain of impact analysis, like the project on study visits and the impacts of lessons learnt.

d. Strengthening the capacity of the Secretariat and the network as a whole in the domain of advocacy is recommended. Face-to-face liaising with policy makers and politicians seems to be successful. Training should be considered.

e. In relation to advocacy it is recommended to start from a common framework with objectives, tools to be used, etc.

f. Capacity building should not only be considered in view of policy influencing, but also in view of helping to strengthen the capacity of (non-member) peer organisations and this to strengthen the quality of their work.

g. It would be useful to analyse the intervention strategy used in relation to the EU Belgian Presidency. The role and functioning of Eurochild in this

framework has been assessed as very successful. Transferability to future work with EU Presidency should be researched.

h. It is recommended to continue to use a mixture of tools in intervention strategies. At the same time, more reflection might be useful on the use of certain tools under certain circumstances with what degree of success. This is useful as a learning process.

i. Focusing advocacy work at the EU level is advised by various members of the network.

j. Within future intervention strategies, not only social aspects should be focused on, but also economic arguments should be taken into account,

Page 57: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

57

considering the current EU2020 Strategy: ―What is the cost of not taking care of children?‖.

k. Members play an important role in assessing outcomes and impact at national level. In order for them to take up this role, it should be clear for members what the bigger context is, what the links are between national and EU policy development, etc. It is recommended in this context to develop and deliver capacity building activities for members.

Page 58: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

58

5. Conclusions and recommendations in relation to

the implementation of the impact analysis framework

5.1. Conclusions

Only the short term indicators could be assessed within the remit of this evaluation. Most of the short term indicators defined are related to improved policies, followed by strengthened organisational capacity.

Figure 4: Level of indicators defined on short term

Level of

outcomes/impact

National level European level

Shift in social norms

Strengthened

organisational capacity

Strengthened alliances

Strengthened base of

support

Improved policies

The darker the colour, the more indicators are defined at that level.

First of all, it should be stated that the implementation of the framework has been a somewhat difficult exercise and mainly for the following reasons. An impact assessment framework should be developed before an intervention takes place in order to be able to keep track of the different variables of the process, like input, activities, etc. Furthermore, the expected outcomes/impact should also be defined at the start of the intervention process in order to be

able to assess whether expectations have been achieved. In this impact assessment, the framework and thereby the expected output, outcomes, impact and related indicators were defined in retrospective. They were defined in the timeframe that some of the outcomes became already visible. Although the impact analysis process should run in parallel with the intervention process. In this way, it is possible to track the variables of the logical framework and to assess in the end. If this is not done, there will be data lacking for a proper assessment. A second important remark relates to the distinction in levels related to indicators: national/European level and short/long term. Within the remit of this evaluation it is difficult to fully report on outcomes/impact at national level. Members interviewed were not always in a position to give data on

Page 59: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

59

national impact, also because they are not required to monitor changes in terms of policy development and impact. Building impact indicators in the work is a prerequisite to monitor changes.

5.2. Recommendations on the basis of the implementation of the

impact analysis framework

On the basis of the findings, the following recommendations can be made:

a. Start to define an impact analysis framework in parallel with the intervention: What is the expected impact to be achieved and at what level(s)? What

are the expected outcomes and at what level(s)?

Within what timeframe are the outcomes and impacts expected to become apparent?

What are the activities and tools needed to achieve this?

What outputs will result from the activities?

What are the necessary investments (time, money)?

b. Build in tracking mechanisms at the different levels (national, European and levels of shifts in social norms, organisational capacity, support base and policy) defined in the framework in order to be able to deliver assessment data.

Page 60: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

60

Bibliography

1. Angelili, R.,, Vice President of the EP, during the 5th European Forum on the rights of the child. Moving forward on the EU strategy on the Rights of the Child, October 2010

2. Belgian Presidency of the EU, (2010), Call for an EU Recommendation on Child Poverty and Child Well-Being, June 2010. Background paper to the EU Presidency Conference: Child Poverty and Child Well-Being, 2-3 September 2010

3. Chapman, J., (2002), Monitoring and evaluating advocacy, PLA notes, 43:p48-52

4. Chapman, J., and A., Wameyo, (2001), Monitoring and Evaluating Advocacy: A Scoping Study

5. Church, M., et al, (2002), Participation, Relationships and Dynamic Change: new thinking on evaluating the work of international networks, Working Paper No 121, Development Planning Unit, University College London, London

6. Consultation document: European Commission‘s Communication on the Rights of the Child (2011-2014) (June – August 2010)

7. Eurochild contribution to the BE Presidency conference ―EU coordination in the social field in the context of Europe 2020: Looking back and building the future‖. Where will action against child poverty and social exclusion fit within the next EU 10-year strategy?

8. Eurochild key messages. 2010 European year to combat poverty and social exclusion

9. Eurochild policy briefing (2009), Indicators: an important tool for advancing child well-being. Supporting document to Eurochild‘s 6th Annual Conference in Cyprus

10. Eurochild policy position (2009), Call for EU action on child poverty and well-being. October 2009

11. Eurochild policy position (2009), Eurochild’s proposals for the development of the EU’s strategy on the rights of the child

12. Eurochild response to the European Commission Consultation on the future “EU2020 Strategy‖

13. Eurochild, (2010), Child participation – how to involve children. Speech for the 5th European Forum on the Rights of the Child, Brussels, 14 October 2010

14. Eurochild, (2010), Terms of Reference of the Policy Steering Group, May 2010, Brussels

15. Eurochild‘s contribution to the EAPN/BAPN conference ―Laying the foundations for a fairer Europe – Ensuring an adequate minimum income for all‖. Ensure adequate resources throughout the life cycle. A children‘s perspective. September 2010

16. Eurochild’s response to the Consultation on the European Commission’s Communication on the Rights of the Child (2011-2012). August 2010

17. European Commission, (2006), Communication from the Commission – Towards an EU Strategy on the rights of the child – COM (2006) 367 final

18. European Commission, (2010), Commission Staff Working Paper. List of Key initiatives. Accompanying document to The European Platform against

Page 61: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

61

Poverty and Social Exclusion: A European framework for social and territorial cohesion, COM(2010) 758 final, Brussels (December 2010)

19. European Commission, (2010), The European Platform against Poverty and Social Exclusion: A European framework for social and territorial cohesion, SEC(2010) 1564 final, Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, Brussels (December 2010)

20. European Council, (2010), The Stockholm Programme – an open and secure Europe serving and protecting citizens, (2010/C 115/01), in: Official Journal of the European Union, 4.5.2010

21. Mc Cawley, F., (2002), The Logic Model for program planning and evaluation, University of Idaho Extension

22. Nuňez, M., and R., Wilson-Grau, (2003), Towards a conceptual framework for evaluating international social change networks

23. Opinion of the European Economic and Social Committee on Child poverty and child well-being. Exploratory opinion. Brussels, 14 July 2010. Rapporteur-General, Ms Brenda King

24. Portes, R., and S. Yeo, (2001), Think-net: the CPR model of a research network CEPR, Prepared for the workshop on local to global connectivity for voices of the poor, 11-13 December 2000, World Bank, Washington

25. Reding, V., Vice President of the EC, during the 5th European Forum on the Rights of the Child, A renewed commitment to children’s rights, October 2010

26. Reinicke, W., et al, (2000), Critical choices; The UN, advocacy networks and the future of global governance, Global Public Policy Institute

27. Reisman, J., Gienapp, A., and S., Stachowiak, (2007), A guide to measuring advocacy and policy, funded and prepared for Annie E. Casey Foundation

28. Roche, Chris (1999), Impact Assessment for Development Agencies: Learning to Value Change, Oxfam

29. Subedi, Nani Ram, (2008), Advocacy strategies and approaches. A training of trainers manual. Second Edition, International Centre for Integrated Mountain Development, Kathmandu, Nepal

30. The Rockefeller Foundation – The Goldman Sachs Foundation, (2003), Social Impact Assessment. A discussion among grantmakers, New York

31. Valuing Children’s potential. How children’s participation contributes to fighting poverty and social exclusion, edited by Mieke Schuurman. September 2010

32. Vanclay, F. 1999b. ―Summary of workshop on International Guidelines and Principles for Social Impact Assessment‖, report to the closing session of the meeting of the International Association of Impact Assessment, Glasgow.

33. Vanclay, F., (2001), Social Impact Assessment, one of the contributing papers to the World Commission on Dams, Cape Town, South Africa

34. Wilson-Grau, R., (2007), Evaluating the effects of international advocacy networks, paper presented at the Advocacy Impact Evaluation Workshop at the Evans School for Public Affairs, University of Washington, 4-6 December 2007, Seattle

35. Yeo, S., (2004), Evaluation of the SISERA network for IDRC, Preliminary report

36. Yeo, S., (2004),Creating, Managing and Sustaining Policy Research advocacy networks, Preliminary paper

Page 62: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

62

Annex 1

Eurochild Membership review

Checklist for (telephone) interviews 2010

1. Do you know the work of Eurochild? 2. Use and relevance of the work of Eurochild. 3. Your involvement in activities/events of Eurochild. 4. Activities of your organisation at EU/international level: now and in

future. 5. (Potential) Added value of work at EU/international level for your

organisation. 6. Feasibility of investments of your organisation in EU level/international

activities (financial, human resources). 7. Suggestions.

Page 63: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

63

Annex 2

Eurochild Impact analysis

Checklist for (telephone) interviews

January-February 2011

Two subjects are selected: children‘s rights and child well-being/child poverty.

1. Children’s rights

The work of Eurochild is underpinned by the principles enshrined in the UN Convention on the Rights of the Child. Therefore, this domain is not the most straightforward one for the implementation of the framework. Limiting the object of evaluation is very important in this case.

Within the domain of children‘s rights, the impact assessment will look at the contribution of Eurochild to impact in the domain of implementation of 2006 communication and the preparation of the EU Strategy on the Rights of the Child17 in 2010.

2. Child well-being/child poverty

The aim of the impact assessment of Eurochild in relation to child poverty is to look at the contribution of Eurochild to the impact on EU level policies and policy documents related to child poverty and child well-being in 2010 (like e.g. documents produced in the framework of the EY2010).

1. What is the intervention strategy used (tools,…) in both domains? What is the comparison (similarities and differences)?

2. What other stakeholders are active in both domains? What is their stake and influence? What are the results of their interventions?

3. To what extent are children‘s rights, child well being and combating child poverty integrated into the EU2020 Strategy? Into the EPAP? What has been the role of Eurochild in this?

4. What has been the role of Eurochild in the consultation process on the future Strategy on children‘s rights? Have members been mobilized? What has been the results?

5. Are members using Eurochild material in influencing national/regional policies in both domains?

17 In the framework of the Communication from the Commission – Towards an EU Strategy on

the rights of the child – COM (2006) 367 final

Page 64: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

64

6. What has been the role of the Eurochild in influencing national/regional policies in relation to the protection of children‘s rights?

7. What has been the role of Eurochild in influencing national/regional policies in relation to child well-being and combating child poverty?

8. To what extent are children‘s rights mainstreamed at EU level (e.g. mentioning of UNCRC in key documents)? What has been the role of Eurochild in this?

9. Did Eurochild cooperate with other NGOs in one or both domains? With what results?

10. To what extent was Eurochild invited by EU, national/regional governments to participate in high-level events?

11. How is the discourse/language of Eurochild used in both domains at different levels of governance?

12. To what extent is child well being and combating child poverty recognised as a political priority at national level (e.g. setting child poverty targets at national level in NRPs, implementing actions) and EU level (e.g. setting child poverty targets in EPAP, implementing actions)? What has been the role of Eurochild in this?

Questions for external stakeholders:

1. Knowledge of work of Eurochild in one of these or both domains. 2. Assessment of work of Eurochild in one of these or both domains. 3. Assessment of outputs and (potential) outcomes and impact of Eurochild

in one of these or both domains (publications, events, etc.). 4. What are the external factors (e.g. other important stakeholders) crucial

for the realisation of the activities, outputs, outcomes and finally impact but which are beyond the control of Eurochild?

Page 65: Research, Evaluation and Consultancy - Lastekaitse Liit · Research, Evaluation and Consultancy Dominique Danau SAGO Research Dorpstraat 22 NL – 6255 AN Noorbeek Chamber of Commerce

65

Dorpstraat 22 NL – 6255 AN Noorbeek Tel. + 31 43 457 45 67 [email protected] Registered in NL No. 14083101

SAGO is a small, independent organisation based on more than 20 years

experience in research and evaluation in regional, national and European context.

The core domains of expertise of SAGO are welfare, labour market, gender and competency development.

Core concepts in the work of SAGO are:

Quality

Cooperation

Result-orientedness

SAGO is specialised in research and evaluation projects and offers (temporary) support to programme and project teams.

Research, Evaluation and Consultancy