Simone Bruder 1 , Silke Hertel 1 , Samuel Greiff 2 & Bernhard Schmitz 3

Simone Bruder1, Silke Hertel1, Samuel Greiff2 &

Bernhard Schmitz3

1 German Institute for International Educational Research, Frankfurt am Main2 Heidelberg University, Germany3 Technical University Darmstadt, Germany

Observing Teachers` Counselling Competence – Evaluation of a Teacher Training Program with a Behavior Observation Instrument

Theoretical background: Relevance of the research

• Research on teacher competencies strengthened only recently since the research

focus lies mostly on students (Beck & Zlatkin-Troischskaia, 2010; Kunter & Klusmann, 2010)

• Most research on teachers in the core area of teaching (Herzog & Makarova, 2011)

• Counselling in addition to teaching, educating & assessing is a central task of

teachers (KMK, 2004)

• Teachers are the most important factor in providing counselling in the school

system (Landesinstitut für Schule und Weiterbildung des Landes Nordrhein-Westfalen, 1998)

• Counselling competency in models for teacher performance (Baumert & Kunter, 2006;

Rambow & Bromme, 2000)

EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 2

Theoretical background: Current situation

• Increase of situations in which counselling concerning learning strategies is

needed (Schnebel, 2007)

• Intensive cooperation between parents and school is positive for school

achievement and social behaviour (Cox, 2005; Hoover-Dempsey & Sandler, 1997)

• Teachers are often not well prepared for counselling tasks: often feel stressed

when counselling parents (Hitzinger, 1987; Hertel, 2009)

• Education in counselling prerequisite for working successfully together with

parents (Krumm, 1996; Wild, 2003)


Theoretical framework – Basic of this study

What has been done in research until now?

• It is important to specify aspects of counselling that are of particular relevance for

teachers

Bruder (2011) developed a 4-dimensional model of counselling

competence for teachers when counselling parents in learning

strategies

• Although being included in German teacher education, counselling is still not

taught as much as needed (KMK, 2004)

Hertel developed a teacher training program to foster teachers´

counselling competence (Hertel, 2009)


Theoretical background: Model of Counselling Competence (Bruder, 2011)

EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz

Counselling skillsDiagnostic/ Pedagogical Knowledge

Collaboration/ Perspective taking

Coping

Structuring Searching for

reasonsCooperative actions

Coping with criticism

Empathetic listeningDefining the

problemPerspective taking

Dealing with difficult situations

Paraphrasing Strategy knowledgeResources/Solution

orientation

Goal orientation

Results of confirmatory factor analysis: X2 = 56,16 / df = 45 CFI = .96, RMSEA = .03 / SRMR = .39

5

Based on: West & Cannon (1988); Reid (1990); McLaughlin (1999); Strasser & Gruber (2003); McLeod (2003); Schwarzer & Buchwald (2006); Hertel (2009)

Theoretical background: Counselling Competence Training Program (Hertel, 2009)

• 4 sessions of 210 min

• comprehensive in terms of competence dimensions

• covering aspects from communication theory to theory of self-regulated

learning

• included role-plays to simulate counselling talks



Session Issue Topics

1 Counselling skills Communication theory, Empathetic

Listening, Paraphrasing, Structure

of Counselling Talks

2 Collaboration/ Perspective taking Factors causing problems with

learning, cooperation of parents and

teachers

3 Diagnostic/ Pedagogical Knowledge

Theory of self-regulated learning,

strategies for parents to support

child’s learning

4 Coping Coping with Criticism, Meta-

Communication, Reflection of

Counselling Talks

Theoretical background: Counselling Competence Training Program (Hertel, 2009)

Theoretical background: How Measuring Counselling Competence?

Teachers’ Counselling competence

Paper & Pencil Behavior Observation

Knowledge Test

Work probe

SelfRating

Participating Observer

Non-particip.Observer

(Hertel, 2009) EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz

Behaviour observation

• The instrument was developed based on the model and consisted of 15 categories

• 12 categories refer to the competence scales / 3 categories capture all other behaviour

during the talk

• Raters were provided with detailed information on how to score the categories

• One rating interval takes 20 seconds

• Interrater-reliabilities for rating intervals of 20 seconds showed very good results: Cohen’s

Kappa (Cohen, 1968) κ > .80 for all categories


9

Research Aim and Questions

Aim:

• To observe the counselling competence of the teachers taking part in the

teacher training program and to evaluate the program

• To develop and test a behaviour observation instrument with a categorical

system based on the model

Research Question:

• Do the participants of the study improve in the four dimensions of the model (to

which the categories belong)?


Method

• Pre-post design with an experimental and a control group (Data for the behaviour

observations only available for the exp. group)

• Role-plays were audio-taped at the beginning of the first unit (pre-test) / end of

the fourth unit (post-test)

• Role-plays were recorded and analysed with the instrument

• Training program consisted of four units (each 3.5 hours)



Variables

Sample of the Training program

Sample of teachers with pre- and post behavior observations

N 81 14

Gender

Male 25 (31%) 6 (43%)

Female 37 (46%) 5 (36%)

No Information 19 (23%) 3 (21%)

Age

< 31 years 5 (6%) 1 (7%)

31 – 40 years 16 (20%) 4 (29%)

41 - 50 years 16 (20%) 1 (7%)

51 - 60 years 25 (31%) 5 (36%)

> 60 years 0 0

No information 20 (23%) 3 (21%)

Job experienceM (SD)

15.58 (10.3) 12 (9.22)

Descriptive Data for the samples

12

Results of ANOVA with repeated measuresfor the four dimensions


M Pre SD Pre M Post SD Post F(1,13) p

Counselling Skills .009 .028 .042 .051 12.66 .04

Diagnostic/Pedagogical Knowledge .085 .078 .211 .12 19.98 .00

Collaboration/Perspective Taking .069 .063 .101 .068 .222 .16

Coping² - - - - - -

²Not observed, no measures possible

Discussion

• The goal of the present study: (1) to develop an instrument to observe teachers

counselling competence before and after (2) a training program

• The observation instrument fills the gap of research that arrogates instruments to

measure teacher competences related to behavior (Kunter & Klusmann, 2010)

• Hypothesis: the participants would improve in 12 categories and the 4

dimensions of counselling

results showed that the instrument was able to indicate changes in

counselling competence

The instrument shows good interrater-reliability

Data from the pre- and post-role-plays showed significant increase in a

substantial number of categories -> promising result


Discussion

• The study shows that teachers can be supported in increasing their counselling

competence through training programs

• The information of the behavior observation can be used to optimize the training

program in the categories in which no improvements could be observed

• Limits of the study:

• Sample size decreased considerably from pre-test to post-test

• No control group design

• The instrument should be validated with other instrument on a bigger

sample


Thank you for

your attention!

[email protected]

16 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz


Questions to be discussed

1) How important is counselling competence for teachers?

2) How can counselling competence be more implemented into

teacher education?

3) How can the loss in data from pre- to post-test in the observation

instrument be reduced? (teacher often are afraid of giving such

sensitive data)

4) How can a larger sample be recruited to validate the instrument?

17


Teacher supports the relationship

development to the parents through

verbal pronouncements

.007 .028 .017 .02

0 2.82 .12

Teachers asks parents for their point of view of the problem

.061 .049 .073 .060 .38 .55

Teacher picks out the resources of the parents

.000 .00 .010 .017 5.02 .04

Teacher works out and defines concrete goals

.005 .013 .002 .005 .77 .40

Teacher thinks about barriers which could occur with the realisation of the task at home

.002 .007 .005 .013 .62 .44

Teacher makes agreements for the further counselling process

.017 .025 .068 .073 5.78 .03

M Pre SD Pre M Post SD Post F(1,13) p

18

2/4


Teachers structures the information of the

parents, outlines them and repeat them with

his own words

.000 .000 .011 .016 6.92 .02

Teacher notices the emotional aspects in

the information of the parents and repeats

them in his own words

.000 .000 .005 .017 1.00 .34

Teacher sticks to facts in a critical

counselling situation²- - - - - -

Teacher communicates learning strategies

and help for their realization.000 .000 .031 .064 3.36 .09

M Pre SD Pre M Post SD PostF(1,13) p

²Not observed, no measures possible

3/4


Teacher gives a hint for the support of the

children with learning at home.062 .065 .106 .050 4.59 .05

Teacher talks about the behavior of the child

at school.122 .052 .120 .116 .00 .97

Teacher communicates through verbal

pronouncements that he is listening.010 .028 .026 .044 4.58 .05

Teacher not possible to categorize (Parents

taking/ silence / not possible to understand

what is spoken)

.262 .142 .170 .112 4.99 .04

Every other reason which can´t be categorized .453 .159 .357 .128 3.18 .10

M Pre SD Pre M Post SD PostF(1,13) p

4/4

Results

• To test the hypothesis ANOVAs with repeated measures were calculated

• As the counselling talks differed in length the measures of the categories were

relativized at the length of the total talk


Theoretical background: Behaviour Observation

• It is one of the basic methods in psychological research and provides objective

data

• It measures not only self-assessment but behavior

• Scientific observation is characterized through a standardized, controlled and

structured procedure

• A systematic behavior observation is characterized through its clear

regimentation


Theoretical background:Summary of the current situation

All in all little research in teacher competences in general – and especially

concerning counselling competence

Some further studies:

• Training programs for teachers in conversation techniques (Aich, 2006)

• Training programs for teachers in conflict management (Stadler, 2009)

• Assessment of professional conversation techniques (Gartmeier et al., 2011)


Bibliographical Reference

• Bruder, S., Keller, S., Klug, J. & Schmitz, B. (2011). Ein Vergleich situativer

Methoden zur Erfassung der Beratungskompetenz von Lehrkräften [A

comparison of situative methods for measuring teachers´ counselling

competence]. Unterrichtswissenschaft, 39 (2) [Instructional Science], 121-135.

•

• Cohen, J. (1968). Weighted Kappa. Nominale scale agreement with provision

for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220.

• Cox, D. D. (2005). Evidence based interventions using home school

collaboration. School Psychology Quarterly, 20 (4), 473-497.



• Eccles, J. S. & Harold, R. D. (1996). Family involvement in children´s and

adolescents´ schooling. In A. Booth & J .F. Dunn (Eds.), Family-school links: How

do they affect educational outcomes? (pp. 3 – 34). Mahwah: Lawrence Erlbaum.

• Epstein, J. L. & van Voorhis, F. L. (2001). More than minutes: Teachers´ roles in

designing homework. Educational Psychologist, 36, 181-193.

• Epstein, J. (1991). Effects on Student Achievement of Teachers´ Practices of

Parent Involvement. In S. B. Silvern & B. A. Hutson (Eds.), Advances in Reading

- Language Research (Vol. 5, pp. 261-276). Greenwich, Conneticut: Jai Press

Inc.



• Gartmeier, M., Bauer, J., Fischer, M. R., Karsten, G. & Prenzel, M. (2011).

Modellierung und Assessment professioneller Gesprächsführungskompetenz

von Lehrpersonen im Lehrer-Elterngespräch. In: O. Zlatikin-Troitschanskaia

(Hrsg.), Stationen Empirischer Bildungsforschung. Traditionslinien und

Perspektiven (S. 412-426). Wiesbaden: VS.

• Guli, L. A. (2005). Evidence-based parent consultation with school-related

outcomes. School Psychology Quarterly, 20, 455-472.



• Hertel, S. (2009). Beratungskompetenz von Lehrern. Kompetenzdiagnostik,

Kompetenzförderung und Kompetenzmodellierung [Teachers’ counseling

competence. Assessment, fostering, and modeling of counseling competence].

Münster: Waxmann.

• Hoover-Dempsey, K. V., Walker, J. M. T., Jones, K. P. & Reed, R. P. (2002).

Teacher involving parents (TIP): Results of an in-service teacher education

program for enhancing parental involvement. Teaching and Teacher Education,

18, 843-867



• Kultusministerkonferenz (KMK). (2004). Standards für die Lehrerbildung:

Bildungswissenschaften. Beschluss der Kultusministerkonferenz vom

16.12.2004. Retrieved from

http://www.kmk.org/doc/beschl/standards_lehrerbildung.pdf

• Kunter, M. & Klusmann, U. (2010). Kompetenzmessung bei Lehrkräften –

Methodische Herausforderungen [Assessment of competence in teachers –

Methodological challenges]. Unterrichtswissenschaften, 38 (1) [Instructional

Science], 68-86.



• Kunter, M., Klusmann, U., Dubberke, T., Baumert, J., Blum, W., Brunner, M.,

Jordan, A., Krauss, S., Löwen, K., Neubrand, M., & Tsai, Y.-M. (2007). Linking

aspects of teacher competence to their instruction: Results from the COACTIV

project. In M. Prenzel (Ed.), Studies on the educational quality of schools: The

final report on the DFG priority programme (pp. 39-59). Münster: Waxmann.

• McLaughlin, C. (1999). Counselling in school. Looking back and looking

forward. British Journal of Guidance and Counselling 27 (1), 13-23.



• Schnebel, S. (2007). Professionell Beraten: Beratungskompetenz in der

Schule [Counseling professionally: counseling competencies in school].

Weinheim: Beltz.

• West, J. F. & Cannon, G. S. (1988). Essential collaborative consultation

competencies for regular and special educators. Journal of Learning

Disabilities, 21 (1), 56–63.


Simone Bruder 1 , Silke Hertel 1 , Samuel Greiff 2 & Bernhard Schmitz 3

Documents

Transcript of Simone Bruder 1 , Silke Hertel 1 , Samuel Greiff 2 & Bernhard Schmitz 3