Post on 02-Feb-2016
description
Simone Bruder1, Silke Hertel1, Samuel Greiff2 &
Bernhard Schmitz3
1 German Institute for International Educational Research, Frankfurt am Main2 Heidelberg University, Germany3 Technical University Darmstadt, Germany
Observing Teachers` Counselling Competence – Evaluation of a Teacher Training Program with a Behavior Observation Instrument
Theoretical background: Relevance of the research
• Research on teacher competencies strengthened only recently since the research
focus lies mostly on students (Beck & Zlatkin-Troischskaia, 2010; Kunter & Klusmann, 2010)
• Most research on teachers in the core area of teaching (Herzog & Makarova, 2011)
• Counselling in addition to teaching, educating & assessing is a central task of
teachers (KMK, 2004)
• Teachers are the most important factor in providing counselling in the school
system (Landesinstitut für Schule und Weiterbildung des Landes Nordrhein-Westfalen, 1998)
• Counselling competency in models for teacher performance (Baumert & Kunter, 2006;
Rambow & Bromme, 2000)
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 2
Theoretical background: Current situation
• Increase of situations in which counselling concerning learning strategies is
needed (Schnebel, 2007)
• Intensive cooperation between parents and school is positive for school
achievement and social behaviour (Cox, 2005; Hoover-Dempsey & Sandler, 1997)
• Teachers are often not well prepared for counselling tasks: often feel stressed
when counselling parents (Hitzinger, 1987; Hertel, 2009)
• Education in counselling prerequisite for working successfully together with
parents (Krumm, 1996; Wild, 2003)
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 3
Theoretical framework – Basic of this study
What has been done in research until now?
• It is important to specify aspects of counselling that are of particular relevance for
teachers
Bruder (2011) developed a 4-dimensional model of counselling
competence for teachers when counselling parents in learning
strategies
• Although being included in German teacher education, counselling is still not
taught as much as needed (KMK, 2004)
Hertel developed a teacher training program to foster teachers´
counselling competence (Hertel, 2009)
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 4
Theoretical background: Model of Counselling Competence (Bruder, 2011)
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Counselling skillsDiagnostic/ Pedagogical Knowledge
Collaboration/ Perspective taking
Coping
Structuring Searching for
reasonsCooperative actions
Coping with criticism
Empathetic listeningDefining the
problemPerspective taking
Dealing with difficult situations
Paraphrasing Strategy knowledgeResources/Solution
orientation
Goal orientation
Results of confirmatory factor analysis: X2 = 56,16 / df = 45 CFI = .96, RMSEA = .03 / SRMR = .39
5
Based on: West & Cannon (1988); Reid (1990); McLaughlin (1999); Strasser & Gruber (2003); McLeod (2003); Schwarzer & Buchwald (2006); Hertel (2009)
Theoretical background: Counselling Competence Training Program (Hertel, 2009)
• 4 sessions of 210 min
• comprehensive in terms of competence dimensions
• covering aspects from communication theory to theory of self-regulated
learning
• included role-plays to simulate counselling talks
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 6
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 7
Session Issue Topics
1 Counselling skills Communication theory, Empathetic
Listening, Paraphrasing, Structure
of Counselling Talks
2 Collaboration/ Perspective taking Factors causing problems with
learning, cooperation of parents and
teachers
3 Diagnostic/ Pedagogical Knowledge
Theory of self-regulated learning,
strategies for parents to support
child’s learning
4 Coping Coping with Criticism, Meta-
Communication, Reflection of
Counselling Talks
Theoretical background: Counselling Competence Training Program (Hertel, 2009)
Theoretical background: How Measuring Counselling Competence?
Teachers’ Counselling competence
Paper & Pencil Behavior Observation
Knowledge Test
Work probe
SelfRating
Participating Observer
Non-particip.Observer
(Hertel, 2009) EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Behaviour observation
• The instrument was developed based on the model and consisted of 15 categories
• 12 categories refer to the competence scales / 3 categories capture all other behaviour
during the talk
• Raters were provided with detailed information on how to score the categories
• One rating interval takes 20 seconds
• Interrater-reliabilities for rating intervals of 20 seconds showed very good results: Cohen’s
Kappa (Cohen, 1968) κ > .80 for all categories
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
9
Research Aim and Questions
Aim:
• To observe the counselling competence of the teachers taking part in the
teacher training program and to evaluate the program
• To develop and test a behaviour observation instrument with a categorical
system based on the model
Research Question:
• Do the participants of the study improve in the four dimensions of the model (to
which the categories belong)?
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 10
Method
• Pre-post design with an experimental and a control group (Data for the behaviour
observations only available for the exp. group)
• Role-plays were audio-taped at the beginning of the first unit (pre-test) / end of
the fourth unit (post-test)
• Role-plays were recorded and analysed with the instrument
• Training program consisted of four units (each 3.5 hours)
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 11
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Variables
Sample of the Training program
Sample of teachers with pre- and post behavior observations
N 81 14
Gender
Male 25 (31%) 6 (43%)
Female 37 (46%) 5 (36%)
No Information 19 (23%) 3 (21%)
Age
< 31 years 5 (6%) 1 (7%)
31 – 40 years 16 (20%) 4 (29%)
41 - 50 years 16 (20%) 1 (7%)
51 - 60 years 25 (31%) 5 (36%)
> 60 years 0 0
No information 20 (23%) 3 (21%)
Job experienceM (SD)
15.58 (10.3) 12 (9.22)
Descriptive Data for the samples
12
Results of ANOVA with repeated measuresfor the four dimensions
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 13
M Pre SD Pre M Post SD Post F(1,13) p
Counselling Skills .009 .028 .042 .051 12.66 .04
Diagnostic/Pedagogical Knowledge .085 .078 .211 .12 19.98 .00
Collaboration/Perspective Taking .069 .063 .101 .068 .222 .16
Coping² - - - - - -
²Not observed, no measures possible
Discussion
• The goal of the present study: (1) to develop an instrument to observe teachers
counselling competence before and after (2) a training program
• The observation instrument fills the gap of research that arrogates instruments to
measure teacher competences related to behavior (Kunter & Klusmann, 2010)
• Hypothesis: the participants would improve in 12 categories and the 4
dimensions of counselling
results showed that the instrument was able to indicate changes in
counselling competence
The instrument shows good interrater-reliability
Data from the pre- and post-role-plays showed significant increase in a
substantial number of categories -> promising result
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 14
Discussion
• The study shows that teachers can be supported in increasing their counselling
competence through training programs
• The information of the behavior observation can be used to optimize the training
program in the categories in which no improvements could be observed
• Limits of the study:
• Sample size decreased considerably from pre-test to post-test
• No control group design
• The instrument should be validated with other instrument on a bigger
sample
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 15
Thank you for
your attention!
bruder@dipf.de
16 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Questions to be discussed
1) How important is counselling competence for teachers?
2) How can counselling competence be more implemented into
teacher education?
3) How can the loss in data from pre- to post-test in the observation
instrument be reduced? (teacher often are afraid of giving such
sensitive data)
4) How can a larger sample be recruited to validate the instrument?
17
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Teacher supports the relationship
development to the parents through
verbal pronouncements
.007 .028 .017 .02
0 2.82 .12
Teachers asks parents for their point of view of the problem
.061 .049 .073 .060 .38 .55
Teacher picks out the resources of the parents
.000 .00 .010 .017 5.02 .04
Teacher works out and defines concrete goals
.005 .013 .002 .005 .77 .40
Teacher thinks about barriers which could occur with the realisation of the task at home
.002 .007 .005 .013 .62 .44
Teacher makes agreements for the further counselling process
.017 .025 .068 .073 5.78 .03
M Pre SD Pre M Post SD Post F(1,13) p
18
2/4
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 19
Teachers structures the information of the
parents, outlines them and repeat them with
his own words
.000 .000 .011 .016 6.92 .02
Teacher notices the emotional aspects in
the information of the parents and repeats
them in his own words
.000 .000 .005 .017 1.00 .34
Teacher sticks to facts in a critical
counselling situation²- - - - - -
Teacher communicates learning strategies
and help for their realization.000 .000 .031 .064 3.36 .09
M Pre SD Pre M Post SD PostF(1,13) p
²Not observed, no measures possible
3/4
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 20
Teacher gives a hint for the support of the
children with learning at home.062 .065 .106 .050 4.59 .05
Teacher talks about the behavior of the child
at school.122 .052 .120 .116 .00 .97
Teacher communicates through verbal
pronouncements that he is listening.010 .028 .026 .044 4.58 .05
Teacher not possible to categorize (Parents
taking/ silence / not possible to understand
what is spoken)
.262 .142 .170 .112 4.99 .04
Every other reason which can´t be categorized .453 .159 .357 .128 3.18 .10
M Pre SD Pre M Post SD PostF(1,13) p
4/4
Results
• To test the hypothesis ANOVAs with repeated measures were calculated
• As the counselling talks differed in length the measures of the categories were
relativized at the length of the total talk
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 21
Theoretical background: Behaviour Observation
• It is one of the basic methods in psychological research and provides objective
data
• It measures not only self-assessment but behavior
• Scientific observation is characterized through a standardized, controlled and
structured procedure
• A systematic behavior observation is characterized through its clear
regimentation
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 22
Theoretical background:Summary of the current situation
All in all little research in teacher competences in general – and especially
concerning counselling competence
Some further studies:
• Training programs for teachers in conversation techniques (Aich, 2006)
• Training programs for teachers in conflict management (Stadler, 2009)
• Assessment of professional conversation techniques (Gartmeier et al., 2011)
EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz 23
Bibliographical Reference
• Bruder, S., Keller, S., Klug, J. & Schmitz, B. (2011). Ein Vergleich situativer
Methoden zur Erfassung der Beratungskompetenz von Lehrkräften [A
comparison of situative methods for measuring teachers´ counselling
competence]. Unterrichtswissenschaft, 39 (2) [Instructional Science], 121-135.
•
• Cohen, J. (1968). Weighted Kappa. Nominale scale agreement with provision
for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220.
• Cox, D. D. (2005). Evidence based interventions using home school
collaboration. School Psychology Quarterly, 20 (4), 473-497.
24 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Bibliographical Reference
• Eccles, J. S. & Harold, R. D. (1996). Family involvement in children´s and
adolescents´ schooling. In A. Booth & J .F. Dunn (Eds.), Family-school links: How
do they affect educational outcomes? (pp. 3 – 34). Mahwah: Lawrence Erlbaum.
• Epstein, J. L. & van Voorhis, F. L. (2001). More than minutes: Teachers´ roles in
designing homework. Educational Psychologist, 36, 181-193.
• Epstein, J. (1991). Effects on Student Achievement of Teachers´ Practices of
Parent Involvement. In S. B. Silvern & B. A. Hutson (Eds.), Advances in Reading
- Language Research (Vol. 5, pp. 261-276). Greenwich, Conneticut: Jai Press
Inc.
25 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Bibliographical Reference
• Gartmeier, M., Bauer, J., Fischer, M. R., Karsten, G. & Prenzel, M. (2011).
Modellierung und Assessment professioneller Gesprächsführungskompetenz
von Lehrpersonen im Lehrer-Elterngespräch. In: O. Zlatikin-Troitschanskaia
(Hrsg.), Stationen Empirischer Bildungsforschung. Traditionslinien und
Perspektiven (S. 412-426). Wiesbaden: VS.
• Guli, L. A. (2005). Evidence-based parent consultation with school-related
outcomes. School Psychology Quarterly, 20, 455-472.
26 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Bibliographical Reference
• Hertel, S. (2009). Beratungskompetenz von Lehrern. Kompetenzdiagnostik,
Kompetenzförderung und Kompetenzmodellierung [Teachers’ counseling
competence. Assessment, fostering, and modeling of counseling competence].
Münster: Waxmann.
• Hoover-Dempsey, K. V., Walker, J. M. T., Jones, K. P. & Reed, R. P. (2002).
Teacher involving parents (TIP): Results of an in-service teacher education
program for enhancing parental involvement. Teaching and Teacher Education,
18, 843-867
27 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Bibliographical Reference
• Kultusministerkonferenz (KMK). (2004). Standards für die Lehrerbildung:
Bildungswissenschaften. Beschluss der Kultusministerkonferenz vom
16.12.2004. Retrieved from
http://www.kmk.org/doc/beschl/standards_lehrerbildung.pdf
• Kunter, M. & Klusmann, U. (2010). Kompetenzmessung bei Lehrkräften –
Methodische Herausforderungen [Assessment of competence in teachers –
Methodological challenges]. Unterrichtswissenschaften, 38 (1) [Instructional
Science], 68-86.
28 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Bibliographical Reference
• Kunter, M., Klusmann, U., Dubberke, T., Baumert, J., Blum, W., Brunner, M.,
Jordan, A., Krauss, S., Löwen, K., Neubrand, M., & Tsai, Y.-M. (2007). Linking
aspects of teacher competence to their instruction: Results from the COACTIV
project. In M. Prenzel (Ed.), Studies on the educational quality of schools: The
final report on the DFG priority programme (pp. 39-59). Münster: Waxmann.
• McLaughlin, C. (1999). Counselling in school. Looking back and looking
forward. British Journal of Guidance and Counselling 27 (1), 13-23.
29 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz
Bibliographical Reference
• Schnebel, S. (2007). Professionell Beraten: Beratungskompetenz in der
Schule [Counseling professionally: counseling competencies in school].
Weinheim: Beltz.
• West, J. F. & Cannon, G. S. (1988). Essential collaborative consultation
competencies for regular and special educators. Journal of Learning
Disabilities, 21 (1), 56–63.
30 EARLI 2012| Bergen | Simone Bruder, Silke Hertel, Samuel Greiff & Bernhard Schmitz