Fremdsprachenkompetenz – Der Schlüssel zur Tür nach Europa

Kloster Banz26. Oktober 2006

Language Testingand Teacher Training

—A Swedish Example

Gudrun EricksonGöteborg University, Sweden

Dept. of Education; Language and Literature Unit

gudrun.erickson@ped.gu.se

The Swedish School System

• 9-year compulsory school — Gesamtschule

• 98% continue to three-year upper secondary education (c. 75 % graduate) — Core subjects

• National curricula and syllabuses for subjects• Grading from year 8 — to be year 6…• Goal/criterion referenced grading system; four

grade levels — to be more…• Teachers responsible for grading• Grades used for selection for higher education• Extensive national testing programme

”Levels” in national test development process

Government

The Ministry of Education

The Swedish National Agency for Education

Different universities in the country

Göteborg University: Foreign languages

www.ped.gu.se/sol

Aims of national assessment—

[Strong] Advisory function

• Enhance educational achievement• Clarify goals; indicate strengths and

weaknesses in individual learner profiles• Concretize goals and grading criteria• Enhance equity and fairness in

assessment and grading• Provide statistics for local and national

analyses of educational achievement

Formative and summative materials

Foreign Languages in Swedish Schools

• English compulsory from primary level

• [Optional] Second FL in compulsory school; approx. 80 % start, 62 % complete their studies – German, Spanish, French

• English compulsory in upper secondary education — a core subject

• Second FL compulsory in more academically oriented programs in upper sec. (Fr, Ger, Sp + e.g. Italian, Russian, Chinese…)

National Test Materials of FL

English• Subject test for grade 5 (11-year-olds)• Diagnostic material for grades 6-9 (12-15-year-olds)

• Subject test for grade 9 (16-year-olds) / end of compulsory school

• Summative tests for two courses in upper sec. ed. (A + B)• Self-assessment material for course A• Illustrative materials for [advanced] productive skills (C)

French, German, Spanish• Summative materials for two courses in upper sec. ed.• Illustrative materials for productive skills• Under development: formative (diagnostic) + summative

materials for compulsory school

Focal areas in the national syllabuses of FL

Receptive competence

Productive competence

Interactive competence

Intercultural communicative competence

Reflective competence

7 stages of proficiency

The Swedish syllabus for FL and the CEFR:A tentative [textual] comparison

Stage Level (entrance levels)

1 A1 – A2

2 A2

3 A2 –B1

4 B1

5 B1 – B2

6 B2

7 B2 – C1

The typical national FL test Four subtests — Target language only (instructions, questions, answers)

• Oral production and interaction (paired or group oral)

• Reception: Listening comprehension

• Reception : Reading comprehension

• Written production (and interaction)

— — — — — — — — — — — — — — — —

• Culture in a wide sense, an integrated aspect of test content and design• Reflection, e.g. self assessment, focused upon in several materials • Results / Sub-scores presented in profiles• Model provided for aggregation into total score

Extensive guidelines for teachers. Benchmarked examples of oral and written production and interaction, incl. reasoning and comments based on goals, grading criteria and pre-defined analytic factors.

Basic principles for national test development

• Pedagogical considerations – Assessment of and for learning

• Enhancing validity and reliability, avoiding bias• Broad coverage of the construct – multi-faceted,

varied, monolingual tests• Giving students the opportunity to show what they

can do with their language, rather than focusing on what they cannot do

• Making what is most important assessable, not prioritizing what is easily measurable

Basic principles for test development (cont’d)

• Presenting individual results in profiles

• Focusing first on strengths, then on weaknesses

• Distinguishing between errors that [may] disturb,

and errors that destroy communication

**********• A collaborative approach to test development• Large-scale pretesting of all materials

• An eclectic approach to method (“Q +Q”)

Development Process

• Analyses of relevant literature and research• Continuous cooperation with ’broad’ expert groups• Small-scale piloting > Adjustments• Large-scale pretesting in randomly selected

groups of students in the country (n ≈ 400 per task)

• Analyses of results AND of students’ and teachers’ views and suggestions (“the q + q approach”)

• Compilation of tests• Standard setting and benchmarking in broad

groups of experts

Reactions to national tests• Students generally positive to varied, authentic

tasks – not too easy, not too difficult

• > 95 % of teachers positive to the national testsfunction, content, level of difficulty, guidelines, profiles, layout

• [Teacher] discussions aboutworkload; practical aspects of oral testing; standards; linguistic requirements

• [Political] discussions aboutrole of national tests (correlation with final grades); confidentiality; costs

Four Lines of Research

Language

Validation

Test-taker feedback

Dimensionality

National assessment in Sweden: Challenges

• Coping with multiple aims• Balancing implementation and prescription• Bridging the gap between formative and summative

assessment• Further developing methods of collaboration with different

stakeholders, e.g. students• Emphasizing further the pedagogical potential of

assessment for efficient and positive learning• Increasing assessment competence and confidence

among teachers• Analyzing further the relationship between the Swedish

syllabuses and the CEFR

Language (Subject) Teacher Educationin Sweden:Two models

I. Language studies, didactics and teaching practice integrated (4.5 – 5.5 years)

II. Language studies —

Didactics and teaching practice as a separate programme (1.5 years)

(often older students with work experience)

Subject teachers normally teach two subjects

Testing and Assessment in Teacher Training

Considerable variation between different universities

Strong criticism from the Swedish National Agency for Higher Education of too little emphasis on assessment and grading in teacher training programmes

Growing interest in assessment issues, in pre- and in-service training

Considerable increase of budget from the new government for teacher pre- and in-service training, including assessment and grading

Example of structurePre- and inservice training

• General lecture(s) on assessment and testing(construct/rubric; validity/reliability; formative/summative; “the joint

venture”; assessment of/for learning; washback; rating/grading, …)

• Literature seminars and workshops(Designing tasks for different constructs and different purposes; rating tasks — focusing on inter- and intra- rater reliability; national tests;

benchmarks/ standards; CEFR / ELP, …)

• Assignments during teaching practice / in practice

• Follow-up seminars based on practical examples

EALTA* Guidelines for Good Practice in Language Testing and Assessment

Three categories/“audiences”:

• Teacher pre- and in-service training• Classroom assessment

• Test development in national or institutionaltesting units or centres

Basic principles the same for all categories:Respect for students/examinees, responsibility, fairness, reliability, validity, collaboration among the parties involved

* European Association for Language Testing and Assessmentwww.ealta.eu.org

EALTA GuidelinesConsiderations for teacher pre-service and in-

service training in language testing and assessmentExamples

• How aware are trainees made of the range of assessment procedures appropriate to their present or future needs?

• What is the balance between theory and practice in the training?

• How far are the trainees involved in developing, trialling and evaluating assessment procedures?

• How far do assessment procedures used to evaluate the trainees follow the principles they have been taught?

EALTA GuidelinesConsiderations for classroom testing and

assessment Examples

ASSESSMENT PURPOSE(S) AND SPECIFICATION• What is the purpose of the assessment?• How does the assessment purpose relate to the curriculum?

ASSESSMENT PROCEDURES• Who designs the assessment procedures?• What account is taken of students’ views on the assessment

procedures?

CONSEQUENCES• What use is made of the results?

• What are the consequences of the results of the assessment for learners?

EALTA GuidelinesConsiderations for test development in national or

institutional testing units or centres Examples

• TEST PURPOSE AND SPECIFICATION

• TEST DESIGN and ITEM WRITING

• QUALITY CONTROL and TEST ANALYSES

• TEST ADMINISTRATION

• REVIEW

• WASHBACK

• LINKAGE TO THE COMMON EUROPEAN FRAMEWORK

European student 2005:

“I think that a good language test/ assessment should get students chance to show what they know and devolope their skills. It should contain of listening and reading comprehations, as well as writing and grammar part. But the most important thing is to speak in that language, so I think there should be also oral exams. Speaking is the most important, because without it, we wouldn’t communicate. Because of that students should also learn pronunciation.”

Matters to discuss

• Formative and summative assessment / classroom assessment and exams: similarities and differences?

• Influence from tests and testing on learning and teaching: positive, negative — inevitable…?

• Effective ways of supporting teachers in their different assessment activities and practices

• The role of students in testing and assessment

• “Making what is most important assessable, not prioritizing what is easily measurable” – effects and implications?

• The EALTA Guidelines for Good Practice in Language Testing and Assessment