专家视点 | Chris Davison: 教师测评素养：我们如何了解需要提高什么？

Chris Davison 外研测评 2022-04-24

收录于合集 #专家视点 10个

教师测评素养对教学质量和教师职业能力发展至关重要。如何更有效地提升语言测评素养，是英语教师们都关心和关注的话题。“他山之石，可以攻玉”，本期专家视点为大家转载澳大利亚新南威尔士大学Chris Davison教授的文章《教师测评素养：我们如何了解需要提高什么？》。Davison教授以在线评估工具包（TEAL）的应用以及澳洲和其他一些国家和地区的教师测评素养培训项目为例，展示了教师测评素养提升的路径和方法，值得大家研究和学习。

Teacher Assessment Literacy:

How do We Know What We Need to Improve?

Introduction

Teacher assessment literacy is regarded as one of the most influential factors in improving student learning in the classroom, in particular a teacher’s ability to collect, interpret and use a range of assessment information to help students monitor and evaluate their learning needs, set achievable goals, and use targeted feedback from teachers and peers to improve their learning (Black & Wiliam, 1998; Hattie, 2008). This paper will first unpack some of the key concepts underpinning teacher assessment literacy and the development of an assessment for learning culture, building on the author’s work in China, Singapore and Brunei. The paper will then focus on a case study of one Australian collaborative approach to building teacher assessment literacy, the Tools to Enhance Assessment Literacy for Teachers of English as an Additional Language (TEAL) project, which is designed to help teachers of students with English as a Second or Additional Language (ESL/EAL) to use assessment tools and techniques more effectively so as to improve teaching and learning. The tools include four main components: first, a set of sequenced teacher professional learning resources about English language learners and assessment designed for small group or self-directed study; secondly, an assessment tool bank containing a range of assessment tools and tasks, including computer-adaptive tests, organized around the three broad macro-skills (oral, reading and writing), three macro-functions (informative, persuasive, imaginative), three stages of schooling (early elementary, mid to upper elementary, and lower secondary) and a range of English language proficiency levels; thirdly, a range of assessment-for-learning and teaching exemplars including a selection of annotated units of work across a range of subject areas and year levels showing assessment tasks with formative feedback embedded within a teaching/learning cycle, and finally, an online teacher discussion forum, including a password-protected area for teachers to share problems and strategies and to moderate work samples in order to build a community of assessment practice. The paper discusses the rationale for the selection of the resources for teacher assessment literacy in English language education and their potential to make a difference to teachers and students. The implications in terms of the process of defining and describing teacher assessment literacy for other systems and settings will also be discussed.

What is teacher assessment literacy

(and why is it important)?

Teacher assessment literacy is regarded as one of the most influential factors in improving student learning (Black & Wiliam, 1998; Hattie, 2008), however there is no common definition of assessment literacy among researchers and policy-makers. Some see it as data literacy, in particular, the ability to interpret results of standardized tests; others talk about it as the possession of knowledge about sound assessment principles and practices, for example, as in New Zealand schools (Ministry of Education, n.d.):

“Assessment literacy is the possession of knowledge about the basic principles of sound assessment practice, including its terminology, the development and use of assessment methodologies and techniques, and familiarity with standards of quality in assessment.”

Why is assessment literacy seen as the “new” Holy Grail, or as Popham (2009) ironically describes it, a bona fide “magic bullet” for education? Much research evidence suggests assessment is the key to improvement in learning. Black and Wiliam (1998) have convincingly demonstrated the learning gains that can be achieved through well focused teacher-based formative assessment enhance students’ learning more than any other strategy across age levels and in different contexts. Hattie (2008) in a study of major influences on educational achievement (using more than 800 meta-analyses) found that formative practice, in particular self-assessment and feedback, had the highest effect sizes (i.e., impact on student outcomes) out of more than 100 different instructional and contextual factors. For these reasons worldwide there has been a push to improve teachers’ assessment literacy, as well as that of other stakeholders, in part due to concerns that teachers may lack sufficient training in what educational assessment entails and/or may lack the confidence or skills required to perform their assessment duties competently. As Popham (2009) says, “…assessment-literate teachers will typically make better decisions … because we want students to be better taught. It should be obvious that today’s teachers must acquire more assessment literacy.”

This raises the question of what kind of “assessment” we want teachers to be literate in? Perhaps not surprisingly, given the traditional and high-stakes role of assessment in educational systems, researchers do not agree on how assessment literacy should be defined nor what it might comprise. Although many have attempted to describe this construct (De Luca & Klinger, 2010; Malone, 2013; Mertler, 2009; Plake, 1993; Popham, 2009, 2011; Siegal & Wissehr, 2011; Volante & Fazio, 2007; Xu & Brown, 2016), Stiggins (1991) convincingly argues that the meaning of assessment literacy varies due to “the needs of the decision maker and the practical realities of the decision context”. This implies that each educational system needs to develop its own definitions and descriptions of teacher assessment literacy to suit its own particular assessment context, as there is so much variation in what constitutes assessment knowledge and skills (for example, see Black & Wiliam, 2005; Davison 2004, 2007). Only then will we know what needs to improve.

This is the case for the Australian school context, in which teacher-based assessment accounts for most evaluation that takes place. In terms of its assessment demands and practices, Australia is very different from other contexts, as it tends to sit in the middle when comparisons are made between it and other countries, both in terms of its performance on international assessments; and in terms of its assessment system which is balanced between classroom level assessment and standardised assessment, with a strong focus on classroom assessment, and a long history of prioritizing assessment for learning over assessment of learning. In assessment for learning cultures, research suggests that teachers spend from one-quarter to one-third of their professional time on assessment-related activities, so assessment literacy needs to focus more on a teacher’s ability to collect, interpret and use a range of assessment information to monitor and evaluate learning needs, provide targeted feedback and help students set achievable goals than in interpreting and using the results of standardized tests, though that is still important.

For example, in the largest and most diverse state educational system in Australia, New South Wales (NSW) (NSW Education Standards Authority, n.d.), the syllabuses advocate assessment for learning:

“…this is a type of quality assessment that has had world wide success in enhancing teaching and improving student learning. Assessment for learning gives students opportunities to produce work that leads to development of their knowledge, understanding and skills. Teachers decide how and when to assess student achievement, as they plan the work students will do, using a range of appropriate assessment strategies including self-assessment and peer assessment.”

Hence, assessment for learning emphasizes the interactions between learning and manageable assessment strategies that promote learning, clearly expresses for the student and teacher the goals of the learning activity, and reflects a view of learning in which assessment helps students learn better, not just achieve a better mark. It provides ways for students to use feedback from assessment, helps students take responsibility for their own learning, and is inclusive of all learners.

Research shows that NSW teachers are more likely to use formative assessment (including feedback to students) than the average of OECD teachers and teachers are encouraged to use a variety of assessment techniques that are valid, reliable and appropriate to the age and stage of learning, although Australia still has some way to go to ensure that teachers understand how to interpret and understand assessment data and effectively embed assessment within teaching and learning. A 2013 Staff in Australia’s Schools survey reported that 25.7% of primary teachers identified the need for more professional learning in making effective use of student assessment information. The findings were similar for secondary teachers. In Australia, concerns have been raised about the trustworthiness and reliability of teacher assessment decision-making processes and teachers’ ability to be both “accurate” and “fair”; and also about teachers’ capacity to be able to collect and use appropriate information to improve learning. However, they are not mutually exclusive—both are important for effective assessment, with the teacher equally, if not more, accountable to individual learners, not just systems, and systems highly motivated to improve learning, not just rank learners.

Teacher assessment literacy in English language education

Paradoxically, the concept of teacher assessment literacy has only relatively recently been widely discussed and promoted in the English language education field (for example, see Davison, 2017, 2019; Inbar-Lourie, 2008; Lam, 2015; Scarino, 2013; Taylor, 2009; Tsagari, 2011; Tsagari & Vogt, 2017; Xu, 2019). This is partly because of the traditional dominance in the field of large-scale standardised externally set and assessed tests, but also because assessing ESL/EAL learners is a particularly challenging area for most teachers.

In Australia, the primary focus in English language education until relatively recently has been on developing more accurate, consistent and transparent descriptions of EAL development to improve reporting systems, especially definitions of the target groups for funding purposes. Much less attention has been paid to improving teacher assessment literacy at all levels, despite the growing emphasis on benchmarking student performance against standardised assessment outcomes which has created particular difficulties for teachers working with learners from diverse linguistic and cultural backgrounds generally, especially those teachers with a variable level of language knowledge and assessment literacy. In Australian schools, about 30% of students require systematic and regular English language support, hence increasingly all teachers—not just EAL teachers—need access to appropriate and useful assessment tools and advice to enhance assessment literacy in order to support learning and teaching.

Enhancing teacher assessment literacy: One approach to improving teacher knowledge and skills in Australia

To respond to this need, researchers at the School of Education at the University of New South Wales, in partnership with the Victorian education system and Educational Assessment Australia (EAA), developed an innovative online assessment “tool-kit” to help all teachers develop pedagogically sound approaches to assessing the English language development of all students. Drawing on Assessment for Learning (AfL) principles and Vygotskian theory, the Tools to Enhance Assessment Literacy for Teachers of English as an Additional Language (TEAL) is for use by all Victorian school teachers to help assess the stage of development for EAL students in speaking and listening, reading and writing, to improve learning and teaching. All tools are aligned against the Victorian EAL curriculum, with potential for alignment to other standards by other jurisdictions.

Following a practice-what-you-preach model (Hill et al, 2014), TEAL was developed by adopting the same principles and promoting the same practices for teachers that needed to be implemented with students in Australian schools, that is:

by finding out initially where teachers were in terms of their assessment literacy;
sharing learning intentions, setting clear and coherent success criteria and achievable timelines, modelling desired outcomes and leading—by example—to sustainable improvements in assessment, learning and teaching;
putting the learners and teachers at the center of the change process;
involving students, parents and the wider school community in understanding and supporting reforms;
maintaining confidence in the assessment system, and striving to be theoretically and philosophically consistent.

At the beginning of the project development, over 150 experienced EAL teachers were surveyed to gather their own perceptions of their assessment literacy, using a suitably contextualised description of teacher assessment literacy developed by Alonzo (2016), in collaboration with this author, which prioritized collaboration and student engagement, that is:

“Teacher assessment for learning literacy (comprises the) knowledge and skills in making highly contextualised, fair, consistent and trustworthy assessment decisions to inform learning and teaching to effectively support both student and teacher learning. Teachers aim to build students’ and other stakeholders’ (such as parents and school leaders) capabilities and confidence to take an active role in assessment, learning and teaching activities to enable and provide the needed support for more effective learning.”

The survey showed that the EAL teachers participated in professional development related to assessment and many were undertaking further education/ training in assessment. The majority demonstrated a belief in the ability of every student to improve, and felt confident they could identify appropriate teaching methods and tailor lessons to available resources, maintain confidentiality in dealing with assessment results, develop an environment of trust, use assessment to build students’ interest to learn, and reinforce the positive learning attitude of students. However, somewhat unexpectedly the same group of teachers indicated very low self-efficacy in a number of key components of assessment for learning, with a confidence level below 60% for the following very common assessment activities:

gathers a range of evidence of student learning, 2.36;
designs English language assessment tasks, 2.45;
engages students in peer-assessment, 2.45;
engages students in self-assessment/ reflection, 2.68;
involves students in the development of learning outcomes, 2.87;
gives feedback related to criteria, 2.87;
assists students in using feedback to feed forward , 2.98;
collaborates with family to establish home activities to support students, 2.98.

This survey informed the development of tailored assessment tools and advice through a process of researcher-teacher collaboration involving the active input and training of hundreds of EAL specialists from selected government, Catholic and independent schools in Victoria and over 10,000 EAL students and their families. The action-research included the collaborative collection of detailed video and documentary records of over 1500 exemplars of student language use in authentic primary and secondary classroom settings, and the trialling and validation of over 60 different oral and written assessment tasks, and associated rubrics, for the assessment tools bank, consisting of a range of assessment tools and tasks organized around macro-skills cross-referenced by assessment type, EAL stages and year levels, as well as the development, trialling and validation of an innovative computer-adaptive test for vocabulary and reading assessment.

However, in developing the online teacher-mediated ESL assessment resource center, priority was also given to the collection and provision of teacher professional learning resources to enhance teacher assessment literacy, including background material on assessment for learning principles and processes, and video and text-based resources to showcase various aspects of assessment literacy. In addition, the site included a selection of annotated units of work across a range of subject areas and year levels, showing assessment tasks with self and peer assessment and formative feedback embedded within a teaching/learning cycle, and, most importantly, an online teacher discussion forum—a password-protected area for teachers to share problems, strategies and work samples and engage in moderation/benchmarking. Nearly five years later, this has resulted in a sustainable teacher-based assessment system aligned with the EAL curriculum to provide longitudinal data and reports to all key stakeholders on students’ English language and literacy development over time, but at the same time innovative and dynamic scaffolding for the continuing development of teacher assessment literacy. The underpinning Vygotskian framework of teacher assessment literacy as activity system and scaffold is described by Michell and Davison (2019) .

In the final stages of the project implementation, a pilot professional learning program was developed to evaluate and enhance the capacity of schools to use the TEAL website to improve their learning and teaching of EAL students, and assess and report on their progress. It also functioned as a field trial to refine and improve the TEAL website, and to track any changes in teacher assessment literacy over time and identify areas requiring improvement. From February to November 2016 there were six rounds of professional learning with 10 groups of three teachers from each school (EALD, content-area, leader), funded by the Department of Education and Training (DET) Victoria, and taught by the University of New South Wales (UNSW), consisting of 6 x 3-hour or 3 x 6-hour modules, focusing on:

becoming familiar with TEAL, its rationale and aims, in particular, developing an understanding of the philosophy of assessment for learning and giving appropriate feedback to students;
developing a clear understanding of the materials and assessment advice on the website;
reflecting on how to incorporate TEAL materials into the assessment of EAL learners;
having opportunities to give feedback on the TEAL website.

All sessions incorporated time for professional dialogue and for sharing of strategies to enable school-based implementation, plus action-oriented activities in the form of between-session tasks, such as trialling and evaluating TEAL common oral and written assessment tasks, experimenting with feedback or self and peer assessment, as well as readings and online reflections. Overall, a total of 182 teachers enrolled in the professional learning programs— a mix of primary and secondary government, Catholic and Independent schools, along with some teachers from P-12 Colleges and intensive English language centers.

Teachers were asked to complete a pre/post program survey which showed that over the course of the program all teachers gained a greater understanding of TEAL as well as feeling more ready and confident to implement it in their schools. Comparison of pre/post program feedback showed a doubling of participants’ understanding of the TEAL website, assessment for learning principles and their use in their classroom as well as significant improvements in their confidence in improving teaching and learning for EAL learners in their school and their understanding of EAL learner needs as well as their knowledge and skills in providing EAL students with feedback and understanding of how to implement student peer and self reflection with EAL students.

In the post program survey, teachers identified the most useful aspects of the TEAL resources, including the actual assessment tasks, the criteria sheets, assessment samples, advice on feedback and self and peer assessment, the readings and resources, the links to the EAL Curriculum and the sample unit planning formats. Participants also provided feedback on the professional learning program, indicating that they gained most from:

1. Getting to know and use the TEAL website:

Spending time exploring the resource and implementing it into my program. If we’d merely been told about it, it would still be sitting untouched in a folder somewhere.
As a school leader, TEAL is useful for leading other staff and delivering general EAL advice and PD. The site has some great resources for this.
Great opportunity to try out a range of assessment tasks, criteria sheets, readings and resources.

2. Networking with colleagues and sharing ideas and expertise,eg:

Good to have all this professional learning with a large group of EAL teachers and learning from the presenters; Time to talk with others, share ideas and make connections.
The time to sit with colleagues and discuss assessment issues and structures within the school.
Ability to take this information back to inform whole-school planning going forward and to provide PD for teachers.

3. Acquiring knowledge about assessment for learning and how it empowers EAL learners and maximizes their learning, eg:

A wake up call about assessment for learning and not just data collection. I can do much better!
Promoting success for those who need it most.
Strategies and awareness of assistance to EAL students will benefit the entire cohort of students by catering for all needs.
The program has inspired me and helped me recognise the need to develop a whole school vision and implement less formal yet consistent and varied forms of assessment for our EAL learners.

Conclusion

TEAL is seen as an important resource for building teacher assessment literacy and professional learning in English language education, not just in Australia but internationally. It also provides an innovative mechanism for the process of defining and describing teacher assessment literacy and a model of what could be developed by other systems and settings wishing to develop their own unique support structures for the improvement of assessment literacy in their context.

注：本文内容基于作者在第三届英语教学与测评学术研讨会上的主旨发言。

往期精华

01 重磅干货

重磅干货 | Chris Davison：教师测评素养：我们如何了解需要提高什么？

重磅干货 | 罗少茜：素养教育背景下的语言测评：变与不变

重磅干货 | 刘建达：基于中国英语能力等级量表的外语教师测评素养发展

重磅干货 | 案例论坛：基于诊断性测评提升高中英语写作教学的案例研究

重磅干货 | 案例论坛：诊断性测评背景下思维导图运用于高中英语阅读教学实践

重磅干货 | 案例论坛：有效利用过程性评价, 以评促学

重磅干货 | 案例论坛：高中英语词汇教学中的测试任务设计

重磅干货 | 专题研讨: 基于核心素养的北京市初中英语学业标准的开发与应用

02 专家视点