What do we want to know?

Assessment by teachers has the potential for providing summative information since they can build up a picture of students' attainments over the full range of activities and goals.  However, teachers' judgements are sometimes seen as unreliable and subject to bias.  This review aims to provide research evidence on the reliability and validity of summative assessment by teachers and the conditions that affect these qualities.

Who wants to know?

Policy-makers, those involved in teacher education and professional development, employers, parents and pupils.

What did we find?  

• Fine specification of criteria, describing progressive levels of competency, is required for reliable teacher assessment. The clearer teachers are about the goals of students' work, the more consistently they apply the assessment criteria.   Teachers who have participated in developing criteria are able to use them reliably.
• Teachers' judgements of students' performance are likely to be more accurate in aspects more thoroughly covered in their teaching.  They can predict with some accuracy their students' success on specific test items and on examinations (for 16-year-olds), given specimen questions. There is less accuracy in predicting 'A' level grades (for 18-year-olds).
• Without training in assessment,  teachers' judgements can be biased by behaviour (for young children), gender and special educational needs; students' overall academic achievement and verbal ability may influence judgement when assessing specific skills.
• Moderation through professional collaboration is of benefit to teaching and learning as well as to assessment. Reliable assessment needs protected time for teachers to meet and to take advantage of the support that others, including assessment advisers, can give.

What are the implications?

• When deciding summative assessment methods, the shortcomings of external examinations should be borne in mind.  Teacher assessment should not be judged in terms of how well it agrees with test scores but by how well it reflects learning goals.
•  Clarity of learning goals is needed for dependable teacher assessment. Resources need to be put into identifying detailed criteria that are linked to learning goals, not specially devised assessment tasks. 
• Professional development and moderation are essential.  Teachers should be made aware of sources of potential bias, and school assessment procedures should guard against unfairness.
• The process of moderation should be seen as having value beyond refining assessment, in helping to develop teachers' understanding of learning goals.

How did we get these results?

Thirty studies were synthesised; all were written in English, and they were conducted mostly in England and the United States.  All were concerned with students between the ages of 4 and 18.

This report should be cited as: Harlen W (2004) A systematic review of the evidence of reliability and validity of assessment by teachers used for summative purposes. In: Research Evidence in Education Library. London: EPPI Centre, Social Science Research Unit, Institute of Education, University of London.