PublicationsSystematic reviewsAssessment and ICTAssessment and ICT - teacher
A systematic review of the impact on students and teachers of the use of ICT for assessment of creative and critical thinking skills. User perspective: a teacher

Unlocking potential: computer technology taking assessment further

'How has modern computer technology been used to assess pupil's creative and critical thinking? Have these approaches been productive? Are there lessons to be learned, about unlocking the potential of computer technology for extending the range of educational outcomes that we can usefully assess?'

As a practising teacher, I was delighted to take the opportunity of participating with a team of leading educational experts reviewing research on these questions. This summary details what we found.

This report summarises the findings of a research review undertaken in response to three evolving features of the school curriculum:

assessment_fig1_2003[2].jpg

 

Teachers and policy-makers need to recognise the potential of assessing critical and creative thinking through information and communications technologies (ICT). By deploying computers in this way, school technology will not simply remain an expensive substitute for 'pen and paper'. Through developing reliable summative assessment of creative and critical thinking, these aspects of higher level thinking will become more valued within the curriculum. In promoting formative approaches to assessing creative and critical thinking, pupils can expect to learn with greater understanding and retention.

1. There has been rapid development in technology over recent years. Not only has there been an increase in technical sophistication, but there has also been a significant increase in access and deployment of facilities in schools. This has led to a variety of uses of ICT in the classroom: for learning and for assessment. Many of these approaches have replicated existing 'pen and paper' activities, but some have sought to exploit the particular affordances of computer technology, including:

a. tracking, recording and displaying the 'audit trail' of a learner's activity
b. simulating real-life scenarios for decision making and problem solving
c. storing, analysing and responding to visual, dynamic and non-linear data
d. assessing learning outcomes of tasks needing higher level thinking
e. providing high-speed processing and feedback, and providing learners with an 'external' memory.

2. These developments in technology have both created the need for education to provide students with what are described as 'higher level thinking skills' and the opportunity to teach and assess these skills. Although it is difficult to define higher level thinking skills without some dispute, it is accepted that 'creative thinking' and 'critical thinking' are essential elements. Assess these and you will be close to assessing higher level thinking: relating ideas in new and original ways (creative thinking), and evaluating statements with reasoning (critical thinking).

3. Recent research into effective assessment has served to illustrate that what is assessed for summative purposes (i.e. examined) becomes what is valued in the curriculum. In contrast, the same research demonstrates that it is the formative assessment of what is taught that can lead to improved learning.

So (a) if the development of higher level thinking matters, it makes sense to ensure that it is assessed; (b) assessment that offers feedback and the opportunity to refine and improve work is likely to foster better learning in this area; and (c) ICT offers the potential to implement the assessment of higher level thinking.

This reasoning prompted a systematic review of research evidence: to identify the impact of the use of ICT for assessment of creative and critical thinking skills on students and teachers. Any evidence of impact was subjected to further questions in order to establish:

a. how any impact might vary with the experience of the students and the conditions of the assessment
b. how any impact might vary according to whether the assessment is formative or summative
c. the implications for policy and practice.

The procedures involved searching as thoroughly as possible for relevant studies and then applying criteria to find those focusing directly on the question addressed in the review. In this way 103 studies were whittled down to 12. Between them these studies considered a number of aspects of computer-based assessment. The main findings emerge from one or more individual studies.

Claim

The use of ICT can help teachers by:

  • Storing and recording information about how students are developing understanding of new material.
  • Taking over some of the role of assessing and providing feedback to students so that the teacher can focus on ways of supporting learning that are beyond the scope of current programs.

Evidence

Research has shown that:

  • Feedback from the computer during the use of test material improved student performance in later use of the same test material.
  • Using a computer program both to test and give feedback to students increased the level of performance as compared with students taking the same tests on paper.
  • Using technology probed students' understanding to a greater degree than conventional tests.
  • The impact of human mediation may be increased by computer mediation due to the computer reinforcing the understanding developed with the teacher.
  • Studies showed that real life problems, requiring creative and critical thinking, can be presented and represented using ICT.
  • The use of computer programs can result in improved motivation for the assessment task, either through the use of the computer per se or through the feedback it provided.

Examples

  • Computer-based concept mapping with automated scoring can be used to provide summative assessment of critical and creative thinking about complex relationships.
  • Using a computer-based program for solving problems of balancing chemical equations enabled students to achieve a higher level of performance than solving equations on paper.
  • A conventional multiple choice test gave a false impression of understanding compared with the analysis of multimedia presentations on the same topic.
  • Use of a program involving diagrammatic representation provided useful information about the development of students' causal reasoning through analysis of their diagrams.
  • Computer-based assessment enabled students to achieve at a higher level than in equivalent paper-based tasks.
  • The visual representation afforded by the use of computer programs in several studies was suggested as supporting students meaning making.
  • The concrete form of abstract relationships provided by computer programs, particularly in concept/knowledge mapping, was considered to be a factor in improved performance.

Caution

  • There was evidence that the choice of content influences the performance of girls to a greater degree than boys.
  • Students' experience with computers and attitude towards them can influence computer-based test performance.

Advice

  • The use of any form of ICT in education should start from consideration of how it can be integrated into learning, teaching and assessment.
  • The route to effective use of ICT for learning, teaching and assessment in the classroom is through the teachers; and the development of their own knowledge and understanding is a precursor to the effective use of ICT.
  • There is need for professional development regarding the role that ICT can take in learning, teaching and assessing higher order thinking.
  • Teachers need to be clear about the relationship/distinction between the roles that ICT can take in learning and assessment.
  
Copyright 2019 Social Science Research Unit, UCL Institute of Education :: Privacy Statement :: Terms Of Use :: Site Map :: Login
About::Projects::Publications::Research Use::Courses & seminars::Resources::Databases::Blog