Research question
What is the evidence for the effectiveness ICT on literacy learning in English, 5-16?
Summary of results
Millions of pounds have been spent on ICT equipment, software and teacher training. This review focuses on the use of ICT in one area of the curriculum - literacy learning. A small number of tiny pilot studies identified in this review provide the basis upon which to justify or refute the potential effectiveness of ICT in this area. They show little high-quality evidence of ICT producing clear benefits in the area of literacy teaching, using narrow definitions of literacy. Large-scale high-quality research studies are urgently needed using broader definitions of literacy.
Background
This review is set against a background of growing international concern that the investment in ICT in schools is not in fact having a positive impact on literacy development, despite the widely held belief that ICT is beneficial to learning in general and literacy learning in particular. Since the mid-1980s, successive governments have invested heavily in the development of ICT in schools. However, there is a lack of secure research-based evidence of its effectiveness in literacy or numeracy. So far research in the field of literacy has produced rather patchy results and there is, therefore, a clear need for a thorough review of recent ICT effectiveness research on all aspects of literacy. Specifically, no systematic review of effectiveness has reviewed studies in all aspects of literacy learning using the most appropriate study design - the randomised controlled trial (RCT).
Methods
The structure of this review involved a two-stage 'mapping' process plus an in-depth review. The earlier systematic literature review mapped the research on the impact of literacy from age 5-16 for the years 1990-2001. For this review, the search was updated to include 2001-2002 and inclusion/exclusion criteria were refined on the basis of effectiveness. Therefore only papers using relevant RCTs were included - i.e. they had randomly allocated pupils to either an ICT or no-ICT treatment for the teaching of literacy. Effectiveness was calculated by checking test and mean gain scores, and studies that did not present these findings, or those in which data had been poorly analysed, were excluded from the review. Studies previously identified in the earlier systematic review were re-screened for possible inclusion in the in-depth effectiveness review. The review team assessed the research methods of each targeted trial and only those seen as 'medium' or 'high' on overall weight of evidence were included. The subsequent narrative synthesis of the trials focused on both the interventions and outcomes described.
Results
2,319 potentially relevant reports were identified for the current review, and over 80 percent were immediately excluded by screening titles and/or abstracts. After reading the full reports, 212 were deemed to meet the criteria for inclusion - less than 10 percent of the original total. Forty-two RCTs were included in the effectiveness map, only three of which were undertaken in the UK - the rest were from the USA. Once these studies were checked against the established exclusion criteria, only 12 remained.
A range of five different kinds of ICT interventions emerged:
- computer-assisted instruction (CAI)
- networked computer system (classroom intranet)
- word-processing software packages
- computer-mediated texts (electronic text)
- speech synthesis systems.
The three literacy outcomes identified were (1) reading (including pre-reading understandings), (2) writing and (3) spelling.
Overall, when comparisons of the five different ICT interventions were analysed, there was little evidence to support the widespread use of ICT in literacy learning in English. In the three principal meta-analyses undertaken, two (reading and spelling) showed no evidence of either benefit or harm, while some weak evidence for a positive impact upon writing was detected.
Publication bias is a key issue for any systematic review. While negative studies have been included, the fact that unpublished studies tend to have negative results may mean that the results are in fact overly optimistic.
Implications
Specifically in the field of literacy learning, policy-makers should avoid investing further in ICT and literacy until there is clear evidence of its effectiveness. Teachers should be aware that there is no evidence that non-ICT methods of instruction and non-ICT resources are inferior to the use of ICT in literacy learning. One interpretation of this review's data is that its results may be over-optimistic due to publication bias. Given this possibility, it seems clear that a series of large-scale, rigorously designed tests to evaluate ICT and literacy learning across all age ranges is urgently required.
The writer is an experienced teacher of English Language and Literature at secondary level and a part-time PGCE tutor in the Department of Educational Studies at the University of York. She is also a senior examiner, moderator and coursework adviser for the Assessment and Qualifications Alliance examination board. The writer is not a member of the Review Group nor an adviser for the Review, and is writing in a personal capacity.