Background
The last few years have seen an increase in research studies on the impact and effectiveness of information and communication technologies in the teaching and learning of English as a school subject. Andrews (2004) provides an overview of such studies and contains chapters on five systematic reviews of research published between 1990 and 2003 on the impact of information and communication technology (ICT) on literacy learning in English for 5- to 16-year-olds. It is against that research background, and recent developments in policy and practice in the UK, that the present systematic review of the effectiveness of different ICTs in the teaching and learning of English has been undertaken.
Aims
The aim of the present review is to shed light on whether ICTs are effective in the teaching and learning of English for 5- to 16-year-olds.
Review questions
The review is in two parts. The first asks what research has been published since 1998 on the topic of effectiveness of different ICTs in the teaching and learning of English, 5–16, and draws a systematic map of the field as a result. The second question is one that is answered in an in-depth review: what is the evidence for the effectiveness of different ICTs in the teaching and learning of English (written composition), 5–16?
Methods
The systematic review (both the map and the in-depth study) used the methodology set out in guidelines and tools devised by the EEPI-Centre. In short, a protocol or plan for the research was drafted, including a provisional research question for the initial map of research in the field. Exclusion and inclusion criteria for the literature search were written. The protocol was peer reviewed, revised and then published on the Research Evidence in Education website. Research papers were searched for, identified, screened for relevance and then keyworded to create an initial database. A map of research studies in the field was generated. From the map, one area of research was identified for in-depth review: written composition. Papers in this area were data-extracted and assessed for quality and weight of evidence with respect to the research question. A narrative synthesis of the results was produced.
Results
A total of 2,103 papers were found in the initial search for studies published between 1998 and 2003 on the topic of the review. Of these, 56 met the inclusion criteria for the review. Of the 56 papers, 14 were reviews of research and 42 reported primary research. Because four of the primary research papers contained more than one study, a total of 14 reviews and 53 studies were examined.
Nine of the 14 reviews were systematic reviews, and the countries of origin of the 14 reviews were the UK (seven) and the USA (seven). They also divided equally between a focus on reading, writing and other aspects of literacy. The general consensus of the reviews with regard to written composition was that computer-assisted instruction or learning and word-processing appeared to have a beneficial effect on student’s written composition, but only when combined with strategic instruction.
Of the 53 primary studies, 36 were from the USA, 15 from the UK, and one each from Australia and Canada. The majority of the studies focused on learners between the ages of 5 and 10 in a primary or elementary school setting; and two-thirds involved learners of both sexes. Again, these studies were evenly divided between an emphasis on reading, writing or other aspects of literacy. The principal focus in terms of ICT across the 53 primary studies was on computer-assisted instruction (CAI) or computer-aided learning (CAL), and on software. Almost two-thirds of the studies on writing focused on composition. More specifically, in terms of those 20 studies that focused on written composition, CAI/CAL, software and multimedia were the most popular ICT interventions, each with six studies; five studies investigated word processing. (CAI/CAL as computer-assisted instruction and computer-assisted learning are often difficult to unravel in studies on ICT's effect on written composition. When we refer to them separately, we mean them as separate concepts; but when we splice them together, it is because there is no clear distinction between 'instruction' ('teaching') and 'learning' in the research we have reviewed.)
The in-depth review on the effectiveness of ICT in the teaching and learning of written composition in English concentrated on nine studies. A meta-analysis was not conducted as the samples, intervention measures and outcomes were considered too heterogeneous, so a narrative synthesis was undertaken. As eight of the nine studies were judged to be of medium weight of evidence and also different from each other in nature, it was not possible to arrive at a clear answer to our in-depth research question. Rather, we wish to report that the field is in a pre-paradigmatic state where definitions of English, literacy and ICT are still relatively unclear and where the causal and/or symbiotic relationship between them have yet to be fully theorised. The most authoritative study in terms of the present review is that by Lewis et al. (1999) which showed that ICT made little difference to an experimental group of ‘learning disabled’ students in terms of writing quality, but that, for lower-order writing skills, improvements happened at a faster rate for such students as well as there being an increase in self-esteem for these students.
Conclusions
Such results can provide neither clear answers to our research questions nor firm conclusions. Despite a rigorous searching and screening exercise and a systematic, explicit and replicable process of distillation that involved inter-rater reliability and peer review on three occasions, the results are non-conclusive. The review suggests that there is insufficient research of high quality to answer the specific research questions we set ourselves. It is fully acknowledged that the research questions are about effectiveness, and therefore require studies of a particular kind to answer them. It does not mean that this is the only kind of question that can be asked in the field, but our results (which are consistent with previous reviews in the field) suggest that researchers, practitioners and policy-makers need to accept that questions about effectiveness or impact with regard to ICT and literacy/English in schools have yet to be answered; and that other questions also need to be asked.
Implications for policy, practice and research are set out in detail in the report. Essentially, from the individual studies we have examined (rather than through a synthesis of the studies for in-depth review) policy-makers might think about ICT as a range of technologies that can have particular effects on particular parts of the English syllabus under particular circumstances, rather than as a main tool for literacy development. The next version of the National Curriculum for England could view ICT as less peripheral, but more limited and specific in its contribution to learning and teaching. Procurement policies should follow from curricular needs, rather than driving them.
In practice terms, ICT is best seen as another tool in the repertoire available to learners and teachers for expression and communication. Custom-made word processing and other software programs should be considered by teachers, as some of these prove to be more attuned to the writing process than others. Teachers also need to be aware that there are times when the use of ICT is appropriate for a particular writing task (or part of that task), and other times when different media are more appropriate. Continued and prolonged exposure to ICT can be demotivating.
Further research needs to undertake some large-scale, well-designed randomised controlled trials if it is to answer the questions set in the present review. Ethical issues need to be considered, and the limitations of controlled trials and pre- and post-test studies acknowledged. In substantial terms, research needs to look at the symbiotic relationship between ICT and literacy; the role of teachers in mediating between ICT and learning; and the use of curriculum time with regard to technologies. Further work also needs to be done on the theoretical foundations of research in the field: in particular, building on the work of the New London Group (Cope and Kalantzis, 2000). Finally, one of the implications of the present (and previous) reviews on the topic is that a new kind of research is needed in the field: one that is at the cutting edge of interface design and pedagogical application, and which adopts a research and development approach.
References
Andrews R (ed.) (2004) The Impact of ICT on Literacy Education. London: RoutledgeFalmer.
Cope B, Kalantzis M (2000) Multiliteracies. London: Routledge.
Lewis RB, Ashton TM, Haapa B, Kieley CL, Fielden C (1999) Improving the writing skills of students with learning disabilities: are word processors with spelling and grammar checkers useful? Learning Disabilities: A Multidisciplinary Journal 9: 87–98.
This report should be cited as: Andrews R, Dan H, Freeman A, McGuinn N, Robinson A, Zhu D (2005) The effectiveness of different ICTs in the teaching and learning of English (written composition), 5–16. In: Research Evidence in Education Library. London: EPPI Centre, Social Science Research Unit, Institute of Education, University of London.