Prefatory note
This report is the result of two linked pieces of work undertaken between March 2001 and May 2002. A mapping study was conducted up to November 2001 and identified the range and type of research studies addressing the impact of Information and Communication Technologies (ICT) on literacy learning for ages 5 to 16. An in-depth review then addressed one area identified in this mapping - the impact of networked technologies on literacy learning in English aged 5 to 16. The in-depth review is the first of a number planned by the English Review Group in the field of ICT and literacy. Further in-depth reviews will take place in 2002 and beyond on evidence from experimental studies from 1990 to the present; the moving image and visual literacy; English as an additional language (which includes English as a second language); literature teaching; and word-processing.
Background
There is considerable interest in the impact of information and communication technologies (especially computers, networked computers, mobile phones) on learning in young people. The present study focuses on the impact of ICT on literacy learning in the English language for 5 to 16 year-olds.
There is particular significance in the selection of such a topic for education policy in England and Wales. The year 2003 marks the end of the New Opportunities Fund/Teacher Training Agency initiative on training teachers in the subject application of ICT. A computer-literate teaching profession will need to know about the impact of ICT on literacy learning in order to encourage the best use of resources in the raising of literacy standards. Policy-makers will need to know the results of our research in order to shape future policy with regard to ICT in the curriculum. Parents concerned about their children's education in the digital/information age will find the results of the study useful, especially with regard to the home/school dimension. So, too, might young people find the study helpful in avoiding unnecessary time spent with practices that are less than useful or enjoyable.
There has been much literature on the topic that is exciting and speculative. Much of it emerged in the early 1990s as the internet began to become more widely used, especially in schools. The Government in England, for instance, has invested a large amount of money over a number of years in the provision of computers to schools. Equally, many (not all) families use computers at home and young people are increasingly using them to research, word process, compose and present homework - and for a number of other functions. Questions remain, however, on what impact ICT has on schooling and in particular on literacy learning. The particular focus of the present study is on the impact of networked technologies - the internet, email - on literacy learning.
The English Review Group, as one of the review groups set up by the EPPI Centre, has investigated the impact of ICT on literacy learning by systematically reviewing an international range of research studies dating from 1990 to the present. It started its work with the definition of key terms in the study: 'ICT', 'literacy' and 'impact':
- ICT is taken to include stand-alone computers, networked technologies with a multimodal interface, mobile phones with the capacity for a range of types of communication, and other technologies which allow multimodal and interactive communication.
- Literacy can be defined narrowly, as the ability to understand and create written language. It is, however, frequently defined in two broader senses, and both are included in the present study. Firstly, the scope can be expanded so that written language becomes written language and graphical or pictorial representation. Secondly, the skill can be treated as social rather than psychological; in this view, literacy is the ability to operate a series of social or cultural representations.
- Impact was defined as the result on end users (here, children between 5 and 16 years old) of an intervention aimed at improving the teaching or learning of literacy. It may also be the result of a non-intervention activity which could reasonably be expected to increase or decrease literacy.
Aims of the review and review question
The present report's main aim is twofold: first, to identify a number of studies that might shed light on the impact of ICT on literacy learning in English for 5 to 16 year olds. The fulfilment of this aim has resulted in a study mapping the field of ICT's impact on literacy learning.
The second aim has been to undertake an in-depth review of the papers that were identified as being on the impact of networked ICT on literacy learning in English for ages 5 to 16.
Review question
Deriving from the above aims, the main review question for the overall study (which will continue for a further year) is:
What is the impact of ICT on literacy learning in English for 5-16 year olds?
The main review question for the in-depth review presented in the present report is:
What is the impact of networked ICT on literacy learning in English for 5-16 year olds?
Methodology
Initially, a protocol or plan was written to set out the parameters for the systematic literature review. This protocol established the main aims of the research, its research questions and the methods that would be employed to answer the questions. A key document in the development of the protocol was one that set out the inclusion and exclusion criteria for the review.
Searches of international literature then took place, both electronically and 'by hand', exploring a wide range of journals, reports, books and other formats. From searches in a number of internationally recognised databases, 1,871 post-1990 studies were identified that were of possible relevance to the larger review.
These 1,871 studies were screened, following the explicit criteria in the protocol, to identify relevant reports. Most of the screening was completed on the basis of abstracts provided by databases or with the papers identified as being of interest to the study. 188 studies were found that met the criteria in the protocol. These studies were then keyworded, or indexed, and the results filed on an electronic database (a resource that is also publicly available on the REEL database).
From the database, a map of the field was drawn. This 'mapping exercise' is an important stage in the process of the review and is reported in the sections of the report headed 'Identifying and describing studies'. The map provided the review team with an overall picture of the field, which enabled us to identify one particular area that we would examine in an in-depth review.
Of the 188 studies relevant to the mapping study, 16 pertained to the topic of the impact of networked ICT on literacy learning. Data were extracted from these 16 studies and have provided the basis for an in-depth review. Of these, half were outcome evaluations (evaluations of the results of an experiment or innovation), seven were process evaluations (evaluations of how an intervention was delivered, rather than whether it worked or not) and one was a needs assessment.
Methodologically, there is a wide variety of study types in the in-depth review, characteristic of education research in general but not reflective of the types of study found in the overall mapping study on the impact of ICT on literacy learning - the majority of which, unsurprisingly, are outcome evaluations. The range of study types in the in-depth review on the impact of networked ICT on literacy learning makes synthesis difficult. Meta-analysis was not possible because only one of the studies with quantitative outcomes was judged to be sound; and also because of the heterogeneity of the studies. The synthesis is therefore narrative in nature.
The methodology was devised by the Evidence for Policy and Practice Information Co-ordinating Centre at the Social Science Research Unit, Institute of Education, University of London. At every stage of the process, the English Review Group as a whole has been consulted. As it consists of parent governors, teachers, parents and policy-makers as well as researchers, it provides a broad constituency for ensuring that the direction of the review is sound. Furthermore, there has been independent peer reviewing of the process at the stages of the protocol writing and the submission of the draft report. Considerable improvements have been made as a result of such input.
Results
The study identified 188 papers published since 1990 that examine the impact of ICT on literacy learning in English for 5 to 16 year olds. Most of these originate from the USA, though a significant minority arise from research in the UK, Canada, Australia and New Zealand. Of the total, 67 percent are set in primary/elementary schools (especially in the 7-11 age range), with about 44 percent set in secondary/high schools (some studies were conducted in both types of setting). About two-thirds of the studies assume a psychological representation of literacy: that is, they assume that literacy development is an individual matter concerned with writing and reading processes. One-third adopt a more sociological conception of the practice: that is, one that assumes that literacy development is a matter of the academic and social communities in which you learn. Of the 188 studies, 57 percent are focused on writing, graphical or pictorial production, whereas 46 percent have an interest in reading (again, studies could have more than one focus).
As far as the in-depth study on networked ICT goes, results are inconclusive.
Many of the studies focus on the primary/elementary school sector, with at least four concentrating on 4th/5th graders (i.e. 9 to 10 year olds). Three of the studies look at out-of-school activities, and only two (and both indirectly) turn their attention to the impact of ICT on literacy at secondary or high school level. The principal areas of interest for the studies are reading and writing, but those twin aspects of literacy are often narrowly conceived, so that we are looking at the impact of new technologies on old practices rather than at the symbiosis between new technologies and new forms of literacy. Four of the studies look at word processing; two at new conceptions of literacy; and one each on speaking and listening, and on special educational needs.
Few of the studies in the in-depth review provide a firm basis for accepting their findings and therefore can have little bearing on the answering of the main research question for the in-depth review. Of the remainder, two provide theoretical and practical insights into widening conceptions of literacy; five suggest increased motivation and/or confidence in pupils as a result of ICT use with regard to literacy development; and one sees empowerment and ownership as an important factor to bear in mind in an increasingly diverse digital world. In general, these studies assume that a positive impact is made on literacy development by networked ICT, and explore how that impact is made; in most of the studies, the conception of literacy is narrow, based on pre-digital notions of reading and writing. The results are therefore suggestive rather than conclusive.
The strengths of the review are in its transparent processes, its breadth of reference and its systematic approach. These have resulted in an accountable, authoritative attempt to answer the main research questions. It has maintained a balanced and open view of study types, not wishing to commit itself to either the quantitative or qualitative camp, but preferring to ask a question - on impact - that enables it to gauge a social and educational phenomenon with a degree of rigour.
Its weaknesses lie in the fact that although there were a large number of studies identified for the mapping stage of the review, not many were identified as being relevant to the focus on networked ICT in the in-depth review. Of those that were about networked ICT, the quality was not high overall. An analysis of the weight of this evidence reveals that there cannot be much confidence in the results of these studies in terms of answering the questions about impact. The clear implications of such a result are that there needs to be more and better research in the field; that we need to revisit the main questions to take account of the symbiotic nature of the relationship between ICT and literacy; and that we cannot, at this stage, come up with a clear answer to this review's research questions.
Conclusion and recommendations
As far as policy and practice go, the recommendations of this review are highly tentative and take the form of implications or pointers. With regard to policy, they are to focus research funding for large-scale studies; to give consideration to the balance of study type expertise in research teams; and to give consideration to the fact that the provision of computer hardware and software to schools, and the application of ICT in teaching and learning, need to be informed by research and evaluation. In terms of practice, more attention needs to be given to the ways in which ICT is used in the classroom in support of teaching; teachers need to take more account of the ways in which young people work at home on computers; and ICT needs to be seen as one tool among many for the improvement and support of literacy learning.
Recommendations for research are more robust.
Recommendations are made to investigate further by covering the ground mapped by the in-depth review. There needs to be a stronger link between theoretical models and data from primary research, as well as more explicit accounts of methods used to gather and analyse data.
In terms of content areas to research more fully, the following were identified:
- email, conferencing and the internet in relation to new literacies
- writing and composing multimedia within a wider conception of literacy
- reading on screen and via networked computer systems
- hybridity, especially between the verbal and visual in multimedia.
The maintaining of a range of study types is suggested in order to build a fuller and sharper picture of the interface between ICT and literacy development.
We need large-scale randomised controlled trials (experiments with two parallel randomly-selected groups, one of which receives the intervention while the other acts as a 'control' group) in order to gauge effect. We also recommend that a series of small pilot trials be undertaken to evaluate other uses for networked technologies with a view to informing the design of large pragmatic trials. Given the paucity of experimental data, more randomised trials are needed.
We also need more research with qualitative outcomes. Small-scale studies (e.g. evaluated case studies) are needed as they are able to provide multi-factor analyses of the use of ICT in literacy teaching and learning. Because the field of research in ICT applications is young, there is also a need for more process evaluations.
There is a need for a large-scale longitudinal study, focused on literacy development, to complement the work currently being undertaken by the ImpaCT2 project, described in the background section (Chapter 1) of the technical report.
Furthermore, more work needs to be done on the compatibility of results from quantitative and qualitative research in education.
This report should be cited as: Andrews R, Burn A, Leach J, Locke T, Low G, Torgerson C (2002). A systematic review of the impact of networked ICT on 5-16 year olds' literacy in English. In: Research Evidence in Education Library. London: EPPI Centre, Social Science Research Unit, Institute of Education, University of London.