Background

This review has been carried out on behalf of the Training and Development Agency for Schools which has identified a number of key areas in which systematic reviews of the research literature are desirable. One of these is the effectiveness of Information and communications technology (ICT) applications in teaching and learning in the core curriculum subjects of English, Science and Mathematics.

The issue of the effectiveness and impact of ICT in the core curriculum subjects is important. In science, ICT has opened up a whole range of potential applications. At the same time, a wide range of potential benefits resulting from the use of ICT has been claimed for both students and teachers by a number of groups (policy-makers, researchers, some teachers, employers).

Although there is a significant literature on ICT in science education, much of it takes the form of articles on applications for use in teaching situations: the emphasis is on how to use ICT, rather than exploring its effects. There is a sense in which it is taken rather for granted that ICT is a ‘good thing’, with students being motivated when they use it, and this leads to better learning. Thus a central purpose of this review is to assess the strength of the evidence base to support the notion that the use of ICT activities in science lessons enhances students’ understanding of science ideas.

Aims

The review aims to assess the impact of the use of ICT on students’ understanding of science..

Review questions

The main review question is as follows:
What is the effect of using ICT teaching activities in science lessons on students’ understanding of science?

The term understanding of science encompasses scientific knowledge and explanations (facts, laws, theories), the scientific approach (evidence, scientific methods, prediction, problem-solving and so on) and ideas about science (its limitations, the scientific community, risk and so on). Attitudes to science were not included in the review.

Certain criteria were applied to the studies in the systematic map in order to ensure that only potentially good quality and appropriate studies were included in the in-depth review. These were as follows:
• the type of ICT activity most frequently evaluated in the research reports
• the most appropriate study designs for an evaluation (researcher manipulated/controlled)
• studies involving a pre-post design
• studies involving representative or average/typical students

This gave the following in-depth review question:
What evidence is there from controlled trials of the effects of simulations on the understanding of science ideas demonstrated by students aged 11-16?

The term ‘simulation’ is used in somewhat different ways in different studies. For the purposes of this review, it was understood in two ways:
• the use of the computers to imitate particular experiments
• the use of computer and other resources to imitate a wider situation and thus provide a virtual environment with a range of facilities The term was not used in the sense of mathematical modelling.

Methods

The review methods are those developed by the EPPI Centre for systematic reviews of educational research literature. Such a review has four main phases:
• Searching and screening: developing criteria by which studies are to be included in or excluded from the review, searching (through electronic databases) for studies which appear to meet these criteria, and then screening the studies to see if they meet the inclusion criteria
• Keywording and generating the systematic map: coding each of the included studies against a pre-agreed list of characteristics which is then used to generate a systematic map of the area where studies are grouped according to their chief characteristics
• In-depth review and data-extraction: summarising and evaluating the contents of studies according to pre-agreed categories
• Synthesis: providing an overview of the quality and relevance across the studies in the in-depth review and compiling the weighted findings of the collective studies

Results

The studies identified through the searching and screening process established that ICT was being used to teach science education in a variety of ways. The focus of the studies was very largely on teaching scientific understanding and scientific approach. Very little research was carried out on the applications of science, or on the use of ICT for stimulating ideas about science, such as its limitations or risk. A number of the studies were interested in other aspects: for example, attitudes to science, but these were not included in this review.

Thirty-seven studies met the inclusion criteria developed for the overall research review. These studies were keyworded and formed the basis of the systematic map. The map revealed a number of characteristics of the use of ICT in science education:
• The majority of the studies reported work that has taken place in the USA and Taiwan.
• A little over one-third of the studies concerned Biology topics and just under one-third concerned Physics topics. Very little research has been done in relation to Chemistry education.
• Few authors gave explicit details of the ability range of their participant students. (It was therefore assumed that students were mixed ability or average for their age, unless otherwise stated.)
• In one-third of the studies, students worked individually with the ICT and in eight studies (22%) students worked in pairs. Two-fifths of authors (15 studies) did not give details of how the students interacted with the computers.
• Close to 90% of the studies focused on the students’ understanding in respect of scientific knowledge/explanations and one-half on scientific approach; 12 studies investigated both. This interest was spread across Earth Science, Biology and Physics.
• Types of ICT used varied, but half were referred to as simulations, either of experiments or of virtual environments. Virtual environments included a range of other ICT activities and non-ICT resources, and could be defined as ‘multimedia’. Thus there is some overlap and flexibility in how the various forms of ICT are described or named.
• Fifty percent of the studies were carried out in one school with several classes. Only four studies (11%) involved large samples over several schools. Nine studies did not give full details of how many schools or classes were involved, although they all gave student numbers.
• Three-quarters of the studies used pre-post testing and half used questionnaires. Test results (that is, post- but no pre-test) were used in a quarter of the studies, as were interviews. Eight studies (22%) observed the student activities.
• Three-quarters of the studies were published in academic journals, seven (19%) as conference papers and one in a book chapter.
• Simulations/virtual environments, multimedia and moving images were used in the same or similar proportions to teach both scientific knowledge and scientific approach. (This is not too surprising given the overlap in these three ICT categories.) Data-logging, databases, the internet and tutorial applications were used more often to teach scientific approach than scientific knowledge.

The ICT activity most frequently evaluated proved to be simulations (19 of the 37 studies).

Nine studies met the criteria for the in-depth review. Following data-extraction and the ICT in science teaching application of the weight of evidence criteria, seven studies were identified as being of sufficient standard to use in the synthesis; one was rated as medium high and six as medium.

The overall findings are listed below. However, as the sample size was small, the simulations variable, the learning objectives diverse and some of the following observations are based on only one study, a number of the findings should not be used for generalisations.

1. Students’ use of ICT simulations helped to improve their understanding of science ideas significantly more effectively compared with their use of non-ICT teaching activities (based on six studies).
2. Students’ significantly better understanding of science ideas when using ICT simulations versus their use of traditional (non-ICT) activities can lead to understanding of science knowledge (based on seven studies) and to understanding of scientific approach (three studies).
3. The simulations fell into two main categories: (i) simulation of specific experiments and (ii) simulations of a wider scientific situation, commonly known as ‘virtual environments’, which could include experimental simulations.
4. The positive effect of students’ use of ICT simulations on their understanding of science ideas is independent of the type of simulation, that is, simulations as virtual experiments (four studies) or simulations of a virtual environment (three studies).
5. Students’ use of ICT simulations was more effective than using non-ICT teaching activities for supporting basic science ideas (from three studies), including the improvement of:
• Bloom’s lower levels of understanding (two studies)
• understanding of basis aspects of the scientific approach (one study)
• science knowledge of less advanced reasoners (one study).
6. The improvements in higher understanding (for example, application) of more advanced aspects of the scientific approach (for example, the design of an experiment) and for more advanced (formal) reasoners can be achieved to the same extent with or without simulations.
7. The gains from the students’ use of ICT simulations were even further increased when teachers actively scaffolded or guided students through the ICT simulations (two studies). The extra gains resulting from teacher guidance through the ICT simulation included further improvement of lower levels of understanding of science (knowledge) and of the scientific approach, including the application of science knowledge to new situations (two studies).

Thus simulations can bring benefits to students in respect of scientific knowledge/explanations and approach, but not in all situations and with all students and teachers. Care needs to be taken in establishing the particular benefits for particular learners and learning objectives in particular situations.

Conclusions

Strengths of the review
The review has a number of strengths:
• The focus is one that is very relevant to the increased use of ICT in science teaching and learning. In particular, simulations are shown to be used in a wide range of situations. Evidence for this comes from the review map, in which 19 of the 37 studies (51%) have simulations as their core mode of ICT.
• The evaluation studies considered student achievement in the spheres of scientific
understanding and scientific approach.
• The approach to the review set high standards for the in-depth sample as only evaluation studies that had a control and pre-post test design were included. Additionally, the review only involved those studies that ensured their measures and their methods of analysis were valid and reliable.
• Quality-assurance agreements are high for all stages of the review.

Limitations of the review
There are four main limitations:
• Although 19 evaluation studies involving simulations were found for this review, only seven were of a sufficient standard to include for the synthesis. These can thus only present successful examples of possibilities for teaching and learning in science education and highlight pedagogical points for consideration when using simulations; generalisations cannot be made.
• Some of the terms used in the field of ICT and education appear to be rather fluid. Thus model/modelling/a model can be used in the sense of ‘to mimic or represent’ or could mean to provide a predictive facility or process. Similarly, simulation can be used to mean that something has been modelled. In this review, the predictive and more mathematical use of modelling was not included. (It did not feature as a topic for evaluation studies.) Multimedia can also include Summary simulations, in which case it is necessary to tease out the particular contribution of the simulation to learning effect.
• The in-depth studies covered the subjects of Earth Science, Biology and Physics. Only two studies of the 37 in the map and none in the in-depth sample were in Chemistry education. There appears to be a gap in this area of research which has impacted on this review.
• None of the in-depth studies was carried out in UK schools and thus the findings might not be directly applicable to the British educational system. However, the fact that similar findings did emerge from three different countries (USA, Taiwan and Israel) does suggest that there is a measure of robustness in the findings that would make them of use in the United Kingdom.

Implications for policy

Evaluation studies have found that ICT, and simulation in particular, can be helpful in teaching science understanding in respect of both scientific knowledge and scientific approach. However, it should be noted that there is a scarcity of high quality research in the area in which the in-depth study focused.

Teachers will also need training in the use of the simulations to obtain the greater benefit for student understanding. In particular, this review has shown that the use of ICT simulation needs to be carefully integrated into the teaching and learning process, and informed guidance provided. This guidance may be built into the software so that the students may work semi-independently, or it may be provided by the teacher. However, teacher guidance is the more effective. This has implications for policies for initial teacher training and continuing professional development (CPD).

Implications for practice

The review has indicated that there is a lack of clarity in the way that ICT and especially simulations, models and multimedia are interpreted. One implication for practice is that teachers should be made aware of this.

The development of ICT simulations for a large variety of virtual experiments and virtual environments would provide a number of teaching and learning benefits. These include, interalia, saving experimental time and resources, reducing the need to kill animals for dissection, allowing students to repeat experiments with ease, and providing experiences (through virtual environments) that would not otherwise be
available to students.

The importance of the structured or guided use of ICT in particular simulations needs to be stressed to teachers. It is not sufficient just to provide the software, unless it has in-built guidance or a virtual mentor. Without either of these, the teacher needs to provide that support. Teachers may also need induction or training if the simulation is part of a complex teaching programme.

The inclusion of simulation activities within science Post Graduate Certificate in Education (PGCE) programmes would also encourage their use.

The newly established Regional Science Learning Centre could provide ideal opportunities for CPD in the use of ICT in science education.

Implications for research

The low numbers of high quality research studies into the value of using ICT in science education, especially in Chemistry, was surprising given the potential benefits. The use of ICT is likely to increase rather than decrease in schools in the near future. It is also likely that curriculum developers and commercial enterprises will increasingly develop software packages for science education. It would therefore be of significant advantage if any science education ICT, of whatever origin, is carefully evaluated before it is adopted.

This report should be cited as: Hogarth S, Bennett J, Lubben F, Campbell B, Robinson A (2006) ICT in science teaching. Technical report. In: Research Evidence in Education Library. London: EPPI Centre, Social Science Research Unit, Institute of Education, University of London.