Background
Language learning, perhaps more than many other curricular subjects, depends on interaction between individuals, particularly for the oral and aural aspects of language relative to reading and writing. However, it is not always practical, easy or desirable to get groups of learners in the same place at the same time. The advent of the virtual classroom over networked systems liberates both the learner and providers of training from these constraints to varying degrees. It means, for example, that a group of learners (whether 3 or 20 individuals) can get together and extend their language learning activity beyond the regular confines of time (scheduled classes) and space (the physical classroom). Not only does this virtual learning environment possibly replicate the traditional environment, it may offer other, different benefits or even disadvantages. Around the beginning of the 1990s, a number of computer technologies (platforms, software, programs) generally developed for the business world and online meetings, started to be used in education. They were combined in various configurations of elements – including text chat (email) and instant messaging, web cameras, screens for writing and drawing, channels for speaking in real time, and shared internet browsers. In the field of education, they have been called variously ‘e-learning tools’, ‘online tuition’, ‘computer-mediated communication’, but, more precisely, they have become known as ‘audiographic conferencing systems’ or ‘synchronous web-conferencing’. When used in real time, the term ‘synchronous’ is added and together these form the topic of interest of this review: synchronous audiographic conferencing (SAC).
Aims
The aim of this review is to identify and evaluate available research evidence on the use and effectiveness of synchronous audiographic conferencing in modern language learning and teaching.
Review questions
The primary question addressed in the map is as follows:
What empirical research (including reviews of such research) could be found on the use and effectiveness of synchronous audiographic conferencing in language learning and teaching?
The primary question of the in-depth review is as follows:
What is the evidence concerning effectiveness of SAC in language learning as demonstrated by the available empirical research?
Methods
Inclusion criteria
Studies were included in the descriptive map of research if they met the following criteria:
- of empirical research (including systematic reviews of such research)
- about language learning
- about a synchronous audiographic conferencing (SAC) intervention, where SAC is defined as bi-directional multipoint speaking/listening that involves text manipulation (e.g. whiteboard, concept map, document and file display area)
- carried out in a formal setting, such as groups of learners in schools, universities and language centres
- carried out since 1990.
Search strategy
Reports were sought via database searches and searches of specific journals or conference proceedings. Key databases were identified and a draft search strategy was developed during March and April 2004. A cut-off date of 31 January 2005 was set for the retrieval of papers to be considered in this version of the review. The searches were carried out between May and August 2004. References were held and managed in an Endnote database. A summary of the search strategy is presented here.
The databases searched were as follows:
- Academic search premier
- ArticleFirst
- Australian Education Index
- British Education Index
- Dissertation Abstracts
- Education-line
- ERIC
- Index to theses
- ISI Web of knowledge
- Linguistics Abstracts online
- MLA international bibliography
- PapersFirst
- ProceedingsFirst
- ScienceDirect
- WorldCat
- Zetoc Electronic Table of Contents
The terms used for searching were as follows:
- computer mediated communication
- language learning or language-learning
- audio-graphic or audiographic
No systematic effort was made to identify relevant studies in the non-English language research literature, although any non-English language reports found would have been included in the review. Updates of the review will include handsearches of non-electronically indexed journals and conference proceedings, and searches of the non-English literature.
Characterising studies
Retrieved reports that met the inclusion criteria were classified according to a standardised keywording system developed by the EPPI Centre. This classifies studies in terms of the type of study; the country where the study was carried out; the educational focus of the study; and the study population.
An additional set of review-specific keywording questions was developed by the reviewers and these were also applied to each study. These criteria described the conferencing software, details of the intervention, the language of study, the participants involved, the outcomes of interest and the reported findings.
In-depth review
In the published protocol for the review, the authors specified the methods for describing and appraising a potential subset of studies within the map.
In establishing the criteria for which studies to include in the in-depth review, the team balanced the need to focus on research, such as large scale studies that control for various sources of bias, with other descriptive research that also forms part of the overall picture for the purposes of policy considerations.
In order to be included in the in-depth review, studies not only needed to meet all the criteria for inclusion in the map but also to be primary reports of experimental studies testing the effect of a language learning intervention against another intervention, or standard practice or no intervention. Screening for the in-depth review was carried out by two reviewers per study. These reviewers worked independently of each other and then conferred to reach consensus over inclusion and exclusion.
Further methods for description, quality appraisal and synthesis within the in-depth review were developed by the reviewers and can be found in the review protocol.
Results
Mapping of all included studies
Altogether, the map included 14 reports on studies conducted since 1990. Four of the studies considered both learners and teachers while another three studies focused primarily on the teachers but also gave information on the learners.
What sorts of studies were found?
All 14 studies described the interventions being evaluated or observed, and so generically can be considered as descriptive intervention studies. There was a range of researcher manipulation in these studies as some were designed as experimental and others were more naturalistic and observational. Nine had specific interventions, although some of these were not set up as prospective studies, while five were naturalistic observational studies. Not all the studies looked only at language learning, but all included it, and over half covered the use of SAC in the standard curriculum as the primary interest point. The studies had various descriptive or evaluative aims, including investigating interaction, effectiveness, satisfaction, practice of the language, attitude and suitability of the tool in question. The majority of the studies were in the secondary or post-secondary sectors and were carried out in Australia, the UK, USA and Canada. The conferencing platforms involved were Electronic Classroom, NetMeeting, Lyceum, PC + telephone, Telelearning + Optel, QuikCams + C-USeeMee, VoxChat + email. Broadly speaking, the studies all reported positive findings (i.e. none found against SAC), but as none was included for in-depth review, assessment of reliability of the results was not possible.
Studies for the in-depth review stage
No studies were included for in-depth review as none met the inclusion criteria (comparative studies testing the effect of the intervention).
Conclusions
Implications for teaching practice and policy-makers
Because no experimental studies are currently available for in-depth review, research-based implications cannot be drawn regarding policy and practice at present. However, this does not mean that synchronous audiographic conferencing should be excluded from teaching practice and policy making but that such decision cannot be based on experimental research evidence.
Implications for research
There is obviously a need for larger scale, robust studies looking at the effectiveness of SAC in relation to various outcomes. However, it remains to be seen if a certain antipathy towards comparative studies and randomised trials will prevent much experimental research evidence becoming available. The USA government (Whitehurst, 2003) and that of the UK (Morrison, 2001) have expressed to an extent a need for large-scale randomised field experiments, but the research establishment, at least in the UK, has yet to respond.
Rather than looking directly at synchronous audiographic conferencing (SAC) effectiveness, an alternative angle of approach might be to consider how best to design tasks for SAC within the computer-mediated communication (CMC) interactionist paradigm.
While research can rarely provide definitive answers to education questions, the more that it can be brought together, the more likely it will serve to focus the issues, highlight the gaps, test theories and interventions and contribute to an increase in the probability that intended educational aims may be reached.
Strengths and limitations
The map of research and the description contained in this review lay down a baseline for the research evidence relating to the effectiveness of synchronous audiographic conferencing in language learning. While definitive statements on the effectiveness of SAC are not possible at this moment, the studies identified in the map address the issue and make available important information regarding the question.
The time and resources have not been available to do any extensive handsearching of journals not indexed electronically, although this remains a long-term objective. The results will be incorporated into updates of the review. For updates of this review, effort will be put into searching for reports of studies published in the non-English language research literature and any ongoing postgraduate projects.
It should be borne in mind that the studies included in this review have not been assessed for their reliability or strength in addressing the question and at best can only really be considered pointers towards a sound evidence base. Their proposals in most cases still need testing on larger numbers of learners and learning situations.
There may still be technical difficulties with SAC, depending on individual systems, internet provision and technical capability. Both teachers and learners wishing to use SAC will need to persevere and appropriate support will need to be given.
In conclusion, until there are high-quality comparative controlled studies, with strong qualitative studies to provide detail and situated explanations of events and mechanisms, and examining the effects of interventions on individuals rather than populations, the picture cannot be fully understood. Without such research, there will always remain more doubt than is necessary: were the outcomes achieved because of SAC, or would they have been achieved in a traditional face-to-face situation in any case?
References
Morrison K (2001) Randomised controlled trials for evidence based education: some problems in judging ‘what works’. Evaluation and Research in Education 15(2): 69–83.
Whitehurst G (2003, April) The Institute of Education Sciences: New wine and new bottles. Presentation at the Annual Meeting of the American Educational Research Association, Chicago, IL.
This report should be cited as: Hassan X, Hauger D, Nye G, Smith P (2005) The use and effectiveness of synchronous audiographic conferencing in modern language teaching and learning (online language tuition): a systematic review of available research. In: Research Evidence in Education Library. London: EPPI Centre, Social Science Research Unit, Institute of Education, University of London.