What do we want to know?

As a result of the Millennium Development Goal of Universal Primary Education (UPE) and the related Education for All goals, massive recruitment of untrained and less educated schoolteachers has taken place in many low and middle income countries. In many places this has succeeded  in meeting the demand caused by vastly expanded pupil enrolment, but there are concerns that it has led to poorer teaching and learning outcomes. Against this background, there is an urgent need to understand better the processes and outcomes involved in the classroom performance of these kinds of teachers and to investigate the various ways in which such teachers may be provided with a belated education, training or upgrading.

This review explored the following research question:

What are the impacts and cost-effectiveness of strategies to improve performance of untrained and under-trained teachers (UUTs) in the classroom in low and middle-income countries?  

This was done through the following sub-questions:

• How do UUTs perform in the classroom, and what factors affect their performance?
• What forms of intervention have been used to attempt to improve the performance of these teachers?
• How have these interventions affected these teachers’ methods, skills and motivation, the performance of their pupils, and the satisfaction of parents, headteachers and other stakeholders?
• What is the available evidence for the effectiveness and cost-effectiveness of such interventions, and what are the factors that may influence these in different settings?

Who wants to know?

The study was funded by the Australian Agency for International Development (AusAID) as part of a joint call for systematic reviews with the Department for International Development (DFID) of the UK, and the International Initiative for Impact Evaluation (3ie).

What did we find?

No one strategy was shown to be directly related to impact on teacher performance; rather, interventions included different combinations of strategies which were applied and evaluated as a package. This makes it hard to disaggregate the significant elements of the overall intervention. Successful interventions in the studies rated as more robust and trustworthy each included several strategies: training workshops, independent study and in-class support were most frequently applied in these studies, followed by in-school support and school clusters.

• Eight of the studies found change in teaching methods as a result of face-to-face workshops and classroom support. All bar one also included a significant and explicit element of self-study.
• Two studies very convincingly demonstrated that coaching by highly qualified, experienced and expert coaches was effective in changing UUTs’ teaching methods, but raised concerns about its cost and labour-intensiveness.
• It is difficult to make claims about the effect of interventions on pupil learning. Of the four studies that convincingly reported some form of objective measures on pupil performance, three showed improved attainment (Tatto et al. 1991; O’Sullivan 2003; Binns and Wrightson 2006) and one showed no improvement (Stuart and Kunje 1999).
• Although many studies reported that the improvement of teacher subject knowledge was an aim of the intervention, this often proved problematic: studies found gains failed to meet expectations. Mathematical learning seemed to be particularly difficult to improve through distance education.
• The in-depth review synthesis also found nine considerations key to successful interventions identified in those studies rated highly contextually robust. These were: the regularity of UUTs’ engagement in the strategies; the proximity to the classroom of the sites where strategies took place; the extent to which workshops and self-study modules were focused on discussion of practice and enhancement of subject and pedagogical content knowledge; the importance of well-produced resources and appropriate use of ICT; the quality of tutors and their briefing or training, including where  possible and more cost-effectively, employing competent experienced teachers from the UUTs’ own schools in a mentor role; recognising that effective interventions might need to last for several years; gaining support for the interventions from school management and external administration; balancing the burden of study and workshop attendance with the UUTs’ work, family and community commitments; and recognising the constraints to classroom improvement, including critical reflection, within the specific contexts in which the UUTs worked.
• Engagement in training activities can have a positive impact on the professional confidence and motivation of the teacher within their school and community. This was reported in twelve studies.
• The figures presented in the studies reviewed strongly suggest that distance education methods are cheaper than conventional methods . However, they may not necessarily be the most effective.
• One of the most trustworthy and robust studies noted that distance learning was effective in educating large numbers of untrained teachers at low cost and might be suitable in other contexts where resources were limited, but where there were large numbers of untrained teachers. As the backlog of untrained teachers was reduced, provided no new untrained teachers entered the system, the cost-effectiveness of distance education would decrease.
• One study (Mehrotra and Buckland 2001) argued that increasing the proportion of trained teachers increased average teacher earnings and thus might not be sustainable in many low-income countries.

What are the implications?

1. Interventions should clearly prioritise their aims to differentiate between those of improving pupil learning, enhancing teacher subject knowledge and granting certification. Interventions designed with unclear goals or outcomes make it difficult to appraise accurately their success, impact or causes of failure.
2. The success of interventions depends on a set of factors located within the school, the education system and society, indicating that it is better to target school clusters and individual schools, as well as individual teachers, to eliminate the effects of some of these variables.
3. Interventions which are based on analysis of the knowledge, experience and needs of UUTs in the contexts in which they work are more likely to result in positive outcomes. The active involvement of teachers and their mentors in the design of any intervention is similarly more likely to have positive outcomes.
4. Interventions should involve frequent, regular engagement with self-study materials, and/or taught sessions, and/or discussions with peers, and/or the opportunity to apply new learning and skills in the classroom, to give the greatest opportunity to improve performance.
5. Proximity of the site of inputs to the UUT’s classroom supports engagement and application of learning, and affects the availability of support from local tutors, peers or experienced teachers. Options that might be considered here include housing taught sessions in nearby teacher resource centres or a key school within a geographic cluster, or planning how best to facilitate easy access to materials or mentor/tutor support.
6. Initially, workshops can be most effective when they are used for specific purposes, such as introducing tutors and other students to each other, presenting a programme structure and/or explaining the purpose of self-study modules or the use of new technology within them. Once a programme is up and running, workshops used to discuss teaching methods in the reality of the UUTs’ classrooms and critical reflection with smaller numbers of participants may be more effective than the more traditional use of workshops to transmit subject knowledge in a lecture mode.
7.  Well-planned and -produced self-study materials, focused on subject knowledge but including practical activities can support teacher learning at a distance; new technology shows much potential but should only be used where realistically and reliably accessible to users. Print-based materials are otherwise more reliable. However, many governments are committed to improving ICT provision, and it is likely to become an integral part of many interventions in the near future.
8. The quality of trainers is important, so interventions should include plans for their training. Briefing, resourcing and capacity building of other stakeholders such as teacher colleagues, headteachers, administrators and community members might create conditions to support interventions more effectively. The effectiveness of such support is maximised when there is closer proximity to the contexts where UUTs are working, greater familiarity with the languages they use, and an understanding of the need for sensitivity in one-to-one contexts in terms of gender relations.
9. Demands on the time and resources of UUTs must be balanced with their daily commitments to their jobs, family and community responsibilities, in particular for women, for whom the burden of care and training can be most onerous.
10. There needs to be recognition of the constraints to classroom improvement, including developing more learner-centred pedagogies and critical reflection, within the specific contexts in which the UUTs work. Sustainable change to professional practice occurs slowly over a substantial amount of time.

How did we get these results?

We searched 22 major databases using a variety of search terms and strategies. We also handsearched seven key journals and looked at fifteen websites, in addition to making use of citation tracking and personal contacts to identify potentially relevant studies. This initial search yielded 3,499 references, of which 130 studies met the inclusion criteria and were obtainable. A systematic mapping exercise was conducted on these by applying a standardised set of codes, giving a broad characterisation or picture of the research field. We then moved to an in-depth review by applying additional, stringent inclusion criteria that ensured that the final set of 23 studies focused centrally on the review’s primary focus of interest: the effectiveness and/or cost-effectiveness of interventions to improve classroom performance of UUTs.

The EPPI Centre reference number for this report is 2104.

This report should be cited as:
Orr D, Westbrook J, Pryor J, Durrani N, Sebba J, Adu-Yeboah C (2013) What are the impacts and cost-effectiveness of strategies to improve performance of untrained and under-trained teachers in the classroom in developing countries? London: EPPI Centre, Social Science Research Centre, Institute of Education, University of London.
ISBN: 978-1-907345-48-7