What do we want to know?

This systematic review explores how school accountability policies operate locally to improve school systems and children’s learning outcomes in low- and middle-income countries (LMICs). These policies include:

• Assessment: student examinations used to monitor the quality of the educational system, some of which (high stakes examinations) also carry direct consequences for performance for schools, school teachers as well as individual students
• Monitoring: the system-level processes designed to collect, compare and report school-level information about the composition, organisation and function of schools
• Inspection: formal site visits to schools by education authorities to observe classroom and management activities.

What did we find?

• Assessment may improve the quality of teaching and learning when the following mechanisms are triggered by specific conditions prevalent in the local school context:

1. Trust in the pedagogical authority of the assessment approaches is triggered by system- and school-level support for teaching tied to assessment approaches.
2. Teachers’ close attention to results in ways that improve teaching follows from customized guidance around interpreting results.
3. Incentives prompt teachers’ desire for reward and improvements in teaching quality when incentives are focused on individual (not collective) performance and are perceived as high-value.
4. Parental oversight of quality of teaching and learning promotes student performance gains when individual student incentives are perceived by parents as high-value.

• Key barriers to assessment activities that aim to improve teaching and learning can include

1. School staff fearing the consequences of poor performance
2. Lack of individual teacher incentives,
3. Lack of training and support to use and interpret assessment results effectively by school staff

• Monitoring could lead to improvement in school management and school performance when one or more of the following conditions are prevalent in the local school context:

1. Interpreting information: Sustained effects on school management and student attendance when there is consistent and clear feedback about results that is accompanied by training to interpret results across district, sub-district and school-levels
2. Accuracy of information: Timely and accurate reporting of school- and district-level information occurs when those at higher levels of the system place value on understanding system performance rather than rewarding positive results (“reality testing”).
3. Local school development planning: Local school development planning is likely to be effective when school leaders and teachers are given opportunities and abilities to learn from failure
4. Acting on information: School management committees use information effectively to improve school conditions when parents develop capacity for interpreting results and pressure schools to improve teaching quality and learning
5. Parental involvement: Service delivery and learning outcomes improve when parents participate in monitoring activities

• Inspection generally has limited impact on systems and school-level outcomes. Key barriers to successful inspection may include limited coordination between the Inspectorate of Education and other national stakeholders, or some specific attributes of inspection feedback (e.g. disrespectful tone of voice, or recommendations out of school’s control) and inspectors providing the feedback (e.g. lack of credibility of inspectors).

What are the implications?

• For educators at the system and school-levels: A key insight of this review is the ways in which development of capacity may need to occur within and across levels in order for accountability activity to yield desirable school, system, and student outcomes.  Our initial theory, based on existing literature, suggested the opposite, implicitly assuming that development of educators’ capacities would follow from school accountability activities.
• For policymakers: The most salient implications of our review concern ways of resolving what we identified as a tension between the monitoring functions of accountability activity and the development functions that aim to cultivate educators’ capacities (see the preceding point) around improving service delivery. Typical approaches to accountability activity assume that establishing performance standards and providing feedback based on results make expectations about performance improvement explicit at the school-level. However, across our review we found that expectations need to be accompanied with proactive and consistent guidance around improving school management and teaching practice. School-level service delivery did not change in the studies we examined when those at the local level did not have capacity or resources to fulfil implicit or explicit demands.
• For researchers: Research has yet to trace clear connections between change in processes at the school-level that occur as a result of accountability activity and changes in student learning outcomes, particularly for the poorest and most marginalized students.

How did we get these results?

We selected an approach to systematic review known as realist synthesis (Pawson, 2006; Pawson, Greenhalgh, Harvey, & Walsh, 2005; Wong, Greenhalgh, Westhorp, Buckingham, & Pawson, 2013) with the intention of identifying mechanisms that might help us understand how accountability approaches operate in diverse contexts. Realist synthesis enabled us to outline the mechanisms that lead to service-delivery, or school-level, outcomes and to characterize the local school contexts under which those mechanisms operate. This is a necessary precursor to explaining why student-learning outcomes do or do not result from accountability activities. The concentration of our review on school-level service-delivery processes and outcomes means that implications for practice at the local level are highlighted, emphasising school management and instructional practices.

We included 68 studies that investigate the three accountability elements in primary and secondary education in LMICs. We included studies published on/after 2001 and in English.  The evidence base for school accountability relevant to this review is largely from sub-Saharan Africa with a smaller portion of the papers coming from South Asia, Latin America, and East Asia and Pacific.

The connection between descriptions of conditions of the local school context and reported outcomes (when available) was identified in the reviewed studies, guiding by the initial theoretical framework. We did this by systematically identifying relevant papers, and then coding and summarizing included papers. From each paper, we extracted information to describe the key features of each study and the accountability activities reported in the study, including details on conditions of the local school context, outcomes and suggested or inferred mechanisms. Syntheses were then conducted for each accountability element. We elaborated the synthesis findings through additional mining of existing papers. The findings about conditions of the local school context and outcomes were then used to elaborate a more refined model of potential mechanisms, within and then across accountability elements.

This report should be cited as:

Eddy-Spicer D, Ehren M, Bangpan M, Khatwa M, Perrone F (2016) Under what conditions do inspection, monitoring and assessment improve system efficiency, service delivery and learning outcomes for the poorest and most marginalised? A realist synthesis of school accountability in low- and middle-income countries. London: EPPI Centre, Social Science Research Unit, UCL Institute of Education, University College London.