Sandy Oliver discusses whether the worlds of academia and humanitarianism can combine to improve the delivery and understanding of the processes and benefits of humanitarian aid through use of evidence.
Interest is growing, when making decisions within the humanitarian sector, in drawing on systematic reviews of studies that assess the effects of different policies or practices. As in other sectors, such research evidence is considered alongside what else is known, such as about competing priorities, social norms, available resources or ease of implementing a programme. Professor Sandy Oliver argues that in contexts where rigorous studies of effects are few and far between, perhaps because conducting research is difficult in such circumstances, it is useful to learn from systematic reviews that encompass other forms of knowledge that are commonly held by individuals and organisations delivering humanitarian aid. These broader systematic reviews increasingly come from partnerships of academics and humanitarian organisations. Strengthening the links between research and work in the field helps create evidence-informed policy/practice, and policy/practice-informed evidence.
Finding evidence of effects
Humanitarian crises, whether ‘man-made’ or following natural disasters, seem overwhelming not only to the people caught up in them, but also to the individuals and organisations wanting to help. How do we know what will help? Humanitarian workers have first-hand experience. Well established organisations have tried and tested procedures. Researchers offer findings from studies designed to understand the nature and scale of the challenges being faced, to compare the effects of different programmes or systems, and to investigate how aid is delivered by organisations and individuals. This knowledge, whatever its source, is widely dispersed and can be difficult to find. One solution is to seek research findings to inform policy and practice by drawing on systematic reviews of research findings. There is growing enthusiasm for systematic reviews in the humanitarian sector. As with any new movement there are new challenges. We are encountering some of these challenges as we help to inform the efforts of humanitarian organisations and evidence champions who try to improve the lives of displaced populations in areas of conflict.
An immediate challenge for humanitarians is how to access systematic reviews quickly. We have found two particularly valuable types of resources: database of systematic reviews maintained by Evidence AID and 3ie; and evidence gap maps prepared by 3ie and the International Rescue Committee.
These resources signpost systematic reviews offering high quality evidence (often from controlled trials) of ways to improve access to health care, such as through involving communities in planning, implementing and or monitoring health service, or by training traditional birth attendants (Obuku et al., 2017). Similarly, we can find high quality evidence showing the benefits of cash or vouchers that give families access to food (Doocy and Tappis, 2016); and WASH (Water, Sanitation and Hygiene) interventions for reducing the spread of disease (Yates et al., 2017).
When evidence of effects is lacking
Particularly challenging, humanitarian organisations tell us, are the logistics and organisation required to deliver supplies and coordinate multiple agencies. Learning how to address these challenges requires different sorts of studies. Frustratingly, not only are there fewer systematic reviews addressing these challenges, but those that exist find relatively little relevant evidence.
In some cases systematic reviews tell us more about what not to do. Efforts addressing violence against women and girls have to be made in the absence of counterfactual studies offering evidence about the effects of prevention programmes. Instead, findings of observational or qualitative studies warn of potential harms. Livelihood programmes that raise women’s earned income may lead to increases in sexual violence, which is already common in areas of conflict. Women can experience: coercion by partners to hand over their earnings; increased risk when collecting firewood to sell; and insecure employment where lack of regulation exacerbates vulnerability (Spangaro et al., 2013). Where justice systems are weak encouraging survivors of sexual violence to testify can be counterproductive. Rather than perpetrators being convicted, survivors may experience retaliation, stigma and ostracism (Fulu et al., 2014). Although they do not cite the research, international guidelines recognise these challenges (Inter-Agency Standing Committee (2015).
In this way, where uncertainties about effective policies or practices are high, and the available research is meagre, learning has come from systematic reviews that address how organisations work, not only the effects of their work. For instance, where research has not addressed the effects of different models or mechanisms for coordinating humanitarian efforts (Akl et al., 2015), learning is available from a systematic review of studies describing how organisations have worked together (Lotfi et al., 2016). The most common model is clusters of national and international humanitarian agencies liaising around the main humanitarian sectors (such as food security, WASH or education) and within a country under the overall leadership of a humanitarian coordinator. They are designated by the Inter-Agency Standing Committee (IASC) and have clear responsibilities for coordination.
Knowledge from practice, organisational procedures and research
Conventionally research knowledge is seen as distinct from organisational or practice knowledge. However, the boundaries are becoming blurred as researchers involve humanitarian organisations in designing their studies, and conversely as humanitarian organisations take the lead in conducting or commissioning research. Examples of the former are the research team in Beirut systematically reviewing coordination of health services (Akl et al., 2015). They began their work by convening key stakeholders to discuss key questions needing answers. Similarly, reviewers addressing sexual violence in conflict and post-conflict zones convened an advisory group of consultants and programme directors with in-depth knowledge in the field. (Spangaro et al., 2013). Both reviews benefited from having direct involvement of people well-placed to inform the project and feed the findings directly into policy processes and practice. Examples of the latter are Oxfam and UNICEF. Both these organisations have commissioned or conducted systematic reviews to inform their own humanitarian work. Oxfam, in partnership with the Feinstein International Centre, has its own humanitarian evidence programme. UNICEF’s Baby Friendly Initiative is supported by systematic reviews of the effects of infant healthcare where much of the evidence comes from randomised controlled trials. In the area of peacebuilding, where RCTs are rare, UNICEF’s systematic review conducted in partnership with academics assesses how educational interventions may contribute to enhancing the agency of youth as peacebuilders (Lopes Cardozo et al., 2015).
Areas of work that are least developed have most to gain from systematic reviews conducted with or by humanitarian agencies who are able to draw on internal reports of the agencies themselves. This is well illustrated by UNICEF’s systematic review of humanitarian programming and monitoring in inaccessible conflict settings (Chaudhri et al., 2017). This review identified only 14 academic papers, largely observational studies. These were supplemented by many more organizational guidelines, programming notes, program evaluations, presentations, and meeting notes which, between them, offered concepts and definitions, and addressed the preliminary steps required to advance this burgeoning field.
Strengthening the evidence ecosystem
Evidence-informed humanitarian aid has had a low profile. Yet many of the essential components mentioned above are already in place. Systematic reviews are readily accessible through databases and evidence-gap maps. Guidelines are developed by consensus at international, regional and national levels. Academics and humanitarian agencies work together to pool resources from their different worlds. The next step to developing an evidence ecosystem for humanitarian aid may be strengthening the links between these components: guidelines informed by evidence; and humanitarian-academic partnerships shaping evidence syntheses. A likely mechanism for doing so could well be the existing system of clusters. The Inter-Agency Standing Committee already designates the clusters, and develops guidance for agencies working in the field and for inter-agency working. The potential for bringing evidence and guidance closer together is apparent in the area of gender based violence, where an overview of the evidence (The Global Women’s Institute and International Rescue Committee 2016) mentions the Inter-Agency Standing Committee (2015) guidance. An even closer relationship between guidance and evidence is illustrated by a systematic review in the area of mental health, (Bangpan et al., 2017) which employed the IASC’s (2007) intervention framework to present the available evidence. Lastly, the potential role for clusters is apparent from the systematic review of humanitarian work in inaccessible conflict settings, which was supported by the World Health Organisation as the Health Cluster Lead Agency and drew on guidance and research evidence (Chaudhri et al., 2017).
Learning how to deliver humanitarian aid, and about the effects it has, benefits from applying the principles of systematic reviewing to a broad range of documents that report research, policies, practices or discussions. Although the field of Evidence-based Humanitarian interventions is relatively young, all the basic components that can allow the emergence of a specialised evidence ecosystem are already in place. At this stage, it seems crucial to focus on inter-sector collaboration and cross fertilisation of ideas. In this way academics and humanitarian agencies working together can strengthen the links between their two worlds to develop further evidence-informed humanitarian aid.
About the image: Internally displaced person’s camp in Dili, Timor-Leste, 2006. © Anthony Zwi.
About the author
Professor Sandy Oliver is the Director of the Social Science Research Unit at UCL, and Deputy Director of the EPPI-Centre.
Akl EA, El-Jardali F, Bou Karroum L, El-Eid J, Brax H, Akik C, et al. (2015) Effectiveness of Mechanisms and Models of Coordination between Organizations, Agencies and Bodies Providing or Financing Health Services in Humanitarian Crises: A Systematic Review. PLoS ONE10(9). DOI: 10.1371/journal.pone.0137159.
Bangpan M, Dickson K, Felix L, Chiumento A. (2017). The impact of mental health and psychosocial support interventions on people affected by humanitarian emergencies: A systematic review. Humanitarian Evidence Programme. Oxford: Oxfam GB.
Chaudhri S, Cordes K, Miller N (2017) Humanitarian programming and monitoring in inaccessible conflict settings: A Literature Review. WHO - Health Cluster.
Doocy S, Tappis H. (2016) Cash-based approaches in humanitarian emergencies: a systematic review. 3ie Systematic Review Report 28. London: International Initiative for Impact Evaluation (3ie)
Fulu E, Kerr-Wilson A, Lang J. (2014) What works to prevent violence against women and girls? Evidence Review of interventions to prevent violence against women and girls. DFID, London.
Inter-Agency Standing Committee (2007) IASC Guidelines on Mental Health and Psychosocial Support in Emergency Settings. Geneva: IASC.
Inter-Agency Standing Committee (2015) Guidelines for Integrating Gender-Based Violence Interventions in Humanitarian Action: Reducing risk, promoting resilience and aiding recovery. Geneva: IASC.
Lopes Cardozo MTA, Higgins S, Maber E, Brandt CO, Kusmallah N, Le Mat MLJ. (2015) Literature Review: Youth Agency, Peacebuilding and Education, Research Consortium Education and Peacebuilding, University of Amsterdam.
Lotfi T, Bou-Karroum L, Darzi A, Hajjar R, El Rahyel A, El Eid J, Itani M, Brax H, Akik C, Osman M, Hassan G, El-Jardali F, Akl E. (2016) Coordinating the Provision of Health Services in Humanitarian Crises: a Systematic Review of Suggested Models. PLOS Currents Disasters. Edition 1. DOI: 10.1371/currents.dis.95e78d5a93bbf99fca68be64826575fa.
Obuku EA, Stewart R, Mijumbi R, Ochan M, Achana F, Akena D, Nakitende A, Ssemata A, Kinengyere A, Semakula D, Ssenono R, Nsangi A, Lalitha R, Mwesiga E, Akite J, Basaza R, Newbrander W, Okello J, Sewankambo N, with Dickson K, Oliver S. (2017) Working with non-state providers in post-conflict and fragile states in primary healthcare service delivery: a systematic review. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London.
Spangaro J, Zwi A, Adogu C, Ranmuthugala G, Davies GP, Steinacker L (2013) What is the evidence of the impact of initiatives to reduce risk and incidence of sexual violence in conflict and post-conflict zones and other humanitarian crises in lower and middle-income countries? A systematic review. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.
The Global Women’s Institute and International Rescue Committee. (2016) Evidence brief: What works to prevent and respond to violence against women and girls in conflict and humanitarian settings? Washington DC: George Washington University and London: IRC.
Yates, T., Allen, J., Leandre Joseph, M. and Lantagne, D. (2017). WASH interventions in disease outbreak response. Humanitarian Evidence Programme. Oxford: Oxfam GB.