What do we want to know?
Web based research evidence portals provide decision-makers access to research findings without needing to consult the research. These portals can be useful for those using research findings to help inform policy, practice or personal decision making.
But for evidence portals to be useful and widely-used they need to be credible. We need to know how trustworthy and relevant the findings are. That is what evidence standards are being used. This study therefore summarizes and assesses the evidence standards being used by 14 English language evidence portals in different areas of social policy.
Who wants to know?
The study was funded by the Centre for Homelessness Impact. The Centre is building a web based evidence portal (the ‘Intervention Tool’) which will be informed by learning from the practices of existing portals. This study is also relevant to others creating or using web based evidence portals across social policy fields.
What did we find?
Evidence standards are primarily being used for evidence claims about the effectiveness of different social interventions. The evidence standards are for different levels of assessment, including: (1) the individual research studies; (2) reviews of an evidence base bringing together what is known from many studies; and (3) for guidance on recommended policy and practice informed by an evidence base. Portals vary on the level at which they make evidence claims and the sophistication of the evidence standards used at each level.
What are the implications?
We make seven recommendations for those creating evidence portals: 1: Specify the purpose and methods of making evidence claims and ensure consistency across levels of evidence (guidance, evidence base, included studies). 2: Consider using broad rather than narrow evidence base questions. 3: Use explicit rigorous methods of evidence synthesis to make claims about the existing evidence base. 4: Specify and justify the different evidence standards for making different claims about the existing evidence base including impact, strength, extent and consistency of evidence, process and contexts and costs. 5: Specify and justify evidence standards for included studies. 6: Specify methods as well as criteria for achieving evidence standards. 7: Develop methods and standards for policy and practice guidance informed by the evidence base.
How did we get these results and what are the key limitations?
The study is based on a purposive sample of 14 English language web based research evidence portals. The evidence standards reported by the portals were summarised using a structured framework. The structured information on the web presentation of each the portals is provided in Appendix 1. A structured summary of the nature of the portals’ evidence standards is provided in Appendix 2. The structured information was used to inform a brief discussion of the nature and variation in evidence standards in the portals.
The study is based on a purposive sample to provide an insight into the range of approaches to evidence standards being used. It is therefore not a comprehensive survey of approaches used by evidence portals.
The report is also available on the Centre for Homelessness Impact website via: https://www.homelessnessimpact.org/post/evidence-standards-in-web-based-research-portals
For more information please email David Gough: david.gough@ucl.ac.uk
This report should be cited as: David Gough, Howard White (2018) Evidence standards and evidence claims in web based research portals. London: Centre for Homelessness Impact. ISBN: 978-1-9995928-3-7
© Copyright 2018