This report presents the findings of a methodological study on the inclusion of diverse groups of children and young people in health promotion (HP) and public health (PH) research.
The need for diversity in research is increasingly recognised by researchers, funders and other stakeholders. Recent and ongoing changes to equalities legislation and health policy in the UK foreground the importance of the inclusion in research of diverse groups. An inclusive approach to research also improves the generalisability of research findings. However, there is relatively little guidance for researchers as to how diversity in research might be promoted. While valuable and relevant work has been done, few researchers have attempted to draw together material relevant to diversity across different populations and research designs. This report describes such an attempt.
We reviewed the methodological and conceptual literature on diversity, and identified ways in which it is appropriate for scientific and/or ethical reasons that HP and PH research should include socially diverse populations. They looked at 174 studies included in nine previous systematic reviews concerned with HP and PH research relating to children and young people. We found a mixed and patchy picture. Many studies do not provide sufficient socio-demographic detail to judge the nature of research samples. Rationales for sample selection are often unclear, and attention to diversity issues in data analysis appears to be driven more often by convenience than by science. Guidelines are suggested for improving the value and generalisability of HP and PH research – for all age groups as well as children and young people – by helping researchers to think critically about how social diversity can most appropriately be reflected in policy-relevant research.
Background
There are both scientific and ethical reasons why research should reflect the diverse and multicultural nature of many societies today. The inclusion of diverse groups in research increases the generalisability to the population at large of research findings, and helps to provide valuable information about differential patterns of ill-health and wellbeing. Apart from the benefit which people may gain directly from being involved in research, everyone has an equal right to contribute to research findings: the exclusion of certain groups is both unjust in itself and likely to contribute to injustice on the broader socio-political level. These rationales for the inclusion of diverse groups in research have been given added impetus in recent years in the UK by the development of a legal and policy agenda for diversity.
Previous studies have investigated the inclusion of diverse groups in research and found that this varies enormously. Poor reporting often makes it difficult to tell who was included. Most of this research has focused on clinical trials. Research using other methodologies, such as surveys and the many forms of qualitative research also needs to reflect diversity if it is to be reliable, ethically sound and relevant to policy and practice.
Research questions
This review seeks to answer the following two questions:
- What methods are appropriate for sampling, recruiting and retaining diverse groups of children and young people in HP and PH research, as well as for collecting and analysing data?
- To what extent have these methods been used in HP and PH research with children and young people?
The review was conducted in two parts. Firstly, theoretical and methodological literature was drawn upon to develop a framework for understanding diversity in research. Secondly, 174 studies included in nine previous reviews conducted by the EPPI Centre were reanalysed to determine how HP and PH research has addressed diversity issues. The studies included in these nine reviews focused on HP and PH research related to mental health, healthy eating, physical activity, walking and cycling, accidental injury, and teenage pregnancy and parenthood. All the reviews were concerned with research involving children and young people. The studies included a range of research designs: 53 were evaluations of HP and PH interventions, using a controlled or randomised controlled trial design; and 121 studies examined people’s views and experiences using a range of quantitative and qualitative methods (surveys, interviews, ethnographic and action research).
Findings
It was found that reporting of demographic data on research participants is frequently insufficient to determine whether or not studies included diverse groups of children and young people.
In around one in twenty studies, information was lacking on the number of participants or their age or gender; more than half provided no data on ethnicity, and more than two out of five gave no information about participants’ socioeconomic position. Measures used (for instance relating to socioeconomic position) were often different between studies, making comparison of samples and findings difficult. Sampling strategies were unclear in most of the studies. A common claim, especially in the views studies, was that participants represented diverse groups, but data supporting this claim was often missing. More than one in ten of the studies did not specify the settings used to recruit participants, and over half did not mention whether or not consent was obtained. It was uncommon for the tools and measures used to have been validated with diverse groups. Subgroup analysis was carried out in half of the intervention studies, but, in most of these, no rationale was given. In almost half the qualitative studies which quoted material directly from interviews or focus groups with participants, no demographic descriptors were offered, and a similar proportion of studies gave no clear description of data analysis methods. In general, the intervention studies were better reported and paid more attention to diversity issues than the views studies.
Conclusions
Different areas of research vary widely in the extent to which they have investigated diversity-related questions, and it is difficult to establish a single set of methods or criteria that can capture the diversity-related questions relevant to research in the HP and PH field. The most appropriate means of achieving diversity will vary depending on the research questions or hypotheses, the methods employed, and the population and setting under study. At present, poor reporting makes it difficult to judge the inclusiveness or appropriateness to diversity issues of much HP and PH research.
A distinction needs to be drawn between research whose central questions focus on diversity, or on particular disadvantaged groups, and research whose questions relate to the general population. Much discussion of diversity in research to date has focused on the former, without considering implications for the latter. Including diverse groups in research means both conducting highly focused research with particular groups, to ensure that diverse voices are heard, and conducting more broadly framed research projects which genuinely reflect the diversity of the population.
This review identified areas in which research practice could improve. We recommend that it should be normal practice for all research to report major demographic factors for samples at baseline. In terms of strategies, to widen the research base, it may be useful for researchers to employ specific strategies such as, for example, testing sampling and recruitment strategies for their capacity to promote the inclusion of diverse groups; recruiting outside institutional settings, such as schools; and using data collection tools piloted with, or previously validated for, diverse populations.
More detailed reporting of sampling procedures (both of institutional settings, where relevant, and of individuals) would be valuable for all study designs, as would reporting of data analysis methods in qualitative research. More empirical work is needed on the effect of different research practices, particularly at the sampling and recruitment stages, on the diversity of samples obtained and the contribution of diverse groups to research findings. Other areas in which methods development would be valuable include sampling strategies for trials and qualitative research; how research synthesis, both quantitative and qualitative, can take account of diversity; the balance between foregrounding diversity and observing common themes in the collection and analysis of qualitative data; and the formation of research agendas and the research-policy relationship.
The contribution of this review
The work described in this report is a contribution to ongoing debates about the extent to which the social diversity of populations is, or might be, reflected in research, in order to improve the scientific credibility and policy relevance of research findings. The review focuses on HP and PH research related to children and young people, but many of the issues discussed here are relevant to social research more generally.