This separate ‘sector’ of our living map of research on generative large language model- (LLM-) based tools for health and social care applications contains bibliographic records of studies that focus on one or more of the following classes of applications of this type of artificial intelligence (AI) tools: medical education, health or social care professional development and training; answering medical (and related) exam questions/ answering medical questions in general; and/ or answering patient or service user questions/ patient or service user education -- without also focusing on clinical health care, public health, or social care applications. The records in this 'sector' have minimal coding assigned to them, compared with records published in the 'main map'. Currently, this 'sector' of our living map contains more records than the 'main map'. 

How to cite this 'sector' and the main living map

Shemilt I, Hollands GJ, Khouja C, Raine G, Thomas J. (2024). Generative large language models for health and social care applications: a living map of research. London: EPPI Centre.

The published protocol that covers this map is available here.

How to use this 'sector' of the map

There are several ways for users to interact with this 'sector' of our living map using this web database and with the bibliographic records it contains. Please note that for the 'application class(es)' code set (dimension) all applicable 'child' codes have been assigned to each record, so the frequencies of those codes will not add up to the cumulative total number of records in the current version of this 'sector'.

Quick Start

There is one pre-configured map in this 'sector' (web database) that can be displayed by clicking on the 'View map' button located directly below this information pane. The title of the pre-configured EGM shows its x- and y-axes (columns and rows) and its 'segments', which are each drawn from among the expandable lists in the left-hand pane of the web database. The records are displayed in bubbles in each cell of the pre-configured map with sizes proportionate to the number of articles of different types. Different bubble colours represent the different article types (the 'segments'). Only one article type code is assigned to each record so the frequencies of the article type codes should add up to the cumulative total number of records in the current version of this 'sector'. Select 'Home' (top right) to return to the previous screen. 

Lists records and get frequencies

To view a list of all records in this 'sector', click on the button in the top right-hand corner of the screen 'All records'. Every record has been assigned to codes in the expandable lists in the left-hand pane of this web database. Use the black triangles to expand each code set to display the codes. Then, for example, to view a list of all the records that are about the use of generative LLMs for answering medical (and related) exam questions, click on the latter code in the 'application class(es)' code set and then on the button above the headings 'List records'.

To see how many records have each of a set of codes assigned to them (frequencies), highlight the 'parent' code (e.g. 'application class(es)') and click on the 'Frequencies' button above. This will open a frequencies table located below the pre-configured map in the central panel (scroll down). You can then list each set of records by clicking on the corresponding number, or switch to displaying the frequencies as a bar chart, or pie chart.

Record details view

Clicking on an individual record in any list will show detailed information about that record. This defaults to basic bibliographic information, with the abstract when available, but can be expanded to include all bibliographic fields in the database. Clicking the 'show coding' button will open up the display to show the individual codes assigned to that record. Bibliographic fields include the DOI link when available, providing an easy pathway to locating the full-text article in most cases. 

Export references

Using the buttons at the top of the list to the left-hand side, the bibliographic records displayed can be downloaded as a plain text file (Citations), in Excel format (Excel), or as a RIS file (just one page - RIS (page); or all records - RIS (all)) for importing into reference manager software (such as Zotero or EndNote).

Get crosstabs

To show a crosstabulation of frequencies for all codes in a code set in this 'sector' (e.g. 'application class(es)') across the top (x-axis/ columns) and all codes in another code set (e.g. 'article type') down the left-hand side (y-axis/ rows), click on the heading for the first code set (headings in the left-hand pane) and, at the bottom right of the screen, click on the 'Set X axis' button; then click on the heading for the second code set (left-hand pane) and click on the 'Set Y axis' button; finally, click on the 'Get Crosstab' button (bottom right of the screen). This will display a matrix (crosstab) of the intersections of the codes in the two selected code sets. By clicking on the 'Bubble map' button, this matrix can be changed from a table to a bubble map, with bubble size indicating the number of records in each cell. The numbers in the table or the bubbles are clickable, and clicking on them will display a list of the records in that bubble or cell. This list will appear below the table or bubble map.

Self-cross tab will display all the options within a category against the same category - for whichever category is selected as the x-axis.

Search

There is a free-text search box at the top of the 'Home' screen. This defaults to searching the title and abstract fields, but specific fields can be selected using the drop-down menu next to it.

Acknowledgements

We would like to thank members of our project advisory group for their helpful suggestions on the presentation of this living map, including its evidence and gap maps: Caroline Green (University of Oxford), Chloe Park (Department of Health and Social Care), Ellen Coughlan (The Health Foundation), Iain Marshall (King's College London), Nell Thornton (The Health Foundation), Pritesh Mistry (The King's Fund) and Sam Taylor (Department of Health and Social Care). We would also like to thank our EPPI Centre colleagues who were co-authors of the protocol for this living map, and who also provided feedback on its presentation: Alison O'Mara-Eves, Dylan Kneale and Katy Sutcliffe. Finally, we would like to thank our other EPPI Centre colleagues for their feedback on our searches for eligible studies: Claire Stansfield and Hossein Dehdarirad.

Funding

This work has been commissioned by the National Institute for Health Research Policy Research Programme (NIHR PRP) for the UK Department of Health and Social Care (DHSC). It is being funded through the NIHR PRP contract with the EPPI Centre at University College London (UCL): Reviews facility to support national policy development and implementation, NIHR200701. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the DHSC.

Funder involvement

This work is part of an ongoing programme of research funded by the DHSC (see also ‘Contributions’, below). DHSC policy team members will be consulted at various stages to help us understand the policy context of the issue under study and identify the focus and scope of the analysis.

Conflicts of interest

None.

Contributions

This work is being undertaken by a team of researchers based at the EPPI Centre and the Centre for Reviews and Dissemination (CRD) under the auspices of the UK NIHR Policy Research Programme Reviews Facility (NIHR PRP Reviews Facility). The NIHR PRP Reviews Facility is a collaboration between the EPPI Centre, University College London, the Centre for Reviews and Dissemination (CRD), University of York, and the Department of Public Health, Environments and Society (PHES), London School of Hygiene and Tropical Medicine.