About the long COVID 'segment' of the COVID-19 Living Map

Overview
This 'segment' contains bibliographic records of articles included in our living map of COVID-19 research [1] that are about long COVID - partitioned by 'Topic'.

Our (much broader) living map contains >200,000 research articles reporting primary empirical data, systematic review, modelling, full economic evaluation, or novel analysis on COVID-19. (Please see 'Introduction' here for full details of methods used to identify eligible articles for inclusion in the broader map).

This long COVID 'segment' is restricted to studies concerning people who have had SARS-CoV-2 infection and who have any symptoms at >4 weeks after diagnosis. It does not aim to include studies that investigate antibody response >4 weeks if no information on symptoms is reported. Studies investigating recurrence of symptoms are included in the 'segment', while those investigating reinfection or test positivity alone are not intended to be included.

The current version of this 'segment' - published on 10th March 2023 - encompasses research articles about long COVID included in our living map up to Version 101 (also published on 10th March).

Until April 2022, we published an updated version of this 'segment', including further research articles on long COVID, approximately every four weeks. From May 2022 onwards, we publish further, regularly updated versions of the 'segment' when ready (i.e. each time a new set of records from each sequential update of the OpenAlex dataset has been auto-coded and added to this 'segment'). See Identifying articles on long COVID, below for further details.

Suggested Citation
Lorenc T, Khouja C, Raine G, Shemilt I, Sutcliffe K, D'Souza P, Burchett H, Hinds K, Khatwa M, Macdowall W, Melton H, Richardson M, South E, Stansfield C, Thomas S, Kwan I, Wright K, Sowden A, Thomas J (2020). COVID-19: living map of the evidence. London: EPPI-Centre, Social Science Research Unit, UCL Social Research Institute, University College London.

Using the long COVID 'segment'
There are various ways to use the long COVID 'segment' of the living map. Like the broader living map, this 'segment' is organised by 'Topic' and 'Version'.

The number of tiles in Mosaic view shows the number of studies in each cell. To view these records, click on the cell of interest. You can also click on a column or row header to view all the records relating to that row or column.

To view the full set of records, click on the Reader tab above the Map. Records can be filtered, and exported as a RIS file, from here. Further details about each study can be found by clicking on the URL(s) and exploring the corresponding full text, where available.

Each record (study) is assigned to one of eleven 'Topic' codes, based on the main focus of the study. Records in the long COVID 'segment' can therefore be filtered by topic. Topic codes are:

- Treatment Evaluation
- Transmission / Risk / Prevalence
- Diagnosis
- Health Impacts
- Vaccine Development
- Treatment Development
- Genetics / Biology
- Case Reports (Patients)
- Case Study – Organisation
- Social / Economic / Indirect Impacts
- Mental health Impacts

In addition to filtering records by 'Topic', records can also be filtered by the 'Version' of our living map in which they were first published (or by 'All Versions').

Identifying articles on long COVID
Version 73 to Version 97
Prospective - Semi-Automated
From September 2021 (Version 73) until April 2022 (Version 97) we prospectively applied a supplementary code 'About long COVID' to records of studies manually selected for inclusion in our living map of COVID-19 research (i.e. in addition to assigning each eligible record to one of eleven (primary) ‘Topic’ codes – see above). Each record was coded by a single researcher. If a researcher was unsure about whether a study was about long COVID (or about which ‘Topic’ code to assign) they had the option of referring it for a ‘second opinion’, to be resolved by team discussion and consensus.

For the inaugural version of the long COVID 'segment' (up to Version 75), records prospectively coded as 'About long COVID' were also reviewed by a second researcher before being added to this 'segment', by 'Topic' and 'Version'. From the first updated version (up to Version 79) until the last version that included manually coded records (Version 97), we automatically assigned records prospectively coded as 'About long COVID' to the 'segment'.

During the same period, to safeguard against missing 'new' articles about long COVID in our prospective study identification workflow, we also periodically searched for, and then reviewed, records of articles published in recent versions of our living map without the supplementary code in EPPI Reviewer-Web [2] using the following keyword search strategy:

TITLE OR ABSTRACT=("long covid" OR "long term covid" OR "post covid syndrome" OR "post covid 19 syndrome" OR "post acute” OR "PASC" OR "chronic covid" OR "ongoing covid" OR "long haul" OR "long hauler" OR "post discharge" OR "postdischarge" OR "long term symptom" OR "persisting symptom" OR "persistent symptom" OR "prolonged symptom" OR "long term sequelae") OR TITLE=( "long term effect" OR "long term impact" OR "long tail")

Once reviewed (including referral for 'second opinion' if needed) any further records about long COVID were added to the 'segment', by 'Topic' and 'Version'.

From December 2021 (Version 83) onwards, we have been using a Bidirectional Encoder Representations from Transformers (BERT) model [3] to automatically assign some records from each sequential update of the OpenAlex (formerly MAG – see below) dataset to either one of our eleven (primary) 'Topic' codes, or an 'exclude' code (see our living map of COVID-19 research for further details), with other records retained for manual screening-coding by a researcher, as before (as described above in this section). From December 2021 (Version 83) until March 2022 (Version 97), those records automatically assigned to one of the eleven (primary) 'Topic' codes by the main BERT model – which were then published in our main living map – were also sent to a supplementary, semi-automated workflow, comprising use of: (i) a bespoke BERT model (binary classifier) that automatically assigns the supplementary code 'About long COVID' to some records; (ii) a bespoke machine learning classifier designed to identify records of articles about long COVID (see our living map of COVID-19 research for further details), combined with the keyword search strategy shown above in this section; and (iii) manual screening of selected (prioritised by ii) records. Further records about long COVID that are identified using this supplementary, semi-automated workflow were also added to the 'segment', by 'Topic' and 'Version'. After Version 97 (March 2022) we discontinued manual coding. We continue to import records when the OpenAlex dataset is updated, and upload those records coded by the BERT model.

Following its final update (on 6th December 2021), the MAG dataset and the academic.microsoft.com website have been discontinued. We have therefore migrated our study identification workflows for the main living map of COVID-19 research from MAG, to the OpenAlex dataset (MAG’s drop-in replacement) from January 2022.

On 28th February 2022, we added 41 further study reports (articles) to the long COVID ‘segment’ that have been identified as eligible for consideration in an update of a living systematic review of research on long COVID [4], of which: 32 were already included in our broader C-19 living map but not yet in the long COVID ‘segment’; and 9 were not yet included in our broader C-19 living map (now added).

Retrospective - Manual
For the inaugural version of this long COVID 'segment' (up to Version 75), we also needed to identify articles about long COVID that had been published in previous versions of our living map of COVID-19 research from its inception up to Version 72. We therefore searched the map for these articles in EPPI Reviewer-Web using an earlier version of the keyword search strategy shown above (available from the authors on request). Retrieved records were then reviewed (including referral for 'second opinion' if needed) and those records about long COVID were added to the 'segment' by 'Topic' and 'Version'. In addition, we identified, coded and added (both to this 'segment' and to the broader living map) a set of study reports (articles) included in a recently published version (27th September 2021) of a living systematic review of research on long COVID [4] that were not already included in our living map of COVID-19 research.

Version 98 Onwards
Prospective – Fully Automated
From May 2022 (Version 98) onwards we discontinued manual coding for the main living map of COVID-19 research and suspended manual coding for this long COVID 'segment'. We continue to import records from automated searches of each sequential update of the OpenAlex dataset and we add those records automatically coded by our machine learning classification (BERT) models as ‘included’ (i.e. assigned to one of eleven 'Topic' codes and ‘On topic for long COVID’) to the main living map and to this ‘segment’.

N.B. The study identification workflow for our main living map of COVID-19 research also incorporates bespoke, supplementary automated searches of sequential updates of the OpenAlex dataset, which are designed to increase coverage of relevant articles in this long COVID 'segment'.

Results
The inaugural version of the long COVID 'segment' (up to Version 75, published 7th October 2021) contained 261 records of research articles about long COVID, partitioned by 'Topic' and 'Version'.

The first update of the long COVID 'segment' (up to Version 79, published 4th November 2021) contained 470 records of research articles about long COVID, partitioned by 'Topic' and 'Version'.

The second update of the long Covid ‘segment’ (up to Version 83, published 9th December 2021) contained 816 records of research articles about long Covid, partitioned by 'Topic' and 'Version'.

The third update of the long Covid ‘segment’ (up to Version 85, published 6th January 2021) contained 1,362 records of research articles about long Covid, partitioned by 'Topic' and 'Version'.

The fourth update of the long Covid ‘segment’ (up to Version 89, published 3rd February 2022) contained 1,646 records of research articles about long Covid, partitioned by 'Topic' and 'Version'.

The fifth update of the long Covid ‘segment’ (up to Version 93, published 4th March 2022) contained 2,292 records of research articles about long Covid, partitioned by 'Topic' and 'Version'.

The sixth update of the long Covid ‘segment’ (up to Version 97, published 31st March 2022) contained 2,454 records of research articles about long Covid, partitioned by 'Topic' and 'Version'.

The seventh update of the long Covid ‘segment’ (up to Version 98, published 5th May 2022) contained 2,488 records of research articles automatically classified as being likely to be about long Covid, partitioned by 'Topic' and 'Version'.

The eighth update of the long Covid ‘segment’ (up to Version 99, published 19th October 2022) contained 3,362 records of research articles automatically classified as being likely to be about long Covid, partitioned by 'Topic' and 'Version'.

The ninth update of the long Covid ‘segment’ (up to Version 100, published 20th January 2023) contained 4,095 records of research articles automatically classified as being likely to be about long Covid, partitioned by 'Topic' and 'Version'.

The tenth update (current version) of the long Covid ‘segment’ (up to Version 101, published 10th March 2023) contains 4,646 records of research articles automatically classified as being likely to be about long Covid, partitioned by 'Topic' and 'Version'.

Funding
The living map of COVID-19 research, including this long COVID 'segment', was commissioned by the National Institute for Health Research (NIHR) Policy Research Programme (PRP) for the Department of Health and Social Care (DHSC) and Public Health England (PHE). It is funded through the NIHR PRP contract with the EPPI Centre at UCL (Reviews facility to support national policy development and implementation, PR-R6-0113-11003). Any views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the DHSC.

Conflicts of interest
None.

Contributions
Any opinions expressed in this publication are not necessarily those of the EPPI Centre or the funders. Responsibility for any views expressed remains solely with the authors.

References
[1] Lorenc T, Khouja C, Raine G, Shemilt I, Sutcliffe K, D'Souza P, Burchett H, Hinds K, Khatwa M, Macdowall W, Melton H, Richardson M, South E, Stansfield C, Thomas S, Kwan I, Wright K, Sowden A, Thomas J (2020). COVID-19: living map of the evidence. London: EPPI-Centre, Social Science Research Unit, UCL Social Research Institute, University College London.

[2] Thomas J, Graziosi S, Brunton J, Ghouze Z, O'Driscoll P, Bond M (2022). EPPI-Reviewer: advanced software for systematic reviews, maps and other evidence synthesis [Software]. https://eppi.ioe.ac.uk/CMS/Default.aspx?alias=eppi.ioe.ac.uk/cms/er4

[3] Devlin J, Chang M-W, Lee K, Toutanova K (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805

[4] Michelen M, Manoharan L, Elkheir N, Cheng V, Dagens A, Hastie C, O'Hara M, Suett J, Dahmash D, Bugaeva P, Rigby I, Munblit D, Harriss E, Burls A, Foote C, Scott J, Carson G, Olliaro P, Sigfrid L, Stavropoulou C. (2021). Characterising long COVID: a living systematic review. BMJ Global Health 6: e005427.

Copyright
Authors of the systematic reviews on the EPPI-Centre website [http://eppi.ioe.ac.uk] hold the copyright for the text of their reviews. The EPPI-Centre owns the copyright for all material on the website it has developed, including the contents of the databases, manuals, and keywording and data-extraction systems. The centre and authors give permission for users of the site to display and print the contents of the site for their own non-commercial use, providing that the materials are not modified, copyright and other proprietary notices contained in the materials are retained, and the source of the material is cited clearly following the citation details provided. Otherwise, users are not permitted to duplicate, reproduce, re-publish, distribute, or store material from this website without express written permission.

The NIHR Policy Research Programme Reviews Facility is a collaboration between:

- UCL Social Research Institute
- EPPI Centre
- London School of Hygiene & Tropical Medicine
- University of York Centre for Reviews and Dissemination