EPPI-Reviewer contains a number of automation tools -:

• Deduplication (not glamorous, but important!)
• Automatic clustering of studies
• 'Priority screening' for screening efficiency
• Six study type classifiers (RCT, systematic review, economic evaluation, COVID-19 categories, Long COVID, and PubMed Study Types)
• 'Robots': the RobotReviewer, and Human Behaviour Change Project tools
• Automated review updates using Microsoft Academic

Many of the tools are extremely useful and can save valuable time. Some should also be considered as being under development and tested carefully before you use them in a review.

Machine Learning, Priority Screening and Classifiers

We have six pre-built machine learning classifiers in EPPI-Reviewer (with more in development). They are:

• The Cochrane RCT classifier, which divides records into two categories: those that are highly unlikely to be randomized trials and those that might be. This is a sensitive classifier, calibrated to achieve 99% recall.
  • An additional classifier that identifies randomized trials, but returns probabilities in order for users to determine their own thresholds for use. (This classifier was built with the same data as the Cochrane RCT Classifier: data from Cochrane Crowd.)
• Two classifiers with data kindly supplied by the Centre for Reviews and Dissemination at the University of York: one to classify systematic reviews; and the other, economic evaluations.
• Two classifiers that have been built with data from our ongoing work of COVID-19 evidence survillance: one to classify research into the categories used in our map; and the other to identify research on Long COVID.
• A classifier that attempts to identify the type of study based on the title and abstract, and assign records into categories based on PubMed indexing. Documentation on this classifier can be found here.

A specific guide to the Machine Learning facilities available within ER Web is available here. A separate document on Priority Screening can be found here.

OpenAlex

During 2020 and 2021 we used the Microsoft Academic dataset to keep reviews and maps up to date automatically. We have now moved this facility over to use the OpenAlex dataset from its end-January 2022 update. We have a recent presentation on using OpenAlex available here.

Our introduction to using OpenAlex in EPPI-Reviewer is available here, with further information available here. (These will shortly be updated to reflect recent changes made in the user interface and move from MAG to OA.) 

The 'Robots' in EPPI-Reviewer

EPPI-Reviewer enables you to use one automatic classification tools to characterise the research included in documents uploaded into the system (with one under development). You can access RobotReviewer in the same area as you upload documents to an individual record. Clicking 'Robot' will open a dialog box where you can select the robot you want to use, and the coding tool where you want to place the results.

For the RobotReviewer robot, you need to have copied the 'RobotReviewer' coding tool into your review. (To use the Human Behaviour Change robot, when available, you need to have brought the 'Human Behaviour Change Project prioritised codes' into your review.)

RobotReviewer can characterise important aspects of randomised trials of health interventions. You may find it particularly useful for assessing the study's risk of bias. It also identifies the 'PICO' characteristics of the study.  For further information about RobotReviewer, please visit the RobotReviewer website.

The Human Behaviour Change robot is still under development, and this early version is currently being evaluated. It can also characterise randomised trials - most accurately in the field of smoking cessation. It is particularly focused on identifying the specific approaches to behaviour change that are used in the interventions evaluated in randomised trials. For futher information about this robot, please see the project website, and its growing set of papers on Wellcome Open Research.