Version 4.6.2.0 includes important new features, opening up some of the machine learning abilities of EPPI-Reviewer to all users. A new GRADE assessment form is available in the Meta-Analyses section. Some other tweaks and improvements are included as well. Note that the "priority screening" features are still made available on request in order that we can support how they are set up. Version 4.6.2.1 fixes a single bug.

New Features.

Machine learning Classifications:

In the "Codes" tab, a new "Machine Learning Classification" button is now present on the top-right toolbar. This provides access to Machine Learning features. It can be used to apply a pre-existing model, designed to identify Randomised Controlled Trials (RCT); it can also be used to build new, user-specified models and then apply them to lists of references.

General:
the machine learning system available in the EPPI-Reviewer classification system belongs to the general "supervised" machine learning family of technologies. Its use requires two phases.
First stage (training): the machine needs to be ‘taught’ how to recognise the studies it is looking for (for example, needs to be shown lots of examples of RCTs and non-RCTs in order to learn how to tell the difference between them). The machine learning algorithm receives title, abstract and keywords for a list of reference, along with their pre-established classification (whether they have or don't have the characteristic shown – in the above example, whether they are, or are not, RCTs). Based on these data, the machine learning algorithm will ‘learn’ how to distinguish between the two classes, and create and save a "model" which can then be used to classify other references.
Second stage (classify): once a model is available, data about references (again: Title, Abstract and Keywords) can be used to "Apply" the model. The result will be an ordered list of references, ordered by the probability (score) of their having the modelled characteristic. The probability score will also be returned, providing a quick heuristic way to evaluate two things: if the model performance is likely to be good and if the returned scores are widely distributed. Scores ranging between 99 (%) and 0 (%) usually indicate that the model was able to distinguish the chosen quality from studies without that characteristic quite well. Scores grouped around 50 (%) normally indicate that the model couldn't discriminate well between the two classes.

Randomised Controlled Trials (RCT) model: EPPI-Reviewer now includes a pre-trained model (stage 1, training) based on more than 280,000 references screened by the Cochrane Crowd. This model is available to all reviews, allowing people to skip the training stage and apply the model directly. For details on how to get the modelling results see "Stage 2" below.

Custom Models: users can now create and apply their own models.

Stage 1:
First of all, training data must be prepared. This is done in a screening-like fashion. Two codes must be created, one representing references with the characteristic of interest, and the other those that don't. The quality of the sample of data which will be used for training is critically important for the performance of the classifier; it should be representative of the studies on which it will be applied, so it might be a good idea to think of labelling manually a random sample of the references that will be classified. Once at least 5 references have been applied to both codes, a model can be created.
Once the preparation is over, the Machine learning window will allow you to select the two codes used above. Type in a name for your custom model and click "Create Model". You can’t see what happens next, but first, the data are uploaded to the machine learning environment on Azure; and then the technology gets to work, ‘building’ your new model. This process can take a few minutes or much more (depending on how much data are being used). On the "stage 2" side, the "refresh" button allows you to track progress. Occasionally, building a new model will fail and the model will be listed with the "(failed)" suffix. This almost always happens because not enough data were provided.
Stage 2:
Once a custom model is ready, you will be able to apply it to all items in your review, or to the ones applied to any code of choice, or to the items imported with a given source. You will need to select a custom model from the list, indicate to what items it should be applied to and then click "Apply model". Close the window and go to the search tab. The machine learning results may take some time (a couple of minutes at minimum) to appear in the form of a new search. Listing the new search results will show the main documents list with an additional "Score" column. This column will show the scores returned by the classifier, allowing to quickly get an idea of the model performance (see above). Items in the list will be ordered by score, from higher to lower. How to use these results depends on specific requirements, naturally.

GRADE assessments:

The GRADE evaluation tool is a well-known, widely used standard to evaluate the quality/reliability of evidence. EPPI-Reviewer now allows you to use the GRADE system to rate the reliability of data included in Meta-Analyses, via a new tab in the "Edit/Run Meta Analysis" Window. We have designed this screen to be completely compatible with the MAGICApp platform for guideline development. Our next stage of development will support the export of structured data from your meta-analyses and GRADE assessments straight into MAGIC. A new ad-hoc report is available to export the grade data, via a "GRADE report" button found in the GRADE tab, bottom toolbar. Note that GRADE data are only saved when users click "Save Changes" (unlike other EPPI-Reviewer features where data is saved in real time).

Main documents list, "Info" column:

Users can control what columns to show in the main documents list via a "Select the field you want to display below" button in the main toolbar. A new column is now available, called "Info". This column will show the contents of "Info" boxes when items are listed based on a particular codes ((i.e. Right click a code on the code tree on the right hand side of the screen and "List items with this code"). Whenever data are present in the Info box for a given code and reference combination, it will be shown in the "Info" column.

Main Documents list, new "Export" option:

The "List formatted bibliography / Export table" button ("Print" icon, main toolbar) allows you to export listed/selected references in a few different formats. Starting from the current version a new "Export Current Page/Selection" option is available. This produces an HTML file, which will show the current page / list of references (aor "selected" references) in tabular form. This format might be useful for re-use via external tools (Word, Excel, SPSS, etc.) especially in combination with the new "Info" column mentioned above or for the export of classifier-assigned scores.

Version 4.6.2.1 Bugfix:

The filter box in the main toolbar stopped working, version 4.6.2.1 fixes the problem.