Version V 4.12.0.0 is a major upgrade that introduces a number of new features which we expect can have profound effects on how we all work.

First of all, it contains major improvements to our developing support for keeping reviews up to date using the latest technologies to minimise unnecessary manual work. These utilise the Microsoft Academic dataset. Many features have been redesigned and the User Interface (UI) has been simplified and made more powerful. As a result, we are now widening experimentation with these “keep your review up to date” features to all users, on request. Should you wish to experiment with them (see below for an overview), please get in touch and we'll activate them on a per-review basis.
This release also includes what we call EPPI-Visualiser (provisional name, we might change it in the future), which is a separate application that can be used to publish review data if, when and how review authors wish. Unlike the current webdatabases application (see here for an example), EPPI-Visualiser can be configured entirely from within EPPI-Reviewer Web. Like the webdatabases application, the data it provides access to is "live", obtained directly from the EPPI-Reviewer database in real time. The version released today is an "alpha" version (not feature-complete, still provisional) and therefore available "on invitation". We at the EPPI-Centre will use it without delay for multiple projects, including to complement the living systematic map of the evidence about COVID-19 research we maintain, please do watch that space!

Finally, this release includes a number of small improvements and bugfixes, mostly for EPPI-Reviewer Web.

Features to keep reviews up to date

Microsoft Academic Graph (MAG) is a large Open Access repository containing about 250 million bibliographic records of research articles from across science – connected in a large network graph of conceptual and citation relationships. It lists almost all research publications available anywhere on the World Wide Web and also describes various kinds of relationships between references. EPPI-Reviewer has been able to "tap in" this data for quite some time; however, until now, we have kept these features available "on invitation" only. This was necessary because the potential of these features is overwhelming, and thus we needed a fairly long period of experimentation, in order to identify the most useful ways to access and process the data, as well as to find out how to make these "ways" accessible to end-users.
With this release, we feel we've reached the stage where what is available is solid enough to be used by (potentially) all EPPI-Reviewer users. However, the number of features that this new section of EPPI-Reviewer contains is vast, and, behind the hood, extremely complex. Concurrently, formal evaluation studies of how effective the methods it enables are still ongoing, thus, we decided to make it available On Demand to reviewers who would like to try the system out on the understanding that this is still work in progress. We can enable MAG-features in a matter of a few clicks, but we do want end-users to contact us directly, so that we can share some documentation and discuss their use-case on an individual basis. This allows us to learn more effectively how people will use and how they understand the new functionalities and allows users to receive direct and customised guidance from us.

The MAG features are fairly "distinct" from the rest of EPPI-Reviewer: in EPPI-Reviewer Web, they are accessible via a button in the Review Home tab called "Update Review". In EPPI-Reviewer 4 the corresponding button is available in the "Documents" tab. Both versions provide access to the same functionalities.
The MAG UI is divided in "tabs", each with specific objectives and ways of interacting with MAG data. The purpose of "accessing" this data is ultimately to import references into EPPI-Reviewer itself. Thus, most people will use these features by eventually import lists of references from MAG into their "normal" review, where they could be screened "normally". Below, you can find a brief description of the new functionalities; more documentation, online help and how-to videos will follow in the next few days/weeks.

"Bring up-to-date" tab:

As the label suggests, the aim of this tab is to support users in bringing their reviews up to date. It enables users to "search" MAG data, based on the studies that are already present in the review. It uses the network of relationships between documents in MAG to find other references that may be relevant to the current review. For example, it finds references that cite references already screened and included, and/or references that appear to cover similar topics. It is possible to filter the documents returned by publication date, with the objective of using the studies already included in a given review to find related records that were published since the original searches were carried out. We will expand the search functionality here soon and include some r&d features for users to test out the performance of the automated searches in their reviews.

"Keep up-to-date" tab:

This tab allows users to "subscribe" to a service that will automatically "learn" the scope of a review and identify newly published records as and when they appear in the MAG dataset (currently, every two weeks). At each update, it produces a list of new candidate records that are "scored" using a machine learning model that represents how closely related they are to the items currently included in the review. It is possible then to refine these results by applying additional machine learning models and ordering / importing the results accordingly.

"Match records" tab:

The functionalities of the two previous tabs require a link between Items (references) as represented within EPPI-Reviewer and their corresponding record in MAG. This tab allows users to trigger the semi-automated task of producing this kind of "matching". It is possible to refine and to explore/evaluate the results. The "Item details" window of the "normal" EPPI-Reviewer, contains a corresponding tab that allows users to inspect the "matching" data on a per-reference basis and to manually validate or invalidate the matches proposed by the machine.
Note: the features offered in the previous two tabs will remain disabled until at least some matching data is present.

Search and Browse tab:

MAG data can also be searched via more traditional "text" based searches and complex Boolean logic. This tab also allows users to search MAG by topic, to combine searches via Boolean operations and to apply filters based on publication dates and publication types.

Browsing Lists of MAG references:

All the tabs provide access to different kinds of "lists" of MAG references. Whenever retrieved, these lists are shown in what we call the "MAG browser" (which does not have its own tab). The MAG browser is a complex component in its own right; it allows users to explore networks of references following "Cited by" lists (both directions), it lists and allows to explore topics that refer either to a single reference or the currently loaded list thereof and, while doing so, allows to "select" individual references for importing.

Show History tab:

The purpose of this tab is to allow researchers to quickly retrace their steps and move back and forth across different, previously visited, lists of references. It is important to note that this data is not saved and will be lost when the user logs out or changes review.

EPPI-Visualiser

The rising popularity of Evidence (Gap) Maps testifies the richness of the data produced while conducting a systematic review, as well as the importance of making this data visible and "explorable". Transparency and accountability are also important aspects of systematic reviewing work. The evidence maps that can already be produced via EPPI-Reviewer, while feature rich and extremely powerful, have two intrinsic limitations: they are based on data-snapshots (maps do not "update" automatically) and are (mostly) limited to 3 dimensions. All these factors point to the need for new ways to visualise and explore review data, which is why we have written EPPI-Visualiser.

Features include:

• Any review in EPPI-Reviewer can be used to "publish" one or more "Visualisations", each presenting arbitrary subsets of the review data, including "all data", if the review authors wish to make some/all data public.
• Each visualisation can be made public along with its own visual branding and rich-text description.
• Data is provided in real-time, straight from the EPPI-Reviewer database.
• Possibility to password protect any visualisation.
• The data visualised revolves around EPPI-Reviewer coding tools, however these can be made visible in customisable ways, hiding parts of the published coding tools, renaming any node therein and also adding "public" descriptions.

Review data is then visualised and explored in multiple ways. The current version of EPPI-Visualiser allows users to:

• List references "with this code".
• Visualise and list references by publication year.
• Produce frequencies tables and access the corresponding lists of references.
• Produce, visualise and explore crosstabs (including some features not yet available in EPPI-Reviewer itself!).
• Export any list of references to Excel and RIS formats.
• View the details for each individual reference, including its (published) coding data.

In the upcoming releases, we expect to add the possibility to both pre-specify three dimensional maps (crosstabs with an added dimension inside each cell) and to (optionally) allow visitors to create their own maps on the fly. We expect to also support the creation of lists of references on the basis of "with/without these codes" criteria, where "these codes" are picked freely by users while exploring a given "Visualise" page.

Bugfixes and enhancements

In the item details window of EPPI-Reviewer 4, when a PDF is open in the view tab, it was possible to trigger a spurious (but annoying) error message, while interacting with the coding data (adding/editing the "info" text, and other actions). Starting with this release, it is now almost impossible to produce the spurious error message.

In EPPI-Reviewer 4, Meta-Analysis window: adding a custom column to the outcomes table and then filtering it could (temporarily) make the window fail to load different Meta-Analyses. This problem has been resolved.

EPPI-Reviewer Web: following a recent addition to version 4, it is now possible to "rebuild" a pre-existing custom classifier also in EPPI-Reviewer Web.

Account Manager: the "summary" page is opened right after logging on and is starting to show very long "loading" times, especially for long-standing EPPI-Reviewer users, who might have participated in many reviews. To make the Account Manager more pleasant to use, we have subdivided the data shown in the "summary" page in separate tabs, which should make it easier to "find" the data needed and concurrently does make the page itself faster to load.

EPPI-Reviewer Web, bulk assign/un-assign codes to items. We have redesigned the confirmation dialog that appears when performing bulk un/assignments of items to codes. The "selected" code name is now made to "stand out" (much) more, so to minimise the chances of error. Also the number of currently selected items now appears in bold, making the whole dialog easier to "read".

EPPI-Reviewer Web: in the item details window, "activating" a code in the tree could make the scroll position "jump up". According to our analysis, this could happen in Chrome and Edge, and, although ultimately erratic, it happened when many nodes in the coding tools were expanded. This problem should be resolved now, although, being erratic, we cannot be absolutely sure.

EPPI-Reviewer Web: coding reports. Coding reports include "Numeric Outcome tables", whenever this kind of data is present. In EPPI-Reviewer Web, these tables were not reporting Arms data, which is now included in its own new column.

EPPI-Reviewer Web: deleting timepoints. In certain situations, deleting a timepoint could temporarily make the wrong timepoint disappear in the UI. The correct timepoint was being deleted, but the user interface could react by removing a different one. Reloading the data for the item would show that the correct change had been applied, but this bug could nevertheless produce quite a bit of confusion and alarm.

EPPI-Reviewer Web: deleting a source "forever". Permanently deleting a source and all its associated data is a slow operation in EPPI-Reviewer; thus, it can sometimes timeout. In EPPI-Reviewer Web, such situations did not produce any visible error message, which gave the (somewhat correct) impression that nothing at all happened. This problem is now resolved and an error message is shown.

EPPI-Reviewer Web: managing sources. The "import items" page also contains the "manage sources" tab. This has been enhanced in two ways: first of all, the list of sources on the right now has its own scrollbar, thus, in big reviews with many sources, it's possible to keep the "source details" visible, while scrolling up and down to find the desired source. Secondarily, clicking on any one source name on the right would automatically activate the "manage sources" tab. However, the first time this was done, the details for the first source in the list were loaded, instead of the ones for the source that the user selected.

EPPI-Reviewer Web: searching in the "Year" field. Creating a new search for text in the "year" field was not working in EPPI-Reviewer Web and is now fixed.

EPPI-Reviewer Web: panels in "search" and "items" tabs could remain open when changing reviews. This produced the potential for problems, because the contents of the affected panels (view scores distribution in search results, coding reports) would refer to the previous review and not the one currently open. Concurrently, we have reviewed and enhanced the data-cleanup routines that happen when changing reviews and when clicking on the "logout" button.