Version 4.11.5.0 involves a range of new additions to EPPI-Reviewer Web and a big addition to the Microsoft Academic functionalities (version 4 only, these features are "on invitation" for the time being, see here for more details). In EPPI-Reviewer Web this release introduces full support for setting-up Priority Screening, as well as a new system to produce HTML and JSON quick coding reports, which is faster and removes the 4000 items per-report barrier. In EPPI-Reviewer 4, the Microsoft Academic area is still changing rapidly, and therefore remains "experimental / on invitation", this release includes the possibility of running boolean searches against the whole Microsoft Academic database of over 240 million records.

EPPI-Reviewer Web, new functionalities

Setting up Priority screening

Before this release, it was possible to use Priority Screening in EPPI-Reviewer Web, but only after setting it up in version 4. This release includes a new "Screening" button in the "Collaborate" tab. This button will appear in all reviews where the Screening feature is activated (activation is still done it the Account Manager). The new screening button provides access to three functionalities.

1. A summary panel that shows how screening is configured and visualises the progress made. This is available to all users that have access to the full user interface (i.e. any user that does not have the "Coding Only" role) and allows users to "begin screening" and to manually trigger a new training round.
2. A "Configure Priority Screening" wizard, which is the main system that people should use to set up priority screening. This is available to any review member that has the Administrator role.
3. A "Change all Settings" functionality, accessible from the summary page (1) and is, like the wizard, restricted to review administrators.

Features (2) and (3) are enriched with a number of checks and warnings that are designed to help users to understand the somewhat hidden implications of some of the available options. These checks are the result of the experience accumulated with priority screening in the last few years, and should ensure the systems will be user-friendly to set-up and use.

Quick coding reports

Quick coding reports are produced in the references tab, and previously required users to start by selecting some references from the current list and then open up the "Coding report" panel. This mechanism had two shortcomings: first of all, it was built on the assumption that it would not be used on many items, so behind the scenes, it worked by collecting the coding data for each selected item in sequence. This made the process slow as it's sequential, downloading data on one selected item at a time. Secondarily, it inherently limited the maximum number of items that a coding report could include to a maximum of 4000 (the largest size for a single page in the references list).

As people need to use the coding report to export data in bulk (especially in JSON format), sometimes on a weekly basis, for example to create and update big Evidence Maps, we've extended the "Coding report" functionalities, making it possible to:

1. Get a coding report for the currently shown page. Behind the scenes, this now happens all at once, making the process of collecting the report data much faster.
2. Get a coding report for the whole currently loaded list (all pages). This will happen on a per-page basis, making the overall speed much better, but is also dependent on the page size.

The side effect of these new capabilities is that it is now possible to create huge coding reports. For this reason, the user interface now includes a number of checks that are aimed at preventing predictable problems that could be generated by excessive size. Reports that are quite big, will not be "previewed" within EPPI-Reviewer Web anymore, as this could slow down the whole app; users are still able to save such reports or preview them in a separate tab. Even bigger reports will only allow "saving" them, as even opening them within the context of the app can cause problems. Finally, while testing the limits to identify the appropriate thresholds, we found that some JSON report took a long time to render in Chrome; as a result, we have disabled the in-app "preview" option for JSON coding reports generated within Chrome.  

The "per selected item" coding report feature remains available and is still working as before (save for the additional preview rules which apply to all coding reports). Finally, the "Quick Question" reports (available via a sub-option for the same "Coding Reports" button) are unaffected and still work exactly as before.

Enhanced selection of references in the "references" tab

In the references tab, individual references can be "selected" via the provided checkbox; this is useful to assign or remove items from a specific code, to change their status flag (I for included, E for excluded or D for deleted) and a number of other reasons. Until now, selecting several individual references required users to manually tick their checkbox for each reference. The new "enhanced selection" of references speeds up this process in the following ways:

• Clicking anywhere in a row will select the current reference and un-select any other reference, with two exceptions (below).
• Clicking on a checkbox itself will affect the checkbox itself only, leaving the selection status of other references in the list unchanged. [Clicking on the checkbox itself also bypass the additional functionalities described below.]
• Clicking anywhere in a row while pressing either "CTRL" or "Alt" will alter the default behaviour:
  • The "CTRL" key makes each row clicked change it's "ticked/selected" status, without changing what other items are already selected.
  • The "Alt" key is used to select a range: the last clicked row is highlighted in bold. Clicking on a different row while pressing "Alt" will select all references between the one highlighted in bold and the reference that has been clicked.
  • [Pressing both "CTRL" and "Alt" will ignore the effect of the "Alt" key.]

The "enhanced selection" feature is turned on by default and can be turned off at any time via the references list options. In a future version, we plan to make this a per-review/user preference that will be "remembered" across sessions.

Known issues: this feature is known to be affected by two (hopefully minor) problems.

1. Sometimes the checkboxes take some time to "change" their status after clicking. Moreover, when using the "modifying" keys, sometimes nothing will happen until the user releases the modifying key.
2. The feature itself makes the list or references more computationally expensive to display and keep up to date. This can make EPPI-Reviewer Web visibly slower when the list shows more than 500 items and enhanced selection is turned on. For this reason, a warning is shown when this is the case.

Sorting and Paging Coding assignments (Collaborate tab)

Until now, EPPI-Reviewer Web would not explicitly sort the coding assignments shown in the "Collaborate" tab. This meant that assignments would normally get ordered roughly by creation date, with some fairly unpredictable exceptions. With this release, the most recent assignment is now shown on top by default. Moreover, the ordering can be changed at any time, by clicking on the header cells. The list is also shown in a more compact format and automatically paged for reviews that contain more than a hundred assignments.

Sorting and Paging Searches

Having thousands of searches could visibly slow down EPPI-Reviewer Web. This is now resolved by "paging" the list of searches so that EPPI-Reviewer Web does not need to render thousands of complex "data-rows", which is why it could slow down significantly.

URLs and DOIs in "item details"

For both the URL and the DOI fields, in the "Item Details" page, the field name will now become a clickable link whenever the corresponding data is present and appears to be valid. Clicking on these links will open the destination page in a new tab.

Outcome tables in "per item" comparison reports

Per item comparison reports are used in EPPI-Reviewer to reconcile coding disagreements for double coding exercises done against complex coding tools, this is typically useful in data-extraction scenarios. Until now, numeric outcome tables were not included in these reports, making it harder to spot and correct disagreements in numeric data-extraction. In EPPI-Reviewer Web, these reports now include the outcome tables, if any. The same kind of report in EPPI-Reviewer 4 has not changed and does not show this data.

Enhancement: priority screening list

In EPPI-Reviewer Web, to access the priority screening list (i.e. "start screening") from the main user interface, people had to expand the "My work" panel in the review home tab. From now on, whenever the list is present and ready, the "click here to start screening" message now appears in its own, always visible, space in the "review home" tab.

Deduplication features

The deduplication page has acquired a number of "ported" functionalities. These are the minimal set of features that is required to disentangle overlapping groups. Although overlapping groups are not created by the new grouping algorithm, the ones created by the old system still need to be manually handled. With this release, doing so is now possible in the Web version. The newly implemented features are:

• Find Related Groups, used to find overlapping groups.
• Delete a group, used to remove the older overlapping group(s), thus resolving the problems they cause.
• Find groups by list of Item IDs, useful to check if one or more items has been included in any group.
• Soft Reset: deletes all groups without changing the "duplicate" status of existing items.
• Hard Reset: deletes all duplicates data, including groups and the "duplicate" status of all items in a review.

EPPI-Reviewer 4: direct boolean search against Microsoft Academic data

For those with access to the Microsoft Academic features, it is now possible to search Microsoft Academic data for references. This brand new search functionality is implemented as a powerful engine that allows to conduct and combine boolean searches against a number of frequently used fields (including title, abstract, authors and date).

Bugfixes

Uploading sources in EPPI-Reviewer Web. In EPPI-Reviewer Web, an undocumented 30MB maximum file-size limit existed for "source" files (RIS files and similar). Users were allowed to "try" uploading larger files, but the attempts would fail, producing a cryptic error message (stating: "Error: Object, object"). To resolve this, we've implemented a few improvements:

• The file size limit has been increased from 30MB to 50MB.
• EPPI-Reviewer Web will not try to upload larger files and will instead mention the file size limit.
• If an error occurs, the error messages should now be more informative (the "Object, object" message was the consequence of its own mini-bug).

Priority screening: "list creation in progress" flag not being set. Priority screening relies on an "active learning" strategy, which allows the machine to keep learning from users' decisions. To do so, the "priority list" gets periodically regenerated after re-training the machine. The priority screening pages have a "read only" flag that signals when the list is being regenerated, this is a "for information" flag, which wasn't always being set as appropriate (could fail to signal when re-training was occurring). This problem occurred on both versions and is now resolved.

Logoff and logon rapid cycle. In EPPI-Reviewer Web, if people clicked "logoff", then logged on again and then paused in the "review list" page for about 20-45 seconds, their authentication state could get deleted, causing two problems (perhaps three):

1. The "Create new review" functionality would get disabled.
2. Trying to open a review would fail with a "forbidden" message.
3. (Unverified) It's also possible that checking in and out reviews from Archie would fail in these circumstances.

The root problem has been resolved and these small annoyances shouldn't happen again.

New groups of duplicates could be created with the wrong "master" item. When a new group of duplicates was being created and two or more "group members" had already been coded, the newest coded member was being picked as the master for the new group. This is not useful, as the oldest coded member is usually more likely to be enriched with more and more useful data. The problem is now fixed and the oldest coded member will be used as the "master" of newly created duplicates groups.

Critical bugfix: in very rare and hard to reproduce situations (EPPI-Reviewer Web only), it was possible for people to be shown data coming from different reviews. This problem occurred when the client sent to the server a request with outdated authentication data. On the server side, this could make the code misunderstand who created the request and thus access the wrong review-data. The problem could occur only on read requests, which meant people could briefly see the wrong data. Moreover, an internal system checks the connection with the server every 45 seconds, and at each "check", the validity of the authentication data is also checked. Thus, we believe that this problem could be experienced for a maximum of 45 seconds, after which the user would be sent back to the login page. Furthermore, any attempt to change data (when authenticated with outdated data) would also fail, meaning that this problem could not result in data corruption.  

On the server side, the problem is now solved: sending a request with outdated authentication data will now produce an "Unauthorised" response, without sending to the client any review data or "confusing" people's identity.

On the other hand, we could not identify for certain the reason why sometimes the client might start sending requests with the wrong (outdated) authentication data, the only known way to do so is very convoluted (requires to "duplicate tab" after opening a review, change review in one tab and then immediately go back to using the other tab) and is therefore expected to be extremely rare, in the real world. This "known" occurrence is also always short-lived as the 45 seconds limit does apply.

Future plans

We expect to release the Microsoft Academic features for EPPI-Reviewer Web quite soon, which is a prerequisite for opening up these features to a larger public. Should the features prove to be robust on both versions, we will then start "exposing" Microsoft Academic features to more users, probably on a "per-feature" or "per set of features" basis.

The next major release of EPPI-Reviewer Web after the one dedicated to Microsoft Academic is expected to contain what's needed to design and edit Configurable Reports. We will of course continue to improve other areas as well.