Version 4.7.0.0 is an important update, it makes the priority screening features of EPPI-Reviewer available to all users. Previously these features were available on request. It includes significant improvements to the priority screening user interface as well as a few important bugfixes. Starting with this release all the features leveraging the EPPI-Centre Machine Learning infrastructure are available by default to all EPPI-Reviewer users.

Priority Screening.

This feature is made available through a dedicated "Screening" tab on the main screen. Users can activate this tab via the account manager, on a per-review basis: on the account manager "summary" tab, click on Edit for a given review and set the "Priority Screening" flag to "On" - the change will become available the next time the review is opened. Since these features are very powerful, setting up the screening preferences in the Screening tab is possible only for review administrators.

General principles:

The priority screening system uses Machine Learning (hosted on Microsoft's Azure ML platform) to automatically learn from the screening decisions made by the end users. To do so, users must tell the machine which codes should be interpreted as signifying "Include" or "Exclude" (system attempts to do this automatically, but it's good practice to check the preferences manually). At the beginning of a screening round, the system will generate a random list, as users proceed in their screening, when enough Include and Exclude decisions are made, data will be fed to the ML components. These will use existing decisions to produce an ordered list, presenting the Items that are more likely to be included on top of the list. As more decisions are made, the ML component will update itself and silently create new ordered lists at regular intervals, thus, the system keeps learning, producing more accurate results along the way. Data used to let the machine learn is the text included in titles and abstracts, for this reason it is natural to expect that the system performance will be highest in the screening on title and abstract phase, while it will be somewhat lower if used for a screening on full text exercise. A full evaluation of this methodology is available here (see also). The screening tab also allows to leverage some ancillary functions/preferences, to support double coding and automatic reconciliation of agreements. Disagreements need to be reconciled as before, using the existing "Comparison" features available in the "Collaborate" tab. All the features made available on the screening tab are powerful and, if used incorrectly, might produce unwanted results. Thus, the system tries to nudge users in the right direct by automatically setting "safe" options in response of changes initiated by the user.

New Features:

The screening tab is now available to all EPPI-Reviewer users, on a per-review basis. The tab can be activated and de-activated at any time as explained above. It is our expectation that the tab will not be de-activated often. Once activated, we do not see any compelling reason to deactivate it again.

When changing the codeset used for a the screening exercise, system now automatically selects the safest values for "# people screening each item" and "reconciliation mode". If the data-entry mode of the currently selected "Screening Codeset" is then changed (between "normal" and "comparison" modes), EPPI-Reviewer will automatically detect the change, bring the user to the screening tab and change the aforementioned values accordingly. The user will have to review the changes and click on either "Save Options" or "Canel" in order to proceed. Only Review Administrators are allowed to change the Screening Codeset data-entry mode, as this change requires to apply corresponding adjustements to the screening options.

When importing a new source, the "review needs indexing" option is automatically activated and saved, thus, when a new list is created (re-training happens automatically in the background at regular intervals) the newly imported items will be automatically included. Users may want to manually click on "Create list of items to screen" to let the newly imported items to participate in the current priority screening round without delay.

Bugfixes:

• When starting a new Screening exercise from scratch, the system sometimes failed to trigger the Machine Learning components after enough Include/Exclude decisions were made. Previously, users had to tick the "review need indexing" option, save options and manually trigger the creation of a new screening list. This problem is now solved.
• The ordered list uses "item locks" to signal who is currently screening what and thus direct people to the next item to screen accordingly (this is necessary to avoid duplicating efforts). In some situations, the locks failed to be removed, making some items inaccessible to other users (through the ordered list / screening functionalities). The new system avoids leaving unnecessary locks behind. Additionally, all locks now last up to 13 hours, after which they are automatically removed. Thus, no stale locks are left behind, ensuring that no items will be unduly sidelined by the process.
• When using a set in "Comparison" data entry mode, the screening tab allows users to indicate how many users should screen each item (default is 2). When asking for "next item to screen" (or clicking "Begin screening"), the maximum number of people indicated in the afore mentioned option wasn't always respected, leading to some (occasional) duplication of effort. This problem is now solved.
• When users manually deleted one or more items (including deleting whole sources), the Machine Learning results (silently produced and collected automatically) failed to be consumed properly. As a result, the priority list would fail to update (would not learn from the proceedings), unless people ticked the "review needs indexing" box manually. This problem is now solved.