Version 6.16.1.0 makes it possible for users to enable LLM-coding functionalities without intervention from the team here and introduces the "Automation Logs" page, where people can inspect the log of long-lasting, review-specific jobs (including LLM-coding batches). This release also includes one new functionality for EPPI Visualiser and contains some important bug fixes (especially to the "update review" pages).

New features

Activating LLM-coding functionalities. It is now possible to activate LLM-coding features autonomously, without having to ask EPPISupport to enable them. To make these functionalities available, it is necessary to provide a way to pay for them, which is done by purchasing Credit from the online shop. The Account Manager, now allows you to associate credit records to either individual reviews or all reviews covered by a site license. Thus, while credit is available, EPPI Reviewer LLM-coding features will be visible and enabled in the selected reviews.

Log of Review Jobs. In EPPI Reviewer, an ever-increasing number of features require to run long-lasting processes; these include building and applying machine learning models, deduplication, deleting sources, and more. Activity of these jobs is logged into a single database table, which we have been using to troubleshoot problems and monitor performance/progress. The contents of this table are now available from the "review home" page, via the new "Automation logs" button. In the resulting page, users now have access to the log of "regular" long-lasting jobs for the present review, as well as LLM-coding jobs (if enabled in the current review).
This feature thus allows you to:

• Find out the current state of recently triggered jobs. For example, when building a classifier, it is now possible to follow its progress, until it either completes or fails.
• Discover what went wrong and why. Naturally, error messages logged in this manner are designed to inform us on the EPPISupport side, so can occasionally be opaque to ordinary users. If in doubt, please do contact EPPISupport whenever experiencing hard-to-explain/resolve recurring failures.

Enhancement: when running batch jobs LLM-coding, sometimes and for a range of different possible reasons, single items will not be coded (for example, when coding against Title and Abstract, and a given item has no abstract). In such cases EPPI Reviewer would skip the affected items, without leaving any trace/clue about why a given item was skipped, which isn't ideal. From now on, the known/predictable reasons for skipping individual items are logged as "errors" in the past-jobs log.

Enhancements: the "Search and Classify" tab has been improved in a number of ways, some very visible, some quite subtle (better control for when buttons get enabled). The most important changes involve the main table of changes, which now can change the page size, has a "per page" "select all" function and improved sorting functionalities. To make room for additional buttons, some text-based buttons (delete, refresh) have been updated to "icon-based" buttons.

Priority Screening Simulations. In the "Search and Classify" tab, the new "Priority Screening Simulations" button provides access to a new page, where it is possible to trigger and then inspect, download and analyse priority screening simulations. To produce a new simulation (using existing data of already screened items), you will need to give it a meaningful name, select one code assigned to all the includes, select one code assigned to all excludes and then click on the "run" button. This triggers a long-lasting machine learning job, which simulates a "whole review" screening exercise powered by priority screening, starting with randomly selected minimal seeds. The same simulation is repeated 10 times, with different randomly selected seeds, and the results saved. Once results are present, the same page allows to download results, visualise them, and compute performance figures at different levels of recall.

In the "Update Review", "search and browse" tab, the "Custom Filter" search method now offers the possibility of pasting the "API query" that OpenAlex makes available from their own "search for works" page. EPPI Reviewer will "parse" the pasted API query, and if successful will then run the relative search. This can be useful because the OpenAlex page has a functioning (and likely to expand) user interface that allows to construct complex queries visually, users can thus build/explore searches there, and then "port" them into EPPI Reviewer effortlessly.

Enhancement: we recently introduced the "enrich item with OpenAlex data" feature in "Item Details / OpenAlex tab", this function however was ignoring authors. It will now fill the Authors field when the current item has no author data and the selected OpenAlex work does.

EPPI Visualiser: the item/reference details page now allows users to "return to the previous visualisation" page. Until now, it was possible to return to a specific list of items, but not to its parent visualisation. Now, the same page offers both possibilities, so to either return to the specific list that contains the current item or to the original visualisation, which could be a Map, a Crosstab, or a Frequencies table.

Bug Fixes

Matching Items to "OpenAlex Works" was affected by a bug when comparing authors, which had the effect of significantly lowering similarity scores. The side effect of this is that far too many possible matches were routinely assigned scores low enough to require manual checking. This fix reduces the number of "matches to be checked manually" by a factor of around four or five.

In the "Update Review" pages, the "search and browse" tab allows users to run (and combine) complex searches against the live OpenAlex database. Since this Summer, when OpenAlex introduced a number of "breaking changes", some/most "custom filter" searches didn't work in EPPI Reviewer. It was possible to "run" them, but then results could not be retrieved. This problem is now solved.

In Item Details, entering Timepoint values that represented extremely long periods (like "999999 weeks") would "break" all timepoints for the affected items, so that they wouldn't appear in the UI and produce an error instead. This problem is now fixed.