Version 184.108.40.206 is a release focussed on how the "Update Review" features work internally. In December 2021 Microsoft discontinued the Microsoft Academic Database (MAG), which was driving all these features. Shortly after, EPPI-Reviewer was updated to use OpenAlex as its data source. However, this "change" still relied on OpenAlex MAG-like snapshot updates (bi-weekly releases for "new/updated data/references"), which OpenAlex produced as a courtesy system for those who previously relied on MAG.
Unfortunately, for technical reasons, these "snapshots" proved to be unreliable for our purposes, which in effect stopped EPPI-Reviewer from receiving up to date data since March 2022. OpenAlex is also discontinuing them now and has shifted over to a new ‘OpenAlex’ format. The present version is thus the result of a major rewrite/refactoring effort, aimed at removing the need to use these snapshots, replacing them with (almost) equivalent calls to the OpenAlex native API and snapshots. This means that the "update-review" functionalities are now fully functional again (albeit with some "new" limitations) and that the "update review" flow of "new references" will re-start very soon.
Abandoning "snapshots" and switching to the OpenAlex API
Most importantly, as we are now using the live OpenAlex API, all searches are using the most up to date dataset available. One effect of this is that search results (in particular: the number of "hits") are now mostly indicative, as the actual number of references found may increase (in most cases) at any time. The one exception to this "rule" is "combined" searches, which instead are "fixed" in time (more below).
The other effect of the above is that the date shown in the Update Review pages (was: "OpenAlex Dataset: YYYY-MM-DD") is now of much reduced importance. It still matters, because from now on, it refers to the last time that "Auto update tasks" were run, producing "new" lists of references that the Machine Learning algorithms have identified as possibly relevant for the reviews that use this feature. For all other features, it now has no relevance, as they feed directly from the OpenAlex API, which uses data that gets updated in real time, and not at fixed intervals.
Moreover, because we now rely on the OpenAlex API, some pre-existing functions are now "adapted", which mostly means, "somewhat limited" by what is currently supported by API. We do expect to expand on these, as and when new API functionalities are implemented on the OpenAlex side.
Specific changes: "bring up to date" tab
This page works as before, with the only difference that some "search" options are not supported by the OpenAlex API, so have been removed. These are: "Papers that 'recommended' known items ('That recommend')" and "Bi-directional recommendation relationships ('Recommendations')". Moreover, the last option ("Bi-directional citation AND recommendations ('Bi-Citation AND Recommendations)'") is now slightly different in that it does not "find" bi-directional recommendations, as this functionality is not available anymore. "Searches" produced in this page are (as before) fixed in time, meaning that the list of papers they "find" will not automatically "grow" as new data are added to OpenAlex, this is different from how most now searches work in the "searches" page/tab.
Specific changes: "keep up to date" tab
Nothing changed on the user interface, but pretty much everything changed on the server/hidden side. Most importantly, with this update, data can resume flowing in, meaning that new "task results"/"auto update runs" will start appearing again soon. The main functional difference is that from now on, we (at the EPPI Centre) control when the tasks will be executed, as we don't depend on receiving (functioning) snapshots.
Specific changes: "search and browse" tab
As hinted above, the "search" functionalities have also been re-written. The most notable effect is the disappearance of some search options, but some other subtle changes are also worth mentioning.
Changes to "New searches":
- Options to search in the "journal name" and "authors" fields have been removed. Also the option to use a "MAG specific" search string has been removed, for obvious reasons.
- The "publication date" filter is not available when searching against "title and abstract" fields (works for searches against title only).
- It's now possible to filter by "OpenAlex date after" which refers to the date when a given publication was added to the OpenAlex database (can be useful for "updating" purposes).
Regarding the loss of features, we expect that most will re-appear in due time as and when the required functionalities will be added to the OpenAlex API, although of course, we can't be sure about exactly when this might happen.
- Combining searches is now a "slow" operation, and has a limit of 40,000 maximum hit-results, due to the fact that we have to download every record from the OpenAlex API. This limit only applies to "new searches"; searches that are themselves the result of "combining" do not participate to it.
- The option of applying a "publication date" filter when combining searches has been removed.
List of searches:
- Old searches, created with the previous system are now entirely "inert", they are still visible for "reference" only, but cannot be browsed, imported or combined. Only "deleting" them is still possible.
- New searches produced via the "Search" button, still include a "hits" count. This count however only reports how many references were found at the time the search was created, and thus, might underestimate the real (current) number, especially when the search is old.
- By contrast, searches produced by combining existing searches are fixed. They represent the "combination" results as they were at execution time and do not change over time.
- As searches don't "expire" when a new "updated" snapshot is released, the "re-run" and "import" functionalities are not mutually exclusive. "Re-run" can be used to obtain the up-to-date figure about "hits", by creating a "new" search in real time. "Import" can be used as before, keeping in mind that it might import more items than the number shown in the "hits" column (if the search is not recent).
Finally, this release also contains a bugfix and a minor "enhancement":
- Bugfix: URLs in ER-Web "reference" reports were not working in some combination of styles and reference type (Harvard and Chicago styles, versus "report" and other publication types).
- Enhancement: one phase of "get new duplicates" could fail for very large reviews, due to a timeout. To mitigate this problem, the timeout for it has been raised from 90s to two minutes, which should allow the successful execution in all cases. (Please contact EPPISupport if "Get new duplicates" is failing consistently, for any given review).