Dear Julie,

the most likely reason is that the code-set was set for "multiple user" (double) coding when the "now disappeared" coding was applied. It was then moved to "single user" mode, but without completing the coding for some or all items. If you'll look at the review-statistics tab (bottom right of the main screen), you'll find the exact numbers that apply to your specific case (look in particular at the "Incomplete coding" box).

There are two ways to proceed now, which one is best depends on how items were allocated amongst the team members:

If each single reference was allocated to one and only one reviewer, then you can use the review statistics tab to complete the coding for all items in just a few clicks. In "review statistics", "Incomplete coding" box, expand the appropriate Code-set and click on the green V for each reviewer, this will mark their coding as complete. The end result will be equivalent to what would have happened if the the code-set was never configured for "multiple user", which is the correct option for this (one reviewer per reference) type of scenario.

If each single reference was allocated to more than one user, then you should revert the code set to multiple-user data entry and proceed in the usual "double coding" way (create comparisons, complete agreements, evaluate and complete each single disagreement), please see the double coding video for details.

If you are in a "mixed" situation, where some references have been coded by two people, and some by one only, then the situation may require some more attention, please let us know if that's the case.

I hope this helps,

Sergio

Hi Julie,

this case appears pretty complex, because you have many review members and many work allocations with a good number of them not completed. From the work allocation details I couldn't figure out which ones may have overlapping items lists, so I figured that creating all the needed comparisons would be quite time consuming.

Anyway, let me explain first:

the situation is that you will need to go through disagreements (where reviewer X has coded an item in a different way than reviewer Y). The standard way to do so would be: put the code-set in multiple user mode, create all relevant comparisons (and what comparisons are relevant depends on your specific case, I couldn't figure it out on my own), complete the agreements in bulk, sift through the disagreements and complete each one by hand.

This is the only way forward in case you need to measure the number of disagreements that came out of your double/multiple coding exercise.

If you don't need to count disagreements, there is an easier way forward: I've looked into the raw data and extracted the list of items that have different codes applied by different reviewers (on the "Q1 Screening on Title and Abstract" set) and found out it only contains 29 items (if I did this in the right way). I've then created a Search for these, so that you'll be able to get the list of these "troublesome" references (it's the one and only list created by me). At this point, you should be able to walk through all of them, evaluate the differences and apply the the "winning" code manually. Shouldn't take too long.

Once this is done we'll have a look at how to complete the remaining ones, will get back to you about this.

Best wishes,

Sergio