HelpForum

Forum (Archive)

This forum is kept largely for historic reasons and for our latest changes announcements. (It was focused around the older EPPI Reviewer version 4.)

There are many informative posts and answers to common questions, but you may find our videos and other resources more informative if you are an EPPI Reviewer WEB user.

Click here to search the forum. If you do have questions or require support, please email eppisupport@ucl.ac.uk.

<< Back to main Help page

HomeHomeUsing EPPI-Revi...Using EPPI-Revi...Questions about...Questions about...Allocating tasks to groupsAllocating tasks to groups
Previous
 
Next
New Post
22/11/2013 14:08
 

Hello,

We are two groups consisting of 3-4 reviewers in each group doing the review. We want to divide the task in the ratio of 75:25 (roughly) between the groups based on the source database, i.e. a group will look at the data extracted from medline only and the other group will look at other data sources. We are not able to undestand how we do that.

We created a new code 'Allocation Groups'  and child codes Group 1 and Group 2. Then going to the tab 'Assign documents to specified codes randomly' we wanted to distribute the tasks accordingly between the groups, but got stuck up here.

Our queries are:

1. If we are to allocate to, say, Group 1, the database extracted from medline only, how do we do that?

2. How do we further subdivide the tasks between individual reviewers within a group?

3. Is there any other way to allocate the tasks other than randomly?

We would be grateful for early resolution of this issue as we are not able to progress further due to this issue.

Regards

Tekendra

 

 
New Post
22/11/2013 16:28
 

Hi Takendra,

Work allocation is done in two phases. First phase is to assign some items to a code (any code that is in a "single user" code-set will do), the second phase is to create a work allocation associated to the code mentioned above.
This gives users total flexibility: you can use the automatic "random" feature to generate the allocation codes and assign them to items, all in one go, but you don't have to.

Since you wish to allocate your work based on the source information, and since your sources information is already "coded" in the "Source codes" code-set, I am guessing you can use the codes you already have to "inform" the work allocations. The complexity in this case is that you probably want to use both mechanisms: the latter will be used to separate between group A and group B, the former will be used to spread the work across different reviewers (in case you don't want all members of each group to look at all items assigned to their group).

Let's see the simple case first. If you want all members of a group to look at all items assigned to their group:

In the collaborate tab, click "create new (coding assignment)"
In "code studies in this group" select the appropriate source code, for example "Medline".
In "Using this code set" select the appropriate code set, for example "Screen on title and abstract"
Pick the first person from your appropriate group for the "To this person" field and click Assign Work.
Repeat the same for all people in the first group. Repeat again for the other group(s) using different "source" codes as appropriate.

The more complex case is when you want something like: for items coming from medline, have 200 items double coded, have all the other items coded from people in your GroupA, but strictly one item per person; have all other items screened by GroupB, again some double coded, some not. This can get messy if you don't plan ahead. I won't give you precise click-by-click instructions because how to proceed depends on exactly what you want to achieve. Instead, I'll explain the strategies, and let you work out the details.

First thing to note: when you code using a set in "multiple user data entry" mode, the codes assigned from each reviewer will be visible to their creator only. To see what the others have coded you need to use the comparison and coding record features. In this mode, it is assumed that you want the different reviewers to remain blind to each-other's coding, so to be able to compare the differences and then reconcile. At the end of this kind of exercise you will have "completed" the coding of all items involved with the coding version of one reviewer or the other. Once the coding is completed, any new code assigned to a "completed" item will be visible to all reviewers immediately.
In other words, the "multiple user data entry" mode should be used only when you want to conduct double- or multiple-blind coding exercises, for all other cases, the "single user" mode should be active.

This means that using the same set to double code some items and at the same time single code some other items is possible but tricky. Ideally, you would do all your double coding first. Evaluate and "Complete" the coding of all the items that have been double coded and then set the code-set back to the normal (single user) data-entry mode. At this point you will be able to create new work allocations to spread the remaining items to each reviewer (the random assignment will come in handy), making sure that no item gets assigned to more than one person.

If you can't proceed in this way, you will need to be careful about what you assign to who, so to be sure you will be able to easily isolate the items that have been coded by more than one person from the items that are getting attention from one reviewer only.
You will then make sure that the double coding exercise is done as soon as possible and that the corresponding coding is evaluated and completed. Once this is done you will be able to use the review statistics tab (lower right corner) to "bulk complete" the remaining "incomplete" items (that is: the items that have been single coded while some other were double/multiple coded). Once all items will be completed (first manually, the double coded ones; then in bulk, the remaining) you will be able to change the code-set back to "Single user data entry".

What does this mean for work allocations? It will all turn around creating the correct codes via the Random Allocate system. I'll give you an example. Say that groupA (your own first group of reviewers) should go through the 7000 items coming from medline. This group is formed by 4 people (RA, RB, RC and RD). You will allocate 700 items to both RA and RB, while RC and RD will get another 700 items each, this time with no overlaps.

To do this, and get the work started you will:
in the main documents tab, click on the "hand" button ("assign documents to specified codes randomly")
from the first drop down, select "all with this code" and pick the "Medline" code in the second drop-down.
in "create codes below this code-set" select somewhere in your "allocation groups" code-set, pick 100 in "percentage to allocate" and 10 in "number of groups".
make sure the "Excluded items" option is ticked (as you have imported your items as excluded) and click "go!". This will create 10 groups called "group 1" to "group 10". You may now rename "group 1" to "double coding: RA and RB", "group 2" may become "First batch: RC", and "group 3" may be "First batch: RD" (RA-RD are the names of the actual reviewers).
go to the "collaborate" tab and create 4 "coding assignments", the first two will use the "double coding: RA and RB" code and be associated with RA and RB, the other two will go to RC and RD and their own allocation code.

You will now be able to keep track of proceeding: coding assignments tell you how many items are allocated, started (have received some codes) and remaining (have not been coded). When RA and RB will finish their allocation, create a comparison and complete the coding for the affected items. Then complete the coding from RC and RD (via the review statistics tab) and if possible (no other double-blind coding is expected for this code-set) turn the code-set back to "single user" mode.

At this point you'll be able to use the other 7 "randomly allocated" codes to distribute the rest of the items between RA, RB, RC, and RD as needed. You may also re-distribute items between these codes to evenly spread the work or to assign more items to the people that are able to proceed faster.

I hope this makes sense to you. It is easier to do it than to explain it: the overall system is designed to allow you to do whatever you may wish, and once you understand why, it doesn't look too complicated.

If you want more precise step by step instructions, I'll need to know exactly how you wish to proceed, please do get back to us with more details if needed.

Sergio

 
Previous
 
Next
HomeHomeUsing EPPI-Revi...Using EPPI-Revi...Questions about...Questions about...Allocating tasks to groupsAllocating tasks to groups


Copyright 2021 by EPPI-Centre :: Privacy Statement :: Terms Of Use :: Site Map :: Login
Home::Help::EPPI-Mapper::RIS Export::About::Account Manager