HelpForum

Forum (Archive)

This forum is kept largely for historic reasons and for our latest changes announcements. (It was focused around the older EPPI Reviewer version 4.)

There are many informative posts and answers to common questions, but you may find our videos and other resources more informative if you are an EPPI Reviewer WEB user.

Click here to search the forum. If you do have questions or require support, please email eppisupport@ucl.ac.uk.

<< Back to main Help page

HomeHomeUsing EPPI-Revi...Using EPPI-Revi...Questions about...Questions about...Multiple Coding and ComparisonMultiple Coding and Comparison
Previous
 
Next
New Post
18/05/2011 15:27
 

 Hi,

We are trying to set up a pilot so we can compare how two users are coding the same items in a review (I am working on the review with Mat Tromme). We have randomly allocated the documents uploaded into two groups. We have then assigned coding allocation group one to both users. The items were then coding using the screening criteria. However, when we try and create a comparison of the coding results we do not get any results coming up.  Also, when we try and view the stats in the collaborate tab no documents are listed as coded by either reviewer.

We have followed the manual but it appears we have clearly missed something here. Please let me know what you think the problem may be.

Thanks,

Sam

 
New Post
19/05/2011 10:59
 

Hello Sam,

I just had a quick look at your review. The issue is in how you are setting up your comparison.

When you click on the Create comparison button (in the Collaborate tab), after selecting Reviewer 1 and Reviewer 2, for the Code set you want to select your 'Screening criteria' codeset. For the 'Set attribute' option, you want to select your Group one code. This means you are running a comparison how Reviewer 1 and 2  coded the items in 'group 1' using the 'Screening criteria' codeset. I think you may had this turned around.

Then, when you 'run'  a quick report, select the 'Screening criteria' codeset for the 'Select this code(set)' option. You will then get a report showing how Reviewer 1 and 2 applied the codes from 'Screening critieria' against the items of in group 1.

I noticed that you have all of your codesets set for multiple data entry. You only need to set a codeset for multiple data entry if the intention is for 2 or more people to apply the codes from that codeset against the same item. If you are double coding/screening and you want to compare how each person coded the same item, then you will want your screening codeset to be set for multiple data entry. If you have an allocation codeset that is used to identify items for allocation, you would want that set for single data entry. That way you don't end up with items marked as 'not complete' that you actually want, and require. to be marked as complete.

Normally I keep all codesets set for single data entry unless 2 or more people will be assigning codes from that codeset to the same item and then comparing how each person coded that item .

Best regards,

Jeff

 

 

 

 
New Post
19/05/2011 11:35
 

 Hi Jeff,

many thanks for your timely help - all seems so simple when you know how!

We're just learning the software at this stage prior to starting our real systematic reviews. But I wanted to ask, if we were to do multiple screening as our pilot stage (gauging discrepancies in reviewer codings), then reconcile the differences between reviewers, could we then revert that code set (screening criteria) to single screening, splitting the remaining documents between the two reviewers, and then complete the screening? This would mean that the pilot was part of the actual review, thus saving us time and keeping everything in one review.

Best,

Sam

 
New Post
19/05/2011 12:15
 

Hello Sam,

What you are suggesting would be fine. Many groups will double screen a number of items at the start of the project. Once they are sure that each coder is applying the criteria in the same manner they resort back to single coding to do the rest of the items.

What I would make sure of is that before you change back to single coding that you 'complete' all of your agreements and reconcile and complete all of your disagreements. Reviewers can get into difficulties if they switch back to single coding but still have a number of double coded items that have not been completed and reconciled. The 'un-completed' items do not show up up reports and can result in misleading reports. The functions to recover from this are available in the software but it is better to sort it out before you change your codeset from multiple to single data entry.

Best regards

Jeff

 
New Post
19/05/2011 15:15
 

 Hi Jeff,

yes, we certainly need to make sure of that. Otherwise I can see that the numbers will not add up at the end, which would be a major problem for our results.

Thanks again for your help,

Sam

 
Previous
 
Next
HomeHomeUsing EPPI-Revi...Using EPPI-Revi...Questions about...Questions about...Multiple Coding and ComparisonMultiple Coding and Comparison


Copyright 2021 by EPPI-Centre :: Privacy Statement :: Terms Of Use :: Site Map :: Login
Home::Help::EPPI-Mapper::RIS Export::About::Account Manager