HelpForum

Forum (Archive)

This forum is kept largely for historic reasons and for our latest changes announcements. (It was focused around the older EPPI Reviewer version 4.)

There are many informative posts and answers to common questions, but you may find our videos and other resources more informative if you are an EPPI Reviewer WEB user.

Click here to search the forum. If you do have questions or require support, please email eppisupport@ucl.ac.uk.

<< Back to main Help page

HomeHomeUsing EPPI-Revi...Using EPPI-Revi...Questions about...Questions about...Managing codeset for repeated inter-rater reliability exercise?Managing codeset for repeated inter-rater reliability exercise?
Previous
 
Next
New Post
24/06/2014 15:19
 

Hi there,

This is a general question, and I'm hoping that people will share their experiences, as well as offer up technical solutions.  We have carried out one review which involved an inter-rater reliability exercise at three points of the screening (at the beginning, in the middle, and at the end).  To carry this out we copied our original screening codeset three times, and made it multiple data entry so that we could do the IRR exercise blind (our original codeset was set to single data entry so we could generate frequencies and coding reports).  We repeated this in a shorter format for our coding phase of the review and it ended up quite messy with a lot of codesets.  

We are about to start another review and I am wondering if there is a more elegant solution.  Can anyone offer any insight?


Thanks.

Lisa

 
New Post
30/06/2014 15:52
 

Hello Lisa,

You could do this using one codeset but it would require a bit of co-ordinating when the coding tool was set for multiple data entry.

At the beginning stage your screening codeset would be set to multiple data entry and you would create coding assignments in the ‘Collaborate’ tab with items that would be used for inter-rater reliability. You would double screen the items, run comparisons to find your agreements and disagreements and then ‘complete’ the agreements and ‘reconcile and complete’ the disagreements.

You would then change the codeset back to single data entry, create new coding assignments with half of the remaining items and screen them.

When those coding assignments were finished you would then change the coding tool back to multiple data entry, create new coding assignments for the inter-rater reliability phase and repeat what you did at the beginning (double code, compare, reconcile and complete) but with the new coding assignments.

When that inter-rater reliability phase was finished you would return the codeset to single data entry, create new coding assignments with the remaining items (minus a batch that you will use for the final inter-rater reliability test) and carry on coding.

The final operation would be the final inter-rater reliability batch of items that you would double code just as you did previously.

When that was all done I would then return the coding tool to single data entry and you would have all of your items screened and completed, have 3 double screening sessions, and still be working with just one screening tool.

Best regards,
Jeff
 

 
Previous
 
Next
HomeHomeUsing EPPI-Revi...Using EPPI-Revi...Questions about...Questions about...Managing codeset for repeated inter-rater reliability exercise?Managing codeset for repeated inter-rater reliability exercise?


Copyright 2021 by EPPI-Centre :: Privacy Statement :: Terms Of Use :: Site Map :: Login
Home::Help::EPPI-Mapper::RIS Export::About::Account Manager