How to document consistency in coding in teamwork

If you are working in a group, checking the consistency of your codes is good research practice. Consistency is important for projects where the group needs to be in agreement about why particular codes are being used in the project as well as their definitional boundaries and situational applicability. In these projects, it may be useful to agree on a code structure early and have regular discussions about how the structure is evolving.

NVivo provides two easy ways in the PC version to check consistency or coder reliability though out the project:

1. Coding stripes—in NVivo for PC you can display coding stripes for users to see what individual team members have coded and the codes they have coded at. If you assign colors to users, this is also reflected in the coding stripes. 
   
   
   

   Screen shot Feb- 2024: Lumivero (2024) NVivo (Version 14)  www.lumivero.com

   
   
   

2. Filtering in a code—in NVivo for PC, when reviewing the references in a code you can filter the content to show only the references coded by specific users. 
   

Screen shot Feb. 2024: Lumivero (2023) NVivo (Version 14)  www.lumivero.com Links to an external site.

• The Coding Comparison query—this query enables you on both NVivo for PC and NVivo for MAC to check for coding consistency among team members. The coding comparison query produces a statistic that represents where the team has used a code on the same piece of text and where they have applied the code in a different place. This statistic is measures  'inter-rater reliability'—the degree of coding agreement between them. It is up to you and your group to decide the implication of the level of agreement. For the statistic to be valid, your coding process has to fulfill the following assumptions:
  • codes have to be mutually exclusive,
  • your group has assessed the same data,
  • your group has coded independently,
  • the same members of the group have applied the codes throughout the project to the data