Inter-coder Agreement Analysis with ATLAS ti
This video shows you how you prepare data for an inter-coder agreement analysis with ATLAS.ti and how to run the analysis. You learn about the methodological issues that you should be aware of, about the various measures that are available and why ATLAS.ti favors Krippendorff's alpha over for example Cohen's kappa.
The table of content helps you navigate through the video, or to just watch the part you are interested in.
TABLE OF CONTENT
00:00:00 Intro
00:00:28 Methodological Introduction
00:03:42 Preview of how to run an ICA analysis in ATLAS.ti
00:10:14 Inter-coder Agreement: Definition and requirements
00:12:24 Inter-coder Agreement: A critical perspective
00:13:08 Inter-coder Agreement: A pragmatic perspective
00:15:31 Why reliability matters?
00:16:39 Reliability in the context of ATLAS.ti coding
00:18:23 Development of semantic domains
00:20:55 On mutually exclusive coding
00:22:22 On multi-valued coding
00:23:01 Common mistakes
00:25:31 Measuring Inter-coder agreement
00:31:31 Krippendorff's family of alpha coefficients
00:34:51 Other measures for analyzing ICA (percent agreement and Holsti index)
00:35:22 Cohen's Kappa
00:36:07 Acceptable levels of reliability
00:36:52 ICA in ATLAS.ti: Project set-up (prepare sub projects for coders)
00:39:15 Merging sub projects from coders to run an ICA analysis
00:45:40 Running the actual analysis
00:50:58 Fixing problems with non-mutually exclusive coding
Link to Krippendorff Appendix:
https://downloads.atlasti.com/docs/IC...
Link to video on team projects:
• A8 Team work in ATLAS.ti (chapter 9)