Download to read offline
Uploaded byMuhammad Awais
Reliability in Content Analysis and How to Measure It - Mass Media Research.pptx
The document discusses the concept of reliability in coding, emphasizing the importance of agreement among independent coders. To achieve acceptable levels of reliability, it recommends defining category boundaries, training coders, and conducting a pilot study. It also highlights the calculation of intercoder reliability using specific methods and formulas.
Reliability in Content Analysis and How to Measure It - Mass Media Research.pptx
- 1. Reliability refersto levels of agreement among independent coders who code the same content using the same coding instrument. If the results fail to achieve reliability, something is amiss with the coders, the coding instructions, the category definitions, the unit of analysis, or some combination of these. To achieve acceptable levels of reliability, the following steps are recommended: 10/14/2022 Reliability Muhammad Awais (facebook.com/awwaiis)
- 2. 1. Definecategory boundaries with maximum detail. Protocol sheet should be given with ample comprehensive. Examples of unit of analysis 2. Train the coders The result of the training sessions is a “bible” of detailed instructions and coding examples, and each coder should receive a copy. 3. Conduct a pilot study. 10/14/2022 Muhammad Awais (facebook.com/awwaiis)
- 3.
- 4. When theinitial test of reliability yields satisfactory results, the main body of data is coded. After the coding is complete, it is recommended that a subsample of the data, probably between 10% and 25%, be reanalyzed by independent coders to calculate an overall intercoder reliability coefficient. Intercoder reliability can be calculated by several methods. Holsti (1969) reports this formula for determining the reliability of nominal data in terms of percentage of agreement: Reliability= 2M divided by N1+N2 10/14/2022 Muhammad Awais (facebook.com/awwaiis)