# Prompt 6 — Compute inter-coder reliability

## Context
Attached are two kinds of file. `dataset_ai_coded.xlsx` (sheet `instances`) is the AI-coded dataset from Step 4, produced under codebook version 2.0. The files named `human_coded_<name>.xlsx` are blind human codings of a sample of rows drawn from that dataset; each has the same columns as the dataset and a `coder_name` column. Rows are matched across files on `instance_id`. The coded variables are `relevance`, `sector`, `event_type`, `leader_visit` and `province`; `relevance` is compared on every matched row, and the other four only on rows that every coder marked `relevance = 1`, because a non-relevant row carries no codes.

## Task
1. Load every file and match rows on `instance_id`. Report how many rows are matched across all coders and list any `instance_id` that is missing from a file or duplicated within one.
2. For each coded variable, compute:
   - percentage agreement and **Cohen's κ** for every pair of coders (the AI with each human, and each human with each other human);
   - **Fleiss' κ** and **Krippendorff's α** (nominal) across all coders together;
   - the mean of the AI–human κ values, and the band that mean falls in (≥ 0.80 reliable; 0.67–0.80 tentative; < 0.67 revise).
3. For each coded variable, produce the **agreement matrix** (confusion matrix) between the AI and the human coders: categories of the AI coding in rows, categories of the human coding in columns, cell = number of rows, with the pooled humans in one matrix and each human coder in a separate matrix. Mark the diagonal.
4. List every row on which any two coders disagree, with `instance_id`, the variable, and each coder's code side by side.
5. State the overall verdict under the workshop rule: **validated** if the mean AI–human κ of every coded variable is at or above 0.67, otherwise **iterate**, naming the variable(s) that fail.

Compute, do not estimate: use the standard formulas (or a statistics library) and show the n for every statistic. Where a statistic is undefined (for instance a variable with a single category in the sample), say so rather than reporting a number. Do not change any code in any file.

## Output
A report with: (a) the matching summary; (b) one table per variable with every pairwise κ, Fleiss' κ, Krippendorff's α, mean AI–human κ and band; (c) the agreement matrices; (d) the disagreement list; (e) the verdict and two sentences on what the pattern of disagreement suggests about the codebook. Also write the tables to `icr_report.xlsx`, one sheet per section.