Step 3 · Framework and Codebook

This is the step where the human decides. The model has described the corpus; now the researcher states what will be measured, at what unit, with which categories. The model's role is to propose typologies and draft the codebook in a form it can itself follow later, but every category in the approved codebook is one the researcher has read, understood and accepted.

Section 1From exploration to framework

Three sentences from the end of Step 2 define the framework for the workshop example:

• Unit of analysis. The instance: one named enterprise reported in one article. An article naming two enterprises contributes two rows. An article that names no specific enterprise contributes one row marked not relevant.
• Research question. What do North Korean state media report about industrial enterprises, which sectors and provinces receive attention, how often does the leadership appear, and how has this changed between 2012 and 2025?
• Variables. A relevance screen, the enterprise's name, and four coded variables: sector, event_type, leader_visit, province. Plus, for every row, a verbatim evidence excerpt and a coder-confidence score.

Two of these are design choices worth pausing on. The instance rather than the article is the unit because the exploration showed that a quarter of articles name more than one enterprise and that sector and province are properties of enterprises, not of articles. The evidence excerpt and confidence score are not variables of interest; they exist so that every row can be audited (Step 4) and so that validation can be stratified by how sure the coder was (Step 5).

Section 2Prompt 3a: typologies for each variable

A typology is a small set of categories that is exhaustive (every case fits somewhere), mutually exclusive (each case fits in one place) and decidable from the text. The prompt asks the model to propose one per variable, to defend it with examples from the corpus, and to say where it struggled, so that the researcher's review is directed at the hard cases.

# Prompt 3a — Candidate typologies for each variable

## Context
Corpus: `corpus_nk_enterprises_2012-2025.xlsx`. The exploration memos from Step 2 are in the project folder. The unit of analysis is the **instance**: one named enterprise reported in one article. I have settled on four variables — sector, event_type, leader_visit, province — plus a relevance screen (is a named enterprise the subject of the article?).

## Task
For each variable, propose a typology — a small, exhaustive, mutually exclusive set of categories — and defend it.

1. Propose between 3 and 8 categories. Each needs a machine-readable code (lower_snake_case), a label in English and Korean, and a one-sentence rule that says when to assign it.
2. Show, with `doc_id`s, at least one clear example per category and at least two articles that sit on a boundary between categories. State the rule that resolves each boundary.
3. State the precedence order to apply when an article reports two events of comparable weight.
4. Estimate the share of instances that would fall into each category in the full corpus, flag any category expected to hold fewer than 2% of instances, and propose merging it.
5. Propose the residual category ("other" / "unknown") and say what should *not* go there.

Then propose the supporting fields each row should carry: evidence excerpt, coder confidence (define the scale), notes.

## Output
Markdown, one section per variable, each with a table (code · label · rule · example doc_id) followed by boundary cases. Finish with a numbered list of decision rules that apply across variables (multiple enterprises per article, leader quotations versus visits, military enterprises, and so on). Do not write the codebook file yet.

Review the memo this produces as you would a student's proposal. Merge categories that the model cannot reliably tell apart; split a residual category that is absorbing too many cases; and insist on a rule for every boundary the memo flags. Save your edits as codebook/typologies_reviewed.md, because the next prompts refer to them.

Section 3Researcher intervention: recoding the typologies

For the workshop corpus, Prompt 3a proposed eight sectors (metal and machine-building; chemical and building materials; energy and mining; light industry; agriculture and fisheries; transport and construction; defence; other) and seven event types (leadership visit; production; construction and modernisation; technical innovation; mobilisation; commendation; other). Each was defensible as a description of the corpus. Neither was the right instrument for the research question, and this is the moment in the workflow where the researcher, not the model, decides.

The researcher's choices are recorded in the codebook so that a reader of the dataset knows the categories are the researcher's, not the model's:

The point is not that the model's typologies were wrong. Collapsing categories trades detail for reliability: four sectors can be coded consistently by three independent coders and by a room of humans in twenty minutes; eight cannot. That trade is a research-design decision, and it belongs to the researcher. Prompt 3c records the decisions and asks the model to apply them without arguing them.

# Prompt 3c — Record the researcher's revisions to the typologies

## Context
The Step 3a memo proposed eight sector categories and seven event types for the NK enterprises corpus. I have reviewed them against the research question (sectoral attention and reported leadership attention since 2012) and against what a two-hour workshop can sustain. I am collapsing both typologies. These are my decisions as the researcher; your task is to apply them consistently, not to argue them.

## Decisions
1. **sector → four industrial sectors.** `heavy_industry` (metal, machine-building, chemical, building materials, munitions), `light_industry` (consumer goods, food, local industry), `energy` (power stations and grid), `mining` (coal, ore, salt). The v1 code `energy_mining` must be split by the enterprise's name (발전소 → energy; 탄광/광산 → mining).
2. **Non-industrial enterprises are out of scope.** Farms, fisheries, transport, construction firms, utilities and services are coded `relevance = 0` with the reason in `notes`. There is no `other` sector.
3. **event_type → four codes.** `leadership_visit`; `production` (now including technical innovation); `construction_modernization`; `political_ceremonial` (mobilisation, commendation, anniversaries and the former residual).
4. **leader_visit** (`kim_jong_un` / `premier_cabinet` / `none`) and **province** are accepted as proposed.

## Task
Rewrite `typologies_reviewed.md` so that every category table, boundary case and decision rule reflects these decisions. Where a v1 boundary case no longer arises because two categories merged, delete it; where a new boundary arises (energy vs mining; a farm that also runs a food factory), add a rule. Add a section **Researcher interventions** that lists each change above with a one-sentence rationale, so that the codebook README can carry it and a reader of the dataset knows the categories are the researcher's, not the model's.

## Output
The revised memo and a mapping table from every v1 code to its v2 code (or to `relevance = 0`).

Section 4Prompt 3b: the codebook

The codebook is the contract between the researcher, the human coders and the model. It is built from the revised typologies, not the model's first proposal. The prompt specifies its form exactly: an Excel workbook with a README, a Variables sheet, one sheet per categorical variable, a sheet of cross-variable decision rules, a sheet of dropdown values for data validation, and a blank coding template. The same content goes into a JSON file, which is the schema the extraction prompt in Step 4 reads.

# Prompt 3b — Write the codebook as an Excel workbook

## Context
We have agreed the typologies in the Step 3a memo, with my edits marked in `typologies_reviewed.md` (Prompt 3c records those edits). Now formalise them as a codebook that both a human coder and an AI coder can follow without asking questions.

## Task
Write `codebook_<project>_v2.xlsx` with these sheets:

- **README** — title, version and date, unit of analysis, research question, list of sheets.
- **Variables** — one row per variable: name, label, type (binary / categorical / ordinal / text), definition, allowed values.
- **One sheet per categorical variable** — columns `code`, `label`, `code_when` (the rule), with the residual category last.
- **Decision_Rules** — the numbered cross-variable rules, one per row.
- **Dropdown_Values** — one column per categorical variable listing its codes, so Excel data validation can be built from it.
- **Coding_Template** — a header row only: `doc_id, instance_id, article_date, article_long_title, article_url, relevance, enterprise_name, sector, event_type, leader_visit, province, evidence_text, coder_confidence, notes`.

Rules for writing it:
- Every category rule must be decidable from the article text plus the enterprise name; say explicitly when background knowledge is allowed (for example province).
- Define `instance_id` as `<doc_id>-<n>`, numbering enterprises in order of appearance.
- Define `evidence_text` as a verbatim excerpt of one or two sentences; no paraphrase.
- Define `coder_confidence` as 3 / 2 / 1 with a rule for each level.
- Keep every rule to one or two sentences. No examples longer than a title.

Also write the same content to `codebook.json` so the extraction prompt can read it programmatically.

## Output
The two files, then a short list of the places where you had to choose between two reasonable rules, so I can confirm or overrule each.

Section 5The completed codebook (version 2.0)

The workshop codebook, as approved for the extraction, is summarised below. Version 1.0 held the model's eight-sector, seven-event typologies; version 2.0 applies the interventions above and is the version every coder in Steps 4 and 5 reads. Its README sheet lists the interventions. Download the workbook to see the full rules and the coding template.

• codebook_nk_enterprises_v2.xlsxVersion 2.0, for human readers and coders. Sheets: README (with researcher interventions) · Variables · relevance · sector · event_type · leader_visit · province · coder_confidence · Decision_Rules · Dropdown_Values · Coding_Template15 KB · xlsx
• codebook.jsonThe same codebook as machine-readable JSON: version, variables with allowed values and definitions, decision rules and the recorded interventions. This is the file the extraction prompt in Step 4 loads.11 KB · json

Section 6Reviewing and approving the codebook

Before the codebook is used for extraction, test it by hand. Pick ten articles across years and code them yourself from the workbook alone, without the exploration memos open. Every hesitation is a missing rule. The questions that most often surface:

• Can every rule be applied from the text? Where the codebook allows background knowledge (here, only for province) it must say so, because a coder without that knowledge will otherwise code differently from one who has it.
• Is the residual category small? If more than about one case in ten lands in other, the typology is missing a category.
• Are the precedence rules in the right order? Rule 4 says a visit outranks a completion ceremony. For a study of construction policy you might reverse that.
• Is the scope rule defensible? Rule 1 removes farms and fisheries from the study. A reader who wants to know about agricultural enterprises will not find them in this dataset, and the methods section must say so.
• Does the unit still hold? Subordinate mines inside a 련합기업소, or enterprises mentioned only as the audience of a speech, test the instance definition. Version 2.0 settles the second (rule 2) but not the first; Step 4 shows what happened.

When you are satisfied, freeze the version. The codebook that goes into Step 4 is version 2.0; whatever Step 4 and Step 5 teach you goes into version 2.1, never into a silent edit of 2.0. Keep the version in the file name, in the README sheet and in the JSON.