Exploring the Corpus

An unsupervised survey answers one question: if you knew nothing about this field except these 434 works, what would you say it was about? The answer is rarely the one you expected when you built the corpus, and the difference is the first thing worth knowing.

Section 1Why explore first

A corpus built from seeds and search terms reflects your interests, but it also pulls in the field's own structure: the disciplines that publish on the topic, the debates they cite, the cases they study. Reference-chasing, in particular, brings in whatever the core works cite, whether or not it is on your topic.

Exploring first does three things. It shows which disciplines own the topic, so you know whose conventions your review must meet. It shows which subtopics are crowded and which are thin, before you commit to one. And it exposes the corpus's own problems — off-topic drift, duplicates, missing texts — while they are still cheap to fix.

The survey is unsupervised in a loose sense. You give the AI starting categories, because a survey with no categories at all comes back as an unreadable list of themes. But you tell it to merge, split, and add categories as the corpus demands, and the changes it makes are part of the result.

Section 2The prompt

This is the prompt that produced every table in this chapter, with the lab's folder paths shortened. It was given to a local agent with read access to the review folder.

You are running a survey of a literature corpus. Accuracy matters more than polish: every article you name must exist in the files.

Files (read-only):
- 3 PDFs/citation_index.csv: 434 works, one row each.
- 3 PDFs/text/<item_key>.txt: full text for 309 of them. Do not read all in full; use titles and abstracts, plus the first ~3,000 characters of a text where the abstract is missing.
- 2 Expanded Corpus/citation_index.csv: 1,261 works; scores.csv has relevance scores.

(1) Count works overall and by decade, for both corpora.
(2) Cluster the PDFs corpus by discipline (economics, political science, sociology, area studies, history, other). Report counts by cluster and by cluster × decade. For each cluster, summarise main topics, typical research questions, data and methods, with 3–5 example works (title + item_key).
(3) Cluster the PDFs corpus by topic. Start from: fiscal decentralization and local government; state enterprise reform and soft budget constraints; central–local relations and cadre control; marketization and informal economy; sanctions and trade; regional inequality and regional development; transition strategy. Merge, split or add clusters as the data demands. Same counts and summaries. Also cross-tab topic × country focus (North Korea, China, Vietnam, Eastern Europe/USSR, comparative/general).
(4) Name the 10 most cited or most central works, and say which measure you used.

Use Python for the counts and your own judgment for the classification. Classify each work into one discipline, one topic and one country, save that per-work table, and compute every count from it. Say how many works you classified from the title only.

Write: survey.md (the report), classification.csv (item_key, title, year, discipline, topic, country, basis), and process_notes.md: what you did step by step, where your classification was uncertain (five concrete examples with keys), what a human must spot-check, and any data problems in the corpus.

Section 3Counting by decade

Start with the dullest table. It tells you when the field was active and whether your corpus has a hole in it.

Read the table against what you know of the field. The 1990s peak is the comparative-transition literature on China and Eastern Europe. The 2000s and 2010s add the North Korea literature, which barely exists before 2000. If your question is about the 1990s debates, the corpus is well stocked; if it is about the last five years, 69 works is thinner than it looks, because the 2020s are not over.

Section 4Clusters by discipline

Discipline was assigned from the journal or press and the approach of each piece, not from the authors' departments. Each work received exactly one label.

The table says something a reader of any one article would not see. The China literature in this corpus is written mostly by economists. The North Korea literature is split between economists and area specialists, who publish in different journals, use different evidence — refugee surveys and night-lights on one side, official announcements and defector testimony on the other — and rarely cite each other. A review that bridges China and North Korea has to speak to both.

The survey also summarized each cluster. Two extracts show the form:

Section 5Clusters by topic

The prompt gave seven starting topics. The AI kept all seven, widened several, and added four. The additions are findings in their own right: they are the parts of the corpus the starting list did not anticipate.

Two of the additions deserve attention. North Korean reform and regime politics was added because about 56 North Korean works fit none of the starting topics: they ask whether, how, and why the regime reforms, and much of the cluster is evaluative commentary on thin evidence. And Other / off-topic — 28 works on Africa, Latin America, medieval guilds, and coltan — is the cost of chasing references. Some are deliberate background reading; several are probably accidents of the expansion step. Either way, they should be looked at before anything is excluded.

The largest cluster hides a similar problem. Of the 90 works on fiscal decentralization, 33 are not about a socialist or post-socialist case at all: they are growth regressions on OECD countries, US states, Spain, Indonesia, and Latin America. They inflate the cluster and would mislead anyone who read its size as a measure of interest in China.

Section 6The cross-tab that matters

The single most useful table in the survey is topic by country. It shows not only what the field studies, but where its tools have not travelled.

This is the most visible structural feature of the corpus, and it is the kind of finding an unsupervised survey exists to produce. The North Korea literature asks about markets from below, sanctions from outside, and reform at the top. It almost never asks how the state divides authority and revenue between centre and localities — the question that organizes most of the China literature.

A pattern like this is not yet an entry point. It might be a real gap, or it might reflect the corpus: perhaps the relevant North Korea work exists but was not captured by the search terms, or is in Korean and was classified from an English title. Chapter 4 returns to how to tell the difference. For now it is a hypothesis about the field, recorded with the table that produced it.

Section 7Which works are central?

Every review needs to know which works the field is built on. The prompt asked for the ten most central works and for the measure used. The AI used two, and the lists disagree almost completely.

Measure A is led by general works cited across all of economics — a theory of vertical integration, a paper on global integration. They are famous, but not because of this field. Worse, the count is blank for 256 of the 434 works, including most of the hand-picked seeds, so foundational works such as Kornai's 1986 article on the soft budget constraint cannot appear on the list at all.

Measure B is led by the works this corpus actually builds on: market-preserving federalism, the M-form hierarchy, Kornai's socialist system, Haggard and Noland on the North Korean famine. For a literature review it is the better guide. But it is a string-matching heuristic the AI wrote for itself. It overcounts common surnames, cannot score authors without a Latin-script surname, and excluded 79 works whose first author is called Kim, Lee, Li, Wang, or Zhang — which in a corpus on Korea and China is a large exclusion.

Section 8Checking the survey

A survey of 434 works cannot be checked work by work. It can be checked where it is weakest, and the process notes say where that is.

Where the labels are least reliable

The agent estimated that a careful human coder would disagree with perhaps 10–20 per cent of its topic labels, and named five cases. Three of them illustrate the usual problems:

1. A wrong abstract. North Korea's Economic Reform and Inter-Korean Economic Relations [7LGRX62E] carries, in the citation index, the abstract of a different book. A classifier that trusts the abstract labels the wrong work.
2. A misleading container. Fiscal decentralization and public investment [VHVHNTMZ] sits in a volume on China, so the keyword pass coded it China. The text is about Latin America.
3. A work that belongs in two places. International isolation and regional inequality: Evidence from sanctions on North Korea [2CF6VDWH] is equally about sanctions and about regional inequality. Moving it would change the small regional-inequality cluster noticeably.

The third kind matters most for small clusters. Regional inequality (22), agriculture and famine (14), and stratification (12) can each shift by several works with a handful of re-codings, so do not read much into their relative sizes.

What to spot-check

• The 33 works labelled from the title alone. They are listed by key in the process notes. A title is weak evidence of topic.
• The Other / off-topic cluster, before excluding anything. Some items are deliberate theory picks.
• The Korean- and Chinese-language works, classified mostly from English metadata.
• Both top-ten lists. For Measure B, open three or four of the citing texts for a work and confirm the matches are real citations.

What the survey found wrong with the corpus

The survey is also a data audit, and this run found real problems: one misattached abstract; nine empty or near-empty text files and one text that is mostly a repeated copyright line; five probable duplicate pairs; three works with no year; and 17 repeated titles in the Expanded cut. None of these would have been found by reading the articles one at a time, and all of them would have quietly distorted the next chapter's analysis. Fix them, or at least record them, before moving on.

Section 9AI: opportunities and challenges

The survey ends with a picture of the field and a list of its problems. Chapter 2 brings a research question to it.