Keyword searches find works that use the review's vocabulary. They miss the older books and articles that a field takes for granted, works written before the vocabulary settled, and works in venues the journal lists do not cover. The references of the seeds are the most direct route to them, because the seeds' authors have already judged them relevant.
Section 1Why follow citations
A work cited by several seeds is part of the conversation the review is joining, whatever words it uses. Following citations is also how a reviewer finds the classics of a field, the books on market-preserving federalism or embedded autonomy that every later article cites and few keyword searches return. The agent does in one pass what a reviewer would do by hand, reading every reference list and tallying how often each work is cited.
Section 2The source set
The agent extracted references from the seeds and reference documents and from the 50 highest-scoring search results of Step 3, 171 source records in all. A source needs a PDF, or a DOI through which its reference list can be looked up, so 130 were eligible, and 98 of these yielded references.
| Sources | Records |
|---|---|
| Seeds and reference documents (comparison corpus) | 121 |
| Top keyword candidates from Step 3 | 50 |
| Source records | 171 |
| with a PDF or DOI | 130 |
| that yielded references | 98 |
08_extract-citations, run again on 12 October 2026 to replace 05_extract-citations.Section 3Extracting and matching references
The agent read 4,005 references from the reference lists and then tried to match each one to a Crossref record, comparing titles, authors and years. A match supplies a DOI and clean metadata. A reference with no confident match is kept, with the details the citing article gives, since many books, chapters and non-English works have no Crossref record.
| Outcome | References |
|---|---|
| Matched to a Crossref record | 924 |
| No confident match | 2,913 |
| Too little information to search | 160 |
| Ambiguous, unresolved or failed | 8 |
| Extracted references | 4,005 |
Fewer than a quarter of the references matched a Crossref record. Crossref covers books, working papers and Chinese and Korean articles poorly, and the seeds of this review cite many works of these kinds. A low match rate is a property of the literature, not a failure of the extraction, but it means that the cited works have much less metadata than the search results.
Section 4Scoring the cited works
Before scoring, the agent looked up missing abstracts and metadata. This raised the number of cited works with an abstract from 183 to 463. It then scored the 3,684 cited works that had a title with the same method as Step 3. With a fixed cutoff of 0.90, 26 cited works were recommended. Another 13 scored above the cutoff but failed the evidence gate, which requires an abstract or citation evidence before a score is trusted.
| Measure | Count |
|---|---|
| Cited works scored | 3,684 |
| with an abstract after enrichment | 463 |
| above the 0.90 cutoff and passing the evidence gate | 26 |
| above the cutoff but failing the evidence gate | 13 |
09_score-extracted-references.Most cited works have a title and nothing else, and a title says little about similarity to the seeds. Selecting the citation pool by score alone would keep only 26 works and drop the classics the seeds rely on. That is why Step 5 also selects by how often the seeds cite a work, not only by how it scores.
- raw_references.csvEvery reference read from the source records, with the source it came from.
- citation_edges.csvWhich source cites which work, used to count citations by seeds in Step 5.
- ranked_articles.csvOne row per cited work, with its signals, combined score and evidence gate.
Terminology
- Source record
- An article whose reference list is read, here a seed, a reference document or a top search result.
- Cited work
- A work that appears in the reference list of at least one source record.
- Crossref match
- A reference linked to a Crossref record, which supplies its DOI and standard metadata.
- Enrichment
- Looking up missing abstracts and metadata for candidates before they are scored.
- Evidence gate
- A check that a candidate has an abstract or citation evidence, so that a high score does not rest on a title alone.