Text as Data Approach

Text analysis can examine topics, sentiment, frames, conversation, discourse, named entities, and the relationships expressed among them. The target determines the unit of analysis, coding instructions, and validation strategy.

Computational text analysis has moved from dictionaries and keywords to statistically generated features such as TF-IDF, bags of words and latent topics, machine-learning classifiers, embeddings and transformers, and now large language models. A structured comparison table may be added after its dimensions are approved.

Large language models support more specific and relational coding of topics, sentiment, frames, conversation, discourse, and named entities. Instead of scoring only an entire document or paragraph, researchers can ask how a particular subject evaluates a particular object within a defined context.

Named entity recognition once focused mainly on extracting broad categories such as people, organizations, places, events, and themes. LLM-assisted NER can target more precise combinations—for example, politicians who contested particular offices in specified elections—and can extract relationships alongside entities.

LLM-assisted coding raises provisional concerns about reliability across repeated runs, sensitivity to prompts and model versions, inconsistent handling of ambiguous cases, bias, missing or fabricated outputs, reproducibility, cost, and protection of source data. This filler text will be revised later.

The lab is exploring randomized validation subsets; AI-AI, AI-human, and human-human intercoder reliability; repeated coding and adjudication of disagreements; versioned prompts and model settings; structured output validation; stratified error analysis; and LLM extraction of relevant excerpts for human interpretation alongside quantitative results. This provisional list is editable in Django Admin.

This section will link to a crowd-sourced Google Doc and other practical resources for collecting, coding, validating, and interpreting text as data.