Data Management for Research Databases | Training

A research database is not a record of what happened. It is a record of what we have been able to establish about what happened, together with an honest account of how well we have established it. That second half is the difficult part, and it is what this workshop is about.

Most writing about data management assumes an application database — one that defines its own truth, validates at the point of entry, and treats bad input as something to reject. Research data is the opposite case. It arrives in bulk, from sources of varying quality, long after the events it describes, and rejecting a messy row destroys evidence that a specialist could later recover.

This workshop works through what follows from taking that difference seriously: how to design a schema you will certainly have to change, how to preserve an original while making it computable, how to write an audit rule that produces tractable human work rather than an unreadable backlog, how to decide whether two records are the same person, and how to publish the result so that a stranger can tell what was checked and what was not.

Every table, column, file path, rule code, and count named in these chapters is real. Only the example records are invented, so that nothing here depends on publishing an actual person's biographical data.

Part I Foundations What kind of thing we are building, and who owns the schema

Part II Getting Data In From a researcher's spreadsheet to attributed, structured rows

Part III The Audit Layer Turning data quality problems into tractable human tasks

Part IV Getting Data Out Making trustworthiness legible to someone who was not there

09 Derivation, Release, and DownloadComputed values that are never edited, the five commitments that make a dataset citable, and why a codebook should be generated rather than written. derivation · curated dataset · codebook · disclosure

How to read this

The chapters build on each other and are best read in order, but Parts II, III, and IV each stand on their own if you already know the material before them.

Every chapter is written for two readers at once. The main text assumes no data-engineering background — it is written for a graduate student or a new research assistant. Passages marked IN THE CODE, with a blue rule down the left, give the exact files, functions, tables, and SQL for a developer. You can skip every one of them and the argument still works.

Each chapter closes with the terminology it introduced, defined once.

A note on the examples. The Taiwan Politics database holds records on real named political figures. Publishing their biographical data in a tutorial would raise disclosure questions that have nothing to do with teaching, so all example records here are invented — 林文德, 陳淑芬, 黃建宏, 吳美玲, 蔡承翰 — and recur across chapters as a consistent cast. Table names, column names, file paths, rule codes, route names, model fields, and counts are real.

A note on what is built. This workshop describes a system that is under active development. Chapters 1 through 6 describe machinery that is in production. Chapters 7 and 8 describe the release process and the identity-resolution design as they have been specified and rehearsed; parts of both are not yet deployed. Chapter 9 describes the release and codebook discipline the lab is adopting, before its first versioned dataset has shipped. Where a chapter says the lab does something, read it as the standard the lab holds itself to rather than a claim about what ran this morning.

Organization of Power Lab · National Chengchi University, Taipei. Written 2026.