"
Large language models are being adopted at pace as research instruments in political science — for coding text, scaling ideological positions, classifying discourse, and
detecting disinformation. The appeal is real: LLMs reduce annotation costs, scale to corpora that human coders cannot process, and produce outputs that look like
political science measurements. The problem is that they are not. Political science measurement rests on a conceptual infrastructure — operationalization, construct
validity, theoretical grounding — that LLMs do not inherit from training data alone.
When a model classifies a tweet as expressing extremism, it does so by predicting a
token distribution over a corpus whose normative commitments are unexamined. The
discipline's own measurement tradition, from Sartori's conceptual stretching to
Adcock and Collier's validity framework, provides the tools to identify what is being lost. This paper argues that the danger is not primarily bias or hallucination —
risks that receive substantial attention — but a quieter one: that political science
cedes conceptual authority over its own categories to a statistical model trained on
data it did not curate, for purposes it did not define. The argument is developed in
three steps: (1) why LLMs constitute political categories rather than measuring them; (2) why this represents a qualitative break from prior methodological imports
(survey instruments, formal models, network analysis) which provided analytical
frameworks while the discipline retained conceptual sovereignty; and (3) what
governance a discipline that takes its own measurement tradition seriously would
impose on LLM use. The paper closes with concrete implications for research design,
not prohibitions.