Towards AI Economic History
Multimodal LLMs transform economic history
This week’s release of Gemini 3 Pro makes an ongoing shift even more visible: the nature of low-level work in economic history is changing. Projects that once required armies of research assistants can now be completed by a single economic historian leveraging AI, illustrating in this field what is also unfolding across the wider economy. Nowhere is this more evident than in the task that has constrained progress for decades: turning primary sources into structured datasets.
At almost every research seminar, someone raises a question and the speaker replies that they wish they had the data, but either they have not yet located the archival primary sources or constructing a dataset from the existing material would require years of manual labour. People outside the field are often surprised that much current research in economic history involves building such datasets by hand, even though optical character recognition (OCR) has existed for decades.
Traditional OCR pipelines only yield unstructured text, stripping away the visual information about how content is arranged on an archival image scan. Yet, this visual organisation matters: whether words belong to a column, a table cell, a header, a caption, or a sub-caption is determined by their spatial relationships in the image, not by their characters alone. The resulting OCR text must be transformed into a dataset suitable for econometric analysis, a step that is difficult because some of the image-level information is lost, making it challenging to parse the OCR output into a structured dataset.
Multimodal large language models (LLMs) are transformative because they can process images end-to-end, eliminating the need for layout detection, OCR, or post-OCR structuring steps. By combining strong character recognition with contextual and semantic understanding of the document as a whole, LLMs can extract the relevant information and output a structured dataset directly from a prompt.
In my work in progress, I am using Gemini 3 Pro’s predecessor model to extract information from image scans of German patent sources containing printed Gothic text. I process the images page by page because the pages are largely independent and follow a similar though not identical structure. Each page contains similar information I want to extract, which allows me to use a prompt that incorporates historical domain knowledge. Since I have thousands of pages, I send parallel requests to the model and then concatenate the resulting page-level outputs into a dataset. Constructing the same dataset by hand would have taken not just years but decades, which perfectly demonstrates the massive productivity gains made possible by AI.
Preliminary experiments with this week’s release of Gemini 3 Pro in the Google AI Studio indicate that this new model requires much less guidance and handles the semantic and spatial structure of historical sources even better. Its reported character recognition and long-context performance is also stronger than that of its predecessor. As historian Mark Humphries has shown, the new model can transcribe handwritten historical scripts with near-perfect accuracy and, surprisingly, seems to have some understanding of the material it is reproducing. This provides the foundation for constructing datasets in economic history.
Consider these two archival image scans, which show patent entries from the German patent register in 1896. I sent both images as a single PDF to Gemini 3 Pro along with this prompt:
Create an economic history dataset with patent id, technological class, status, applicant, location, representative, title, and date.
The dataset produced by the AI is absolutely astonishing. The model infers that the patent entries are organised by technological class in a double-column layout, understands which information belongs in the dataset, and distinguishes the elements that should be omitted, such as the introductory section, running headers, class headings, and the lists of patent IDs that sometimes appear after each heading. Gemini extracts all information correctly, even though the text is printed in historical German Gothic typefaces and contains older spellings. The model returns exactly the kind of dataset I would expect a research assistant to construct manually. A close inspection reveals that all transcriptions of the extracted words are completely correct.
Consider for a moment how difficult this would be without multimodal LLMs. Producing the same dataset would require an image-to-dataset pipeline with multiple steps, including image segmentation, layout detection, fine-tuned OCR, and then a rule-based or custom named-entity recognition system to parse the OCR text into a structured dataset. Most of these steps only work well with annotated datasets, the creation of which is very time-consuming in addition to the technical barriers. This should make it clear why economic historians who typically do not have a background in computer science still construct such datasets by hand.
What I find most remarkable is how well the model attends to visual elements that are distant in the PDF but clearly belong to the same semantic unit. The technological class heading “Klasse 2. Bäckerei” appears on the first page and does not appear again on the second page, yet the model still assigns the correct class to the continuing patent entries. Even for the entry with ID 87543, which begins at the bottom-right of the first page and continues at the top-left of the second page, Gemini extracts all entities correctly.
Of course, there are still limitations. Historical sources with irregular layouts, low-resource languages, or complex handwriting are most likely still prone to causing hallucinations. Moreover, the output that Gemini can produce is capped and tends to degrade as the input file becomes larger. This means that one cannot simply upload a multi-hundred-page PDF and expect a dataset in a single request. For these reasons, benchmarking remains absolutely crucial, and longer sources still require a custom LLM-based data pipeline that processes the PDF in smaller chunks. LLM-assisted coding tools now also enable researchers to vibe-code such pipelines quickly, making this more accessible to economic historians. For shorter and more straightforward documents, my suspicion is that LLM-generated datasets are already significantly more accurate than those manually constructed by humans. If current trends in AI development continue, it is easy to imagine a future in which an economic historian simply uploads a PDF to a multimodal LLM and receives a structured dataset ready for analysis.
AI will not only accelerate dataset construction but also the entire empirical research pipeline. This week, a collaboration between OpenAI, Oxford, Harvard, and other leading institutions demonstrated that AI is already accelerating research across mathematics, physics, astronomy, computer science, biology, and materials science, even helping mathematicians generate small but novel results. DeepMind’s AI co-scientist likewise showed that LLMs can generate high-value hypotheses in the biomedical domain. These developments are now openly discussed in the STEM disciplines, yet they are almost absent in economic history. This absence is surprising because the structural conditions in economic history are even more favourable. Natural sciences and other empirical fields rely on field or laboratory experiments for validation; economic history does not. The past has happened, the data is finite, and the answers are, in principle, contained in the surviving primary sources. AI agents could scrape digital archives for image scans, turn them into structured datasets, link these to publicly accessible ones, and then analyse the merged dataset quantitatively.
Economic history will enter a golden age of on-demand dataset construction and analysis. Much rigorous quantitative work will be done, many new causal channels will be identified, and many new stories will be told. What remains the same is the central human contribution, which is to ask original questions that LLMs cannot brute-force by exhaustive sampling and to locate the physical primary sources still scattered across archives worldwide.
Yet this trajectory still feels incremental, because accelerating the existing empirical pipeline produces more causal findings without necessarily bringing us closer to understanding the emergent system we call the economy. However, AI may also give rise to completely new research paradigms. As more surviving sources and artefacts are digitised, economic history could move beyond isolated causal estimates and begin to model past economies as complex systems, with world models grounded in the historical data. In the long run, this may become the defining project of AI economic history.
 | A guest post by
|
Wow, it's fascinating how you explain the shift! I'm super curious, how exactly do multimodal LLMs bridge that gap of lost image-level information for structured datasets? Such a smart analysis!