A PDF is often treated as a document.

For a machine, it is usually a collection of different problems placed next to each other.

One page may contain ordinary text. The next may be an image-only scan. Further on there may be a table, a formula, a chart, a footnote, or two columns that have to be read in the right order.

That difference matters when the goal is not simply to search a folder, but to prepare a private knowledge system.

I have been building the first layer of such a system around a practical question:

How do we turn a messy PDF into evidence that can later support a reliable knowledge graph?

The answer is not to choose one powerful model and send every page through it.

The test started with real pages

I selected pages from real documents rather than constructing a clean demonstration set. The small test matrix included:

  • an ordinary text page;
  • an image-only scanned page;
  • a dense multi-column research page;
  • a table-heavy page;
  • a formula-heavy page;
  • a chart and image page.

The point was not to publish a universal benchmark. Six pages cannot establish that. The point was to expose the kinds of failure that a production pipeline has to handle.

The first observation was simple: the pages did not need the same capability.

A native OCR path handled ordinary pages in well under a second in the earlier test. A specialist layout parser took roughly ten to twenty seconds on many of the more complex pages. A general multimodal model worked, but took longer again. A long-document model had a different promise: process many pages together while preserving structure and reducing the cost of a growing attention history.

Those results did not produce one winner.

They produced a routing problem.

The fastest path should not process every page

If a PDF contains a trustworthy text layer, the system should use it before invoking OCR. Reconstructing text that is already present is unnecessary work and can introduce errors.

If a page is a simple scan containing ordinary prose, a fast local OCR path is usually sufficient.

If the page contains a table, formula, chart, multiple columns, or an unusual layout, the system should escalate it to a layout-aware specialist.

If the result is incomplete, contradictory, or important enough to require a second opinion, another model can adjudicate it.

The routing is deliberately explainable. The system should be able to say:

  • the page had no usable text layer;
  • the layout detector found a table;
  • the page contained several columns;
  • the first OCR result fell below a quality threshold;
  • a second processor was required because the page was high risk.

This is less glamorous than training a mysterious router, but it is more useful at the beginning. We can inspect the reason for every decision, measure the failures, and only later decide whether a learned router is justified.

The impressive model still failed a useful test

One of the models produced exactly the kind of output that attracts attention in a demonstration.

It could emit text blocks with coordinates. It identified tables and equations. It marked page numbers. Its output included records similar to:

text [x1, y1, x2, y2]
table [x1, y1, x2, y2]
equation [x1, y1, x2, y2]
page_number [x1, y1, x2, y2]

That is valuable because a knowledge system needs more than a bag of recognized words. It needs to know where the words came from and what kind of region produced them.

But the model failed another test.

On a chart and image page, the output degraded into repetitive numeric tokens. On a multi-page run, it preserved page markers but began repeating content across page boundaries. The output looked structured enough to be dangerous.

This changed the decision.

The model was not rejected because it was unintelligent. It was rejected as the canonical extractor because the failure mode was not yet controlled. A system that silently assigns the wrong text to the wrong page can be worse than one that reports an explicit failure.

A strong model output is not automatically a trustworthy artifact.

The architecture that followed

The current design has several separate stages:

source policy
    -> document inspection
    -> page-level classification
    -> least-sufficient local processor
    -> specialist escalation
    -> immutable page artifact
    -> lexical index and citations
    -> structured extraction
    -> knowledge lattice

The word “immutable” is important here.

For each processed page, the system should preserve at least:

  • the source identity and content hash;
  • the document version and page number;
  • the rendering settings;
  • the selected route and the reason for it;
  • the extracted text;
  • regions and coordinates where available;
  • model and runtime provenance;
  • confidence values and warnings;
  • start, completion, and failure status.

The derived index can be rebuilt. A routing policy can change. A better model can be installed later. The original evidence and the record of what happened should remain available.

This also makes the worker resumable. If a job fails on page 47, the system should not need to throw away pages 1 through 46. If a model is replaced, we should be able to reprocess only the affected artifacts.

That is ordinary engineering discipline, but it is often missing from AI document demos.

Why the graph needs time

The next layer is a knowledge graph, but not one that simply merges every extracted statement into the latest version of reality.

Documents change.

A policy is revised. A technical specification is corrected. A research paper is followed by an erratum. A contract has drafts, signatures, amendments, and expiration dates. A decision may have been reasonable when it was made and wrong after new evidence appeared.

If the system replaces the old document with the new one, it loses part of the history needed to understand the decisions that followed.

A better representation keeps versions connected:

Document v1 --superseded_by--> Document v2
     |                              |
  claim A                       revised claim A
  valid interval                new valid interval

Old versions do not all need to remain active. Some can be marked obsolete. But they should not disappear merely because a newer file arrived. Historical versions may be required to explain what was known at a particular time.

The graph therefore needs more than nodes and edges. It needs temporal scope, intervals, and causal direction.

A relationship may need to say:

  • this event preceded that decision;
  • this policy caused a workflow change;
  • this claim was valid during a particular interval;
  • this document corrected an earlier statement;
  • this version was the source available when the decision was made.

That is the reason for the name Lattice.

The objective is not another flat collection of connected concepts. It is a structure that can preserve relationships while also preserving their order, duration, and changing validity. A left-to-right attention over causal and temporal sequences can help the system follow how knowledge changes without losing the graph of entities and relationships around it.

The document worker is therefore not the knowledge system itself. It is the evidence layer that makes the later structure possible.

From documents to activities and decisions

Documents are evidence of work. They are not the work itself.

A useful knowledge system should eventually connect documents to the things that matter operationally:

  • activities and workflows;
  • systems and tools;
  • people and responsibilities;
  • policies and constraints;
  • decisions and their rationale;
  • outcomes and later corrections.

That is different from asking a chatbot to answer questions about a folder of PDFs.

The purpose of the extraction pipeline is to make those connections safer to build. A document can support an entity, an activity, or a relationship, but the graph should retain the pointer back to the page and version that provided the evidence.

Machines can propose entities and relationships. Humans or institutional processes can approve, merge, refine, or reject them. The graph can grow gradually, without pretending that the entire enterprise has been modeled correctly on the first day.

Embeddings can add useful similarity search. They should enrich the structure, not replace it.

The order matters:

evidence first
structure second
embeddings third

What this changes about private AI

The private part is not only about keeping files on a local machine.

It is also about retaining control over the processing path:

  • which collections are allowed into the system;
  • which models can see them;
  • which pages require escalation;
  • what is preserved as evidence;
  • when a result is provisional;
  • who or what can approve a relationship;
  • how an answer can be reconstructed later.

A local model can still produce an unreliable knowledge base if the system discards provenance, overwrites versions, or treats every generated statement as fact.

Conversely, a modest local model can be useful when it is placed inside a pipeline that limits its role, records its work, and keeps deterministic boundaries around it.

That is the larger lesson from the PDF work.

The difficult part was not finding a model that could read a page. Several models could do that.

The difficult part was deciding what each page required, recognizing when a model had failed, preserving the evidence, and creating a path from extracted text to time-aware relationships without pretending that the latest document erased the past.

The goal is not a chatbot that remembers the latest file.

The goal is a knowledge system that can explain what was known, when it was known, which version supported it, what changed, and why a later conclusion may be different.

That work starts with messy PDFs.