If you’re building document extraction into a product right now, the interesting architectural decision is no longer whether to run inference in the cloud or on-prem. That question was live in 2023. It’s mostly settled — you do both, and you route. The interesting decision is what you route on, and where the routing topology forces the rest of your architecture to go.

Here’s the scenario that pushed us to think about this properly. A client pipeline: roughly a million invoices a year, about 80% manuscript, hard data-residency requirement that nothing leaves the building. That last constraint isn’t a preference — it’s contractual, and it’s the sort of clause you find in Quebec public-sector work, in French banking, in a lot of healthcare. So local inference isn’t a nice-to-have, it’s the floor for a subset of the corpus. But you also have documents where the residency constraint is softer, the confidence threshold is high, and a cloud model — Claude, GPT-4o, o1 — will give you a materially better F1 on named-entity extraction than the local Llama-2 or Qwen deployment can. Both realities coexist inside the same client, sometimes inside the same document batch.

So the router isn’t choosing between cloud and local as deployment styles. It’s choosing per document, on two axes at once: what’s the residency class of this document, and what confidence do I need to hit on the extraction before it can flow downstream without a human touching it. When we built the router for QoreCapture’s hybrid pipeline, that was the substantive design decision — the intelligent router balances load between cloud and local models based on complexity, cost and latency, but the harder discriminant is the residency-plus-confidence pair. A low-residency, high-confidence-threshold document routes to the cloud model that gives you 90%+ F1. A high-residency document routes to the local Llama-2 stack, even if the F1 is a couple of points lower, because the alternative isn’t a better score, it’s a contractual breach.

Two things fall out of this that are non-obvious.

The first is that the connector layer inherits the routing topology, whether you designed it to or not. If a document was inferenced locally because it can’t leave the perimeter, then the validation calls that follow the extraction — checking the supplier name, resolving the PO number, verifying the GL code — cannot leave the perimeter either. Otherwise you’ve just extracted a sensitive entity locally and shipped it to a cloud API to validate it, which is worse than having sent the document to the cloud in the first place, because now you’ve fragmented the exposure and made the audit trail incoherent. So the same routing decision that placed the inference also has to place the resolution calls. In our architecture, the confidence-thresholded validation queue is bound to the residency class of the source document, not to the convenience of the connector.

The second thing that falls out — and this is where MCP gets interesting for extraction specifically, not for the generic assistant use case that most of the writing about MCP focuses on — is that the highest-value use of MCP in a document pipeline isn’t agent convenience. It’s resolution at inference time.

Consider what an extracted invoice actually is before validation. It’s a probabilistic assertion: the model thinks the supplier is a particular mid-market logistics vendor, the PO number is one specific value, the total is one specific amount, the GL code should be one specific account. Every one of those fields carries a confidence, and every one of them is wrong often enough that you can’t just write to the system of record and hope. The industry answer for the last twenty years has been human validation queues — send everything below threshold to a person. That works, and QoreCapture uses it, but it scales linearly with document volume, and at a million invoices a year the queue itself becomes the bottleneck.

What MCP lets you do — and this is what we’ve been building against on the Sage X3 and Sage 100cloud integration work — is have the extraction step itself call the system of record during inference. Not after. During. The extracted supplier name gets checked against the actual supplier master. The PO number gets checked against open POs. The GL code gets checked against the live chart of accounts. If the extracted supplier name differs from the master only in formatting — a missing “Inc.”, a case difference — and there’s no other close match, you have a resolved assertion; the extraction and the ground truth agree, and the human queue never sees this document. If the extracted supplier matches nothing, or matches three things ambiguously, then and only then does it go to human review, and it goes with the ambiguity already flagged. The probabilistic extraction has become a verified assertion, or a specifically-flagged unverified one. Those are very different objects downstream.

And here’s the piece almost nobody writes about because most MCP thinking is one-directional: it has to run both ways. QoreCapture is an MCP client into Sage, into whichever ERP or CRM the client runs — that’s the resolution direction I just described. But it also has to be an MCP server, so that a partner’s own agent, or a client’s own orchestration layer, can call extraction as a tool and keep its own downstream integrations intact. If we only did the client half, we’d be forcing every partner to accept our integration topology. If we only did the server half, we’d be a passive tool waiting for someone smarter to orchestrate us. Doing both is what collapses the integration problem from N×M — every extraction vendor times every ERP — to N+M: each extraction system speaks MCP, each ERP speaks MCP, and the wiring is a matter of configuration rather than bespoke connectors.

Now the honest tradeoffs, because none of this is free.

Resolution at inference time adds latency. Every MCP call to Sage during extraction is a network hop with a real cost budget. We’ve measured local Llama-2 inference stabilized under two seconds; a naive resolution pattern with three or four MCP round-trips per document can add another two or three on top of that. The mitigation is aggressive caching of the reference data — supplier master, open PO list, chart of accounts — with careful invalidation, because a stale supplier master will happily resolve an extraction against a supplier that was deactivated last week. Our caching layer cut redundant API calls by about 35% on the cloud-model side and the same pattern applies to MCP resolution calls, but caching a chart of accounts is not caching a Reddit thread — the freshness requirements are stricter and the failure mode is silent.

The other real tradeoff: MCP as a protocol is young. The spec is moving, the auth story is uneven across implementations, and if you’re building an MCP server that a partner’s agent will call, you are effectively publishing an API contract to a protocol whose semantics may shift under you. We’ve been treating our MCP server surface with the same versioning discipline we’d apply to any external REST API, which is more work than the “just expose it as a tool” framing suggests.

The practical takeaway, if you’re designing document AI right now: stop thinking about the model as the durable asset. In eighteen months the frontier model you’re using today will be a commodity, and the one after it will be too. The durable asset is the connector topology — the router that decides where inference happens based on residency and confidence, the resolution layer that turns extractions into verified assertions against live systems of record, and the MCP surface that lets you sit on both ends of the wire so you’re not the bottleneck in someone else’s integration. Build those well and the model layer becomes swappable, which is what you want, because it’s going to be swapped whether you want it or not.