Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval

Real-world data spans tables, documents, and semi-structured files with implicit semantics. Querying this data requires integrating evidence across inconsistent schemas and formats, yet existing approaches either demand costly manual engineering or bypass structure entirely. We present a system that automatically discovers an executable schema from raw multi-source data and uses it as a shared contract for knowledge graph construction and query-time retrieval. A closed-world field catalog constrains LLM-based schema discovery to attested fields; deterministic structural analysis infers identity keys, foreign keys, and source hierarchy; and the resulting schema drives extraction, deduplication, and cross-source linking into a provenance-aware knowledge graph. At query time the schema -- optionally extended via a monotonic protocol -- conditions a multi-tool agent routing retrieval across structured lookup, graph traversal, and vector search, returning grounded answers with traceable citations. In controlled zero-shot comparisons using the same LLM, data, and evaluation harness, the system improves over retrieval-only and decomposition-based baselines across four QA benchmarks, with ablations showing that schema-conditioned routing, structural intelligence, and schema-guided construction each contribute to the gains.

Paper

References (40)

Scroll for more · 28 remaining

Similar papers

Agents ingesting contracts, invoices and filings need nested JSON matching a caller-supplied schema, but 2026 extraction benchmarks report frontier models achieving five-percent field-level pass rates on complex nested documents, because repeated arrays must be row-aligned and an absent field is indistinguishable from an invented one. A naive single prompt with schema-constrained decoding guarantees syntactically valid JSON, not correct JSON: it silently fabricates plausible values for missing fields and emits no per-field trust signal, so callers ingest hallucinations. Evaluate a vision-language extractor combining page-level retrieval, per-field self-consistency across repeated re-reads and layout crops, logprob-and-agreement confidence fusion, mandatory source-span grounding with page and bounding-box coordinates, an explicit not_present sentinel, and arithmetic cross-checks on totals. Measure field-level precision, recall and F1, hallucination rate on deliberately absent fields, span-grounding accuracy and risk-coverage AUROC of the confidence, against a hand-labeled gold set of nested documents with injected omissions, beating a naive base-model prompt, constrained-decoding-only extraction, and a commercial OCR pipeline. Deliver as the prototype a minimal runnable Python MCP server (stdio) exposing the priced AI tool extract_schema_with_abstention that invokes a language or ML model to produce its output, with a typed input/output schema, an x402-style pay-per-call metering stub that records a per-call price in USDT and emits a settlement receipt, and one smoke test that exercises the tool end to end.Dogukan Ali Gundogan2026 · Zenodo (CERN European Organization for Nuclear Research) In Library
Agents ingesting contracts, invoices and filings need nested JSON matching a caller-supplied schema, but 2026 extraction benchmarks report frontier models achieving five-percent field-level pass rates on complex nested documents, because repeated arrays must be row-aligned and an absent field is indistinguishable from an invented one. A naive single prompt with schema-constrained decoding guarantees syntactically valid JSON, not correct JSON: it silently fabricates plausible values for missing fields and emits no per-field trust signal, so callers ingest hallucinations. Evaluate a vision-language extractor combining page-level retrieval, per-field self-consistency across repeated re-reads and layout crops, logprob-and-agreement confidence fusion, mandatory source-span grounding with page and bounding-box coordinates, an explicit not_present sentinel, and arithmetic cross-checks on totals. Measure field-level precision, recall and F1, hallucination rate on deliberately absent fields, span-grounding accuracy and risk-coverage AUROC of the confidence, against a hand-labeled gold set of nested documents with injected omissions, beating a naive base-model prompt, constrained-decoding-only extraction, and a commercial OCR pipeline. Deliver as the prototype a minimal runnable Python MCP server (stdio) exposing the priced AI tool extract_schema_with_abstention that invokes a language or ML model to produce its output, with a typed input/output schema, an x402-style pay-per-call metering stub that records a per-call price in USDT and emits a settlement receipt, and one smoke test that exercises the tool end to end.Dogukan Ali Gundogan2026 · Zenodo (CERN European Organization for Nuclear Research) In Library
© 2026 NYSGPT2525 LLC