Agents ingesting contracts, invoices and filings need nested JSON matching a caller-supplied schema, but 2026 extraction benchmarks report frontier models achieving five-percent field-level pass rates on complex nested documents, because repeated arrays must be row-aligned and an absent field is indistinguishable from an invented one. A naive single prompt with schema-constrained decoding guarantees syntactically valid JSON, not correct JSON: it silently fabricates plausible values for missing fields and emits no per-field trust signal, so callers ingest hallucinations. Evaluate a vision-language extractor combining page-level retrieval, per-field self-consistency across repeated re-reads and layout crops, logprob-and-agreement confidence fusion, mandatory source-span grounding with page and bounding-box coordinates, an explicit not_present sentinel, and arithmetic cross-checks on totals. Measure field-level precision, recall and F1, hallucination rate on deliberately absent fields, span-grounding accuracy and risk-coverage AUROC of the confidence, against a hand-labeled gold set of nested documents with injected omissions, beating a naive base-model prompt, constrained-decoding-only extraction, and a commercial OCR pipeline. Deliver as the prototype a minimal runnable Python MCP server (stdio) exposing the priced AI tool extract_schema_with_abstention that invokes a language or ML model to produce its output, with a typed input/output schema, an x402-style pay-per-call metering stub that records a per-call price in USDT and emits a settlement receipt, and one smoke test that exercises the tool end to end.
Direct user-specified research topic: Agents ingesting contracts, invoices and filings need nested JSON matching a caller-supplied schema, but 2026 extraction benchmarks report frontier models achieving five-percent field-level pass rates on complex nested documents, because repeated arrays must be row-aligned and an absent field is indistinguishable from an invented one. A naive single prompt with schema-constrained decoding guarantees syntactically valid JSON, not correct JSON: it silently fabricates plausible values for missing fields and emits no per-field trust signal, so callers ingest hallucinations. Evaluate a vision-language extractor combining page-level retrieval, per-field self-consistency across repeated re-reads and layout crops, logprob-and-agreement confidence fusion, mandatory source-span grounding with page and bounding-box coordinates, an explicit not_present sentinel, and arithmetic cross-checks on totals. Measure field-level precision, recall and F1, hallucination rate on deliberately absent fields, span-grounding accuracy and risk-coverage AUROC of the confidence, against a hand-labeled gold set of nested documents with injected omissions, beating a naive base-model prompt, constrained-decoding-only extraction, and a commercial OCR pipeline. Deliver as the prototype a minimal runnable Python MCP server (stdio) exposing the priced AI tool extract_schema_with_abstention that invokes a language or ML model to produce its output, with a typed input/output schema, an x402-style pay-per-call metering stub that records a per-call price in USDT and emits a settlement receipt, and one smoke test that exercises the tool end to end.. Investigate this topic end-to-end: survey the state of the art, identify a concrete tractable research question within it, design and run an experiment, and report results.
Paper
The full text of this publication is not hosted on 44B due to licensing.
Read it at OpenAlex