Hugging Face researchers have released ProvenanceGuard, a verification system designed to catch a subtle but critical flaw in tool-using AI agents: when they state a fact correctly but attribute it to the wrong source. The problem, which the team calls 'cross-source conflation,' occurs when agents have access to multiple data sources via the Model Context Protocol (MCP)—such as patient records, policy documents, and research databases—and weave facts from different sources into a single answer.
In a customer support scenario, for example, an agent might claim 'According to the account record, this plan includes a 30-day refund window,' when the refund policy actually comes from a separate policy document. To a source-blind verifier, the claim looks sound because both sources are in the evidence pool. But the attribution is wrong, and in sensitive applications like healthcare or finance, a misattributed fact can be as damaging as a false one.
ProvenanceGuard sits as a post-generation verification layer that preserves the identity of each source throughout the checking process. Rather than pooling evidence together, it breaks answers into individual claims, identifies the most relevant source for each, verifies that the source actually supports the claim, and confirms the stated attribution matches. When tested on medical agent traces, the system caught 138 of 139 unsupported claims with only 67 false positives among 361 total claims—reflecting a conservative approach that flags borderline cases for human review rather than letting errors through.
Key Points
ProvenanceGuard addresses 'cross-source conflation'—when AI agents correctly cite facts but attribute them to the wrong source
The system preserves source identity through verification instead of pooling evidence, matching claims against their stated sources
Medical testing showed ProvenanceGuard caught 138 of 139 unsupported claims with minimal false positives
Source attribution errors are especially critical in data-sensitive fields like healthcare, customer support, and finance