Home / Real cases end to end

Verify before believing, four real traps

A report written by a third party's agent, a paid skill sold as a bundle of features, a data connector announced as synced, and a competitor's page displaying a missing feature share the same flaw: each replaces a direct measurement with a claim that would need checking before being cited.

A report written by a third party's agent, a paid skill sold as a bundle of features, a data connector announced as synced, and a competitor's page displaying a missing feature share the same flaw. Each replaces a direct measurement with a claim, and nothing distinguishes a true claim from a false one until someone has verified it independently.

The report that cites itself

An agent tasked with carrying out a job for a third party often produces, at the end of the mission, a report summarising what it did. This report is written by the same system that carried out the task: it describes its own work, with the same authority it would have to describe a successful piece of work or a failed one. A commit message, a changelog or an agent report describe an intention, never a measured state. The only way to know what was actually produced is to open the result itself, the delivered file, the modified database, the completed spreadsheet, and not the text that claims to describe it.

The skill sold as a bundle, the connector and the competitor

A paid skill presented as a bundle of features poses the same problem in a different form. The product page lists capabilities, but a page executes nothing: only the actual execution of each feature, one by one, proves that it exists and works as advertised. A data connector, for its part, displays a counter on its dashboard, a number of records supposedly synced. This figure comes from the same system that carries out the sync: it proves no more than an agent's report does. The count that counts is the one obtained by a direct export of the same data, recounted independently of the dashboard that displays it. A competitor's page claiming that a feature is missing from your product follows the same reversed logic: the claim comes from a party with an interest in it being believed, and only a test run on the competing product itself, not a reading of its marketing page, confirms or refutes the claim.

The move that forces the measurement

In all four cases, the fix is identical: before citing a figure, a feature or a verdict coming from a third party, open that source with a command that produces the same result independently. The example below fabricates its own data to illustrate the principle, a figure reported by an external system compared with a count obtained directly.

node -e "
const rapporte = 1200;
const fixture = Array.from({ length: 947 }, (_, i) => ({ id: i }));
console.log('rapporte par le connecteur :', rapporte);
console.log('compte obtenu par export direct :', fixture.length);
"

The figure reported and the count obtained directly differ by 253 units in this fabricated example. Nothing in the connector's display alone would have shown it. This same requirement for active refutation, rather than past trust, is the one already set out by fan-out research with adversarial verification: a claim kept is a claim that has withstood an attempt to contradict it, not a claim that simply went unchallenged.

What it costs not to verify

The cost of direct verification is almost always lower than the cost of the error it avoids. Recounting an export takes a few minutes. Discovering six months later that some of the records announced as synced never were costs far more. The same proportion holds for a skill feature never tested or a competitive comparison never replayed on the real product.