Home / Checking what it gives back
Proven, or simply not yet refuted
Proven means a precise command has run and its output has been read; not yet refuted only means that nothing has caught a problem so far, a far weaker claim.
A type check proves that no type inconsistency was detected in the files examined, not that the code compiles or runs: module resolution, bundling and native dependencies remain outside its scope. A test suite proves that the paths it exercises behave as written. A headless preview, a simulated run with no human touching a real screen, proves that the components mount without crashing. None of this proves that a product genuinely works for someone holding a real device.
Two words confused under a single label
Most courses on testing blend two very different states under the word tested. Being honest separates two claims. Proven means you have run a precise command and can cite its output. Not yet refuted means nothing has caught a problem so far, a far weaker claim: a green preview or an absence of console errors only means the checks carried out did not happen to reveal a defect, not that no defect exists.
A format that looks correct illustrates the gap well. A phone number regular expression can be run and its output cited: that is proven, in the strict sense.
$ node -e "console.log(/^\+33[1-9][0-9]{8}$/.test('+33199999999'))"
trueWhat this line proves stops there: the string has the expected shape. It proves nothing about the number itself, which remains only not yet refuted until someone has actually dialled the line.
Green signals, and defects all the same
See the figure: batches were shipped with every automated signal green, type checks passing, full coverage, a clean headless preview. The person who actually installed the application on a real device found problems within minutes, none of which had been caught by any check: a slowdown that only shows up under a real touchscreen and real memory pressure, a setting tuned for the wrong hardware, and a contact button that dialled a landline an ordinary messaging app could not reach. Each of these defects was invisible by construction to a headless run, which has no finger, no screen, and no phone line.
The fix is in the report, not in more automation
Adding automated checks does not fix this class of defect, which resists automation by nature. The fix is honesty in the status report, a theme already raised about what a very rigorous review cannot see. Before declaring a piece of work finished, write down what has been proven, with the command and the output line cited, and what remains only not yet refuted, such as a clean preview or a silent console. Name in advance what can only be settled on a real device or by a real user, so that no one mistakes that silence for a verdict.
What each check proves, and what it cannot prove
| Automated check | What it genuinely proves | What it cannot prove |
|---|---|---|
| Type check passes | No type inconsistency detected between the resolved files | The syntax of files outside the typed scope, actual execution, behaviour on a real device |
| Lint passes | The code follows the style rules and dangerous patterns the linter knows about | The absence of type errors, real behaviour on a real device |
| Headless preview, zero console errors | The components mount without crashing | The screen is readable under a real finger, the animation feels smooth |
| The full test suite is green | The coded paths behave as written | The feature does what the user actually needed |
| A phone number field matches the expected format | The string has the right shape | The number reaches an active line on this real channel |
Green signals, defects found only through real use
An application screen is rendered by an automated preview tool that runs on the continuous integration server. This preview relies on a rendering engine that simulates the component structure directly in memory, on that same server. The render finishes, with no errors shown in the console.
Write, in one sentence, what this result establishes, and in one sentence what it does not establish.
What this establishes: This screen's components mount without crashing, under the conditions of this headless preview.
What this does not establish: This result does not establish that the screen stays readable under a real finger, nor that the application behaves normally on a physical device, since no one has installed it yet.
The three most common miscalibrations
- Too broad This result establishes that the screen will work correctly once installed on a real device.
- Too narrow This result establishes nothing, a headless preview never proving that the code runs.
- Beside the point This result shows that the code follows the style rules expected by the team.
- Proven denotes a precise command that has run and whose output has been cited; not yet refuted denotes only the absence of a problem noticed so far, a markedly weaker claim.
- A headless preview proves that components mount without crashing, never that a screen stays readable under a real finger or that a phone number is reachable.
- Several batches shipped entirely green still failed on real installation, see the figure, over a slowdown, a wrong hardware setting and a button dialling an unreachable line.
- Before shipping, write down separately what is proven from what is only not yet refuted, and name what only a real device can settle.
Take the last success claim you made about your own work and split it into two lists: what is proven, with the exact command and the output line cited, and what remains only not yet refuted, such as an absence of errors noticed so far.