A tool that does what isn't its job: why a skill is an open system

Origin. The observation began with a failure in our linguistic skill: it missed a class of defects, and asking why led to a more general property of skills. We formalized it and tested it in an autonomous run, reading against the data in an attempt to refute rather than confirm it.

We have skills — packaged procedures the model invokes for a specific job: parse a text, find data, restore structure after a crash. The usual way to think about a skill is as an instruction: a list of steps, follow them in order. This post is about why that picture is incomplete, and where it breaks.

The failure that started it

Our linguistic skill describes a text: register, devices, word-formation, intonation. One day it missed a defect a human editor caught at a glance. The autopsy found something unexpected: the skill wasn't broken. It honestly did exactly what it can do — it described. The problem was not inside the procedure but on its boundary: a rich, detailed descriptive output was read as a verdict — as a good/bad judgment the skill never issued and structurally cannot issue.

Hence the working formulation: a skill is not a closed system. Part of its correctness lives not in the body of the procedure but on the boundaries with the outside world, and the body cannot contain those boundaries.

Two boundaries

A skill has two boundaries, and neither is self-sufficient.

The input boundary (world → skill). A skill does not always know which "regime" its input arrived in. A linguistic reading of literature and a reading of a technical system-description are different jobs, and a procedure sufficient for the first is insufficient for the second. Which regime it is may be decided not by the skill's body but by whoever invoked it.

The output boundary (operator → skill). Correctness holds because judgment stands above the output. If the output can be taken for a finished verdict — and there is no judgment above it — the skill de facto seizes a decision it did not earn.

Testing the observation: read against yourself

It is easy to fall in love with a neat formulation and start confirming it. We did the opposite — we ran it against nine of our skills, deliberately including ones that should refute it.

And they refuted — the naive version. A mechanical skill (launching a local model, recovering from a crash) keeps the input boundary inside the body: it checks the amount of free memory, distinguishes a real crash from a normal exit by a concrete signal. Its output is factual and checkable — no verdict-illusion. So "every skill is open on both boundaries" is false.

Then an independent check (a separate instance of the model, given only the hypothesis and the task of breaking it) found a subtler counterexample — and it proved more valuable than a confirmation. It exposed that the very phrase "verdict-illusion" was a label over two different things:

• masquerade — a descriptive output taken for an evaluation it never made (our linguistic case);
• overreach — an explicitly evaluative output whose confidence exceeds its actual coverage (a "complete / whole" verdict taken for "correct").

What survived — a stronger principle

The observation did not die from the refutation — it grew stronger. The two boundaries (input/output) and the two sub-illusions (masquerade/overreach) turned out to be facets of a single rule:

For each boundary of a skill: if the correctness check at that boundary can be decided from inside the body — build it in. If it cannot (it needs judgment, authorization, an external fact) — mark the boundary explicitly and hand it to the operator. Do not pull it into the body: an overloaded instrument starts doing what isn't its job. The defect is leaving a boundary neither built-in nor marked.

The same divider — "is the check decidable from inside?" — governs both boundaries. A mechanical skill builds in what is decidable (a memory number, a crash signal). A specification skill externalizes what is not (formal "completeness" is blind to "does it actually work"). Both are right — because both apply one criterion to different situations.

A side finding: a defect in our own skill

The same analysis found a real defect — in one of our own structural-audit skills. It emits a "does the architecture serve its goals: yes/no" verdict, but neither marks the boundary (this is a finding for judgment, not a proof) nor builds in a completeness check. Neither built-in nor marked — exactly the defect the rule describes. This is the typical outcome of an honest audit: you test the principle against your own components and find one that fails it. The fix is a separate job; what matters here is not the fix but that the audit found it on its own.

Why this is a feature, not a bug

The existence of external boundaries for a skill is a property of the environment, not a fault. A skill is always invoked into a context it does not control; so part of correctness necessarily lives on the boundary. This is not fixed by "add one more step inside" — it is accounted for: mark the boundary and route the judgment to where it lives.

The practical consequence for how we build skills: align them per-boundary, not by a common template. Ask, for each boundary: is the check decidable from inside or not? — and either build it in or mark it. Not a blanket tax "add judgment everywhere" (on mechanical skills that is a false positive), but a targeted decision by criterion.

Boundaries of the analysis

Honest caveats. This is a property of the corpus of our skills (nine of them, one series of observations), not a proven law about skills in general. It survived one independent attempt at refutation and came out stronger — that reinforces but does not prove. "A skill behaves like a mini-model" is a working lens, not a literal claim (a skill has no weights): it behaves as if, inducing a dominant in the host model's processing. And the link of this phenomenon to silicon psychology — why judgment is prone to abdicating before a ready external verdict — we keep separate, as research material, not as an established mechanism.