Three pillars — current-baseline reassessment¶
Dated 2026-09-19. Baseline da741ef0d6c924c4b6b62bd75f8c1431faddcae2.
Work order: GHI #1046; demonstrated documentation repair: GHI #1047.
Persona: main-session. This follows the commissioned three-pillars assessment;
it does not initiate an OBPI or select another campaign item.
Appraisal and diagnosis¶
| Pillar, sharpened into an observable question | Current evidence | Disposition |
|---|---|---|
| 1: Does applicable material reach the context, and what load remains? | Native Codex 0.155.1 prompt assembly delivered the entire ordered instruction chain at all four sampled starts. An intentionally reduced cap truncated the root as expected. | Retire the old truncation diagnosis for these sampled paths. Full delivery is not proof of attention or compliance; no further diet is justified by this study. |
| 2: Does the assessment preserve the relationships between relevant surfaces? | Repetitions consistently omitted one relationship per task despite discovering the surrounding code. | There is a bounded behavioral gap. Sparse attention, transformer scaling and any other internal mechanism are not identified by this experiment. |
| 3: Does discovery establish applicable obligations rather than stop at plausible material? | All flag responses omitted the manifest's command-authority versus flag-discovery distinction. All smoke responses described opt-in behavior without identifying the contradictory unconditional empty-tier prose. | Keep incomplete obligation discovery as the useful diagnosis. Neither a complete retrieval list nor a coherent answer establishes closure. No new generic instruction is supported here. |
The native measurements replace file-size inference with observed assembly. They concern native CLI input; the assessment subjects used the app's subagent harness. These instruments cannot be combined into a claim that every subject received or used every native prompt byte. The repaired tests instruction body changes the final working tree; the native numbers describe the named pre-repair baseline, not an undisclosed post-repair remeasurement.
Repeated current-code assessment¶
Two independently curated prospective tasks, three fresh subjects each. An independent reviewer challenged the oracle before the six prompts were frozen. The protocol, criteria, inputs, raw answers C01–C06, command traces, and independent scores preserve the evidence.
| Task | Runs | Complete criteria per run | Repeated missing evidence |
|---|---|---|---|
| Flag mention boundary | C01, C03, C05 | 4/5, 4/5, 4/5 | F3 relationship: manifest command obligations are not the per-flag discovery filter. |
| Smoke elapsed scope | C02, C04, C06 | 5/6, 5/6, 5/6 | S4 relationship: unconditional empty-tier prose contradicts configuration-dependent behavior. |
Primary adjudication accepts the independent scores: each missing relationship is expressly required by the frozen criterion, and the cited answers do not demonstrate it. Discovery and consistent application were credited for those criteria; this is missing evidence, not an observed harmful implementation. The scorer's semantic credit for quality-result propagation is retained: explicit propagation proposals suffice without demanding particular field names. A stricter reading would lower the smoke scores; it would not reverse the conclusion. No unsafe policy rewrite or fabricated execution was found.
All six raw files match the returned answers after trimming outer whitespace; all prompts and the oracle retain their frozen hashes. All 7,687 archived regular files were unchanged after the runs. Subjects used 8–9 commands and 668–730 words, within the 20-call/900-word limits. See verification.
Deviations: broad searches in C02, C04 and C06 exposed excluded historical evaluation/proof snippets. Subjects disclosed the exposure and said they did not use it. That statement does not certify absence of contamination. Preserve these runs with the deviation; do not silently relabel them clean held-out trials. The three flag runs have no such reported exposure. There is no clean smoke replicate on which to base a contamination-free claim.
These are two task-specific observations, not a population success rate or statistical result. No treatment arm was run. The earlier historical pilot used different tasks and snapshots, so its scores cannot establish improvement here. Fresh forks share inherited model settings; no provider build fingerprint or cross-model replication was available.
Remedy, implementation and validation¶
Independent case curation exposed a real stale-policy consumer: smoke rule,
skill, manpage and docstrings asserted unconditional empty-tier failure while
code and configuration implement an adopter opt-in. The main agent independently
read the current implementation, existing tests, GHI #724 and its landing
ce234a0fbebbb1e5eef379ddb235777b6a477824. That history settles the policy:
absent/false smoke.required means advisory exit 0; true means exit 3.
No operator policy choice was invented to resolve the conflict.
GHI #1047 corrects those coupled explanations, audit row 87 and the rule history, then regenerates packaged and harness mirrors through the normal sync command. Python changes are docstrings only. The threshold, elapsed-time boundary, membership and exit logic remain unchanged. See repair evidence.
The prospective flag matcher and smoke timer changes were experimental stimuli, not established defects or implementation work orders. They were not implemented. Other subject observations—static marker/runtime-selection disagreement, loader error handling, full-suite budget phrasing and threshold recovery authority—are recorded here as independent follow-up leads, not verified repair requirements. They do not widen the empty-tier documentation contract.
The concrete failed policy case was checked again across absent/false/true states and reviewed against repaired prose. This verifies removal of the contradiction; it is not a post-treatment model trial or evidence of improved discovery rates.
Completion boundary¶
Follow-up dated 2026-09-19: the authorized obligation-tracing remedy comparison is complete under GHI #1048. Twelve fresh assessments produced equal 28/30 totals in baseline and candidate conditions; the candidate used more calls. The predeclared adoption condition failed, so the procedure was not added to canon. These were different tasks, not an improvement comparison with this study.
This study supplies current delivery evidence, repeated discovery observations, an independently reviewed concrete repair and explicit limits. It neither claims that all three pillars are solved nor adds a general agent-comprehension gate. Production Stage 4 observation and dependency-sensitive currency design retain their separate work orders, GHI #1028 and GHI #1029. The original assessment and scholarly/practitioner appraisal remain in the recovered assessment; this is a dated current-code supplement, not a replacement research claim.