This sheet compares main at sha-2bff81a (Main to RC forecast) against production 17.0.0, run 2026-10-19 10:46 UTC.
Gate · Main to RC forecastFAILsha-2bff81a beside 17.0.0 · 2026-10-19 10:46 UTC full report · the run · scorecard 40 (D) · 2 glance itemsFAIL because 8 differences have no claim and no PR declared them (#501's CHANGELOG line says (image-hardening, image-manifest, metrics); startup moved; #481's CHANGELOG line says (network); #505's CHANGELOG line says (dom, focus); persistence moved). The other 84 are declared in its PR's CHANGELOG line (#505: 82, #501: 2): nothing is owed on main, and the release author claims them from those lines when the release is cut. A CHANGELOG line that names what moved, a Rule that claims it, or a fix on main turns this to PASS; the gap below says where each came from.
Release Confidence ScorenoneThe Gate wins: no number talks a FAIL back on
The eight dimensions (for the root cause, not a decision)
Claim coverage0
Noise health80
Statistical margin100
Rehearsals100
Test signal10
Requirements traceabilityn/m
Change risk0
Post-deploy historyn/m
Quality · a reading aid beside the Gate, never an input to it
!
Speedlookp95 2.03 ms on production, 2.01 ms on main; 0 of 600 requests failed; 4 slowdowns to look at, advice only: catalogue: ready slower on b: median 2366 ms → 5471 ms (+131%), p=0.012, n=5/5; live: ready slower on b: median 2362 ms → 5342 ms (+126%), p=0.012, n=5/5; reader: ready slower on b: median 2370 ms → 5392 ms (+128%), p=0.012, n=5/5; and 1 moreWithin reason when: No journey, page or startup significantly slower (advice: timing never fails the Gate); k6 failure rate 0 on mainstartup:catalogue/ready:105 · startup:live/ready:106 · startup:reader/ready:107 · the load
Testslookharness half: within reason!monorepo half: lookHarness: within reason. Monorepo (record for the judged commit, 68m old): 3 changed packages below 80%: @tutors/ui-navigators 55.11%, @tutors/i18n 72.73%, @tutors/metrics 75.93%; the Nightly within reason.Within reason when: Harness: every journey completed on both sides, the A/A clean and at most 2 days old, every mutant caught. Monorepo: the Nightly green on the commit, no changed package below 80% mutation scorethe A/A · self-test of 2026-10-19 · quality.json · the Nightly run
Tests read the monorepo's quality record for the judged commit, generated 2026-10-19 (68m before the forecast). The strip is a reading aid: it never changes the Gate, the score or an exit code.
What each light read
Light
Check
State
Reading
Speed
No journey or startup significantly slower, or outside production's control limits
look
4 slowdowns (advice, not a failure): catalogue: ready slower on b: median 2366 ms → 5471 ms (+131%), p=0.012, n=5/5; live: ready slower on b: median 2362 ms → 5342 ms (+126%), p=0.012, n=5/5; reader: ready slower on b: median 2370 ms → 5392 ms (+128%), p=0.012, n=5/5; and 1 more; control chart: every journey's median on main inside production's limits (18) startup:catalogue/ready:105 · startup:live/ready:106 · startup:reader/ready:107
Speed
k6 failure rate 0 on main
within reason
p95 2.03 ms on production, 2.01 ms on main; 0 of 600 requests failed the load
Countermeasures open2fixes a 5 Whys asked for: 2 built, waiting on a release to show they work · 1 verified · each one
Fixed on main · good news, never a failure0% with a note2 console errors production shows and main does not, none with a claim saying why yet. These count for main, not against it. A claim is an optional note that the fix was meant; nothing waits on it · which ones
No RCS: Gate FAIL. The Gate wins: no number talks a FAIL back on. The dimensions are shown for the 5 Whys, not for a decision.
The gap, explained every difference main at sha-2bff81a (Main to RC forecast) against production 17.0.0, run 2026-10-19 10:46 UTC found, in one row each
103 differences: claims explain 11; #505 declared 82 in its CHANGELOG line; #501 declared 2 in its CHANGELOG line; #501 contributed 4 (#501's CHANGELOG line says (image-hardening, image-manifest, metrics); startup moved); #478 most likely contributed 4 (#481's CHANGELOG line says (network); #505's CHANGELOG line says (dom, focus); persistence moved); 0 unexplained.
Explained by
Differences
Share
What changed (kinds)
First seen
Claimed: a Rule or CHANGELOG entry says so
11
11%
Rule 0330: When a student opens a learning object, the reader shall send the learning event to the reader's server without publishing it over Realtime from the browser. — Rule 0330: the reader sends each learning event to its own server (POST /api/live/activity) instead of broadcasting it from the browser (#481)
8
8%
Rule 0330: When a student opens a learning object, the reader shall send the learning event to the reader's server without publishing it over Realtime from the browser. — Rule 0330: moving learning events from browser Realtime to the reader's server adds one immutable chunk on reference:course and reference:note (#478, #481)
2
2%
Rule 0331: When the reader's server receives a learning event, the reader shall publish it only to the activity feeds a live dashboard has watched in the last 90 seconds. — Rule 0331: the reader's server claims the topics to publish to, and charges the day's Live budget, through realtime_claim_live_publish (#481)
1
1%
Declared by its PR: the CHANGELOG line names it, claimed when the release is cut
persistence: RPC realtime_claim_live_publish — # row(s) on a, # on b (3) persistence: RPC realtime_watch — # row(s) on a, # on b (1)
2026-10-18, 2026-10-18
Unexplained: the record cannot say who
0
0%
The behaviour behind these Rules moved: Rule 0582 (exercises 0011, 0250, 0252, 0260, 0303, 0330): 9 differences in dom, focus, persistence; Rule 0591 (exercises 0319): 8 differences in dom, focus; Rule 0584 (exercises 0047, 0049, 0050): 6 differences in dom, focus; Rule 0588 (exercises 0052, 0053, 0054): 6 differences in dom, focus; Rule 0589 (exercises 0330, 0335): 6 differences in dom, focus, persistence; Rule 0590 (exercises 0319): 6 differences in dom, focus; Rule 0585 (exercises 0037, 0038, 0039): 4 differences in dom, focus; Rule 0586 (exercises 0011, 0267): 4 differences in dom, focus; Rule 0583 (exercises 0046, 0064, 0280): 3 differences in dom, focus, persistence; Rule 0587 (exercises 0042, 0284): 2 differences in dom, focus; Rule 0592 (exercises 0275, 0276): 2 differences in dom, focus.
A PR is named when the difference first appeared in the forecast after it merged and its files touch what changed; several PRs merged the same night stay one row ("one of"), never split by guesswork. A PR whose CHANGELOG line names the artefact that moved, as CONTRIBUTING.md asks ((axe, dom)), declared it: nothing is owed on main, and the release author claims it from that line. Read from 15 earlier forecasts beside the same production, 2026-10-15 to 2026-10-19.
Timing on control charts advice, never a failure: an outlier against production's own variation is something to investigate
1 of 18 journeys outside production's limits: investigate it, starting with "what to check" in the report.
Journey
Main's median night by night (dots), production's limits (band)
Reading
anonymous-student-searches
special cause: below LCL main 2313ms is below the limits from production's last 14 nights (2348–3180ms): a special cause to investigate, not a failure Limits from production's median on each of the last 14 nights: centre 2,764ms is their mean; the average jump from one to the next (mR̄) is 156ms; UCL = 2,764ms + 2.66 × 156ms = 3,180ms; LCL = 2,764ms − 2.66 × 156ms = 2,348ms.
anonymous-student-reads-course
common cause main 4447ms is inside the limits from production's last 14 nights (4134–5703ms): common cause, nothing to investigate Limits from production's median on each of the last 14 nights: centre 4,918ms is their mean; the average jump from one to the next (mR̄) is 295ms; UCL = 4,918ms + 2.66 × 295ms = 5,703ms; LCL = 4,918ms − 2.66 × 295ms = 4,134ms.
catalogue-loads
common cause main 1227ms is inside the limits from production's last 14 nights (1187–1717ms): common cause, nothing to investigate Limits from production's median on each of the last 14 nights: centre 1,452ms is their mean; the average jump from one to the next (mR̄) is 99ms; UCL = 1,452ms + 2.66 × 99ms = 1,717ms; LCL = 1,452ms − 2.66 × 99ms = 1,187ms.
live-loads
common cause main 1317ms is inside the limits from production's last 14 nights (1170–1796ms): common cause, nothing to investigate Limits from production's median on each of the last 14 nights: centre 1,483ms is their mean; the average jump from one to the next (mR̄) is 118ms; UCL = 1,483ms + 2.66 × 118ms = 1,796ms; LCL = 1,483ms − 2.66 × 118ms = 1,170ms.
reference-course-reads
common cause main 6157ms is inside the limits from production's last 14 nights (5846–8847ms): common cause, nothing to investigate Limits from production's median on each of the last 14 nights: centre 7,347ms is their mean; the average jump from one to the next (mR̄) is 564ms; UCL = 7,347ms + 2.66 × 564ms = 8,847ms; LCL = 7,347ms − 2.66 × 564ms = 5,846ms.
replay-course-urls
common cause main 7250ms is inside the limits from production's last 14 nights (0–43133ms): common cause, nothing to investigate Limits from production's median on each of the last 14 nights: centre 16,849ms is their mean; the average jump from one to the next (mR̄) is 9,881ms; UCL = 16,849ms + 2.66 × 9,881ms = 43,133ms; LCL = 16,849ms − 2.66 × 9,881ms = 0ms.
rule-0582-signed-in-lab
common cause main 4192ms is inside the limits from production's last 7 nights (3496–5924ms): common cause, nothing to investigate; provisional limits, 7 nights Limits from production's median on each of the last 7 nights: centre 4,710ms is their mean; the average jump from one to the next (mR̄) is 456ms; UCL = 4,710ms + 2.66 × 456ms = 5,924ms; LCL = 4,710ms − 2.66 × 456ms = 3,496ms. Provisional: under 10 points, so they will move as more are added.
rule-0583-presence
common cause main 3269ms is inside the limits from production's last 7 nights (1318–8863ms): common cause, nothing to investigate; provisional limits, 7 nights Limits from production's median on each of the last 7 nights: centre 5,091ms is their mean; the average jump from one to the next (mR̄) is 1,418ms; UCL = 5,091ms + 2.66 × 1,418ms = 8,863ms; LCL = 5,091ms − 2.66 × 1,418ms = 1,318ms. Provisional: under 10 points, so they will move as more are added.
rule-0584-quiz-notebook-talk
common cause main 12880ms is inside the limits from production's last 7 nights (7786–15929ms): common cause, nothing to investigate; provisional limits, 7 nights Limits from production's median on each of the last 7 nights: centre 11,858ms is their mean; the average jump from one to the next (mR̄) is 1,531ms; UCL = 11,858ms + 2.66 × 1,531ms = 15,929ms; LCL = 11,858ms − 2.66 × 1,531ms = 7,786ms. Provisional: under 10 points, so they will move as more are added.
rule-0585-preferences
common cause main 4606ms is inside the limits from production's last 7 nights (3085–10501ms): common cause, nothing to investigate; provisional limits, 7 nights Limits from production's median on each of the last 7 nights: centre 6,793ms is their mean; the average jump from one to the next (mR̄) is 1,394ms; UCL = 6,793ms + 2.66 × 1,394ms = 10,501ms; LCL = 6,793ms − 2.66 × 1,394ms = 3,085ms. Provisional: under 10 points, so they will move as more are added.
rule-0586-private-course
common cause main 10604ms is inside the limits from production's last 7 nights (6386–12763ms): common cause, nothing to investigate; provisional limits, 7 nights Limits from production's median on each of the last 7 nights: centre 9,574ms is their mean; the average jump from one to the next (mR̄) is 1,199ms; UCL = 9,574ms + 2.66 × 1,199ms = 12,763ms; LCL = 9,574ms − 2.66 × 1,199ms = 6,386ms. Provisional: under 10 points, so they will move as more are added.
rule-0587-calendar-llm
common cause main 3382ms is inside the limits from production's last 7 nights (1249–9450ms): common cause, nothing to investigate; provisional limits, 7 nights Limits from production's median on each of the last 7 nights: centre 5,350ms is their mean; the average jump from one to the next (mR̄) is 1,542ms; UCL = 5,350ms + 2.66 × 1,542ms = 9,450ms; LCL = 5,350ms − 2.66 × 1,542ms = 1,249ms. Provisional: under 10 points, so they will move as more are added.
rule-0588-content-lock
common cause main 15667ms is inside the limits from production's last 6 nights (5576–23942ms): common cause, nothing to investigate; provisional limits, 6 nights Limits from production's median on each of the last 6 nights: centre 14,759ms is their mean; the average jump from one to the next (mR̄) is 3,452ms; UCL = 14,759ms + 2.66 × 3,452ms = 23,942ms; LCL = 14,759ms − 2.66 × 3,452ms = 5,576ms. Provisional: under 10 points, so they will move as more are added.
rule-0589-live
common cause main 4717ms is inside the limits from production's last 6 nights (0–19111ms): common cause, nothing to investigate; provisional limits, 6 nights Limits from production's median on each of the last 6 nights: centre 7,829ms is their mean; the average jump from one to the next (mR̄) is 4,242ms; UCL = 7,829ms + 2.66 × 4,242ms = 19,111ms; LCL = 7,829ms − 2.66 × 4,242ms = 0ms. Provisional: under 10 points, so they will move as more are added.
rule-0590-time-calendar
common cause main 5482ms is inside the limits from production's last 6 nights (4474–8609ms): common cause, nothing to investigate; provisional limits, 6 nights Limits from production's median on each of the last 6 nights: centre 6,541ms is their mean; the average jump from one to the next (mR̄) is 777ms; UCL = 6,541ms + 2.66 × 777ms = 8,609ms; LCL = 6,541ms − 2.66 × 777ms = 4,474ms. Provisional: under 10 points, so they will move as more are added.
rule-0591-time-labs
common cause main 6245ms is inside the limits from production's last 6 nights (5566–10264ms): common cause, nothing to investigate; provisional limits, 6 nights Limits from production's median on each of the last 6 nights: centre 7,915ms is their mean; the average jump from one to the next (mR̄) is 883ms; UCL = 7,915ms + 2.66 × 883ms = 10,264ms; LCL = 7,915ms − 2.66 × 883ms = 5,566ms. Provisional: under 10 points, so they will move as more are added.
rule-0592-whiteboard
common cause main 13925ms is inside the limits from production's last 5 nights (13213–14360ms): common cause, nothing to investigate; provisional limits, 5 nights Limits from production's median on each of the last 5 nights: centre 13,787ms is their mean; the average jump from one to the next (mR̄) is 216ms; UCL = 13,787ms + 2.66 × 216ms = 14,360ms; LCL = 13,787ms − 2.66 × 216ms = 13,213ms. Provisional: under 10 points, so they will move as more are added.
student-signs-in
common cause main 3381ms is inside the limits from production's last 14 nights (3361–4604ms): common cause, nothing to investigate Limits from production's median on each of the last 14 nights: centre 3,983ms is their mean; the average jump from one to the next (mR̄) is 234ms; UCL = 3,983ms + 2.66 × 234ms = 4,604ms; LCL = 3,983ms − 2.66 × 234ms = 3,361ms.
How to read it: the band is how much production's own time for the journey varies when nothing has changed. The UCL (upper control limit) is the top of that band and the LCL (lower control limit) the bottom, each the centre ± 2.66 times the average jump between consecutive points (an XmR chart; 2.66 puts the limits about three standard deviations out, so a point beyond one happens by chance well under 1 time in 100). A dot inside the band is ordinary variation; a red dot outside it is a special cause, something changed, worth investigating. Limits come from production's nightly medians beside the same production, or from its runs in the newest forecast until there are two nights.
1Background why this matters
Tutors ships four apps (reader, catalogue, live, time) as signed images. The release harness runs each candidate beside production, diffs everything a student can observe, and stops the line unless every difference is claimed by a Rule or a changelog entry.
This A3 reads every run the harness has kept: 54 runs from 2026-09-26 to 2026-10-19 (Release candidate 5, Main to RC forecast 35, Nightly A/A 14), the kaizen register (3 filled 5 Whys), and the workflows' own history on GitHub. It is rebuilt with the site, so it always describes the newest evidence.
Why it matters: production is 17.0.0; main (sha-2bff81a) is what the next release would ship. Every unclaimed difference is a claim or a fix that release will need, and every red check after a deploy is a question about what students are running.
Comparing main at sha-2bff81a (Main to RC forecast) against production 17.0.0, run 2026-10-19 10:46 UTC. 103 differences: claims explain 11; #505 declared 82 in its CHANGELOG line; #501 declared 2 in its CHANGELOG line; #501 contributed 4 (#501's CHANGELOG line says (image-hardening, image-manifest, metrics); startup moved); #478 most likely contributed 4 (#481's CHANGELOG line says (network); #505's CHANGELOG line says (dom, focus); persistence moved); 0 unexplained.
Gate FAIL on the Main to RC forecast sha-2bff81a beside 17.0.0, so no RCS.
Last release candidate kept: 17.0.0-rc.1 beside 16.2.2, WARN on 2026-10-15.
Post-deploy since 17.0.0 was deployed: green, rollback issue #131 open.
A/A: PASS on 2026-10-19.
Quality: Speed look, Metrics look, Tests look.
Real courses: 1 of 5 real courses load on main sha-2bff81a, 1 on production 17.0.0; none is broken by main alone.
3Goal the target condition
Measure
Now
Target
Gate on the newest forecast confidence.json
FAIL
PASS
gap
Unclaimed differences report.json
92
0
gap
Claims: known and gaps report.json
11 claimed, 92 unclaimed (10% covered)
every difference claimed or fixed
gap
Real courses load on main (course corpus) course-capture.yml, course-check.json
1 of 5 courses, 155 of 161 pages (production: 1 of 5)
every corpus course
gap
Nothing a student uses is worse on main (real courses) course-check.json, both sides (src/course/compare.ts)
2 page(s) and 0 journey(s) worse, 0 page(s) better
Post-deploy: production differs from the recorded release1878%
Main to RC forecast: stale claims891%
91%: the vital few above, the useful many below
Release candidate: unclaimed differences497%
Release candidate: stale claims2100%
65 stops over the kept runs and the post-deploy runs since the current release was deployed, about the monorepo and production only: the harness's own A/A and its failures are not counted. 3 causes carry 91%: Main to RC forecast: unclaimed differences; Post-deploy: production differs from the recorded release; Main to RC forecast: stale claims.
Numbers
Cause
stops
Cumulative
Main to RC forecast: unclaimed differences
33
51%
Post-deploy: production differs from the recorded release
Claim coverage5 deductions, e.g. unclaimed: dom /: reader:home: semantic DOM differs (+1 −1 lines at line 10)2030%
Change risk19 deductions, e.g. PR #476 changed Reader, Development but no changelog entry under v2bff81a779424d44aab12c4e4074a75c4457eac8 names it1553%
Test signal3 deductions, e.g. mutation score 72.73% on @tutors/i18n, below 80%13.573%
Requirements traceability (not measured)up to 10: needs the changelog, the EARS files and the claims side by side: pass --traceability <json>; changes.json already reports 13 orphan changes between the tags (a changelog entry with no diff, or a diff with no entry), a floor breach once it is measured1088%
88%: the vital few above, the useful many below
Post-deploy history (not measured)up to 5: needs the last release's post-deploy record: pass --post-deploy <its run dir or report.json>595%
Noise health4 deductions, e.g. mask realtime-rest-fallback-warning fired nothing in this run (a mask that never fires is a mask to delete)3100%
52 points lost on the dimensions measured, and 15 more are not measured on this run (hatched: a ceiling, never scored). The vital few: Claim coverage, Change risk, Test signal, Requirements traceability (not measured).
a diff and its changelog entry disagree130 points1368%
a hotspot file touched9 points384%
84%: the vital few above, the useful many below
churn above the app's median5 points189%
dependency5 points195%
production lines with too few test lines10 points1100%
19 findings over 16 PRs; a diff and its changelog entry disagree: 13 of 16.
Numbers
Cause
findings
Cumulative
a diff and its changelog entry disagree
13
68%
a hotspot file touched
3
84%
churn above the app's median
1
89%
dependency
1
95%
production lines with too few test lines
1
100%
Real courses: what a student would notice if main were released the course corpus, each page read in production's reader and main's, then compared page by page
1 of 5 real courses load on main sha-2bff81a, 1 on production 17.0.0; none is broken by main alone.
Course
Production 17.0.0
Main sha-2bff81a
Pages on main
Median time to title
tutors.json schema
Database databases-2024
fails 39 of 40 pages /lab/localhost:8194/topic-01-Conceptual/unit-1-Introduction/book-a/01: title "Download Visual Paradigm" not shown: locator.waitFor: Timeout 30000ms exceeded.
fails 39 of 40 pages /lab/localhost:8194/topic-01-Conceptual/unit-1-Introduction/book-a/01: title "Download Visual Paradigm" not shown: locator.waitFor: Timeout 30000ms exceeded.
1 worse · 0 better of 40 pages compared journeys: 0 worse, 0 better of 2
691 ms → 586 ms (−105 ms)
168 problems / must have required property 'authLevel' /los/[topic-00-Assignments] must have required property 'authLevel' /los/[topic-00-Assignments]/los/[unit-0-repeat-ca] must have required property 'authLevel' …
fails on both
Computer Systems & Networks setu-hdip-comp-sci-2024-comp-sys
fails 39 of 40 pages /lab/localhost:8193/topic-08-week8/unit-1/book-3/01: title "Raspberry Pi OS" not shown: locator.waitFor: Timeout 30000ms exceeded.
fails 39 of 40 pages /lab/localhost:8193/topic-08-week8/unit-1/book-3/01: title "Raspberry Pi OS" not shown: locator.waitFor: Timeout 30000ms exceeded.
0 worse · 0 better of 40 pages compared journeys: 0 worse, 0 better of 2
691 ms → 622 ms (−69 ms)
164 problems / must have required property 'authLevel' /los/[topic-00-Assessment_Details] must have required property 'authLevel' /los/[topic-00-Assessment_Details]/los/[talk-1] must have required property 'authLevel' …
fails on both
Higher Diploma in Computer Science 2024 wit-hdip-comp-sci-2024
loads 1 of 1 pages
loads 1 of 1 pages
0 worse · 0 better of 1 pages compared
551 ms → 744 ms (+193 ms)
28 problems / must have required property 'authLevel' /los/[unit-1] must have required property 'authLevel' /los/[unit-1]/los/[web-0-workshop-4] must have required property 'authLevel' …
loads on both
Full Stack Web Development wit-hdip-comp-sci-2024-full-stack-1
fails 38 of 40 pages /lab/localhost:8192/unit-1-node/topic-02-hapi/unit-2/book-1-playtime-0-2-0/04: title "Sessions" not shown: locator.waitFor: Timeout 30000ms exceeded. /lab/localhost:8192/unit-6-sveltekit/topic-19-svelte-routing/side-unit/book-2-donation-svelte-1/05: title "Signup" not shown: locator.waitFor: Timeout 30000ms exceeded.
fails 38 of 40 pages /lab/localhost:8192/unit-1-node/topic-02-hapi/unit-2/book-1-playtime-0-2-0/04: title "Sessions" not shown: locator.waitFor: Timeout 30000ms exceeded. /lab/localhost:8192/unit-6-sveltekit/topic-19-svelte-routing/side-unit/book-2-donation-svelte-1/05: title "Signup" not shown: locator.waitFor: Timeout 30000ms exceeded.
1 worse · 0 better of 40 pages compared journeys: 0 worse, 0 better of 2
fails 38 of 40 pages /topic/localhost:8191/topic-00-assignments/hold-unit-2/previous assignments/2022: title "Constraints" not shown: locator.waitFor: Timeout 30000ms exceeded. /lab/localhost:8191/topic-02-selection_and_iteration/unit-02a-if-stmt-and-events/book/05: title "Mouse events" not shown: locator.waitFor: Timeout 30000ms exceeded.
fails 38 of 40 pages /topic/localhost:8191/topic-00-assignments/hold-unit-2/previous assignments/2022: title "Constraints" not shown: locator.waitFor: Timeout 30000ms exceeded. /lab/localhost:8191/topic-02-selection_and_iteration/unit-02a-if-stmt-and-events/book/05: title "Mouse events" not shown: locator.waitFor: Timeout 30000ms exceeded.
0 worse · 0 better of 40 pages compared journeys: 0 worse, 0 better of 2
590 ms → 666 ms (+76 ms)
265 problems / must have required property 'authLevel' /los/[topic-00-assignments] must have required property 'authLevel' /los/[topic-00-assignments]/los/[unit-1] must have required property 'authLevel' …
fails on both
Database: 1 worse on main, 0 better
Worse on main
Assignment Two Videotalk · /video/localhost:8194/topic-00-Assignments/unit-2-assignment-2/talk-1/vNzjCvOApCA?start=5244 new accessibility violation(s): button-name
Full Stack Web Development: 1 worse on main, 0 better
Worse on main
01: Nodetopic · /topic/localhost:8192/unit-1-node/topic-01-node new console error: Permissions policy violation: compute-pressure is not allowed in this document.
Checked 2026-10-19 11:06 UTC by course-capture.yml. Worse on main: a page that does not load, more broken course images, a course file failing, a new console error, a link to a page that does not exist, a lost heading, a quarter less text or a new serious accessibility violation; or a journey main clicks less far through. One check per side: times are a reading, not a benchmark, and nothing here is an input to the Gate. tutors.json schema: 1 of 5 courses conform to the schema the generator on main promises (mono-repo 55f3aff); reported only, since the reader opens courses older generators wrote.
4Root cause analysis each question as deep as the evidence goes
Q1
Why does main (sha-2bff81a) stop the line?
92 unclaimed differences. 2 causes of 6 carry 89% of the 92 unclaimed differences. dom: semantic DOM differs alone is 45%.
Which differences were decisions, not failures, and why were they made?
2 fixes on b across 1 page (console). A fix never gates: it is a decision, and a claim is where its why is written. None has a claim saying why yet. Fixed with no note yet: catalogue:home (Cannot read properties of undefined (reading 'from')). That is good news; a claim is an optional note that the fix was meant, and it switches the glance item off.
52 points lost on the dimensions measured, and 15 more are not measured on this run (hatched: a ceiling, never scored). The vital few: Claim coverage, Change risk, Test signal, Requirements traceability (not measured). The Gate is FAIL, so none of this is a score: it is where the next release has to earn its points.
Evidence stops: 2 dimensions not measured on this run: Requirements traceability (needs the changelog, the EARS files and the claims side by side: pass --traceability <json>; changes.json already reports 13 orphan changes between the tags (a changelog entry with no diff, or a diff with no entry), a floor breach once it is measured); Post-deploy history (needs the last release's post-deploy record: pass --post-deploy <its run dir or report.json>)
Q5
Why is change risk 0 of 100?
19 findings over 16 PRs. 0 of 16 were merged with no approving review, and any one of them breaches the floor, so change risk cannot rise above it until review coverage does. The points themselves go to a diff and its changelog entry disagree (13), a hotspot file touched (3), churn above the app's median (1), dependency (1), production lines with too few test lines (1).
Evidence stops: the next why is the monorepo's branch rules and review load, which the kept reports do not carry; open a 5 Whys with harness why --finding two-of-three-below-75:change-risk once the scoreboard fires it
Q6
Can the A/A be trusted to say the harness sees clearly?
20 of 37 A/A nights were clean, 9 noisy (2 on 2026-09-22, 13 on 2026-09-23, 1 on 2026-09-24, 1 on 2026-09-27, 1 on 2026-10-01, 2 on 2026-10-03, 1 on 2026-10-16, 1 on 2026-10-19, 1 on 2026-10-19) and 8 degraded. 2 clean nights ran on a harness older than 1.4.4, which could not tell a clean night from a blind one (a journey failing on both sides), so their clean cannot be counted as seeing. 4 of 28 masks fired nothing on the latest night: each is a blind spot to review.
Evidence stops: the detection that answers it shipped in 1.4.4; every night since says degraded when it is
Q7
How long does a change wait to reach production, and where?
The oldest merge to main not yet in production has waited 4.1d, while the machine time of the whole stream is 29m: flow efficiency 0.48%. The wait is inventory: 16 PRs merged since v17.0.0, not in production; 92 unclaimed differences the next release owes; 2 open countermeasures.
the value stream map below
Evidence stops: release cadence is a decision, not a defect: the map shows the batch, the team chooses its size
Fixed on main present on production, gone on main: never a failure; a claim records whether it was meant
Cannot read properties of undefined (reading 'from')
fixed
no note yet (optional: a claim records that the fix was meant)
5Countermeasures to the system, never to a person
A countermeasure is a fix to the system that a 5 Whys asked for. It moves from open (not built) to in place (built, with a link) to closed, and it closes only when a release shows it working: its check is read in that release's report and the release goes in Verified by.
Countermeasure
Built
How a release shows it works
Status
claim guidance claims/README.md gains "A tool removed from an image": one narrow `sbom` claim per removed tree, its scope a brace list of the removed packages generated from report.json, citing the CHANGELOG entry, so the next release claims all 708 SBOM hunks without a broad claim. its 5 Whys · answers gate · kaizen/2026-09-29-sha-185e87f-gate.md
the release gate's SBOM differences are all claimed by narrow claims: none unclaimed, and claim hygiene flags no sbom claim. 17.0.0 did not show it (release gate 2026-10-15 12:02): 212 SBOM differences unclaimed, and one sbom claim covers 696.
waiting on a release
SOP change release/SOP.md's deploy step records the build production serves (the pinned digest, or while Netlify serves tutors.dev a /version naming the commit) and post-deploy judges only a confirmed build, reporting any other as "not the tested build" instead of a rollback. its 5 Whys · answers post-deploy · kaizen/2026-09-29-16.2.2-rollback.md
post-deploy names the build production serves and judges only that one: its report reads "production build: match" for the deployed release. 17.0.0 showed it: post-deploy run 38791747525 (2026-10-15 19:11) read "production build: match (production serves the recorded candidate's commit 432d46864626)".
verified in 17.0.0
glance rule a claim that names an info hunk is recorded as matching it, without gating it, so the glance's fixed-on-b rule reads "claimed" and the claim is no longer reported stale. its 5 Whys · answers decisions · kaizen/2026-09-29-sha-185e87f-fixed-on-b.md
a release with a claim that names a fix on b (an info hunk) reports that claim as matching, not stale, and the glance's fixed-on-b item reads "claimed". Not yet shown: 17.0.0 claimed no fix on b.
waiting on a release
6Plan who, what, when
What
Who
When
Status
claims/README.md gains "A tool removed from an image": one narrow `sbom` claim per removed tree, its scope a brace list of the removed packages generated from report.json, citing the CHANGELOG entry, so the next release claims all 708 SBOM hunks without a broad claim.
Leigh Griffin
2026-10-06
in place
a claim that names an info hunk is recorded as matching it, without gating it, so the glance's fixed-on-b rule reads "claimed" and the claim is no longer reported stale.
Leigh Griffin
2026-10-06
in place
Score 3 real releases so the weights can be judged against the team's view (C0)
the release captain
the next three releases
1 of 3
Record 2 reviewer's glances (C3)
the reviewer
the next two releases
0 of 2
7Follow-up how we will know it worked
Check
Now
Every countermeasure closed in a release (Verified by)
1 of 3 closed
not yet
Open countermeasures not rising (run rule)
2 open, 0 overdue
yes
The next forecast's Pareto: the top bar shrinks
dom: semantic DOM differs: 41
not yet
Post-deploy green on a build it can confirm
green
yes
This sheet is rebuilt with the site after every kept run. A countermeasure closes only when a release names it in Verified by; until then it stays on the plan.
4The 5 Whys from the kaizen register, as written and checked by harness why check: each answer checkable, each chain ending at the first process or tool
Unclaimed differences: 893 of 986 hunks (93 more are info: fixes, never gating)
The same Gate on the two forecasts before it: 892 unclaimed (2026-09-28T03:45Z, sha-017ceb6), 865 (2026-09-27T09:14Z, sha-54a8f83): the line has stopped on every forecast since the Main to RC stream began.
Why 2
Why are there 893 unclaimed differences?
Because one kind of difference is most of them. By artefact and change (the A3 Aggregator's Pareto, from report.json): 696 are sbom "package removed" and 12 are sbom "package bumped", so 708 of 893 (79%) are the images' software bill of materials. The 696 removals are 174 distinct packages (npm, corepack, yarn, node-gyp, @npmcli/*, @sigstore/*, tar 7.5.11 and the rest of npm's own dependency tree) counted once in each of the four images. The 12 bumps are node 22.23.2 to 22.23.3, tar and tzdata, again once per image. The next bars are the Paper UI rebuild (network 85, dom 33, screenshot 16, focus 15, logs 15): 185 in all.
Why 3
Why did 174 packages leave every image?
The monorepo's runtime image stopped shipping a package manager. Commit 4070315 "build: patch Debian and drop package managers from the runtime image" removes npm, npx, corepack and yarn from the runtime stage of the Dockerfile (the Trivy gate failed on npm's bundled tar and brace-expansion). It reached main with #143 on 2026-09-19 and is not in v16.2.2 (caa53d0, git merge-base --is-ancestor says no). It is an intended, security-positive change: the same run reports 72 advisories gone on b (18 per image, vulns info hunks), and the Debian patch is the 12 bumps.
Why 4
Why is an intended change like that unclaimed?
Because nothing tells the release how to claim a removed tool. release/claims.yaml on main holds 4 claims, all for the accessibility fixes of #313 (mono-repo #364), and a claim is written in the release PR (SOP step 4), so the forecast shows the claims the next release still owes. For this change the owed claim is awkward to write: claims/README.md shows how to claim one bumped package and a framework bump by glob (*/@sveltejs/*), but a removed tool takes its whole dependency tree with it, 174 packages with unrelated names, one hunk per package per image. The precise claim is not obvious and the easy one (*/**) is a broad claim that needs an approver.
Root cause, in a process.
claim guidance claims/README.md gains "A tool removed from an image": one narrow `sbom` claim per removed tree, its scope a brace list of the removed packages generated from report.json, citing the CHANGELOG entry, so the next release claims all 708 SBOM hunks without a broad claim.
Owner Leigh Griffin · due 2026-10-06 · in place
Built: claims/README.md · Check: the release gate's SBOM differences are all claimed by narrow claims: none unclaimed, and claim hygiene flags no sbom claim. 17.0.0 did not show it (release gate 2026-10-15 12:02): 212 SBOM differences unclaimed, and one sbom claim covers 696.
The harness's own trace (facts, each one a link or a file):
the evidence
Verdict: FAIL: 5 new difference(s) between production and the recorded candidate; open a rollback issue
The differences:network GET {{course}}/course.png on reference:lab; console new messages on reference:course, reference:lab and reference:note; focus keyboard order changed on reference:course (#38)
Production build: NOT CONFIRMED: /version did not answer; production was built at 2026-09-18T08:06:22Z from an unnamed build (run log)
History:post-deploy.yml concluded failure on all 64 of its runs from 2026-09-17 to 2026-09-29: 36 found no recorded release run to compare with (before the first release gate passed on 2026-09-27), 17 could not start on a push (2026-09-17 to 2026-09-21), and 11 found production different, each adding to the one open rollback issue.
Why 2
Why does production differ from the recorded 16.2.2?
Because production is not the build the harness recorded. The recording is the signed image quay.io/tutors-sdk/tutors-reader:16.2.2 (digest sha256:7567d59…, created 2026-09-19T18:07Z from caa53d0). tutors.dev was built at 2026-09-18T08:06Z, three minutes after the release PR #264 merged caa53d0 (08:03Z), by Netlify from source (apps/reader/netlify.toml: pnpm --filter tutors-reader... build). Same commit, a different build and a different host, so the two can differ without any change to the code.
Why 3
Why is production built outside the image the harness gates?
Because the deploy path that ships the gated image is built but does not serve tutors.dev yet. The monorepo's deploy.yml pins signed digests into deploy/k8s/overlays and release/deployed.json (16.2.2, deployedAt 2026-09-21T10:57Z) for OpenShift, and the OpenShift and AWS migration that would make those overlays production is planned, not done. So the "deployed" dispatch that tells the harness what production serves has never been sent: none of the 64 runs was a repository_dispatch, every judged one was the 15-minute schedule.
Why 4
Why has a red check not stopped anything for twelve days?
Because the release SOP has no step that says which build production serves before post-deploy is trusted. The monitor finds the same five differences every quarter hour, the rollback issue deduplicates them by signature, and a standing difference with no confirmed build reads as a standing alarm rather than a stop. The Jidoka rule holds for the gate before a release; after it, nothing in the standard work says what to do when production is not the tested artefact.
Root cause, in a process.
SOP change release/SOP.md's deploy step records the build production serves (the pinned digest, or while Netlify serves tutors.dev a /version naming the commit) and post-deploy judges only a confirmed build, reporting any other as "not the tested build" instead of a rollback.
Owner Leigh Griffin · due 2026-10-13 · closed
Built: src/modes/production-build.ts · Check: post-deploy names the build production serves and judges only that one: its report reads "production build: match" for the deployed release. 17.0.0 showed it: post-deploy run 38791747525 (2026-10-15 19:11) read "production build: match (production serves the recorded candidate's commit 432d46864626)".
Why is this a place to look: reader-auth:sign-in: 2 axe violations fixed on b, unclaimed?
The harness's own trace (facts, each one a link or a file):
the evidence
Glance item: rank 2 of 7 (fixed on b; novelty 1.00 × exposure 0.17): reader-auth:sign-in: 2 axe violations fixed on b, unclaimed; a fix nobody claimed can be a behaviour change
Hunks:axe:reader-auth:sign-in:158 color-contrast (serious) and axe:reader-auth:sign-in:159 document-title (serious), both severity info
Nearest claim:axe reader-auth:sign-in: Rule 0216: The reader shall show the sign-in page with no critical or serious WCAG 2.1 AA violations; why it did not cover: it is info-only (the claim matched nothing in this run: stale)
The same run's Gate reasons: "4 claim(s) matched nothing and should be removed from the changelog"; glance items 2 to 5 say the same of reference:course, reference:note and reference:topic
Why 2
Why do a fix and the claim written for it both read as unmatched?
Because the matcher never offers an info hunk to a claim. The four claims in the monorepo's release/claims.yaml (Rules 0216, 0051 twice and 0036, added by #364) name exactly the five axe hunks, and all five are severity info ("axe violation fixed on b"). src/claims/matcher.ts assigns only failing hunks ("Info hunks are reported but never need a claim"), so the four claims are never used and staleClaims lists them, and compare.matches carries no claim for the fixes.
Why 3
Why does the glance repeat it?
Because the glance reads the same matches. fixedOnB in src/glance/rank.ts looks for the claim of each fixed hunk in compare.matches, finds none, and words the item "unclaimed; a fix nobody claimed can be a behaviour change". The tool asks for a claim on a fix, then calls the claim stale: the Contributor who follows the SOP is told by the Gate to remove it and by the glance to add it.
Root cause, in a tool.
glance rule a claim that names an info hunk is recorded as matching it, without gating it, so the glance's fixed-on-b rule reads "claimed" and the claim is no longer reported stale.
Owner Leigh Griffin · due 2026-10-06 · in place
Built: src/claims/matcher.ts · Check: a release with a claim that names a fix on b (an info hunk) reports that claim as matching, not stale, and the glance's fixed-on-b item reads "claimed". Not yet shown: 17.0.0 claimed no fix on b.
Lean terms on this sheet
Jidoka
the Gate stops the line on an unclaimed difference; no score talks it back on
Andon
the red marks: a stopped gate, a red post-deploy, a stage that is not the tested artefact
Gemba
every number links to the report, the hunk or the run it came from
Pareto
the vital few causes (80% of the effect) above the dashed line
5 Whys
asked until the answer is a process or a tool, never a person
VSM
process time (P/T), first-pass yield (FPY), waits and inventory (I) from merge to verify
Flow efficiency
process time as a share of lead time
Decision
a fix on b is a choice, not a failure: orange, beside the traffic lights, and switched off by the claim that says why
Kaizen
every finding ends in one of seven countermeasure kinds, closed in a release (docs/lean.md)