McRitchie Studio
Say Hi 👋
Agents Builders
← All Tasks
Sizing

Production smoke seal agent

Shipped steffon
production-smoke-seal-agent

Created

Jun 26, 20:28

Started

Jun 26, 20:34

Completed

Jun 26, 23:21

DevOps handoff

Type

Feature

Shape

ui+db

Worktree Slug

production-smoke-seal-agent

Repositories

mcritchie-studio

Release Train

—

Branch

feat/production-smoke-seal-agent

Pull Request

https://github.com/amcritchie/mcritchie-studio/pull/249

Local URL

http://localhost:3005/deployments

QA URL

—

Production URL

—

devops,release,prod-safety

Acceptance Criteria

  • Implement bin/prod-smoke read-only suite against production URL
  • Ship runs smoke agent after up gate passes
  • Seal verdict persists on the Release record
  • Deployments board shows green or red seal badge
  • Discord and release notes include the seal verdict
  • Failed smoke alerts operator and recommends rollback non-blocking

Expected Test Plan

  • [unit] SmokeSeal verdict + rollback; ProdSmoke host resolution; Release seal persistence; Formatter seal line
  • [component] 🟢/🔴 seal badge renders on /deployments Last Release card
  • [integration] post_release_notes carries the seal verdict into notes + Discord
  • [e2e] seal badge on /deployments; bin/prod-smoke @qa-readonly green vs prod

Checks Run

  • [unit] bin/rails test test/models/release/smoke_seal_test.rb test/models/release/prod_smoke_test.rb — SmokeSeal verdict/rollback + ProdSmoke host resolution
  • [unit] bin/rails test test/models/release_test.rb — Release#record_smoke_seal!/smoke_seal persist + survive ship flip + broadcast
  • [unit] bin/rails test test/services/release_notes_formatter_test.rb — Formatter appends the seal verdict line to message + Discord header
  • [component] bin/rails test test/controllers/tasks_controller_test.rb — 🟢/🔴 seal badge renders on /deployments Last Release card (and absent when unsealed)
  • [integration] bin/rails test test/models/release/conductor_test.rb — post_release_notes carries the recorded seal verdict into notes + Discord
  • [e2e] E2E_PORT=3092 npx playwright test e2e/release_seal.spec.js — seal badge on /deployments; bin/prod-smoke mcritchie-studio GREEN vs https://app.mcritchie.studio
  • [full-suite@ee7fa19d12b3a722de04f0fc2bd01eefa3991fd7] bin/rails test green
  • [rubocop@ee7fa19d12b3a722de04f0fc2bd01eefa3991fd7] bin/rubocop clean

Agent Context

Post-deploy production smoke SEAL OF APPROVAL, born from the 2026-06-26 QA-release audit. GAP it fixes: config/devops_test_suites.yml declares a production_smoke lane with command 'bin/prod-smoke <app>' but that script DOES NOT EXIST; today the only prod verification at ship is a single 'curl GET <smoke_url>/up' 200 check in bin/release (~line 1375). DESIGN (operator-aligned 2026-06-26): (1) Floor = implement bin/prod-smoke <app> reusing the EXISTING read-only Playwright spec e2e/qa_readonly.spec.js (@qa-readonly: home/signin/tasks render + /up 200 + /devops & /devops/cycle 404) but pointed at the app's PRODUCTION url instead of QA. Read-only by construction: no content creation, no emails, no provider-state change. (2) Trigger = post-ship, AFTER bin/release ship deploys and the existing /up hard-gate passes (prod live). It is a SEAL, not a gate — the ship already happened. (3) Scope = GENERIC HEALTH ONLY (operator choice) — do NOT read shipped members' acceptance criteria or deep-probe; the @qa-readonly suite is the verification. (4) Seal recorded in ALL THREE: a 🟢/🔴 badge on the release in /deployments + persisted on the Release record (new column e.g. smoke_seal{status,summary,checked_at}); Discord (release-notes thread); and appended to the auto-posted release notes body. (5) On FAILURE: record red seal, ALERT the operator, and surface the exact rollback/revert command (Release#abandon! / git revert -m1 of the merge on release) — but DO NOT auto-rollback; operator stays the gate. HOST NUANCE to resolve: config production_url is https://mcritchie.studio but the ship adapter smoke_url is https://app.mcritchie.studio — confirm canonical prod host for the suite base URL. Reuse Release::Conductor.post_release_notes / the Discord Formatter path. Shape ui+db because the seal persists (DB) and renders a board badge (UI); dor_tiers = unit, component, integration, e2e. Relates to two other audit findings (out of scope here, file separately): the two divergent tier vocabularies (feature_shapes dor_tiers vs Release::STEP_TEST_TIERS), and tier tags being agent-asserted strings rather than machine-credited.

Stage Timeline

Who handled each stage, the time it took (measured), and the model / tokens / cost reported (best-effort) — plus who's on it right now. — means the agent didn't report that metric.

Sizing Avi · PO LARGE Dev LARGE Actual XL ≠ forecast
  1. Created → Designed
    M Machoke
    Machoke
    Model
    —
    Duration
    —
    Tokens
    —
    Cost
    —
    Completed Jun 26, 20:28 · about 13 hours ago
    api
  2. Designed → Building
    M Machoke
    Machoke
    Model
    claude-opus-4-8
    Duration
    6 minutes
    Tokens
    1,608,969
    Cost
    ~$1.82
    Started Jun 26, 20:28
    Completed Jun 26, 20:34 · about 13 hours ago
    cli
  3. Building → Submitted
    M Machoke
    Machoke
    Model
    claude-opus-4-8
    Duration
    about 1 hour
    Tokens
    52,276,348
    Cost
    ~$33.96
    Started Jun 26, 20:34
    Completed Jun 26, 21:29 · about 12 hours ago
    cli
  4. Submitted → Reviewed
    C Carl
    Carl primary
    A Alex
    Alex light
    Model
    claude-opus-4-8
    Duration
    about 1 hour
    Tokens
    424,114
    Cost
    ~$0.37
    Started Jun 26, 21:29
    Completed Jun 26, 22:33 · about 11 hours ago
    cli
  5. Reviewed → Assembled
    S Steffon
    Steffon
    Model
    claude-opus-4-8
    Duration
    6 minutes
    Tokens
    1,750,888
    Cost
    ~$1.50
    Started Jun 26, 22:33
    Completed Jun 26, 22:39 · about 11 hours ago
  6. Assembled → Shipped
    A Avi
    Avi
    Model
    claude-opus-4-8
    Duration
    43 minutes
    Tokens
    17,188,005
    Cost
    ~$12.97
    Started Jun 26, 22:39
    Completed Jun 26, 23:21 · about 11 hours ago

Conversation

QA review feedback, agent handoffs, and follow-up notes for this task.

Comment alex about 11 hours ago

Senior review -> APPROVED (2/2). Reviewers: Carl (backend/migration heavy) + Alex (release-semantics/docs light), both high confidence. Confirmed the EXISTING SHIP FLOW IS SAFE: seal is a post-/up-gate step 5c, read-only against prod, strictly NON-BLOCKING (no new abort/rollback path; board write rescued); migration purely additive (jsonb, schema matches, no backfill); 149 focused tests green. Follow-ups (non-blocking): (1) minor bin/release line ~1474 -- the bin/prod-smoke spawn (Open3) sits OUTSIDE the degrade-to-warn rescue; a narrow env failure (non-exec / wrong CWD) could propagate between deploy and ship!/notes, leaving a release stuck assembled with no notes -- wrap the sh call; (2) medium: docs devops-cycle-design section 1.4 Run Deployment prose not updated for the new seal/badge step (AGENTS.md docs-in-same-pass); add ~2 sentences; (3) low: devops_test_suites.yml production_url comment overstates its role (host actually resolves from release_repos smoke_url). -> reviewed.

Sealed-bid sizing

Edit →

Alex (PM)

—

Avi (PO)

LARGE

Dev

LARGE

Actual

XL