ClubSpark Classic Auth — Stage/Test post-deployment validation

Captured: 2026-07-09 (after the new Duende identity-platform deploy to stage) · Harness: pw-harness (Playwright) · Compare to: pre-rollout baseline

Re-run of the same sign-up / sign-out / sign-in commands as the pre-rollout baseline, against the deployed build, to surface regressions and improvements. Every recording below is shown baseline (pre-rollout) beside post-deploy for direct comparison. All post-deploy interactions are human-like (real keystrokes, clicks, dropdown selections) — see Harness change.

Videos play inline on reports.clubspark.dev; on the GitHub file browser use the ▶ links.

Baseline → post-deploy diff

Legend: ✅ pass · ❌ fail · ⛔ blocked (external/pre-existing) · ➖ n/a · ⧗ recording didn’t complete

# Domain Baseline Post-deploy Δ
1 stg-solo-web.clubspark.io/ 7/7 ✅ 7/7 ✅ — no change
2 stg-solo-web.clubspark.io/RITUVENUES/ 9/9 ✅ 9/9 ✅ — no change
3 stg-solo-custom.clubspark.ninja/ 7/7 ✅ 7/7 ✅ — no change
4 tst-fa-web.clubspark.io/ (FA→B2C) 5/5 ✅ 5/5 ✅ — policy b2c_1a_signup_signin; MFA auto-detected OFF on signin → clean 5/5 (earlier “Mailpit spam” note corrected — see §4)
5 stg-ecb2-web.clubspark.io/ (ECB→Okta) 4/4 ✅ 4/4 ✅ ✅ interim signin regression found + fixed — see §5
6 stage-lta.clubspark.io/ (LTA) ⛔ signup ⛔ signup — no change (reCAPTCHA)
7 stg-fa-web.clubspark.io/ (FA→B2C) 3/5 ❌ 5/5 ✅ 🔼 IMPROVEMENT — signup+signin

Net result: FA on stage is fixed (§7, 3/5 → 5/5) and every stage domain is green. Two STS issues surfaced during validation and were both fixed with follow-up deploys: an interim /Account/Login 500 that broke all Duende-routed sign-in, and the ECB-OKTA callback 500 (IDX10503, §5). No outstanding regressions.

How to reproduce

cd pw-harness && npm install

.env carries CS_BASE, CS_LEGACY_AUTH_HOST, CS_EMAIL_DOMAIN=clubspark.dev, MAILPIT_*. Each domain’s command is in its section.

🔼 §7 · stg-fa-web (The FA) — FIXED (3/5 → 5/5)

Baseline: signup federated to the wrong (tst) relying party → couldn’t complete. Post-deploy: FA federates to the correct stage B2C tenant login-stg.thefa.com (b2cppthefa), and signup + signin + signout all pass.

node fa-suite.mjs https://stg-fa-web.clubspark.io/ 1     # 5/5
  Baseline (pre-rollout) — ❌ 3/5 Post-deploy — ✅ 5/5
signup
▶ old
▶ new
signin + signout
▶ old
▶ new

✅ §5 · stg-ecb2-web (ECB) — interim regression FIXED (now 4/4)

Signup, sign-in, sign-out all pass — 4/4. During validation the STS’s ECB-OKTA OIDC callback returned HTTP 500 (IDX10503: Signature validation failed … Keys tried: SymmetricSecurityKey, KeyId: '' — the STS had no matching signing key for ECB Okta’s RS256 id_token, i.e. the ECB-OKTA JWKS/Authority config hadn’t come through). A follow-up deploy restored the config; the callback now returns 302 and sign-in logs in. Re-verified across runs.

The ECB sign-in flow now carries the same resilience as FA (§4): Okta MFA auto-detection (after the login POST the harness decides between an Okta MFA challenge, the “Confirm Your Preferences” consent modal, or an already-completed federation — so an MFA toggle on ECB’s side won’t silently hang the run; an email factor is satisfied from Mailpit) and timestamped step logging. The preferences modal is also now waited-for-render + retried, so it advances reliably.

CS_BASE=https://stg-ecb2-web.clubspark.io/ node test-ecb-auth.mjs 1     # 4/4
  Baseline (pre-rollout) — ✅ 4/4 Post-deploy — ✅ 4/4 (after fix)
signup
▶ old
▶ new
signin + signout
▶ old
▶ new

§1–3 · Solo / Independent — ✅ unchanged (7/7, 9/9, 7/7)

Legacy WS-Fed local form, self-serve. No change from baseline.

CS_BASE=https://stg-solo-web.clubspark.io/            CS_LEGACY_AUTH_HOST=stg-solo-auth.clubspark.io node test-legacy-auth.mjs 1
CS_BASE=https://stg-solo-web.clubspark.io/RITUVENUES/ CS_LEGACY_AUTH_HOST=stg-solo-auth.clubspark.io node test-legacy-auth.mjs 1
CS_BASE=https://stg-solo-custom.clubspark.ninja/      CS_LEGACY_AUTH_HOST=stg-solo-auth.clubspark.io node test-legacy-auth.mjs 1
  Baseline (pre-rollout) Post-deploy
§1 stg-solo-web (legacy)
▶ old
▶ new
§2 /RITUVENUES (venue)
▶ old
▶ new
§3 custom domain (.ninja)
▶ old
▶ new

§4 · tst-fa-web (The FA) — ✅ 5/5 (MFA auto-detected: OFF on signin)

tst-fa-web was deployed to (its B2C policy changed from b2c_1a_signup_signinmfa → b2c_1a_signup_signin, matching stage’s non-MFA flow; redirect_uri correctly points at the tst RP tst-solo-ids-sts). Re-validated end-to-end: signup, signout, and signin all pass — 5/5. Signup’s own email-verification OTP arrives from Mailpit in ~4s; signin then federates straight through with no code prompt.

Correction to the earlier note (which blamed “Mailpit spam load”): that diagnosis was wrong. The re-run wasn’t stalling on the mailbox — it was the sign-in flow waiting for an MFA email code that this policy never sends. b2c_1a_signup_signin requires MFA only at signup (the account-creation email verification), not at signin. The harness now auto-detects per run whether the current sign-in flow presents an MFA code step before it waits for one (FA/ECB can toggle MFA on/off outside our control), so it federates straight through when MFA is off instead of hanging. Timestamped step logging was also added so any future hang is pinpointed to an exact step. The captured timeline for this run:

signup:  gotoFA → B2C signin → email → Send Code → OTP received (~4s) → profile → password → Create → left B2C (created=true)   [~49s]
signin:  gotoFA → B2C signin (credentials once) → MFA detection → done (no code step) → left B2C → Your Details → venue: loggedIn=true
signout: no :8443 leak

node fa-suite.mjs https://tst-fa-web.clubspark.io/ 1
  Baseline (pre-rollout) — ✅ 5/5 Post-deploy — ✅ 5/5
signup
▶ old
▶ new
signin + signout
▶ old
▶ new

§6 · stage-lta (LTA) — ⛔ unchanged (signup reCAPTCHA-gated)

Local signin form on stage-lta-auth; signup delegated to lta.org.uk behind reCAPTCHA. No change.

CS_BASE=https://stage-lta.clubspark.io/ CS_LEGACY_AUTH_HOST=stage-lta-auth.clubspark.io node test-lta-auth.mjs 1
  Baseline (pre-rollout) Post-deploy
register → reCAPTCHA wall
▶ old
▶ new

Harness change since the baseline: human-like interactions

The post-deploy runs drive forms the way a person would — real keystrokes (typed char-by-char, with self-verify + re-type under load), real clicks (buttons, links, styled checkboxes/radios via their labels), and real dropdown selections (the solo SignUp SelectBoxIt DOB widget, B2C/ECB <select>s) — instead of injecting values/checked state. Consent: only the required T&C is ticked; marketing opt-ins are left unticked. B2C/SPA steps wait for the page to be rendered/ready before interacting, and Mailpit reads are bounded (per-request timeout). This is why the baseline (old) videos look more “instant/injected” and the post-deploy (new) ones look like a genuine human journey. Playwright doesn’t render a mouse cursor in video, so the pointer isn’t visible, but all motion/clicks are real.

Outstanding action items