assign A/B after scoring, lock legacy prospects out, fix arm-B approval
- Owner instruction: "we're only going to do this fifty-fifty split on future
- prospects that come in โ I don't wanna go back and alter emails that have
- already been written that are good to go."
- ASSIGNMENT MOVED discovery-create โ generation time, after research and the
- deterministic lead score and immediately before the writer runs (the arm
- decides what the email says, so the final score must decide the arm).
- Stratified on campaign x lead-score TIER, recorded in outreachVariantTier so
- arms can later be compared at equal tier.
- An adversarial review of the guarantee found three real defects, all fixed:
- 1. LEGACY LOCK-OUT (the requirement violation). The email-based guard was not
- sufficient: ~136 of the 288 existing prospects had no email row yet, so the
- every-10-minutes autopilot would have enrolled them with no operator action
- and written arm-B paid-offer emails for half. All 288 are now stamped
- outreachVariant='LEGACY' (migration 20260824_experiment_legacy_lock.sql) โ
- a stored fact, refused by assignment, filtered out of every comparison.
- The email guard also tightened: ANY email row blocks enrolment, cancelled
- included.
- 2. ARM B WAS DEAD ON ARRIVAL. The approve route and the rewrite path both
- called validateFirstTouch WITHOUT the variant options, so the arm-A rules
- rejected the offer link โ every arm-B draft failed approval, and a bulk
- rewrite silently replaced it with an arm-A fallback that could then never
- pass the send gate. New firstTouchOptionsFor() resolver; both call sites
- fixed; the rewrite writer now receives the arm-B rules too.
- 3. BULK REWRITE ATE APPROVALS. rewriteFirstTouchEmail overwrote APPROVED
- bodies in place and demoted them to DRAFT, discarding the approval and its
- send slot. It now returns skipped_approved.
- UI: an experiment-arm filter (All / A / B) that applies to whichever tab and
- sort is active โ so "Ready to email, arm B" and a per-arm send schedule are
- each one click โ plus a By-variant grouping showing both arms side by side.
- Payment rehearsal: POST /api/admin/prospecting/experiment/test-payment, wired
- to two console buttons โ mint a test offer link, or have the arm-B offer email
- delivered to your own inbox with a working link. Rides a suppressed fixture
- prospect, so no real prospect is touched, and exercises the real Stripe path.
- +15 tests (278 prospecting, 928 lib-wide).
- Also fixes the /admin/saas/prospecting/analytics crash reported in production
- ("Cannot read properties of undefined (reading 'current')"). The page
- server-renders fine and threw on client mount, so all three plausible causes
- are removed rather than guessed between: the chart moves to its own module
- dynamically imported with ssr:false (Recharts measures the DOM on mount);
- it carries an error boundary so a chart failure can never blank the route
- again; and the --viz-* tokens move from an inline <style> in a client
- component into globals.css (React 19 hoists <style> elements, a known cause
- of this exact error). The vitest ResizeObserver/IntersectionObserver mocks
- were also not constructible under vitest 4, which is why no test had ever
- caught this class of bug โ both are now real classes.
63313c9View on GitHub โ