Optimizer

The dashboard’s Engagement → Optimizer page runs a nightly pass over your own campaign, segment, and holdout data and proposes bounded, specific changes — never freeform ideas, never a model deciding what to do. Everything it can ever propose comes from a closed catalog of six actions. Whether a proposal waits in a queue for you to approve, or is allowed to apply itself within a hard weekly limit, is a setting you control per module. Nothing it does is a new kind of write: every applied change goes through the exact same validated path a merchant’s own edit in the dashboard would use, and every applied change can be reverted with one click.

The detectors themselves are pure statistics — SQL and arithmetic over your campaign_sends, message_deliveries, and holdout data, gated by minimum-sample floors so a handful of noisy sends never triggers a proposal. No model ever picks what to propose. When a model provider is configured, it only ever writes the one- or two-sentence rationale narrating the numbers the detector already computed — it cannot introduce a number of its own. This runs as the fourth leg of the existing nightly 30 1 * * * job, alongside RFM scoring, wallet-offer expiry, and the gamification reward sweep — no new cron trigger.

The closed action catalog

These six action_type values (verbatim from packages/db/src/schema/optimizer.ts’s OPTIMIZER_ACTION_TYPES) are the only things the optimizer can ever propose or apply — there is no path to an arbitrary action:

Action (label) Module Target What it does Self-applies in Auto?
adjust_send_delay (“Adjust send delay”) Engagement timing An active post_visit automation PATCHes the trigger’s delay_hours (1–168h — the only timing knob a post_visit automation has) toward the delay that engaged best in that campaign’s own history Yes
pause_underperformer (“Pause underperformer”) Campaigns An active campaign Pauses it — the existing pause action, nothing new Yes
refine_segment (“Refine segment”) Segments An active campaign’s segment PATCHes the segment with a complete replacement definition: your existing predicates plus one new last_transaction_days lt N appended Yes
adjust_holdout (“Adjust holdout”) Campaigns An active campaign Raises or lowers holdout_pct, clamped to 5–20 Yes
switch_channel (“Switch channel”) Campaigns An active automation Creates a new draft campaign on a cheaper/better-performing channel No — always a draft
draft_variant (“Draft a variant”) Campaigns An active automation Creates a new draft campaign with reworded content No — always a draft

Labels are the dashboard’s own (apps/merchant-dashboard/src/features/optimizer/catalog.ts’s ACTION_TYPE_LABELS) — the same strings you’ll see in the proposals queue and applied log.

The three modules (OPTIMIZER_MODULES, each independently configured — see Modes and the weekly cap):

Module Covers
campaigns Pausing a sustained underperformer, nudging a campaign’s holdout percentage, or proposing a cheaper/better channel or a re-worded variant as a draft.
segments Tightening a campaign’s audience segment when engagement lags in a sub-cohort.
engagement_timing Adjusting a post-visit automation’s send delay toward its best-performing timing.

adjust_send_delay/pause_underperformer/adjust_holdout/refine_segment are the four auto-applicable actions. switch_channel and draft_variant are not, on principle — see What Auto never does.

Detectors: statistical, not AI

Every detector is a pure function over rows it fetched itself — deterministic, no randomness, no wall-clock reads (the run’s now is threaded through explicitly, never read fresh inside a detector), no network calls, no model. Each one only fires once its data clears a documented minimum-sample floor, so it never proposes a change on the strength of a handful of sends:

Detector Key floors What it compares
adjust_send_delay ≥200 eligible sends per campaign; ≥20 per 6-hour delay bucket; peak bucket must beat the current bucket by ≥10 points; proposed shift must be ≥2h to bother Buckets the campaign’s own observed send-to-transaction delay (28 buckets across the legal 0–168h range) and proposes moving delay_hours halfway toward the best-performing bucket — not straight to it, so a later re-run keeps correcting as more history accumulates
pause_underperformer ≥200 eligible sends; ≥14 days of runtime; ≤2% engagement rate; ≥20-member holdout group Sustained near-zero engagement and a defined (non-null) holdout-adjusted revenue lift ≤ 0 — the campaign isn’t even paying for itself
refine_segment ≥200 sends; overall rate ≤5%; candidate sub-cohort ≥50 sends and ≤70% of the whole audience; sub-cohort must beat overall by ≥10 points Tries four recency windows (14/30/60/90 days since last transaction, as of send time) and proposes the best-qualifying one
adjust_holdout ≥200 sent+holdout combined; ≥30 in the holdout arm The holdout arm’s relative standard error: >50% raises holdout_pct by 5 (more sample needed); <15% lowers it by 5 (already precise, free up members)
switch_channel ≥200 sends on the automation’s own channel; ≥200 tenant-wide on the candidate channel This automation’s own engagement rate vs. the tenant-wide rate every other channel achieves across all your campaigns, paired with a cost delta from the same estimator POST /v1/campaigns/:id/estimate uses; must beat by ≥15 points
draft_variant ≥100 sends in each of two non-overlapping windows An early window (campaign’s first 30 days) vs. a recent trailing window (last 14 days): early rate must have been ≥5%, and the recent rate must have fallen to ≤70% of it (a genuine ≥30% relative decline, not day-to-day noise)

Cooldowns. After a finding for one exact (target, action_type) triple is proposed, that same triple is suppressed from re-firing for a while — measured from when it was first created, never from when it was decided: 14 days if you dismissed it, 30 days for every other outcome (still sitting proposed, approved, applied, reverted, or naturally expired). Dismissals get the shorter window deliberately — you said “not now,” not “never.”

Evidence, not adjectives. Every finding carries its own numbers in detector_json, captured once at propose time and never recomputed. The dashboard renders these as plain facts, e.g. “Engagement 1.4% over 21d (n=350).” / “Holdout lift −12.0%.” — never a model’s paraphrase standing in for a number. switch_channel’s evidence also discloses that this codebase’s cost model has no per-message rate for email (it’s modeled as a flat $0) so a channel comparison against email is never allowed to look like an undisclosed win.

Rationale: AI-worded when configured, honest either way

The rationale — the one or two sentences explaining why a proposal fired — is the only part of this feature a model ever touches, and even there it can only narrate numbers that are already in detector_json; it’s explicitly instructed never to invent, estimate, or restate a number differently than the JSON it was given. Two paths:

The nightly workers/jobs worker that runs this leg deliberately declares AI_PROVIDER/AI_API_KEY/AI_MODEL as optional and leaves them unset by default (unlike workers/edge-api, which sets a stub provider for local dev because the copilot/campaign-agent are load-bearing on an answer). So out of the box every rationale here is the templated, “Auto-generated” one — that’s the designed default, not a degraded state, and it can be upgraded to a real model with zero code changes by pushing the same three secrets already documented for the copilot (see infra/DEPLOY.md).

Modes and the weekly cap

Each of the three modules has its own mode, set independently:

Mode Behavior
suggest (default) Findings sit in the proposals queue; nothing happens until you Approve or Dismiss.
auto_within_caps Self-applies immediately, through the same domain path Approve would use, while under the weekly cap — see below. Once the cap is hit, further findings for that module fall back to behaving exactly like suggest until older auto-applies age out of the trailing 7-day count.
off That module’s detectors don’t run at all this cycle — no new findings. A proposal already sitting in the queue from before you switched a module off is untouched; only new detection stops.

A tenant with no optimizer_settings row for a module is running it at the built-in default: suggest, cap 5 — the dashboard’s mode table labels this (default) rather than hiding it.

The weekly cap is a 0–20 integer, default 5, configured per module — but what it’s actually checked against is a single tenant-wide running count of this tenant’s auto-applies in the trailing 7 days (a rolling window, not a calendar week), shared across all three modules within one nightly run, not a separate budget per module. With every module left at the shared default (5), this behaves exactly like one tenant-wide weekly budget of 5 — the common case. Setting different caps per module only matters at the margin: modules are evaluated in a fixed order (campaigns, then segments, then engagement_timing), so a module considered earlier in a run can exhaust the shared count before a later module’s own, possibly higher, cap is ever tested. Either way, a capped-out finding is never dropped — it’s inserted as an ordinary proposed row for you to approve by hand, same as suggest. The count is checked before every auto-apply attempt, and a since-reverted auto-apply still counts against the week it happened — revert doesn’t refund the budget, which would otherwise let apply-then-revert cycle around the cap.

switch_channel/draft_variant are never counted here at all: they never attempt to auto-apply in the first place (next section).

What auto_within_caps never does

Same domain paths, no bypasses

Every mutation this feature ever makes — auto or manual — runs through the identical validated function a merchant’s own dashboard action would call: the same campaign PATCH validation, the same pause/activate lifecycle gates, the same segment definition validator. Frequency caps, consent enforcement, and template-approval gates are structurally out of this feature’s reach — it never touches those code paths, so there’s nothing here that could bypass them.

One real wrinkle: adjust_send_delay and adjust_holdout only ever fire on active campaigns (that’s the detectors’ own precondition), but the underlying PATCH function only allows draft/paused campaigns to change their trigger or holdout — a deliberate, unrelated guardrail this feature doesn’t get to loosen. So applying (or reverting) either of these two actions against an active campaign orchestrates pause → patch → reactivate, exactly what you’d click through by hand, wrapped in a database savepoint: if reactivation honestly fails (say, a WhatsApp template that was approved a moment ago just got rejected), the savepoint rolls back the whole choreography, leaving the campaign exactly as it was — still active, still on its old values — rather than stranding it paused.

Proposal lifecycle

proposed  ──▶ approved ──▶ applied ──▶ reverted   (terminal)
   │             │
   │             └──▶ back to proposed, with the failure recorded (an approved proposal
   │                  whose apply attempt honestly failed — re-decidable)
   │
   ├──▶ dismissed   (terminal)
   └──▶ expired     (terminal — 30 days after creation if nobody decided it)

Every transition is compare-and-swap on state (the same pattern the AI agents (MCP) guide’s approval flow uses) — a losing race reports the real terminal state honestly rather than silently double-applying.

Approving, dismissing, and reverting

The proposals queue only ever shows state: "proposed" rows. Using the same dashboard-session cookies.txt from the quickstart:

curl -sS -b cookies.txt http://localhost:8787/v1/optimizer/proposals?state=proposed | jq .

Every decision is audited — except one

Every transition writes an ai_audit row (feature: "optimizer") — except expiry. Proposed, approved, applied, dismissed, and reverted each get their own row, attributed to either the system (system:optimizer-cron, for the nightly leg’s own propose/auto-approve/auto-apply writes) or the approving/dismissing/reverting merchant. Expiry is the one deliberate exception: aging out an undecided proposal after 30 days is a bulk, per-tenant sweep (one UPDATE ... WHERE state='proposed' AND expires_at < now, piggybacked on the same nightly run) with no single decision to attribute — the same “no per-row audit trail for a bulk sweep” precedent the AI agents (MCP) guide’s own approval-expiry sweep already established. It’s still fully legible, just not individually audited: query optimization_proposals for state = 'expired' and you have the complete list.

The dashboard

Engagement → Optimizer has three sections:

API reference

Method Path Auth
GET /v1/optimizer/proposals?state=&module= dashboard or API key — campaigns:read OR settings:read
POST /v1/optimizer/proposals/:id/approve dashboard session — re-checks campaigns:manage
POST /v1/optimizer/proposals/:id/dismiss dashboard session
POST /v1/optimizer/proposals/:id/revert dashboard session — re-checks campaigns:manage
GET /v1/optimizer/settings dashboard or API key — settings:read
PUT /v1/optimizer/settings dashboard or API key — settings:manage
GET /v1/optimizer/report dashboard or API key — campaigns:read OR settings:read

Full request/response schemas are in the API reference under the Optimizer tag.

v1 limits

Next steps