Or the deprecate-the-loser decision is theatre. Cycle time, quality, cost, risk. Same inputs. A name who signs retirement.
A twin that runs beside the core is not interesting because it is new. It is interesting because, after a stated window, someone can retire a path on evidence. Four weeks is a serious first pass. Enough instances to argue with. Not a transformation. A comparison.
If you cannot name the four measures, the window, the inputs, and the signer before the first live file, do not start the run. You will produce a dashboard and call it a verdict.
Same pack, two paths
The official file continues on the old path. The twin receives the identical input — the live RFQ, the live order pack, the live PBC, the live data room — and writes a draft into a review queue. Same customer. Same timestamp. Same system of record after the gate.
Different inputs are not a parallel run. A sandbox of last year’s examples is a demo. A twin that only sees cleaned files the core never sees will win on paper and lose on Monday.
Four measures. Not twelve.
Finance owns the board. The gate owner owns the refusal. Neither may invent a fifth category mid-run to protect a path.
- Cycle time. Arrival to committed output behind the named gate. Days-to-offer, days-to-released pack, days-to-first-draft, time-to-first-action. Start the clock on arrival. Split wait from work if you must. Do not bury wait inside “complexity.”
- Quality. First-pass completeness. Error rate against what was released. Issues found after the gate. Review rounds on the twin’s draft versus the old pack. Quality is what the gate had to fix, not a satisfaction score.
- Cost. Cost per instance on each path: hours that touched the file, tools, rework. Not transformation TCO. Not hours “saved” against a baseline nobody trusted. If the twin is cheaper only because the old path still does the exceptions, write that down.
- Risk. Incidents: wrong release, privilege leak, untraceable number, action that could not be rolled back, gate skipped. Near-misses count. A faster path that cannot be reconstructed is not a win.
Write the operational definition of each measure on the workflow card before day one. If two people can score the same file differently, you do not have a measure. You have a mood.
What the window has to contain
Four weeks only works if the workflow has throughput. Three instances is an anecdote. Hold the 80 / 20 split honest: the stack clears the chainable majority; the named human takes the rest. Score both slices. A twin that wins by refusing the hard files is selecting, not winning.
Do not change the playbook mid-window to help the twin. Do not starve the old path of staff. Do not add a second workflow halfway through to dilute a loss.
Who signs retirement
The mandate already named this person. If it did not, the run is a pilot, and pilots do not retire paths.
The signer is a single executive — CEO or the named deputy — acting on the scoreboard Finance kept and the incidents Legal can read. The gate owner may refuse a draft. The gate owner may not keep the old path alive after the agreed rule is met. A committee that “notes the results” is how losers survive.
The rule is written in advance: when the twin is better on the four measures for the compressible share, that share of the old path is retired. Not considered. Retired. If the twin loses, say so on the same board. Fix the chain or stop. “Share learnings with the core” is how the edge is absorbed without ever beating the incumbent.
What this is not
– Hours saved, adoption scores, prompt counts, seats licensed
– A model-eval leaderboard detached from the file that left the building
– A qualitative readout from a committee that did not see the packs
– A four-week pause on the old path. Both paths run.
– Retirement of a department. You retire a chain, for the instances the numbers cover.
A board you can hang on a wall
One sheet. Two columns. Four rows. Instance count. Incident count. Window-close date. Signature line. If the sheet needs a narrative appendix to be understood, the measures failed.
– Can both paths be scored from the same incoming pack without a meeting?
– Is the baseline already visible — even if it still lives in Excel?
– Will four weeks produce enough instances to argue with?
– Is the retirement rule written, and is the signer a person?
Four yeses. Then run. At the end of the window the firm either retires a path or admits the twin did not win. Everything else is theatre with better tools.