Tool · Independent research
The Cost of Being Wrong
When does an imperfect AI agent become economically preferable to a human process? Adjust the assumptions and find the crossover.
Defaults are illustrative. This is an economic screening model, not permission to deploy.
- Expected savings
- $100,000
- Human route, annual
- $1,200,000
- Agent route, annual
- $1,100,000
- Agent route with no review, annual
- $1,100,000
- Break-even agent error rate
- 11.0 %
- Break-even volume
- Always
- Expected error loss per year
- $1,000,000
- Tail loss if the agent fails together
- $0
Where the money goes
| Component | Human | Agent |
|---|---|---|
| Fixed cost | $0 | $0 |
| Execution | $1,000,000 | $50,000 |
| Verification | $0 | $50,000 |
| Review | $0 | $0 |
| Error loss | $200,000 | $1,000,000 |
| Severe events | $0 | $0 |
| Total | $1,200,000 | $1,100,000 |
Scenarios
The same inputs at three agent error rates: the upper bound, the point estimate and the lower bound of the measured interval, or one and a half times and half the stated rate when nothing has been measured.
| Scenario | Human | Agent | Savings |
|---|---|---|---|
| Conservative | $1,200,000 | $1,600,000 | -$400,000 |
| Base | $1,200,000 | $1,100,000 | $100,000 |
| Optimistic | $1,200,000 | $600,000 | $600,000 |
What the error rate costs
Agent annual cost across the whole range of agent error rates, against the human line. Every point is a full run of the model.
| Agent error rate | Agent annual cost | Human annual cost |
|---|---|---|
| 0.0 % | $100,000 | $1,200,000 |
| 2.0 % | $300,000 | $1,200,000 |
| 4.0 % | $500,000 | $1,200,000 |
| 6.0 % | $700,000 | $1,200,000 |
| 8.0 % | $900,000 | $1,200,000 |
| 10.0 % | $1,100,000 | $1,200,000 |
| 12.0 % | $1,300,000 | $1,200,000 |
| 14.0 % | $1,500,000 | $1,200,000 |
| 16.0 % | $1,700,000 | $1,200,000 |
| 18.0 % | $1,900,000 | $1,200,000 |
| 20.0 % | $2,100,000 | $1,200,000 |
| 22.0 % | $2,300,000 | $1,200,000 |
| 24.0 % | $2,500,000 | $1,200,000 |
| 26.0 % | $2,700,000 | $1,200,000 |
| 28.0 % | $2,900,000 | $1,200,000 |
| 30.0 % | $3,100,000 | $1,200,000 |
| 32.0 % | $3,300,000 | $1,200,000 |
| 34.0 % | $3,500,000 | $1,200,000 |
| 36.0 % | $3,700,000 | $1,200,000 |
| 38.0 % | $3,900,000 | $1,200,000 |
| 40.0 % | $4,100,000 | $1,200,000 |
| 42.0 % | $4,300,000 | $1,200,000 |
| 44.0 % | $4,500,000 | $1,200,000 |
| 46.0 % | $4,700,000 | $1,200,000 |
| 48.0 % | $4,900,000 | $1,200,000 |
| 50.0 % | $5,100,000 | $1,200,000 |
| 52.0 % | $5,300,000 | $1,200,000 |
| 54.0 % | $5,500,000 | $1,200,000 |
| 56.0 % | $5,700,000 | $1,200,000 |
| 58.0 % | $5,900,000 | $1,200,000 |
| 60.0 % | $6,100,000 | $1,200,000 |
| 62.0 % | $6,300,000 | $1,200,000 |
| 64.0 % | $6,500,000 | $1,200,000 |
| 66.0 % | $6,700,000 | $1,200,000 |
| 68.0 % | $6,900,000 | $1,200,000 |
| 70.0 % | $7,100,000 | $1,200,000 |
| 72.0 % | $7,300,000 | $1,200,000 |
| 74.0 % | $7,500,000 | $1,200,000 |
| 76.0 % | $7,700,000 | $1,200,000 |
| 78.0 % | $7,900,000 | $1,200,000 |
| 80.0 % | $8,100,000 | $1,200,000 |
| 82.0 % | $8,300,000 | $1,200,000 |
| 84.0 % | $8,500,000 | $1,200,000 |
| 86.0 % | $8,700,000 | $1,200,000 |
| 88.0 % | $8,900,000 | $1,200,000 |
| 90.0 % | $9,100,000 | $1,200,000 |
| 92.0 % | $9,300,000 | $1,200,000 |
| 94.0 % | $9,500,000 | $1,200,000 |
| 96.0 % | $9,700,000 | $1,200,000 |
| 98.0 % | $9,900,000 | $1,200,000 |
| 100.0 % | $10,100,000 | $1,200,000 |
Model assumptions and formulas
Symbols
- N
- cases handled in a year
- F
- annual fixed cost of one route
- X
- execution cost per case
- V
- verification cost per case
- q
- share of errors detected before they cause loss
- c
- cost to correct an error that was detected
- L
- loss carried by one error that was not detected
- r
- share of cases a person reviews
- d_R
- share of the errors review catches that the automated checks missed
- l
- loss carried by one error, detection folded in
Total cost
TC = F + N · (X + V + p · L)
Every route is priced the same way. F is paid whether or not any work happens. N · (X + V) is the doing and checking of the work. The last term is the expected cost of being wrong: the error rate p times the loss Lcarried by one error that nothing caught.
Detection and the loss folded into one error
l_J = q_J · c_J + (1 - q_J) · L_J
A detected error is a correction bill. An undetected one is the loss itself. For the human route the detection rate is stated directly. For the agent route there are two chances to catch an error, the automated checks and the sampled review, and this model treats them as independent:
q_A = 1 - (1 - q_A,auto) · (1 - r · d_R)
Independence is an assumption, not a measurement. Two checks that share a blind spot catch less than this formula says, and the chart will look better than the process deserves.
u_H = X_H + V_H + p_H · l_H
u_A = X_A + V_A + r · c_R + p_A · l_A + p_S · L_S
Those are the per-case costs, so TC_H = F_H + N · u_H and TC_A = F_A + N · u_A. The agent route is also run with r = 0 to give the cost with no human review at all. Review is a cost, not a saving: a review that costs more than the errors it catches makes the no-review number lower.
Break-even agent error rate
p* = (TC_H - F_A - N · (X_A + V_A + r · c_R + p_S · L_S)) / (N · l_A)
The agent error rate at which the two routes cost the same. If the denominator is zero there is no such rate. If the numerator is negative the agent route cannot win even at a zero error rate. If the ratio is above one the agent route wins at every possible error rate.
Break-even volume
d = u_H - u_A, D = F_A - F_H, N* = D / d
The volume at which the two routes cost the same, read from the sign table: if d and D are both positive the crossover sits at D / d; if the human route is dearer per case and no dearer to set up, the agent route wins at every volume; if both differences are negative the agent route wins only below D / d cases; and if the per-case costs are equal the agent route never catches up.
The tail is reported, not added
tail = blast fraction · N · loss per affected case
A correlated failure is not an average year. It is kept out of the expected total so the headline number stays an expectation, and shown beside it so the tail is not forgotten.
How the decision is reached, in order
- Any deployment limit set means not permitted, and nothing below can change that.
- No measured agent error sample, unknown loss amounts, or a break-even rate that lands inside the agent error interval all mean pilot or insufficient evidence.
- Otherwise positive expected savings mean economically favorable.
- Otherwise the agent route is not economically favorable.
How the uncertainty is handled
Each sample gets a Wilson score interval at 95 %. The three scenarios re-run the model at the upper bound, the point estimate and the lower bound of the agent interval, or at one and a half times and half the stated rate when there is no sample to measure.
Economic advantage is not permission
The model answers one question: at this volume, is the agent route cheaper in expectation? It does not answer whether the work may be done, whether the errors are tolerable, or whether the answer stays true next quarter. A process can be economically favorable and still be the wrong process. Deployment limits are entered separately on purpose, so that a cheap route cannot quietly become an approved one.
What this model does not capture
- Correlated failure beyond the single blast scenario: one scenario is a number typed in by a reader, not a distribution with a tail behind it.
- Statistical confidence beyond the Wilson interval: no test of whether the two routes are actually different, and no correction for trying many volumes until one looks good.
- Regulatory restriction: nothing here reads a rule, a licence or a filing. Deployment limits are a checkbox a person ticks, not a compliance check.
- How long the assumptions hold: model drift, retraining, price changes, staff turnover and the cost of the transition itself all sit outside the arithmetic.
The starting numbers on this page are invented for demonstration and are labelled as such. Nothing here is a benchmark, a client result or a claim about any real process.