VOLLENDATA
Randall Vollen

Type to search titles, summaries and topics.

Tool · Independent research

The Cost of Being Wrong

When does an imperfect AI agent become economically preferable to a human process? Adjust the assumptions and find the crossover.

Defaults are illustrative. This is an economic screening model, not permission to deploy.

Process assumptions

Cases handled in a year. Every variable cost is multiplied by it.

Labour, machine time and overhead to work one case on the human route.

The check applied to a human case before it is released.

Share of human cases that turn out to be wrong.

What one wrong human case costs when nobody catches it.

Model and compute cost to work one case on the agent route.

Automated checks run on every agent case, whether or not they find anything.

Share of agent cases that turn out to be wrong. This is the number the chart turns on.

What one wrong agent case costs when nobody catches it.

Build, integration, licences and running the agent route for a year.

Recruitment, training, tooling and management for the human route for a year.

Advanced risk assumptions

Share of human errors caught before a customer sees them.

What it costs to fix a human error that was caught in time.

Share of agent errors the automated checks catch on their own.

Share of agent cases that still get a human review.

What one human review of an agent case costs.

Of the errors the automated checks miss, the share that review then catches.

What it costs to fix an agent error that was caught in time.

Chance that one case produces an event severe enough to matter on its own.

What one severe event costs. This sits inside the expected total.

Share of a year that goes wrong together if the agent fails in one correlated way.

What one case costs inside that correlated failure. Reported beside the total, not inside it.

Evidence

Leave a pair empty when it has not been measured. An empty sample counts as unknown, not as zero.

Agent error rate: no measured sample, so the rate is taken as stated.

Human error rate: no measured sample, so the rate is taken as stated.

Deployment limits

A limit here ends the question. The arithmetic keeps running so the cost stays visible, but the answer is not permitted.

Pilot or insufficient evidence
  • No agent error sample: evidence insufficient
  • Expected savings are positive at the stated volume.
Expected savings
$100,000
Human route, annual
$1,200,000
Agent route, annual
$1,100,000
Agent route with no review, annual
$1,100,000
Break-even agent error rate
11.0 %
Break-even volume
Always
Expected error loss per year
$1,000,000
Tail loss if the agent fails together
$0

Where the money goes

Annual cost by component, human route against agent route
ComponentHumanAgent
Fixed cost$0$0
Execution$1,000,000$50,000
Verification$0$50,000
Review$0$0
Error loss$200,000$1,000,000
Severe events$0$0
Total$1,200,000$1,100,000

Scenarios

The same inputs at three agent error rates: the upper bound, the point estimate and the lower bound of the measured interval, or one and a half times and half the stated rate when nothing has been measured.

Annual cost under the conservative, base and optimistic agent error rate
ScenarioHumanAgentSavings
Conservative$1,200,000$1,600,000-$400,000
Base$1,200,000$1,100,000$100,000
Optimistic$1,200,000$600,000$600,000

What the error rate costs

Agent annual cost across the whole range of agent error rates, against the human line. Every point is a full run of the model.

$0.0$2.8M$5.7M$8.5M$11.3M0.0 %25.0 %50.0 %75.0 %100.0 %Agent error rate
Agent annual costHuman annual costBreak-even agent error rate
Agent annual cost sampled at 51 agent error rates, 0 to 100 %
Agent error rateAgent annual costHuman annual cost
0.0 %$100,000$1,200,000
2.0 %$300,000$1,200,000
4.0 %$500,000$1,200,000
6.0 %$700,000$1,200,000
8.0 %$900,000$1,200,000
10.0 %$1,100,000$1,200,000
12.0 %$1,300,000$1,200,000
14.0 %$1,500,000$1,200,000
16.0 %$1,700,000$1,200,000
18.0 %$1,900,000$1,200,000
20.0 %$2,100,000$1,200,000
22.0 %$2,300,000$1,200,000
24.0 %$2,500,000$1,200,000
26.0 %$2,700,000$1,200,000
28.0 %$2,900,000$1,200,000
30.0 %$3,100,000$1,200,000
32.0 %$3,300,000$1,200,000
34.0 %$3,500,000$1,200,000
36.0 %$3,700,000$1,200,000
38.0 %$3,900,000$1,200,000
40.0 %$4,100,000$1,200,000
42.0 %$4,300,000$1,200,000
44.0 %$4,500,000$1,200,000
46.0 %$4,700,000$1,200,000
48.0 %$4,900,000$1,200,000
50.0 %$5,100,000$1,200,000
52.0 %$5,300,000$1,200,000
54.0 %$5,500,000$1,200,000
56.0 %$5,700,000$1,200,000
58.0 %$5,900,000$1,200,000
60.0 %$6,100,000$1,200,000
62.0 %$6,300,000$1,200,000
64.0 %$6,500,000$1,200,000
66.0 %$6,700,000$1,200,000
68.0 %$6,900,000$1,200,000
70.0 %$7,100,000$1,200,000
72.0 %$7,300,000$1,200,000
74.0 %$7,500,000$1,200,000
76.0 %$7,700,000$1,200,000
78.0 %$7,900,000$1,200,000
80.0 %$8,100,000$1,200,000
82.0 %$8,300,000$1,200,000
84.0 %$8,500,000$1,200,000
86.0 %$8,700,000$1,200,000
88.0 %$8,900,000$1,200,000
90.0 %$9,100,000$1,200,000
92.0 %$9,300,000$1,200,000
94.0 %$9,500,000$1,200,000
96.0 %$9,700,000$1,200,000
98.0 %$9,900,000$1,200,000
100.0 %$10,100,000$1,200,000

Model assumptions and formulas

Symbols

N
cases handled in a year
F
annual fixed cost of one route
X
execution cost per case
V
verification cost per case
q
share of errors detected before they cause loss
c
cost to correct an error that was detected
L
loss carried by one error that was not detected
r
share of cases a person reviews
d_R
share of the errors review catches that the automated checks missed
l
loss carried by one error, detection folded in

Total cost

TC = F + N · (X + V + p · L)

Every route is priced the same way. F is paid whether or not any work happens. N · (X + V) is the doing and checking of the work. The last term is the expected cost of being wrong: the error rate p times the loss Lcarried by one error that nothing caught.

Detection and the loss folded into one error

l_J = q_J · c_J + (1 - q_J) · L_J

A detected error is a correction bill. An undetected one is the loss itself. For the human route the detection rate is stated directly. For the agent route there are two chances to catch an error, the automated checks and the sampled review, and this model treats them as independent:

q_A = 1 - (1 - q_A,auto) · (1 - r · d_R)

Independence is an assumption, not a measurement. Two checks that share a blind spot catch less than this formula says, and the chart will look better than the process deserves.

u_H = X_H + V_H + p_H · l_H

u_A = X_A + V_A + r · c_R + p_A · l_A + p_S · L_S

Those are the per-case costs, so TC_H = F_H + N · u_H and TC_A = F_A + N · u_A. The agent route is also run with r = 0 to give the cost with no human review at all. Review is a cost, not a saving: a review that costs more than the errors it catches makes the no-review number lower.

Break-even agent error rate

p* = (TC_H - F_A - N · (X_A + V_A + r · c_R + p_S · L_S)) / (N · l_A)

The agent error rate at which the two routes cost the same. If the denominator is zero there is no such rate. If the numerator is negative the agent route cannot win even at a zero error rate. If the ratio is above one the agent route wins at every possible error rate.

Break-even volume

d = u_H - u_A, D = F_A - F_H, N* = D / d

The volume at which the two routes cost the same, read from the sign table: if d and D are both positive the crossover sits at D / d; if the human route is dearer per case and no dearer to set up, the agent route wins at every volume; if both differences are negative the agent route wins only below D / d cases; and if the per-case costs are equal the agent route never catches up.

The tail is reported, not added

tail = blast fraction · N · loss per affected case

A correlated failure is not an average year. It is kept out of the expected total so the headline number stays an expectation, and shown beside it so the tail is not forgotten.

How the decision is reached, in order

  1. Any deployment limit set means not permitted, and nothing below can change that.
  2. No measured agent error sample, unknown loss amounts, or a break-even rate that lands inside the agent error interval all mean pilot or insufficient evidence.
  3. Otherwise positive expected savings mean economically favorable.
  4. Otherwise the agent route is not economically favorable.

How the uncertainty is handled

Each sample gets a Wilson score interval at 95 %. The three scenarios re-run the model at the upper bound, the point estimate and the lower bound of the agent interval, or at one and a half times and half the stated rate when there is no sample to measure.

Economic advantage is not permission

The model answers one question: at this volume, is the agent route cheaper in expectation? It does not answer whether the work may be done, whether the errors are tolerable, or whether the answer stays true next quarter. A process can be economically favorable and still be the wrong process. Deployment limits are entered separately on purpose, so that a cheap route cannot quietly become an approved one.

What this model does not capture

  • Correlated failure beyond the single blast scenario: one scenario is a number typed in by a reader, not a distribution with a tail behind it.
  • Statistical confidence beyond the Wilson interval: no test of whether the two routes are actually different, and no correction for trying many volumes until one looks good.
  • Regulatory restriction: nothing here reads a rule, a licence or a filing. Deployment limits are a checkbox a person ticks, not a compliance check.
  • How long the assumptions hold: model drift, retraining, price changes, staff turnover and the cost of the transition itself all sit outside the arithmetic.

The starting numbers on this page are invented for demonstration and are labelled as such. Nothing here is a benchmark, a client result or a claim about any real process.

Anyone with this link can read these inputs

The link carries every assumption on this page in the address itself, including your volume, your costs and your loss figures. Anyone who has it can read them, and so can anything that reads the page later. Nothing is stored on this site, but the link itself is the copy.

Continue only if these numbers are yours to share.