Cost and abuse

What stops someone running up the AWS bill on dansabin.com, from WAF at the edge to token buckets and a kill switch, and how the site degrades before it refuses anyone.

case study · 2026
~/dansabin.com — chat Ask about this project

Isn't a public AI chat expensive to run?

Not per message. A typical turn is about 2 cents, from 52 turns on production over 14 days. A long conversation can reach about 8 cents, most of it the context sent to the model that writes the reply. With nobody here, it scales to 0 minus trivial hosting fees.

What stops a stranger running it up, without shutting everyone else out?

A public chat with no login spends real money on every message. Scaling isn't the worry. Abusers are. At Amazon we constantly saw people attempting to attack the domain. The last stop is an alarm that shuts the chat when too many tokens go through in five minutes. Set it too high and a stranger spends my money. Set it too low and a few real visitors close the site for everyone.

Seven replies in five minutes

I was moving the chat's replies to a stronger model, whose tokens cost about ten times as much, so every limit on the site was coming down. The agent building it proposed a tighter limit on bursts. It came as a token count with its checks passing and no line saying what it meant for a visitor, and I said ok.

A little later, before the branch went to review, the agent came back to that number and put it in visitors: "about 7 replies in 5 minutes across all visitors refuses every new message." Next to it was what an earlier drill had measured, a site that "took 41 minutes to reopen", and a suggestion to keep the earlier limit. I agreed, and we kept building.

The tighter limitThe earlier limit
How it read in the branchA token count, with its checks passingA token count, with its checks passing
How it reads in visitorsAbout seven replies in five minutesAbout fourteen in the test runs
After the site closes"I've paused my AI for now." for about 41 minutes, and I'm pagedThe same, at the larger wave

Where that came from

Every session the agent runs loads the rules from my handbook. One of them says: "Say what a result means before the number."

What to watch

Two numbers tell me whether the burst limit is sized for the traffic it guards: how many replies in five minutes close the site, and how many minutes it then stays closed. At the tighter limit, a wave of about seven replies would have closed the chat to everyone for about 41 minutes. Seeing that while it was being written took one message.

Replies in five minutes before the site closes One five-minute window, counted in replies across all visitors, in the order it happened: the limit as first proposed would have closed the site at about 7, seen while it was being written. Two days later, production showed real turns running larger than the test ones, so the earlier limit would close it at about 12: "four visitors asking three questions each, trips it." Today's limit sits at about 46, the wave I chose to allow. ONE FIVE-MINUTE WINDOW · REPLIES ACROSS ALL VISITORS about 7 · as first proposed seen while it was written about 46 · today the wave I chose to allow 0 10 20 30 40 50 about 12 · the earlier limit, real turns seen in production's numbers Past the mark, every visitor is refused for about 41 minutes. Replies in five minutes before the site closes One five-minute window, counted in replies across all visitors, in the order it happened: the limit as first proposed would have closed the site at about 7, seen while it was being written. Two days later, production showed real turns running larger than the test ones, so the earlier limit would close it at about 12: "four visitors asking three questions each, trips it." Today's limit sits at about 46, the wave I chose to allow. ONE FIVE-MINUTE WINDOW REPLIES ACROSS ALL VISITORS 0 25 50 about 7 · as first proposed seen while it was written about 12 · the earlier limit, real turns seen in production's numbers about 46 · today the wave I chose to allow Past the mark: refused for about 41 minutes.
One five-minute window, counted in replies across all visitors, in the order it happened: the limit as first proposed would have closed the site at about 7, seen while it was being written. Two days later, production showed real turns running larger than the test ones, so the earlier limit would close it at about 12: "four visitors asking three questions each, trips it." Today's limit sits at about 46, the wave I chose to allow.

On a team, the number I'd count is the share of threshold and config changes whose effect on a user is written down before review, in a line of the pull request. Each one caught there is a defect that never reaches production.

Want the details?

Ask the window anything this page skips.