Cost and abuse
What stops someone running up the AWS bill on dansabin.com, from WAF at the edge to token buckets and a kill switch, and how the site degrades before it refuses anyone.
Isn't a public AI chat expensive to run?
Not per message. A typical turn is about 2 cents, from 52 turns on production over 14 days. A long conversation can reach about 8 cents, most of it the context sent to the model that writes the reply. With nobody here, it scales to 0 minus trivial hosting fees.
What stops a stranger running it up, without shutting everyone else out?
A public chat with no login spends real money on every message. Scaling isn't the worry. Abusers are. At Amazon we constantly saw people attempting to attack the domain. The last stop is an alarm that shuts the chat when too many tokens go through in five minutes. Set it too high and a stranger spends my money. Set it too low and a few real visitors close the site for everyone.
Seven replies in five minutes
I was moving the chat's replies to a stronger model, whose tokens cost about ten times as much, so every limit on the site was coming down. The agent building it proposed a tighter limit on bursts. It came as a token count with its checks passing and no line saying what it meant for a visitor, and I said ok.
A little later, before the branch went to review, the agent came back to that number and put it in visitors: "about 7 replies in 5 minutes across all visitors refuses every new message." Next to it was what an earlier drill had measured, a site that "took 41 minutes to reopen", and a suggestion to keep the earlier limit. I agreed, and we kept building.
| The tighter limit | The earlier limit | |
|---|---|---|
| How it read in the branch | A token count, with its checks passing | A token count, with its checks passing |
| How it reads in visitors | About seven replies in five minutes | About fourteen in the test runs |
| After the site closes | "I've paused my AI for now." for about 41 minutes, and I'm paged | The same, at the larger wave |
Where that came from
Every session the agent runs loads the rules from my handbook. One of them says: "Say what a result means before the number."
What to watch
Two numbers tell me whether the burst limit is sized for the traffic it guards: how many replies in five minutes close the site, and how many minutes it then stays closed. At the tighter limit, a wave of about seven replies would have closed the chat to everyone for about 41 minutes. Seeing that while it was being written took one message.
On a team, the number I'd count is the share of threshold and config changes whose effect on a user is written down before review, in a line of the pull request. Each one caught there is a defect that never reaches production.
Want the details?
Ask the window anything this page skips.