Scaling an app to 6,000 users on a tight budget
· 8 min
We inherited the application days before an expected traffic spike: from a few hundred to around 6,000 concurrent users, with no budget for horizontal scaling and no time for a rewrite. The deadline was one week, and the goal was simple to state and hard to deliver: keep latency stable and the system up under real load.
Finding the real bottleneck
Before touching a single line of code, we measured. A quick profile of the FastAPI backend showed the application itself wasn't the problem, the async routes responded well in isolation. The bottleneck was downstream: PostgreSQL exhausted its available connections within minutes of simulated load, and a handful of queries without proper indexes were triggering sequential scans on tables already past a few million rows.
PostgreSQL: connection pooling and indexes
The first change was separating the application's connection pool from the database's: we put PgBouncer in transaction pooling mode in front of Postgres, cutting the number of physical connections the database had to sustain even as hundreds of FastAPI workers opened connections concurrently. On the application side, we tuned SQLAlchemy's pool size (pool_size and max_overflow) so it wouldn't compete with the limit configured on PgBouncer.
In parallel, we reviewed the execution plans of the most-called queries with EXPLAIN ANALYZE and added composite indexes on the columns used in the most frequent filters and joins. Two queries that were running sequential scans on large tables started using an index instead, cutting response times from hundreds of milliseconds down to a few milliseconds under load.
Validating every change with k6
Every adjustment was validated with Grafana k6 load tests simulating the expected access pattern: a ramp-up to 6,000 concurrent VUs, holding the peak for a few minutes before ramping down. We set explicit thresholds, p95 under a target ceiling and an error rate near zero, so the test itself would fail automatically if stability regressed.
Tests ran after every change, turning the week into a short loop of hypothesis, change, and measurement instead of a single bet at the deadline. That let us isolate the real effect of PgBouncer, the indexes, and the pool tuning, one at a time.
Race conditions under high concurrency
The load tests themselves surfaced a problem low traffic would never have shown: with thousands of concurrent requests hitting the same record (the same order, the same user balance), we started seeing lost updates and, in some cases, duplicate rows, a classic race-condition symptom from a read-then-write pattern with no locking at all.
The fix lived in the database, not the application: critical transactions switched to SELECT ... FOR UPDATE to lock the row through the read-write cycle, we added UNIQUE constraints as a last line of defense against duplicates, and introduced idempotency keys on the most sensitive write endpoints so a client retry after a timeout wouldn't produce a duplicate side effect. We rebuilt the scenario in k6 with multiple VUs deliberately hammering the same resource, until the test stopped detecting any inconsistency.
The result
By the end of the week, the application sustained 6,000 concurrent users with a stable p95 and without exhausting database connections, on the same infrastructure that previously struggled with a fraction of that traffic. No new hardware, no rewrite, just measuring the right bottleneck, fixing the connection problem, and proving every change under real load before trusting it.