Local-First AI: When GPU Rentals Don't Make Sense
When buying matters more than renting. The math for solo founders.
The naive "renting is always cheaper" claim
You hear this everywhere. It's true if your utilisation is low. It's not always true if your utilisation is high. Here's where the line is.
The crossover math
Per-month cost of a rented GPU on a continuous schedule:
hourly_rate * 24 * 30 = monthly_cost
At $0.20/hour spot pricing, that's about $144/month. At $0.50/hour on-demand mid-tier, $360/month. At $1.50/hour for a high-tier card, $1,080/month.
Per-month amortised cost of buying a comparable GPU + box + power, over 24 months, lands in the $40-$120/month range for entry-level consumer cards.
Crossover happens fast — at sustained utilisation, owning beats renting cleanly within 6-12 months for entry-tier hardware.
When buying wins
- You run model load 6+ hours a day, every day. The fixed cost amortises.
- You're doing privacy-sensitive workloads. Some data simply cannot leave your premises.
- You need predictable latency floor. Spot rentals can disappear; your workload may not survive that.
- You're training frequently. Training a small model weekly on rentals can cost as much as buying the GPU outright in 6 months.
When renting wins
- You spike — heavy training once a month, otherwise idle. Fixed cost wastes capital.
- You need a top-tier card occasionally. A rented A100 for a weekend beats buying one for a year of underuse.
- Geographic mobility. If you move/travel, rental beats hauling hardware.
The hybrid that wins for most solo founders
- Buy a modest local GPU (entry-tier consumer card, 8-12 GiB VRAM).
- Run inference, light fine-tuning, dev work locally.
- Rent on demand for big training runs and ad-hoc throughput bursts.
This is the shape that's beaten "all rental" for me on three different setups. The local box pays for itself in saved rental fees within a year, and ad-hoc rentals stay rare and predictable.
The mistakes I've watched
- Buying a $4,000 GPU "to be safe". Utilisation didn't justify it. Six months in, the box was idle most of the time.
- Going pure-rental at high utilisation. The sticker shock at month 12 is real. $1,000+ months on rentals add up.
- Self-hosting one small model and forgetting it. A bigger frontier API would have done it for $20/month.
The decision in three lines
- Less than 4 hours/day average use → rent.
- 4-12 hours/day → buy entry-tier, rent for spikes.
- 12+ hours/day or privacy-required → buy proper hardware.