The hidden cost of tokens
Per-token prices keep falling, yet our bill kept rising. Where the tokens were really going, and the five changes that cut spend by 61%.
Daniel Reyes2 min read
Category
Infrastructure
GPUs, serving, cost and capacity planning.
2 articles
GPU scheduling for small teams
Four GPUs, nine engineers and a lot of idle time. How a simple queue and a few house rules doubled our utilisation.
Daniel Reyes2 min read