Portkey

introducing portkey cli. 1 command to set up claude code or codex with portkey's gateway. add llms, mcps, and skills across your entire eng org. every agent gets the same batteries, no manual setup. try it today: npx portkey setup

the diagram is incomplete. i've fixed it

LLM Pricing Is 100x Harder Than You Think why do @karpathy, @thdxr, @simonw, @swyx, @opeclaw all maintain their own model pricing data? because nobody else has solved it. we've tracked $180M+ in llm spend on Portkey's gateway. here's why it breaks, and how we solve it 🧵 Every project I look at maintains its own model data. A JSON with names, prices, maybe context limits. @karpathy has one in LLM CLI. @thdxr built models.dev for OpenCode because there was no canonical source for pricing and capabilities across providers. @simonw built llm-prices.com and has 67+ blog posts tracking pricing changes. @swyx tracks model releases through AI News at Latent Space almost daily. LibreChat has one. Pi has one. @theo's T3 code has one. Everyone rebuilds this independently. It works for a few weeks, then drifts. And it's not just pricing. Model names change. Context limits get updated silently. New billing dimensions show up and nobody's JSON has a field for them. At Portkey we've tracked $180M+ in LLM spend across 3,500+ models over three years. The naive formula says cost = tokens × rate. That's the tip of the iceberg. Below the surface, 6 patterns break every in-house implementation I've seen: 1/ Thinking tokens @OpenAI's o3, @AnthropicAI's Claude with extended thinking: tokens consumed for internal reasoning that never appear in the response. You still get billed. If your system only counts visible output tokens, you're undercounting agentic workloads by 30-40%. 2/ Cache asymmetry Anthropic charges 25% MORE for cache writes. OpenAI charges nothing for writes. Apply a single "cache discount" multiplier across both and your numbers are wrong for at least one provider. Looks like a rounding error until you're processing millions of cached requests a day. 3/ Context thresholds Cross 128K tokens and per-token cost silently doubles on many models. $0.075/M becomes $0.15/M. Nothing in the API response tells you which tier you hit. Openai, Anthropic, Google all do this. Almost no one accounts for it. 4/ Same model, different prices Kimi K2.5: $0.50/$2.80 on @TogetherCompute, $0.60/$3.00 on Fireworks. @awscloud Bedrock prepends regional prefixes that need stripping before you can look up the price. @Azure returns deployment names instead of model identifiers. You need an extra API call just to figure out what model you're running. 5/ Non-token billing DALL·E 3 bills by resolution. Video charges per second. Realtime audio has separate input/output rates. Fine-tuning is per-token on some models, per-hour on others. "cost = tokens × rate" was never the full picture. 6/ New dimensions keep appearing Started with 2 billing dimensions. Now there are 20+. Web search has per-query pricing. Googles's Grounding with Search has its own rate. Tool use, code execution, each has its own cost model. New ones show up faster than providers update their docs. The whole industry is focused on harness design, agents, benchmarks. Meanwhile every project maintains its own model metadata in a JSON that's stale by the time you ship. There's literally open issues on OpenCode asking for better models[dot]dev integration because manually tracking this is "tedious and error-prone." @stripe clearly sees this problem too. They just launched their own AI Gateway specifically for LLM token billing. Same core insight: the centralized proxy is the natural place to solve cost attribution. We've been solving this at Portkey for three years. We open-sourced all of it 👇