Accelerate inference, model shaping, and pre-training on a research-optimized platform.
San Francisco, CA
Joined November 2022
- Tweets3.3K
- Following279
- Followers65.3K
- Likes2.5K
performance depends on the shape of your traffic
short in / long out → decode speed. long prompts → prefill. tool-heavy agents → end-to-end latency. repeated turns → token economics
measure the bottleneck your workload actually creates
tl;dr of what Together Compiler does, from @realDanFu on @thecompute100:
1/ run the model end to end
2/ find the bottlenecks
3/ let the model fix them
Links below 👇
Learn about Together Compiler: togetherai.link/R7BKfl7
Watch on YouTube: youtube.com/watch?v=V_SbzAxb…
Listen on Spotify: open.spotify.com/episode/0By…
1
695
SVP of Research Dan Fu on @thecompute100 covered:
- what a kernel actually is
- how a 3T-parameter model gets served on day zero
- how Together Compiler uses AI to improve inference
“We’re at a place where the models are good enough now that we can take those models and apply them to our own inference workloads. It’s starting to get to the promise of models that can improve themselves.”
Fun podcast on kernels, origins of @togethercompute, and how we’re using AI to build inference in an RSI inference loop with Together Compiler.
Thanks so much for having me @gabyllorenzi and @BSchech!
2
25
2,732
Up to 2.7× more throughput at the same speed per user.
In our benchmarks comparing Together Compiler vs. the best InferenceX configs on NVIDIA B200 and B300:
→ Kimi K3: 2.7×
→ GLM-5.3: 1.8×
→ DeepSeek V4.1 Flash: 1.8×
→ MiniMax M3.1 Flash: 1.5×
Read more: togetherai.link/R7BKfl7
If you’re comparing models by their price sheet, you’re missing half the math.
Jacob Loewenstein @spatialjlo from NVIDIA explains why token efficiency belongs at the top of your evaluation criteria, whether you’re adopting a model or training your own.
One of the standout moments from Open by Design NYC, where Jacob, Zach Wentz @zkwentz from Reflection, and Hassan El Mghari @nutlope from Together AI chatted with builders digging into open models, post-training, and inference.
Join us at an upcoming event: luma.com/user/together_ai
2
380
public benchmarks are useful for building a model shortlist
the real test is replaying the workloads you actually serve and seeing which models pass your own evals
leaderboards help you shortlist. your traffic tells you what to ship
step 5 throughput more than tripled as we optimized the serving stack against stepfun’s ttft + tps targets
agents helped search across moe routing, scheduling, kernels, spec decode + sparse attention, with every change benchmarked before moving to the next
Introducing Together Compiler: agents that continually optimize kernels for your model, your hardware, and your traffic.
We built this to scale frontier research from the teams behind FlashAttention, ThunderKittens, ThunderAgent, and more to every customer deployment.
So far we're seeing:
👉 +12% to +20% median TPS on serverless endpoints our researchers had already heavily optimized by hand
👉 Customers in production cutting response and generation times by about a third to nearly half
Read the full story: togetherai.link/R7BKfl7
“We are driving towards Super intelligence that is vastly superior to what we have today and will become a part of the labor force to solve kind of problems we have not been able to solve before.”
Our CEO, @vipulved , shares his vision for the future of AI in this conversation with @cheddar .
Watch the full interview: cheddar.com/media/inside-the…
3
11
2,778
collinear ai has been building on together post-training since our beta in april
with the latest release, self-onboarding made it easy for their team to launch full-scale agentic rl runs
together.ai/post-training
4
12
2,402
ripple-65b preview from ricursive is now live on together ai
the 65b dense model is built for agentic coding + reasoning, with ricursive reporting 80.5 on terminal-bench 2.1
trained from scratch on 12t tokens, with up to 1m context + apache 2.0
together.ai/models/ripple-65…
Inference is entering a new era. Models ship weekly, traffic shapes shift daily, and hardware turns over every year. The era of Static Inference, tuned once for one model with one traffic shape on one chip, is ending.
Welcome to the era of Polymorphic Inference: inference that adapts to new models, workloads, and hardware as they arrive.
Today we’re announcing Together Compiler, our first step into that era. It enables continual optimization, so a single overnight run can match or beat the manual expert tuning that used to take weeks.
Read the full vision from our SVPs of Research, @simran_s_arora and @realDanFu, on Polymorphic Inference and our research roadmap 👇
togetherai.link/uQJRZjW
“AI is now improving the systems that run AI.” Thanks to @Forbes for covering Together Compiler and the shift to Polymorphic Inference.
New open models can now get fully optimized for production performance in hours, not weeks.
togetherai.link/OeAIz0u
Introducing Together Compiler: agents that continually optimize inference for production traffic.
Serving trillions of tokens/day. Up to 2.6x faster than leading InferenceX configs, up to 50% faster on customer workloads.
Congrats @reyna_abhyankar @dylan__lim @simran_s_arora and the team!
Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨
Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur.
Together Compiler research highlights include:
1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems.
2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2.
3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation.
4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions.
From an incredible team @reyna_abhyankar @dylan__lim @realDanFu @togethercompute!!
We are live with @Kimi_Moonshot to go through Kimi K3.1 Pro.
Join now!
Everything you need to know about Kimi K3.1 Pro: A conversation with Moonshot AI and Together AI xitter.twitterapi.xyz/i/broadcasts/2dKYBAyRX…
1
8
2,232
Everything you need to know about Kimi K3.1 Pro: A conversation with Moonshot AI and Together AI xitter.twitterapi.xyz/i/broadcasts/2dKYBAyRX…
how clearly a task is defined can help guide model choice
scoped work like writing tests or renaming an API may not need your largest model
open-ended work like a framework upgrade or an auth refactor is where a larger model's reasoning is more likely to pay off
1
6
2,198
Kimi K3.1 Pro has arrived on Together AI
@Kimi_Moonshot’s most capable 3T-class model yet brings native multimodality, 1M context, and major gains across coding and agentic work
Run it as-is, or post-train it with RL/SFT on Together Post-Training for your own workflows
together.ai/models/kimi-k3-1…
Kimi K3.1 Pro has arrived, and the weights are dropping tomorrow.
Tomorrow, hear @Kimi_Moonshot 's Feihu Tang explain the architecture and decisions behind it.
He joins Austin Silveria, Kevin Cui and Zain Hasan from Together AI for a deep dive into long-running agents and production inference.
🗓 Wed, Nov 4 · 9 AM PT / 12 PM ET Register → luma.com/together-1w4x
