Building SkyRL at @BerkeleySky Join the Slack community: https://xitter.twitterapi.xyz/t.co/mSO97T61vR
Berkeley, California
Joined January 2025
- Tweets189
- Following18
- Followers2.9K
- Likes322
Pinned Tweet
We are excited to announce that SkyRL now implements the Tinker API. Run Tinker training scripts on your own hardware with zero code changes.
Try it out today: novasky-ai.notion.site/skyrl…
SkyRL now implements the Tinker API.
Now, training scripts written for Tinker can run on your own GPUs with zero code changes using SkyRL's FSDP2, Megatron, and vLLM backends.
Blog: novasky-ai.notion.site/skyrl…
🧵
NovaSky retweeted
Super excited to share this work from our AWS FDE team! 🚀
With AgentCore RL Toolkit (ART) + SkyRL (@NovaSkyAI), a 9B SWE-Grep agent beats Claude Sonnet 5 on line F1 across two code-retrieval benchmarks after 30 RL steps.
See it in action. 👇 1/6
NovaSky retweeted
"Evolving Agent Teams"
AI coding agents would get stuck optimizing solutions that are fundamentally limited, and simply adding more agents with fixed roles doesn't necessarily solve the problem.
So this paper introduces a supervisor that continuously reorganizes its agent team based on what is blocking progress. It can create specialized agents for debugging, profiling, or exploring alternative solutions, while preserving lessons from previous attempts in shared memory.
With an extended token budget, the system develops a GPU kernel that runs 1.2x faster than the production FlashInfer baseline, outperforming it on 36 out of 47 workloads. It also achieves 3.44x speedup over FlashKDA on another kernel optimization task using NVIDIA B200 GPUs.
alphaxiv.org/abs/2610.36848
NovaSky retweeted
Post-training is now a core workload for AI labs, data providers, and enterprise leveraging proprietary data to build custom models.
Learn from the practictioners doing it at @harborframework, @NovaSkyAI and @mercor
Doors open 5:30 PM 🍕 na2.hubs.ly/H08MNd-0
NovaSky retweeted
We support SkyRL in AgentEnv agentenvframework.com/docs/t… -- train your model on GridSimEnv using SkyRL today!
At @ScaleAILabs we created a high fidelity RL Environment of a trading desk in a real Texas battery grid.
GPT-6.1 Sol made $85,771 trading a real Texas battery. The battery's real operator made $20,647.
Hand the design back to agents! Check our recent work on self-improving multi-agent systems for long-running optimization tasks (GPU kernels, end-to-end systems, and beyond)!
1/n
Introducing our NeurIPS’26 work 💙Evolving Agent Teams (EAT) 💛, where a supervisor diagnoses why an agent team stalls and restructures it to escape local optima, for long-running optimization tasks.
EAT shows superior performance across domains:
🏆 Open Kernel Challenge, decode layer on H200: 2.449× over SGLang, #4 of 1,180 (as of Nov 1, 2026)
⚡️ NVIDIA kernels: KDA prefill on B200, 3.44× over FlashKDA (previous best: 2.96×)
🔥 AMD kernels: KDA prefill on MI350X, 3.01× over vLLM
💻 MoE serving on Mac: Qwen3.8-Flash-Next (Unsloth UD-IQ4_XS GGUF) on M5 Pro, ~3.3× faster prefill (930 tok/s) and 1.22× faster decode (43.7 tok/s) than Strata at the same memory
🦾 RoboDojo: 62.1 on held-out seeds (the best leaderboard entry as of Oct 26, 2026 is 55.3)
- Paper: arxiv.org/abs/2610.36848
- Code: github.com/caoshiyi/EAT
🧵👇
1
404
NovaSky retweeted
Really happy to see SkyRL picking up on day 0 supports; thanks to @AoShen98662303 from @vllm_project ! Congrats to @MistralAI !
🎉 Day-0 RL for @MistralAI’s Mistral Large 4 with @NovaSkyAI’s SkyRL and @vllm_project!
On 8 @nvidia GB200 nodes, LoRA RL on the NVFP4 checkpoint brings AIME 2024 avg@8 from ~64% to ~75% in 75 training steps.
vLLM generates rollouts, while FSDP2 or Megatron handles training. The recipe uses rank-32 LoRA, NVFP4 quantization-aware training, FP8 KV-cache simulation, routing replay (R3), and token-level truncated importance sampling.
The FSDP2 run maintains a mean absolute trainer–vLLM logprob gap of ~0.02–0.026. Megatron reaches similar accuracy with a comparable gap.
NovaSky retweeted
🎉 Day-0 RL for @MistralAI’s Mistral Large 4 with @NovaSkyAI’s SkyRL and @vllm_project!
On 8 @nvidia GB200 nodes, LoRA RL on the NVFP4 checkpoint brings AIME 2024 avg@8 from ~64% to ~75% in 75 training steps.
vLLM generates rollouts, while FSDP2 or Megatron handles training. The recipe uses rank-32 LoRA, NVFP4 quantization-aware training, FP8 KV-cache simulation, routing replay (R3), and token-level truncated importance sampling.
The FSDP2 run maintains a mean absolute trainer–vLLM logprob gap of ~0.02–0.026. Megatron reaches similar accuracy with a comparable gap.
NovaSky retweeted
The release adds low / medium / high reasoning effort, defaulting to high. Pixtral vision supports multiple images per prompt for multimodal workflows.
SkyRL (@NovaSkyAI) also supports day-0 RL training for Mistral Large 4, with vLLM rollouts.
NovaSky retweeted
We have Day-0 RL support for Le Chonk on SkyRL, with LoRA training on FSDP or Megatron, courtesy of @AoShen98662303 and the @vllm_project team!
Today, we release the weights of a new and improved Le Chonk, advancing the open frontier for agentic coding and cybersecurity.
💻 Built to build: Le Chonk ranks #1 on @cognition’s FrontierCode among open models, and sets a new benchmark for Pareto efficiency on @artificialanalysis.
🛡 Built to defend: State-of-the-art in cyberdefense, put through the paces by the world's leading cyber vendors, including @CrowdStrike, @cloudflare, @rubrik, @zscaler, and others.
Built for sovereignty: Make it yours. Download the weights on @huggingface , customize it with Mistral Forge, and deploy it wherever you need it.
Read more from our team and customers 👇:
NovaSky retweeted
Great to see @modal launch this! You can also deploy a Tinker endpoint directly on AWS EC2 with SkyRL @NovaSkyAI for the same cost and flexibility benefits, and start agentic RL training with our AgentCore RL Toolkit! github.com/awslabs/agentcore…
Tinker changed how people do post-training. We're big fans, and so are many of the teams we work with. But as those teams scale up, they keep raising the same two problems: per-token pricing gets expensive quickly, and you’re limited to the models and features the provider decides to support.
We think a good API deserves more than one implementation.
Today we're releasing Spindle, an open-source, Tinker-compatible training server that runs in your Modal workspace. Your existing Tinker scripts run unchanged, and because you own the server, you pay for GPU time instead of tokens.
NovaSky retweeted
FrontierCS 2 is our new benchmark for CS research & development.
Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
NovaSky retweeted
Post-training a 4B model with RL to fly Tesla Roadster. It outperforms recent open models and even GPT-6 Astra in a storm.
The key: have the model learn from the teachers first and then learn itself in practice.
Trained with SkyRL (@NovaSkyAI) and vLLM (@vllm_project), orchestrated by SkyPilot's Job Group (@skypilot_org).
Check it out ⬇️
Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds."
We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind?
We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips.
Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.
NovaSky retweeted
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀
This work was done in collaboration with the amazing folks at @LilaSciences! 🤩
Getting there took key algorithmic and systems optimizations in four main areas:
1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer.
2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU.
3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA.
4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network.
For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…!
This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!
NovaSky retweeted
Check our blog: skypilot.ai/blog/flying-road…
Trained with SkyRL (@NovaSkyAI), served with vLLM (@vllm_project), and orchestrated with SkyPilot (@skypilot_org)
NovaSky retweeted
Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds."
We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind?
We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips.
Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.
NovaSky retweeted
Model customization recipes are live on our website and in the cookbook repo.
Beam is now supported across RL and SFT stacks including @axolotl_ai, NeMo RL, NeMo AutoModel, Megatron Bridge (@NVIDIAAI), Miles (@radixark), SkyRL (@NovaSkyAI), and TRL (@huggingface).
Get cooking 🧑🍳
developers.reflection.ai/cus…
NovaSky retweeted
First day 0 model support on SkyRL. Great work from @sumanthrh !
NovaSky retweeted
🎉 Day-0 RL support for Beam in SkyRL @NovaSkyAI.
LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM rollouts and Reflection’s Megatron-Bridge implementation.
SkyRL reports +62% LoRA decode throughput from right-sizing expert adapters in vLLM: 276 → 447 tok/s/GPU at 32 concurrent requests, PP2×TP8. This is a decode result, not an end-to-end training speedup.
R3 works with pipeline parallelism, and FP32 router weights are preserved during weight sync. Thanks to Sumanth Hegde and the SkyRL team!
docs.skyrl.ai
