Daniel Han

We're releasing new Qwen3.8-27B GGUFs with Unsloth Dynamic V3.1! Expect 5-10% accuracy gains on the same size vs UD3 and other quants. We do better vs GSQ-RCO, Exllama-v3 on the pareto frontier for Terminal Bench, DeepSWE and more. Quants above 2-bit to 6-bit are updated. We did not update 3-bit quants from UD3 to UD3.1 as we found UD3 to perform on par with 3.1. We also decided to benchmark Exllama3 quants as the community requested them. Exllama3 keeps the token embedding layer in BF16, which is different from ISTA and UD3 / UD3.1 and GGUFs. We decided to exclude the token embedding from all plots. This does not mean the RAM and disk space usage of EXL3 is less - it's in fact more if you include the BF16 embedding layer. We provide plots for actual RAM+VRAM usage at unsloth.ai/docs/models/qwen3…, and it shows exl3 quants doing much worse. For LiveCodeBench, GSQ-RCO showed Unsloth UD3 quants doing consistently worse than ISTA's. However we found the opposite - UD3 quants (our OLD quants) do better than ISTA on LCB on all sizes except the smallest. UD3.1 further improves on LCB as well. We approximate that GSQ-RCO used a context budget of 96K, as that best fits the data for the runs we have. At 96K, GSQ-RCO does somewhat do better than UD3, but at 128K, UD3 does much better (and UD3.1 even more) For ISTA GSQ-RCO's AIME25 benchmarks, their quants reach 100%, 100%, 100% and 96.7% accuracy recovery. We estimate ISTA only did 1 trial - we did 14 trials, and we show on average the recovery rae is 93% to 96%. And in fact our UD3 old quants do better on average than their quants on 14 runs vs 1 run of theirs. We are still improving our algorithms to make smaller quants and larger quants (and also 3-bit) much more accurate, and will definitely apply these methods to Qwen4! All blog details at unsloth.ai/docs/models/qwen3… and the new quants are at huggingface.co/unsloth/Qwen3…

Yes our main goal was to make quants better on the Pareto efficiency front! So vs GSQ & V3 quants, V3.1 is much better! Avg recovery of UD-IQ3_XXS is 99.9% & UD-IQ2_XS at 99.4% whilst being smaller than IQ3_S - the diffs are largest on small bits - UD-IQ1_S 97.9% vs GSQ 90.5%

Our new Unsloth Dynamic v3.1 quants for Qwen-3.8-Flash-Next work seamlessly now in Strata! UDv3.1 DeepSeek-V4.1-Flash GGUFs also run great!

We're releasing new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-Flash-Next! They're better by 10-15% accuracy on DeepSWE, Terminal Bench, Aider Polyglot and more vs GSQ-RCO. We also provide a new engram switching guide for BF16, Q8_0, Q5_1, IQ4_NL! This builds upon our DeepSeek-V4.1-Flash UD3.1 work, which improved over 10-20% accuracy vs others. All benchmarks are 256K for agentic ones, and 128K for the rest. We used temperature 1.0, top_p 0.95, top_k 20, min_p 0, no presence or repetition penalty. preserve_thinking and xhigh reasoning, and ran over 26B of tokens for benchmarking. The RMSE vs BF16 for each of the engrams: BF16 102.4 GB 0 Q8_0 54.4 GB 0.0053 Q5_1 38.4 GB 0.0376 IQ4_NL 28.8 GB 0.0757 More details at unsloth.ai/docs/models/qwen3…. We also moved all benchmarking, grading, infra issues and more to unsloth.ai/docs/basics/dynam…

New Unsloth Desktop update! Unsloth now has Glass themes and more customization options for backgrounds, greetings and chats. We also enabled full CPU training (optimized as well!) and make diffusion, inference, training all faster! github.com/unslothai/unsloth…

We ran DeepSWE, Terminal Bench, Automation Bench, HLE, SciCode at 512K context on our DeepSeek UD3.1 GGUFs and show we're >10-20% better than GSQ-RCO We ran all other benchmarks (14 in total) to 128K, fixed grading, infra. AIME2025 64x, DeepSWE/TB 3x, gpt-6-luna xhigh for HLE We addressed some issues people mentioned with these new benchmarks: 1. 32K context was too short - we did 512K context for agentic + 128K for general tasks 2. Saturated benchmarks - new unsaturated agentic style ones including Terminal Bench, DeepSWE and Automation Bench 3. Benchmarks are noisy - we did 35.1B tokens of benchmarking vs 1.3B before so 27x more data. See unsloth.ai/docs/models/deeps… where we did 64x AIME 2025 runs, 3x DeepSWE, 3x TB runs and more. 4. Our graders are wrong - we used Math Verify huggingface.co/blog/math_ver…, improved graders, used gpt-6-luna xhigh for HLE, and all official graders for Terminal Bench (Harbor), DeepSWE and the rest. See unsloth.ai/docs/models/deeps… and unsloth.ai/docs/models/deeps… for infra fixes For DeepSWE v1.1, we used mini-swe-agent and chose 500 max steps (334 steps was the max runs we saw, so none went over). Terminal Bench 2.1 used mini-swe-agent as well, and we set max steps to 150 with no max limit. We had to exclude 7/89 tasks since they couldn't pass on our infra - see unsloth.ai/docs/models/deeps… We also re-checked ISTA's numbers as we have done many more runs (27x more data). GSQ-RCO IQ1_M claims 98.7% recovery, but we measured 97.24 [96.23, 98.21] with a p-value of 0.0013. GSQ-RCO IQ2_XS retaining 100% claimed recovery is possible with a p-value of 0.21. We measured 98.38% [97.52, 99.22]. GSQ-RCO IQ2_XXS claims 99.80% recovery but we measured 98.38 [97.52, 99.22] with a p-value of 0.0001. If we can get more details on how the benchmarks were run, we are more than happy to run more. We provided all our setups and methods in our blog at unsloth.ai/docs/models/deeps…

We're running 128K for all benchmarks - LCB and Aider are finished - the results are even more obvious

We're releasing new Unsloth v3.1 GGUF quants for DeepSeek-V4.1-Flash-1017. We ran benchmarks and v3 + v3.1 do better than GSQ-RCO. We also cannot reproduce numbers ran by GSQ-RCO - AIME25 for 1bit is not 100% but rather 91.88%. Livecodebench and GPQA are similarly incorrect. There are also claims that KLD is not a good metric, and that people can "game" KLD. The pirmary goal of quantization is to minimize KLD but also minimize disk space. If you get a KLD of 0, this means your quantized model == 16bit model, as logits are perfectly modelled. We ran benchmarks on multiple benchmarks including AIME25, Aider Polyglot, LiveCodeBench v6, HMMT 2026, GPQA Diamond, MMLU-Pro and IFEval, and also recorded reasoning length to see if lower bit quants think for longer. In unsloth.ai/docs/models/deeps…, we show that lower bit quants do think somewhat longer than the unquantized model - we plan to investigate and make quants even better. However, we're still much ahead vs GSQ-RCO on even reasoning length as well. We provide full benchmarking data, benchmarking methods (for eg we ran AIME25 16 times per question and took an average) and more in our updated blog at unsloth.ai/docs/models/deeps…

Mistral Large 4 is here! We made some Unsloth quants & tested across many domains! 1bit retains 91% of top-1 accuracy across coding, agentic tasks, math etc whilst being 80% smaller (223GB vs 1.1TB FP8) They all work in Unsloth Desktop now! Thank you Mistral for day zero access!

Unsloth now has browser use for all local models! It can take screenshots and act as well! We also added some cool Halloween themes and effects! If you click Trick or Treat, there are over 11 themes to choose from + animations 👻🎃!

Unsloth works seamlessly in RTX Spark! You can do video, image, audio generation and all multimodal llm coding tasks plus training for all! Native Windows ARM64 + PyTorch + triton kernels are all enabled!

We made Unsloth UD3 MiniMax-M3.1-Flash quants! We tested across multiple domains to ensure our quants work great for them as well! By default, MiniMax-M3.1-Flash uses NVFP4 for MoE layers and BF16 / MXFP8 for the rest. UD-IQ1_M retains ~89.9% top-1%. Works in Unsloth Desktop!

We made Flint Unsloth GGUFs - Recursive's new model - it's an MoE that's the best perf for its size! It works great for tool calls and agentic coding in Unsloth Desktop! We also now have a beta browser + browser-use, skills, more themes and more! Flint can use them all!

We made some Unsloth UD3 dynamic quants for Ornith's new 122B release! UD-Q2_K_XL is 44.5GB and retains around 84% top-1% for wikitext general and 89% ish for chat whilst being 82% smaller (BF16 = 249GB)

We worked on a llama.cpp impl for DS4.1-Flash + made Unsloth dynamic GGUFs for it! The engram / PLE can be swapped out from MXFP8 to Q6_K to Q5_K and IQ4_NL precision - we found Q6_K approx = MXFP8 and IQ4_NL loses a tiny bit of accuracy. The engram can stream fully from disk!

Unsloth now works with ARM64 + Windows + CUDA on RTX Spark! We made [running, training] X [diffusion, multimodal, language, decision] models all run natively + we optimized it to make it 2.7x faster! Get it via winget or unsloth.ai/download/windows

Muse Spark Open from Meta is here! We made Unsloth UD-3 quants for the model with UD-IQ1_S being 176GB We also did some KLD benchmarks and recovered as much accuracy as we could for the smaller quants! Muse Spark Open is fully multimodal and works in Unsloth Desktop!

We created some Unsloth UD-3 GGUF quants for NVIDIA's Nemotron 3.5 Super and Bolt model! Bolt runs great in RTX Spark! We also made NVFP4 GGUFs for Super! We also ran KLD benchmarks for both - docs at unsloth.ai/docs/models/nemot… and unsloth.ai/docs/models/nemot…

We're doing a meetup at Dolores Park, Saturday 2PM - we'll be around the palm trees at the center of the park! Co-hosted with @arankomatsuzaki @iScienceLuvr @gordic_aleksa All are welcome!