Run and train models locally with the Unsloth Desktop app. 🦥 https://xitter.twitterapi.xyz/t.co/2kXqhhvdCD
San Francisco, CA
Joined November 2023
- Tweets829
- Following483
- Followers105K
- Likes15.9K
Unsloth AI retweeted
Unsloths support for Deepseek is here! 👀
+ Improvements on Strix
+ Improvements on parallel requests
+ Improvements on DGX Spark
+ Improvements on Intel B-series
See more at: github.com/Niko1221/Strata/r…
Unsloth AI retweeted
MiniMax M3.1 Flash meets the @UnslothAI community. 🐙🦥✨
Unsloth’s Dynamic GGUFs bring our 428B MoE model to a 143.9GB 2-bit footprint, with 91.5% top-1 token agreement in their evaluation across seven domains.
That means a lower memory barrier to self-hosting, deployment through Unsloth Desktop and llama.cpp, and an OpenAI-compatible API for existing apps and agents.
With 78K+ GitHub stars and over 500M model downloads on Hugging Face, Unsloth gives M3.1 Flash a path into an established community of developers running, evaluating and building with open models.
Open weights. Local control. More ways to build.
MiniMax-M3.1-Flash can now run locally!
We made 2-bit GGUF retain 91.5% top 1 accuracy while being 45% smaller. Run on 150GB RAM.
MiniMax-M3.1-Flash is the strongest open model for its size & among the best overall.
GGUF: huggingface.co/unsloth/MiniM…
Guide: unsloth.ai/docs/models/minim…
Unsloth AI retweeted
Getting started with @UnslothAI on RTX Spark? Just run the .exe.
@danielhanchen and @NaderLikeLadder got hands-on with NVIDIA RTX Spark to talk about running and training AI models locally, exploring reinforcement learning, and making it easier to start building with Unsloth.
Unsloth AI retweeted
We plugged Beam into @opencode, asked it to look at @UnslothAI, and create a fine-tuning notebook for the latest and smallest Gemma-4 model on a Text2SQL task – an out of distribution domain for Beam.
The model successfully researches the model cards, Unsloth documentation, and available READMEs, and implements the fine-tuning logic and surrounding machinery. As a result, Beam is able to successfully finetune Gemma, increasing Gemma’s accuracy on its heldout test set by 66.5%.
Unsloth AI retweeted
🥳 Ornith-1.5-122B Unsloth GGUFs are now available!
🧡 Thanks for the awesome work by @UnslothAI
🔗 huggingface.co/unsloth/Ornit…
Aloha!🌺 Meet Ornith-1.5-122B: No Tradeoff Between Strong and Fast
Stronger. Faster. Leaner. Ornith-1.5-122B delivers frontier-level capabilities with exceptional inference efficiency. With superior instruction following, stronger task decomposition, and deeper understanding of real-world workflows, it achieves state-of-the-art performance among open-source models of comparable size, while running on a single NVIDIA DGX card!!
Open weights under the MIT license.
Coding
✅SWE-bench Verified: 0.845
✅TB2.1: 0.813
✅CyberGym: 0.800
✅NL2Repo: 0.512
Agentic
✅HLE: 0.446
✅BrowseComp: 0.785
✅MCP-Atlas: 0.746
✅Toolathlon: 0.708
🤗Hugging Face: huggingface.co/ornith-ai/Orn…
We can’t wait to see what you build with Ornith-1.5-122B.
Unsloth AI retweeted
how do you fine-tune a decision model on your own data?
in these two notebooks, you'll learn:
• how to prepare the training data
• how to fine-tune our d1-3B decision model
• using @UnslothAI and @huggingface TRL
all within 30 minutes on a free-tier GPU.
unsloth: github.com/Liquid4All/cookbo…
TRL: github.com/Liquid4All/cookbo…
ps: how adorable are these halloween kitties? 👻
Unsloth AI retweeted
Video: Decision model trained in ~2 mins on a single DGX Spark using Unsloth! Qwen3.5-2B as base model.
Just follow instruction here: unsloth.ai/docs/basics/train…
Unsloth AI retweeted
We're co-hosting an open source party with @HuggingFace! 🤗🦥
Come join us, we'll be handing out @UnslothAI merch, showcasing our upcoming features and more!
It'll be a super fun night with lots of demos, DJs, food and more.
Join: luma.com/OpenTogether
Use code: NOSLOTHSHERE to get in.
Unsloth AI retweeted
We got @UnslothAI a DGX Station!
@DanielHanChen and @NaderLikeLadder checked out Unsloth’s new @Dell Pro Max with GB300 and talked about what comes next: support for more models, faster quantization, and more efficient reinforcement learning.
Unsloth AI retweeted
Friendly reminder that you can fine tune 500+ open source models in a free Google Colab
You can even upload PDFs/CSVs and turn them into usable synthetic datasets.
1. Open the Google Colab below
2. Run the blocks to install Unsloth Studio
3. Choose a model
4. Upload a dataset
5. You're good to go!
And you can of course export your model afterwards.
Unsloth AI retweeted
ICYMI, we're celebrating 1 BILLION+ downloads for @googlegemma 💎
How are developers actually using open models? @GoogleDeepMind’s @DynamicWebPaige caught up with devs and collaborators like @UnslothAI and @Qualcomm to hear how they’re building on-device tools, running local fine-tuning, and pushing multimodal breakthroughs.
Unsloth AI retweeted
Local GGUF inference is now up to 3.3× faster at long context lengths
We made GLM-5.3-Flash run 3.3x faster locally!
Local GGUF inference is now 1.6–3.4× faster with optimized decoding and bonus multi-token prediction.
Run 3-bit on 128GB setups via Unsloth Desktop or llama.cpp.
Guide: unsloth.ai/docs/models/glm-5…
GGUF: huggingface.co/unsloth/GLM-5…
Unsloth AI retweeted
⚡ Fine-tune and quantize AI models directly on NVIDIA Jetson.
Our new Jetson AI Lab tutorial shows how to use Unsloth and memory-efficient QLoRA to customize models, export quantized GGUF files, and run them locally with llama.cpp.
Follow hands-on examples for:
🔹 NVIDIA Nemotron 3.5 Lightning on Jetson AGX Thor
🔹 Qwen3.5-4B on Jetson Orin Nano
Start optimizing: nvda.ws/45YagAq
Unsloth AI retweeted
GLM-5.3 can now be run locally!
The 2-bit model retains ~81% accuracy after we shrunk it from 1.51TB to 239GB (-83% size).
Run on a 256GB Mac or RAM/VRAM setups.
GLM-5.3 is the strongest open model to date.
Guide: unsloth.ai/docs/models/glm-5…
GGUF: huggingface.co/unsloth/GLM-5…
GLM-5.3 is now open-weight.
Our most capable model for agentic coding and cyber defense is now available to download, run, and customize.
Weights: huggingface.co/zai-org/GLM-5…
Tech blog: z.ai/blog/glm-5.3
Unsloth AI retweeted
The VRAM barrier is officially dead.
I just ran Qwen 3.8 Flash Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090.
21 tokens/sec decode. 364 t/s prefill.
no mtp. no dflash. no kv cache quantization!
We are running datacenter models on consumer hardware.
Tested on Ubuntu 22 | CUDA 13.0 | PCIe 4.0 x16 | 110 GB DDR4 System RAM with a continuous 28k prompt across all runs.
### The Benchmarks & Scaling
# 1. Hybrid Offload (-ncmoe 40 @ 80k Context)
Offloaded 40 expert layers to the GPU, pushing VRAM to the ceiling.
./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 80000 --port 8080 -v --fit off -b 4096 -ub 4096 -ncmoe 40
Prefill: 383.85 t/s | Decode: 22.52 t/s
Footprint: 23.85 GB VRAM | 97 GB RAM
# 2. Full CPU MoE Offload (-cmoe @ 80k Context)
Pinned all 512 expert layers to DDR4 RAM (-cmoe), keeping attention on the 4090.
llama.cpp flags: (Same as above, replace -ncmoe 40 with -cmoe)
Prefill: 355.72 t/s | Decode: 20.84 t/s
Footprint: 11.66 GB VRAM (12GB+ VRAM freed up!) | 110 GB RAM
# 3. The 180,000 Context Run
Prefill: 357.75 t/s | Decode: 20.98 t/s | VRAM: 15.6 GB | RAM: 110 GB
# 4. The 250,000 Context Absolute Ceiling
./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 250000 --port 8080 -v --fit off -b 4096 -ub 4096 -cmoe
Prefill: 364.29 t/s | Decode: 20.97 t/s
Footprint: 18.3 GB VRAM (Still ~5.7 GB of VRAM headroom!) | 110 GB RAM
### Key Insights:
-b 4096 -ub 4096: doubles the prompt ingestion from ~150 to 364+ t/s.
-cmoe Free Lunch: Shifting expert layers to DDR4 RAM slashes VRAM from 24GB to 11.6GB with virtually zero decode penalty (22.5 -> 20.9 t/s), enabling the 250k context ceiling.
Qwen 3.8 Flash-Next (UD-Q4_K_XL) is a massive 111.4 GB model split across 4 shards. To run this architecture, you must build from the experimental PR branch (#27742) by @danielhanchen:
git clone && cd llama.cpp
git fetch origin pull/27742/head:qwen-next && git checkout qwen-next
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DBUILD_SHARED_LIBS=OFF
cmake --build build --config Release -j $(nproc) --target llama-server
A single 4090 paired with 100 GB of cheap DDR4 RAM will comfortably serve production grade 125B inference.
While Qwen 3.8 27B (dense) still holds the crown for single 3090/4090 rigs, Flash Next proves 125B hybrid models are officially viable on consumer hardware.
Hugging Face GGUF link and complete performance telemetry graphs are dropped in the replies below.
GLM 5.3 Flash VS Qwen 3.8 Flash Next, which one takes the open weights crown this week?
Qwen3.8-Flash can now be run locally! 🔥
The 125B MoE model outperforms Claude-Opus-4.6 (Max).
Run on 75GB RAM via Unsloth GGUFs.
Qwen3.8-Flash-Next enables CPU RAM / unified mem setups to deliver near VRAM speeds.
Guide: unsloth.ai/docs/models/qwen3…
GGUF: huggingface.co/unsloth/Qwen3…
Unsloth AI retweeted
A high-performance 125B model now running locally on just 75GB RAM! Thank you @UnslothAI for the day-0 support.🥳
Qwen3.8-Flash can now be run locally! 🔥
The 125B MoE model outperforms Claude-Opus-4.6 (Max).
Run on 75GB RAM via Unsloth GGUFs.
Qwen3.8-Flash-Next enables CPU RAM / unified mem setups to deliver near VRAM speeds.
Guide: unsloth.ai/docs/models/qwen3…
GGUF: huggingface.co/unsloth/Qwen3…
Unsloth AI retweeted
We did a new Unsloth Desktop release!
* Experimental auto compaction for any model
* LAN & Remote Access tabs
* Faster & smoother chatting
* Over 200 PRs merged!
For compaction, we use a mix of RAG + forced 1st turn RAG + tail - let us know how it goes!
github.com/unslothai/unsloth…
Unsloth AI retweeted
Huge thanks to Unsloth for the great work. This is wonderful news for the community! 🥳Qwen3.8-27B, smaller and sharper than ever. Let's try it! @UnslothAI
We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy.
Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks.
We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.
Blog: unsloth.ai/docs/basics/dynam…
GGUF: huggingface.co/unsloth/Qwen3…
Unsloth AI retweeted
Local AI just got a major upgrade.
Fine-tune and run AI models locally with @UnslothAI Desktop, plus get up to 20% faster llama.cpp performance (@ggerganov) with our new optimizations.
Unsloth AI retweeted
Qwen3.8-27B + Unsloth is wow! Feels like having an Opus-level coding agent running fully locally it edits, sees, fixes, and more. Huge congratulations to Qwen for dropping a true Opus-class model that actually runs on local hardware, and to Unsloth for making it easy to play with!
@Alibaba_Qwen @QwenDevs @UnslothAI