ANEMLL (pronounced like "animal") Advanced Neural Engine Machine Learning Library, Open Source Project
Seattle and San Diego
Joined January 2025
- Tweets3.5K
- Following866
- Followers5.5K
- Likes9.4K
🚀🚀🚀
Apple's Core AI models repo is now accepting third-party contributions 🥳
If you have developed some cool ports, this is the best place to share your recipes with the community. Looking at you @JackdeS11 @mattt @kevinqz 😂
github.com/apple/coreai-mode…
👀
We're releasing new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-Flash-Next! They're better by 10-15% accuracy on DeepSWE, Terminal Bench, Aider Polyglot and more vs GSQ-RCO.
We also provide a new engram switching guide for BF16, Q8_0, Q5_1, IQ4_NL!
This builds upon our DeepSeek-V4.1-Flash UD3.1 work, which improved over 10-20% accuracy vs others.
All benchmarks are 256K for agentic ones, and 128K for the rest. We used temperature 1.0, top_p 0.95, top_k 20, min_p 0, no presence or repetition penalty. preserve_thinking and xhigh reasoning, and ran over 26B of tokens for benchmarking.
The RMSE vs BF16 for each of the engrams:
BF16 102.4 GB 0
Q8_0 54.4 GB 0.0053
Q5_1 38.4 GB 0.0376
IQ4_NL 28.8 GB 0.0757
More details at unsloth.ai/docs/models/qwen3…. We also moved all benchmarking, grading, infra issues and more to unsloth.ai/docs/basics/dynam…
System 1 ANE now supports iPhone Duo!
My first attempt at adapting a simple on-device AI app to the Duo layout:
Jeff (0.8B) playing Tetris on the Neural Engine, nothing in the cloud. Opt in and your games become training data for better System 1 models. Only the moves are shared, no account or device ID.
Same TestFlight, just update:
testflight.apple.com/join/4M…
Open source, iOS app included:
github.com/Anemll/system1-an…
PS video is created from Duo screen capture using ChatGPT 6.1 Terra and voice command
3
326
Good to see KLD defense across multiple datasets 👍
We’re releasing new DeepSeek-V4.1-Flash-1017 GGUFs, ranking highest across benchmarks.
Dynamic V3.1 leads across nearly all benchmarks: GPQA Diamond, AIME25, LiveCodeBench v6, MMLU-Pro, Aider & more.
Run on 64GB+ RAM.
GGUF: huggingface.co/unsloth/DeepS…
Blog: unsloth.ai/docs/models/deeps…
2
12
1,402
More on KLD … yes on private benchmarks.
On Qwen models, I also notice longer reasoning for poor quants, which translates into more tokens to solution
New paper from NVIDIA on 2-bit quantization: arxiv.org/abs/2610.33039
74B-A9B hybrid Mamba-Transformer (cut down from Nemotron 3.5 Super VL 120B-A12B), 2-bit quantization-aware distillation, fits in 24GB. Ships with a speculative drafter, 84 tok/s on RTX Spark. The 2-bit QAD stage alone was 26B tokens.
I spent about a month trying to get 2-bit to acceptable quality for local models... Colab plus my local h/w just wasn't enough compute to get there, so I shipped 3-bit instead. No luck with Google TPU Research Cloud access so far either
Good to see 2-bit actually working with enough distillation. IMO this is where local models are going.
Has anyone gotten usable 2-bit on 27B+ without a big training budget?
Open source for the win 🚀😎
The best way to run DeepSeek 4.1 Flash 1017 on M5 Ultra is to use @localcodeapp. It is fully open-source: github.com/drawthingsai/loca…, it is usable, and it is fast (note the decode can be improved further with DSpark).
3
576
Grok @bot integration with X messaging is quite useful, including audio and alerts . Bots drive multiple Claude agents on multiple local instances and Colabs for my experiments with Metal, CUDA and ANE quants and evaluations 🚀
Plus it’s easy to switch between phones, desktops, notebooks, iPad or AVP
2
844
🤘
Today we're coming out of stealth, introducing Neywa Labs 🚀
We built the infrastructure to build and deploy AI that runs on the devices people already own. Private, offline, unlimited.
One mission: make AI available to everyone in the world.
neywalabs.ai/
15
1,408
