MTS@liquid.ai. Do vision pretaining and architecture design for lfm3 vl. Democratize on-edge intelligence.
Joined July 2022
- Tweets29
- Following28
- Followers49
- Likes4
Tianshu Yu retweeted
Adding vision to a language model is more than one extra modality. With large images or long video, the vision encoder can use more compute than the language model itself, so efficient models need a different design. Our VLM team, including @rshubert98, @TianshuYu6, @sina_semnani, and @iamsamstevens, explains how they teach LFMs to see, from mid-training to the custom vision encoder built for LFM3.
Tianshu Yu retweeted
The LFM3-VL-2B is the best model for its size for video understanding, quite usable compared to all other models of that size for checking security camera footage for example.
Tianshu Yu retweeted
LFM3-VL-2B vs Gemma 4 E2B on 50 real scanned receipts.
LFM3 got 41 of 50 receipts fully right. Gemma got 28. 50 receipts from SROIE, same prompt, JSON out, one 5060 Ti 16 GB.
- LFM3-VL-2B: 95% of fields right, 41/50 receipts perfect, 46 s
- Gemma 4 E2B: 87% of fields right, 28/50 receipts perfect, 62 s
Across both tests: Gemma reads messy chat-style text a bit better (95% vs 90%). LFM3 reads receipts better, and was faster both times.
@liquidai @googlegemma
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵
Tianshu Yu retweeted
pointed lfm3-vl-2b at the whole san fernando valley overnight. 16,692 aerial tiles, one rtx 3060, 5.5 hours.
it found 62,873 pools and drew a box around each one.
not a crazy idea, just “find the pools”. it misses about 1 in 5 and boxes a roof now and then. but it’s a 2B model with zero finetune.
imagine what you can do with this. palantir at home lmao
2/ LFM3-VL-2B
64 slots x 16K. ~143 tok/s on one request, ~583 tok/s total with all 64 describing images at once. 11.5GB VRAM.
config:
llama-server
-m LFM3-VL-2B-Q8_0.gguf
--mmproj mmproj-LFM3-VL-2B-F16.gguf
-ngl 99 -c 1048576 -fa on --jinja
-np 64
--cache-type-k q8_0 --cache-type-v q8_0
-b 2048 -ub 1024
--temp 0.1 --top-p 1.0 --top-k 50 --min-p 0.0 --repeat-penalty 1.1
--reasoning on --no-reasoning-preserve
Tianshu Yu retweeted
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵