Trying to solve sample efficiency prev: Engineering physics @iitbombay '23
Joined June 2023
- Tweets814
- Following668
- Followers4.2K
- Likes2.8K
Pinned Tweet
I got 37% on ARC-AGI-2 Kaggle without LLMs!
100M looped transformer trained from scratch
- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%
thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition!
Link: mvakde.github.io/blog/looped…
(writeup for the paper track)
Mithil Vakde retweeted
we should not like a future where there are 1 to 2 AIs and the rest are illegal. we should want a cambrian explosion of the types and possibilities of minds, experiencing new joys and thinking new thoughts. ensuring that this is compatible with human flourishing is only step 1
Mithil Vakde retweeted
LLMs require very large amounts of data and arguably the path to ASI runs through a different model architecture that’s more data sparse. Amazing what @evilmathkid has done with compute constraints on ARC-AGI-2.
cc @fchollet
I got 37% on ARC-AGI-2 Kaggle without LLMs!
100M looped transformer trained from scratch
- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%
thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition!
Link: mvakde.github.io/blog/looped…
(writeup for the paper track)
Mithil Vakde retweeted
And Mithil strikes again, this time casually beating opus
I got 37% on ARC-AGI-2 Kaggle without LLMs!
100M looped transformer trained from scratch
- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%
thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition!
Link: mvakde.github.io/blog/looped…
(writeup for the paper track)
Mithil Vakde retweeted
THE GOAT (of resource efficient arc impls) RETURNS
I got 37% on ARC-AGI-2 Kaggle without LLMs!
100M looped transformer trained from scratch
- Likely highest score without an LLM
- Peaked at 6th place
- Beats last year's winning LLM
- Looping, added in the last week: 33% → 35% → 37%
thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition!
Link: mvakde.github.io/blog/looped…
(writeup for the paper track)
Why do it without an LLM?
Because I care about sample efficiency. LLMs see so much data, ARC puzzles included, that you can't measure it. We now know that with enough data, ARC-1/2 become trivial
So the interesting question is how far a model gets on a few ARC puzzles alone
1
3
692
This is a scaled up and upgraded version of my previous work on ARC-AGI
xitter.twitterapi.xyz/evilmathkid/status/202…
44% on ARC-AGI-1 in 67 cents!
Trained from scratch in 2hrs on a 5090
Matches TRM, beats HRM and is way faster & cheaper
No recursion, just a transformer
Also, 7% on ARC-2 🧵
1
4
680
didn’t know my botc lobbies and my ai timeline were the same people
I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game).
The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies.
I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump.
Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game.
You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com
1
1
550
Mithil Vakde retweeted
What, if some day or night a demon were to steal after you into your loneliest loneliness and say to you: ‘This life you will have to live in an age of chaos; in the most interesting times; in an era when the foundations of civilization are shaking and few dare to step off the crumbling tracks laid before them.’
Would you not throw yourself down and gnash your teeth and curse the demon who spoke thus? Or have you once experienced a tremendous moment when you would have answered him: ‘You are a god and never have I heard anything more divine.’
Mithil Vakde retweeted
What if your devices, apps and agents knew what you wanted?
You finish breakfast, and your Uber is booked.
Running late, your agent sends a “Running late” text.
You reach home, and your lights turn on.
AI is proactive now. But it still needs your yes. It waits, for you to unlock your phone, open an app and tap approve. Every time, hundreds of times a day.
AI needs its own interface.
Aina is a ring with a screen that predicts and shows you the three things you’ll need next. One flick of your thumb, and it’s done. For everything.
✦ Cues: proactive suggestions, ready before you need them
✦ Voice: talk to your agents, take notes, dictate
✦ Remote: control your music, Mac and TV
✦ Calls: right from your hand
Pre-orders open now
aina.com
Mithil Vakde retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Mithil Vakde retweeted
Hi! this is Rohan Anil, I develop training algorithms among other things - my work trains various frontier models.
At CoreAuto, I was fascinated by the problem of training on compute islands, eventually in space training.
I hope you all enjoy this write up!🪐🛰️
coreauto.com/hitchhikers-gui…
Mithil Vakde retweeted
I manually inspected 40,000 animations to post train an LLM to make demoscenes.
Scraped 10 years of demos, did continued pretraining, curriculum learning and finally RLHF-ed with myself as the reward function.
Yes, models can have "taste".
All you need is data.
1/6
Mithil Vakde retweeted
Launching RNAFoundry.com – we manufacture personalized cancer vaccines.
Mithil Vakde retweeted
Launching Junction: come speed-run the semiconductor tech tree this December in Bangalore.
We’re flying in students from all over India 🇮🇳 to a 2-week fabrication retreat at @_lagrangepoint to build equipment, design chips and engineer processes, with our friends at @hackerfabindia.
India still needs thousands of semicon engineers -- the ecosystem to produce them isn’t mature enough yet -- we're changing that.
Current applicants have built vacuum chambers to study plasma physics, RISC-V processors from scratch, scanning tunneling microscopes, et cetera
At junction you will:
1. Propose a project
2. Figure out a plan with the Junction team.
3. Iterate fast towards a working prototype, with help from the Junction team
4. Be mentored by top semiconductor hackers and specialists.
5. Demo to industry leaders, founders, investors, and your fellow craftspeople
Flights, accommodation and project expenses? All covered. Apply on the link below
wow
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
1
27
3,157