the bunny talisman of yearn
Joined January 2010
- Tweets30.4K
- Following3.7K
- Followers234K
- Likes31.9K
come join the sokobanteg frontier challenge. it has almost reached my own holdout result (dotted line). the real fun begins there!
soko.banteg.xyz/frontier
btw limited time offer. if you find improvements, i will give you a follow back.
introducing sokobanteg frontier challenge
every room opens with the shortest route any model found in the realtime eval runs: opus 5.5 starts with 28 rooms, fable 5.2 with 25, astra with 7, sol with 4, sonnet and carbon with one each. the best segment run of the whole trilogy is 3,819 inputs total.
models already hit proven optimal in 44 of 66 rooms. in the remaining 22, my holdout routes tally to 3,064 keys, so 755 keys of slack, with only the target lengths shown so you can hunt them down yourself.
63/66 rooms are closed by lower-bound math proofs (part i = 748, part ii = 950, trilogy floor = 3,000). three rooms in part iii remain open:
- iii-14: 43 floor / 48 known / 116 model
- iii-20: 74 floor / 85 known / 145 model
- iii-21: 268 floor / 316 known / 589 model
sign in with x to claim first clears on all 66 rooms, shave shortest-route records off the models (by hand or with your own solver), or submit a reproducible lower-bound proof. every key replays in the original game.
soko.banteg.xyz/frontier
3
16
3,501
models played heroes of sokoban. it went like this.
now you can beat them: soko.banteg.xyz/frontier
you can now raw dog the submission on soko.banteg.xyz/frontier by playing the game yourself.
humans also get the undo button, it works like a backspace, so it doesn't make your solution longer.
introducing sokobanteg frontier challenge
every room opens with the shortest route any model found in the realtime eval runs: opus 5.5 starts with 28 rooms, fable 5.2 with 25, astra with 7, sol with 4, sonnet and carbon with one each. the best segment run of the whole trilogy is 3,819 inputs total.
models already hit proven optimal in 44 of 66 rooms. in the remaining 22, my holdout routes tally to 3,064 keys, so 755 keys of slack, with only the target lengths shown so you can hunt them down yourself.
63/66 rooms are closed by lower-bound math proofs (part i = 748, part ii = 950, trilogy floor = 3,000). three rooms in part iii remain open:
- iii-14: 43 floor / 48 known / 116 model
- iii-20: 74 floor / 85 known / 145 model
- iii-21: 268 floor / 316 known / 589 model
sign in with x to claim first clears on all 66 rooms, shave shortest-route records off the models (by hand or with your own solver), or submit a reproducible lower-bound proof. every key replays in the original game.
soko.banteg.xyz/frontier
the full heroes of sokoban trilogy takes between 3,000 and 3,064 keys. i can't close the last 64.
beat any record and it's yours, verified against the original game. beat a proven minimum and you broke my proof, which is even better.
soko.banteg.xyz/frontier
you can watch sokobanteg replay of the livestreamed thinking-only v0.2 run. only opus and fable could clear the game, with astra and carbon getting very close, but running out of 5 hour/part time limit. sol has also shown a very respectable result.
soko.banteg.xyz/watch/?rules…
sokobanteg v0.2 live
gpt-6.1 sol, gpt-6.1 terra, gpt-6 astra
claude opus 5.5, claude sonnet 5.5, claude haiku 5.5, claude fable 5.2
gemini 4 argon, gemini 4 carbon
qwen3.8 max 0902, kimi k3.1 pro, muse wonder, muse ember, grok 4.7 build
xitter.twitterapi.xyz/i/broadcasts/2ZBjVkaLm…
this version of the eval uses no tools, so the models need to reason in their head and solve all puzzles with thinking only.
i added some mitigations against doom loops, namely, the input is returned if 16 of the keypresses in a row result in no changes. previously some models spammed walking into a wall or restart when they deemed the level impossible or misunderstood the rules.
in this version every model runs on the same harness that uses @pidotdev underneath.
5
883
big walk is getting review bombed for 0.1% ai use. their mistake was engaging in purity tests. it's obvious from the framing where try to explain themselves. remember kids, you don't owe anyone any explanations. just let your work speak for itself.
bigwalk.game/faq/#ai
hope you liked sokobanteg livestream and it was entertaining to watch the models trying to solve puzzles in their head.
we hit some huge variance and some unfortunate doom loops this time. it also gave me some very useful observations of where the harness should intervene. for example, i added an early return when 16 inputs result in no screen change. since there is no give up function, some models burn inputs instead of trying harder.
interestingly, in this run, terra survived for a long time and beat 51 level before i manually stopped the eval. this proves smaller models could do well if they believe in themselves.
the next plan is to finish a unified harness based on pi, fix the python repl we give the models, and test newly released gemini carbon, muse models, and some chinese models.
im also trying to prove the lower bounds of perfect solutions, so we have things to compare on when this eval gets more crowded.
there may or may not be another livestream, i haven't decided yet.
12
2,690
with python, opus finished first, and now it’s only two more rooms left for sol and astra. tune in!
in the thinking only run, the last remaining contestants are opus and, surprisingly, terra.
other models were unlucky to fall into a doom loop, which i will investigate.
2
8
3,055
sokoban eval live xitter.twitterapi.xyz/i/broadcasts/2qVjNznjA…
btw if you want an unsaturated puzzle eval that is extremely fun to watch, i made one at soko.banteg.xyz. so far only ran all models at medium effort.
Claude Fable 5.2 from @AnthropicAI on ARC-AGI (Verified):
- ARC-AGI-3: 97.7%, $18.6K (provider adapter harness), 81.2%, $21.9K (standard harness)
- ARC-AGI-2: 93.8%, $0.98/task
- ARC-AGI-1: 98.5%, $0.36/task
Fable 5.2 sets a new high score on ARC-AGI-3 in the standard harness.
will have more to say on this in a couple of weeks 👀
we should make formal verification the "new normal" for DeFi in 2027
ref anthropic.com/research/forma…
