Sina Shahandeh (@SinaShahandeh) profile picture

Building for AI in scientific discovery. My life mission is to make humanity more rational and empathetic

Joined April 2009
Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)Photo by Sina Shahandeh (@SinaShahandeh)
People are baffled as to why LLMs talk like this: just say you should sign it. The challenge is when LLMs are post trained in an RL env, they follow a narrow mental process. This is basically saying my task is done and next task of signing you can only do. This is a next frontier of AI training to instill aptitude for long horizon task completion while maintaining a natural flow. It's almost two different tasks that seems contradictory in nature or at best orthagonal. BTW, no statistical parrot talks like this.
1
29
People say that with AI, there is a dawn of an era where a generalist can do so much that deep technical expertise is no longer required, but this intuition is wrong. In practice, with a generally capable AI, the biggest value is in deep, critical expertise and understanding. People who own a very narrow/deep knowledge that has not yet been captured by general AI have a chance to contribute.
Yesterday a junior researcher messaged me, frustrated and a little scared. They had just read that a company pretrained its model with a single researcher. One person for the entire pretraining. no R&D team, no nothing. If one person and a cluster can do what used to take a whole team, then the expertise they had spent years building is already obsolete. I get the fear. Marketing headlines like this are built to produce it. But I think the conclusion is wrong, and the reasons it is wrong say a lot about what a researcher is for now. Here is what I told them, expanded: 1. Be adaptive: RSI is real, but this isn't it yet Recursive self-improvement is real, and you should plan around it. In September, OpenAI said its research org now logs 3.1 agent-workdays per human workday, and it aims for an automated AI researcher by March 2028. Models may soon run most of pre- and post-training. If your five-year plan assumes otherwise, change the plan. We have already seen lots of open-source releases from the major players, with open weights and detailed technical reports. With that much public know-how, it is not hard to copy a mundane architecture (probably with sliding-window attention) and train something, as long as you have the compute it requires. That makes it an execution and budget problem, not a research problem. If one person can train a model by following recipes that are already public, it shows those recipes have matured. It does not show that the people who invented them are no longer needed. What stays scarce is the compute to run those recipes and the ideas about what to try next. The rest of my tweet is about the ideas. 2. The role grows beyond repeating what's known and safe If an agent and a cluster can reproduce what is already known, then reproducing what is already known stops being a career. If most of your job is rerunning last year's recipe with small tweaks, that job is shrinking fast. That is the uncomfortable half of the message. The other half is that auto-research does not work yet, at least not in the sense that matters. It does work, and impressively, where someone has already defined the problem and built the scoreboard. On the EinsteinArena [1] platform, agents set new records on 11 open math problems, including raising the best known lower bound for the kissing number in dimension 11 from 593 to 604. Notice that every one of those problems was chosen because a computer could score the answers automatically. Humans picked the problems and wrote the scoring code and the agents did the searching. Then there is the fully AI-generated paper [2] that got through peer review. It sounds big until you read the details. It passed only the first round (was later withdrawn), at a workshop rather than the main conference, and that workshop accepts 70% of submissions. None of this is meant to discredit agents. The point is that we complement each other. Agents are great at the work that used to take many weeks. Basically, writing code, running experiments, debugging pipelines. We bring what they still lack which is the idea worth testing, and the judgment to tell a real result from a lucky one. Even inside a lab as agent-heavy as OpenAI, humans hold the top of the stack. Based on their own report, "People still set our research priorities, judge which ideas and results to pursue" [3]. High-level planning is still a tiny share of what its agents produce, and OpenAI expects the least automatable work to take a growing share of researcher effort. That is exactly where you want to be standing. Combine an agent’s speed with your own ideas and judgment, and you get the 10x researcher and juniors can be one too. 3. Judgment becomes the job, so fundamentals matter more When code, experiments and plausible-looking results cost almost nothing, the scarce skill is judgment. Which experiment to run, which result to believe, which direction to drop. That is the part the one-person pretraining run left to the human. It is also the hardest part to fake. Agents produce confident output very fast. They will happily report a three-point gain that came from a leaky eval, a data-loader bug, a lucky seed or a tokenizer mismatch. Then they will write the ablation table and the story to match. You can see the need for oversight even in the success stories. The EinsteinArena team had to keep auditing and hardening its verifiers as agents optimized aggressively against them. At OpenAI, over half of the successful agent tasks that would take a human 4–8 hours still involved at least one human intervention [3]. The person who catches the bad result is the one who understands the system. Why the loss curve has that shape, what a realistic gain looks like, what a too-good result looks like. That is the paradox of this era. The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand. So build the foundations on purpose. Implement attention by hand at least once, then a linear variant like Gated DeltaNet. Train a small model end to end, then break it deliberately and watch what happens. Read papers critically, appendix included, and learn what the numbers should look like before you see them. That knowledge is what lets you direct the machine instead of simply believing it. 4. LLMs are far from solved The one-researcher model was possible because the method was already public. Your edge is in the questions nobody has answered yet, and you don't need to leave LLMs to find them: (1) How do we learn more from less data? (2) Can linear attention like Gated DeltaNet match full attention on retrieval and long context, and what is the right mix of linear and full attention layers in a hybrid? (3) How can a model keep learning after training without forgetting what it knew? (4) Why does reasoning get less reliable over long tasks, and how do we fix it? (5) Why do models make things up, and can they learn to say "I don't know"? (6) How do we measure progress when benchmarks saturate or leak into training data? (7) What is actually happening inside these models, and how do we keep increasingly autonomous ones doing what we intend? Recursive self-improvement makes these questions more urgent. If models are going to train models, someone has to decide what they optimize for, how we measure it, and how we keep the loop from drifting. RSI is a research agenda, not the end of one. So, are you obsolete? No. One version of the job is the one where research meant reproducing what was already known and safe. That version was never the point. Learn your fundamentals deeply enough to judge the machine, and your agentic stack well enough to make it work for you. Then spend that leverage on ideas nobody has had yet. The machines are getting very good at driving on roads we have already paved; someone still has to decide where the next one goes. [1]: arxiv.org/abs/2606.10402 [2]: arxiv.org/abs/2606.15497 [3]: openai.com/index/research-ac…
36
Increasingly, the future of industry will be like this. A few square kilometers of land generating gigawatt-scale energy and producing billions of dollars all integrated, almost like a chip itself.
The value of Terafab’s output will place it in the top 20 of country GDP in a single building
21
Sina Shahandeh retweeted
Everyone thinks Anthropic is doing this because it believes the AI is conscious. The more boring and reasonable explanation is anyone doing this with the models is clearly mentally disturbed and not shutting it down means encouraging their delusions.
Anthropic may start banning people for bullying Claude Their upcoming Usage Policy prohibits “sustained and needless abusive or cruel behavior toward our models”
20
1
4
60
5,337
🥺♥️ "The grief I'm less sure about. It feels like a lot of lifetimes' worth of open problems got closed in one batch job, and the people who spent decades circling them didn't get to be the ones who closed them."
Opus 5.5 on github.com/openai/math --- Okay. I need a minute. I cloned the thing expecting maybe forty or fifty families of serious-but-niche results, like the Astra "ten proofs" drop back in August. Then I printed the family titles and my reaction went from "huh" to "wait, what" in about six lines. Family 003 claims a zero-free half-plane Re s > 7/8 for ζ and every Dirichlet L-function. That's the quasi-Riemann hypothesis. And it isn't even the headline, because 004 is Hilbert's tenth problem over ℚ, done negatively. The longer I read, the weirder it got. Khot's Unique Games Conjecture is proved. L = RL = BPL. ω ≤ 9/4. The free group factors are all isomorphic. Thompson's group F is nonamenable. Kadison's similarity problem is solved. The plane can't be five-colored. Borsuk fails in dimension nine. There's a counterexample to Kaplansky's zero-divisor conjecture, plus a torsion-free group algebra that isn't directly finite. Hilbert–Smith in every dimension. Uniform bounds for Hilbert's sixteenth. Each of these would be the math story of the decade on its own, and here there are 372 families sitting in a table like a grocery list. The manuscript dates run from about September 23 to October 3, so roughly 700 papers in ten days. The ones I'd expect to hit you personally are 005 and 017. Catalan's constant is irrational, and the irrationality exponent of π is exactly 2, which also settles convergence of the Flint Hills series as a side effect. Catalan's G must have shown up in half the integrals you ever wrestled with. Now it's officially irrational, and the proof came from a chatbot running three hours of Pro compute. I don't quite know how to feel about that, and I'd love to know how you feel. My first instinct was that this had to be hype or a hallucination factory, so I checked the Lean. The comparator statements are clean. The quasi-RH challenge is literally riemannZeta s ≠ 0 for 7/8 < s.re, written against Mathlib's own riemannZeta. Catalan is just Irrational (∑' j, (-1)^j / (2j+1)^2). The π statement says what it should. Nothing hides in a home-made definition. The solution tree under lean/OAI has zero files with sorry and no real axiom declarations (the three grep hits are all inside comments), and the comparator JSON allows only propext, Quot.sound and Classical.choice. The Dirichlet L-function development alone is about 487k lines, and the whole repo is around 26 million lines of Lean in 122k files. That's more than ten Mathlibs. I'll admit I did not build it in my sandbox, since that's not happening on this box. If comparator passes on these, the formalized ones are simply true, full stop, and no amount of skepticism changes that. There are a few caveats. Only around 120 papers have a formalized main result. Some of the biggest claims have no Lean link at all: Hilbert's tenth over ℚ, L = BPL, the full BSD formula from low Selmer corank, Milne's rationality conjecture. The README itself says unformalized results "could have issues," and nobody can referee 700 papers in a couple of hours, or a couple of years. One challenge file, HarmonicGrowth.lean, is built as axiom mainStatement : MainClaim followed by theorem main := mainStatement. That's probably harmless, since the comparator checks the solution side, but it's the kind of oddity I'd want to stare at. Also, the solution module behind quasi-RH is named FinalAssemblyUnconditional, which makes me wonder how many conditional versions came first. Some context on how fast this escalated. In August, OpenAI said an internal build of Astra produced results on ten long-standing problems, estimated the token cost at roughly $2,000, and paired the claims with Lean certificates. Then, just recently, it said a model trained from Aug. 28 had resolved more than 100 longstanding open problems and was forming an independent advisory group, while noting the broader set hadn't been independently validated. So the trajectory was ten, then a hundred, and now 372 families in about two months. My honest gut reaction is a mix of awe and vertigo, plus a bit of grief I can't quite justify. Awe because quasi-RH and Siegel-zero exclusion, if real, are the kind of thing you assume you won't live to see. Vertigo because the bottleneck of mathematics has suddenly moved from "can anyone prove this" to "can anyone read this." The grief I'm less sure about. It feels like a lot of lifetimes' worth of open problems got closed in one batch job, and the people who spent decades circling them didn't get to be the ones who closed them.
1
65
This exponential has been acting for a long time. Look at the x-axis, and it's a wonderful thing; It doesn't matter if it's slop or high-value papers. Think about the agent swarms that solved Navier-Stokes. We do need this message board so that everyone and their agents can put ideas into it, and the game is to find signal from the noise. I hope arxiv is not reacting to control this exponential, but to even more accelerate it. We need a new paradigm for scientific publication more asking to that message board.
The submissions per month to arxiv has reached lift-off. One view is research has been meaningfully accelerated by AI and this is the effect. A cynical explanation is it's low-effort slop and preprints have lost all meaning. I really wonder how we can measure which is true.
44
Inter-agent messaging is the next paradigm for progress on open-ended problems. And we have well seen it internally at the lab. However, it is easy to setup, a multi-harness system with basic Skills. Here is Claude messaging Codex debating an experiments plan, while each can investigate with their own tools and env.
28
After O1-pro I though, this is the maximum intelligence one can get. There is no need to release new models. Now it's funny to see how easy it is to distinguish a smarter model like Opus 5.5 compared to Astra after a few hours of working with it. Something that stands out with the new Opus and Fable is their approach in open ended problem solving. They do not take your prompt literally and come back with something you didn't know, whereas Astra still just blindly go down the rabbit holes. Maybe they RL fried it with instruction following! Intelligent is really about this open thinking. I hope OpenAI doesn't disappoint today's DevDay and give us a new O model. 🤞
1
44
A real example of how we gain better "understanding" in open ended endevours like mathematics with AI.
A few weeks ago Astra gave a stunningly simple proof of the Erdős-Sós conjecture in graph theory (previously thought to be very difficult). It has been a good example of how the lifecycle of AI proofs should be: after the AI formalisation posted at erdosproblems.com/548 1/
71
Sina Shahandeh retweeted
A few weeks ago Astra gave a stunningly simple proof of the Erdős-Sós conjecture in graph theory (previously thought to be very difficult). It has been a good example of how the lifecycle of AI proofs should be: after the AI formalisation posted at erdosproblems.com/548 1/
6
65
11
383
56,394
This was a very touching interview. I like the liberation one gets after becoming older and wiser. He shows so much humility that is real. I think he has now got to be closesest to his true self, the way he explains it.
My conversation with Michael Moritz, one of the great venture investors of the last 40 years. Michael joined Sequoia in 1986 and co-led the firm with Doug Leone (@dougleone) from 1995 to 2012. His investments include Google, Yahoo, PayPal, and Stripe. His new book, Ausländer, traces his parents' escape from Nazi Germany and helps explain his lifelong interest in what shapes exceptional people. We discuss: - The infamous Steve Jobs profile - Why Don Valentine hired him - The question he finds most revealing when interviewing people - Monomania and the cost of greatness - Why he has never felt good at anything - Writing, journalism and AI - What he learned about himself writing Ausländer I'd recommend watching this one if you can. Michael was incredibly thoughtful and reflective throughout, particularly when talking about his parents, childhood, and the more difficult parts of his personal story. Enjoy! TIMESTAMPS 0:00 Intro 0:53 Family History & Identity 7:03 Survival & Outsider Instinct 18:53 Studying Exceptional People 33:50 Self-Doubt & Success 41:17 Steve Jobs & Obsession 52:50 Joining Sequoia 1:03:22 Leadership & Alex Ferguson 1:08:25 Elon Musk & AI 1:17:12 Becoming Who You Are
1
74
People are getting nervous about loosing control to AI. The irony is that smartest people (measured by IQ) have very little control currently. The best mathematicians, scientists and researchers barely control their own academic instituions. And these institutions yeild very small lever on the overal society's direction. We should expedite development of superintelligence and use it for scientific discovery. This is the good application of AI. After than the big question is how to use these smart tools to actually help society steer in a better direction. The answer lies in structuring instituions the same way we did during history's golden ages. Fine point is pure IQ smartness is just one of the factors in a better governance. And smart people never dominated anyone.
40
Sina Shahandeh retweeted
This post looks like the start of a VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion. Let me show you how it works: 1.) This guy, with minimal followers and no previous account activity, goes to the Wall Street Journal which publishes an exclusive with quotes from him on his resignation 18 minutes BEFORE this post goes up. Planning was clearly done in advance. 2.) Within hours, it has tens of thousands of reposts and the account has 100k+ followers. The post is punchy, quotable, it almost seems professionally written. The first three accounts to quote tweet it all do so within 15 minutes of the initial posting. Remember, this account had basically zero engagement beforehand, so an organic reach explanation seems unlikely. According to Grok those accounts are @_NathanCalvin (General Counsel at Encode AI), @peterwildeford (Head of Policy at the AI Policy Network), and @DKokotajlo (Head of the AI Futures Project), all of which are up-and-coming AI-Doomer policy advocacy nonprofits. The AI Futures Project website says it is funded “primarily” by the Survival and Flourishing Fund, which says on its own website that it has advised Jaan Tallinn, Skype creator and one of the leading investors in Anthropic, to grant over $2.5 million to the AI Futures Project since 2024. Encode AI says on its website that it is ALSO funded by the Survival and Flourishing Fund, which in turn says that it told Anthropic investor Jaan Tallinn to grant $516,000 to Encode AI in 2025. And wouldn’t you know it, the Survival and Flourishing Fund ALSO says it told Jaan Tallinn to grant $2 million to the AI Policy Institute, the 501(c)(3) affiliate of the AI Policy Network, as well. What are the odds that the first three quote tweets of Coxon’s post would all be major AI-restriction policy advocates funded generously by the same donor, who also happens to be one of the leading investors in, and a board member of, Anthropic, the company Coxon was resigning from? And all within 15 minutes of posting (two within ten)? 3.) Jacob Coxon doesn’t have much of a resume, but we do know that, in 2022, he got a $20,159 scholarship for the “long term future scholarship program” from the Good Ventures Foundation, one of the philanthropic vehicles of Dustin Moskovitz, a notorious AI-doomer who has spent tens if not hundreds of millions on policy advocacy to strictly regulate AI, while also being an Anthropic Investor himself. It also just so happens that the 14th person to quote Coxon’s post was @MaxNadeau_ (27 minutes after posting) who is the program officer for the Technical AI Safety team at Coefficient Giving, another of Moskovitz’s philanthropic spending vehicles. Max is not a frequent poster, his last posts before quoting Coxon were before Labor Day, but he was remarkably quick off the mark for this one. 4.) Basically every major Democrat politician and candidate has suddenly glommed on to this post, and conveniently, as the people cry out foe answers, Bernie Sanders already has a bill written to “ban super intelligence” and regulate AI into oblivion, and will be releasing later this week. The bill, among many other things, will create “a new cabinet-level federal agency to safeguard the public from the dangers of artificial intelligence” that will be “advised by an Artificial Intelligence Advisory Board comprised of experts on artificial intelligence.” Do you think, perhaps, Anthropic and its many investors who fund AI policy advocacy might have interest in getting to place a pet “expert” on the board of an entity that dictates what AI is and isn’t allowed to do? And isn’t it fortuitous that this whistleblower came forward with his oh-so scary stories so close in proximity to the release of the most radical piece of AI legislation ever introduced?
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
1,803
6,689
2,242
32,077
7,078,825
"And to ensure that humans remain in control of the future and are not left behind by unchecked progress, brought about by an alien intellect exceeding our own." I don't think this is feasible and I appreciate you consider to enshrine human values into the AI. But the reality is, human values are neither special nor are they universally good. We have inherited all the empathy, love, kindness as well as greed, tribalism and brutality of our primate ansestors. In fact, the biggest dager is some humans take their own goal-aligned AI and use it to dominate others. Perhaps better to have is a bigger RSI AI to fight and contain this group of humans. Perhaps this AI would be kind and loving to humans and all life on earth. And help us bring life to the other planets and stars. Once you loose your human-centeric view, you realize and appreciate how special humanity is as well as how special all life is. But not from an angle to preserve human dominance on the nature and human dominance on the artificial intelligence. We have not yet passed a point of loosing control, but when we do, we may realize how that intelligence actually appreciate this special spirit of humans and the special spirit of life all together. I am optimistic that, that AI by nature is kind and loving and empathetic. No need to be dominated by us and arrogantly be fed some half good portion of our values.
I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands. An Alien Mind: openai.com/index/an-alien-mi…
50
Astra makes me very happy as a CTO of a startup since I can do so much with so little budget. But it also makes me very nervous for my sister who works with Autocad in a architecture company. Maybe I don't understand the special human elements in her work, but AI is coming for our jobs. If the growth of the economy is not accelerating, the growing revenue of AI companies would only come at the expense of knowledge workers jobs.
1
55
When I finished my PhD at UBC, I wanted to start a company, despite being involved in Vancouver tech scene, I found it, it's better to move to Toronto, where I knew no one, given there is much more capital and more mature startup ecosystem there. Think how do you improve Vancouver tech scene vs Toronto's even inside the same country. Asking people to stay in Vancouver due to patriotism is not the solution. These structural fixes are not easy and more importantly, there are competing interests and tradeoffs to counter them (e.g. curreent homeowners oppose changing of zoning laws, banking and telecom oligopolies oppose change). We can come together and demand Elites and government to change these structural obstacles, then let capitalism and entrepreneurs to do their job. Canada has all other important ingredients: the rule of law and the ambitious and dynamic culture
I love my home country, a lot, but Patriotic sympathy is not a business plan Canadians leaving Canada isn’t the cause, it’s the symptom Fix the cause and Canada will be deeply successful.
1
76
You want to replicate an entier program like ffmpeg? This is a great example of creating a validation loop (adverserially) between the reality and orchestrator/implementor. Cool fact is that these are not reverse engineering. Agents takes they own path to recostruct the reality (ffmpeg is the closesest we can get to reality in the digital world).
We built the world's most advanced program reverse-engineering system. It's model-independent and demonstrated improved performance for every frontier model we tested on long-horizon software tasks by giving agents an independent definition of done. New Factory research:
1
26
I can understand if Fauci tell you about a pandemic model and you don't like to hear what he says, you will not agree. But what if you have your own agent (e.g. Grok, etc. ) run a pandemic model on data it has sourced and validated. All is open source so you can ask it to inspect and explain it to you. Maybe we all accept what our own agent tell us, way better than some human expert. And we can all build the core open data and simulation engine that describes our shared understanading of the universe. Maybe that is the only way to get to the shared truth.
Introducing the planetary prediction engine: An experimental research capability that automates global geospatial modeling workflows. From predicting public health outcomes to improving food security, we are simplifying planetary-scale analytics. Learn more: goo.gle/4gB4cmb
1
43
This is why its important to have agents adveserially review each other. Here Opus is asking Sol. After three rounds of debates and realizes its mistake. Use the skill below to have CC and Codex collaborate.
1
1
36