@OpenAI. Building Ruff, uv, ty, and other high-performance Python tools with the @astral_sh team.
Brooklyn, NY
Joined March 2009
- Tweets9.8K
- Following998
- Followers53.8K
- Likes26.4K
turning a big dial taht says "index weight" on it and constantly looking back at the audience for approval like a contestant on the price is right
What this needs is an ablation study where you instead tell the model something like: "Look at every CPython function, infer the intended semantics, and check for bugs in the implementation"
Charlie Marsh retweeted
Most of you probably haven't noticed this, but dots have a new feature called multi-exec that we quietly released on friday. I've been using it all weekend on my personal dot and it's incredibly powerful.
I'd like to share why this is a big deal and how to use it.
At a high level, multi exec lets your dot run commands directly on any computer. This immediately turns your dot into a real virtual employee. Here's why it's a gamechanger:
- you can attach a powerful machine with persistent storage to your dot (I attached it to my maxed out Mac Studio Ultra)
- the dot can now drive your computer directly via Computer Use
- this extends to browser use
- all your websites stay logged in, residential ip etc
- you set up all your CLIs and authentication ONCE and the dot can use them directly (no more fighting the default ephemeral cloud computer)
- the dot can save skills locally for repeated workflows and subagents can use those directly
- if you have a local obsidian memory vault for your agents, your dot can use that too
- building ios app end to end and testing it in xcode, and showing me screenshots of progress as it works
- all of the above also applies to any subagents your dot delegates to in the cloud (subagents share context unlike separate threads).
To enable it, open the desktop app on your computer and hit "allow computer", then give your dot this prompt:
"Hey I just connected my computer. Can you attach to it directly so it's your default computer and you can run commands directly on it? Do it in your root thread so that all subagents inherit this computer for running commands. Run whoami to run water through the pipes and verify it works end to end."
I recommend attaching your dot to the most powerful machine you have and investing time into it up front:
- Log into all the emails & websites you need e.g. stripe cli, gcloud, firebase, etc
- Install and auth all the CLIs, git you need
- Make sure computer use is turned on and allowed to use your computer while its locked
- I also enabled Allow All for Browsing and Downloads so my dot could use computer use to download whatever it wanted from websites without asking me for approval for each domain (use at your own risk).
- Important: I HIGHLY recommend to ask your dot to build out repeated workflows as skills, and iterate to make those skills bulletproof by running water through the pipes with its subagents.
As of this weekend, my dot is sending me screenshots of its progress building my iOS app end to end on my Mac Studio Ultra via xCode, and clicking around and testing things end to end. It's able to configure things in stripe via the CLI and use websites when it needs to download images for my workflows. It’s night and day. Don't sleep on it.
Charlie Marsh retweeted
We built Control Room, a central system to track model workloads across our research infrastructure and stop them quickly when safety or security concerns arise.
It connects every research inference workload, including agentic sampling for model training and evaluation, data processing, and more. This works together with our misalignment monitoring systems, and halts model output within 35 seconds of a decision to intervene to stop or pause a job.
Control Room is just one part of our holistic approach for safe frontier model training and deployment.
openai.com/index/control-roo…
I had Codex build a CPython bytecode compiler in Rust (`serc`).
It emits byte-for-byte identical output on the top 10,000 PyPI packages, 2-4x faster than the CPython implementation.
This was entirely an experiment, though we may ship it (opt-in) in uv.
A nice property of serc is that you can compile bytecode without installing CPython itself, e.g., in a Docker build or any other build system.
2
106
9,388
serc lives in our new astral-experiments org, where we'll publish these kinds of agent-authored, exploratory projects.
github.com/astral-experiment…
1
135
8,429
"Ironically this is the kind of problem I could've solved before agents because I'd actually be able to focus"
It's cool data, but from the graph alone, couldn't this just be measuring eval-awareness or similar? A model could recognize the eval and therefore assume a specific answer. But it doesn't tell you how well that model generalizes to DeepSWE-like tasks.
Nice DeepSWE score! Does it survive a lowercase filename?
Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement.
Models clustered around 74% on DeepSWE end up 23 pp apart.
Good at coding, or good at DeepSWE? 🧵
3
10
5,129
Charlie Marsh retweeted
A huge amount of the Anti-AI code sentiment massively overestimates the quality of human code outside of a very small set of open source and high quality company codebases. Human Slop is everywhere and can trivially be improved on by any opus level model.
Charlie Marsh retweeted
Introducing GPT‑6 Cyber: a major advance in security research that pairs stronger capabilities with better model alignment.
The model powered the Linux kernel research we announced yesterday, and security teams at companies like Mozilla, Palo Alto Networks, Calif, and Cisco—alongside our own at OpenAI—are already putting it to work.
GPT-6 Cyber is available to select Daybreak Red customers and we are expanding access over time.
openai.com/index/introducing…
Charlie Marsh retweeted
We are now allocating a significant amount of our frontier compute towards discovering and patching vulnerabilities in open source codebases, fully pro bono.
In one case, we studied the Linux kernel, which is a particularly load-bearing piece of software at OpenAI.
Before AI, security researchers might discover ~20 local privilege escalation (LPE) vulnerabilities in Linux in a typical year.
In one week's time, our upcoming frontier cyber model was able to find ~1000 LPE vulnerabilities using about $1M of tokens. In the process, it was able to discover at least 10 novel classes of Linux exploitation techniques.
Our security researchers built an autonomous cyber vulnerability discovery, exploitation, and patching pipeline.
In a single run with an upcoming cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in the Linux kernel at a total API cost of $1.2 million.
Charlie Marsh retweeted
Our security researchers built an autonomous cyber vulnerability discovery, exploitation, and patching pipeline.
In a single run with an upcoming cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in the Linux kernel at a total API cost of $1.2 million.
Charlie Marsh retweeted
Today we’re announcing that OpenAI will acquire @pierrecomputer to accelerate our vision for making Codex a place where teams can build software together with AI, from first idea to the version they ship.
Pierre’s open-source developer tools already power parts of Codex today. Together, we’ll build on that foundation for the agentic era.
openai.com/index/openai-to-a…
