Open infrastructure for building cloud software factories
Joined June 2021
- Tweets3.3K
- Following2
- Followers59.6K
- Likes4K
Pinned Tweet
Introducing Scorers: Agents that grade your agents.
Use LLM-as-a-judge to grade past coding agent sessions on:
• Quality
• Efficiency
• Compliance
• Or any custom dimension
Scores feed into performance measurements and automatic self-improvement for software factories
Warp retweeted
I just wrapped up a keynote at @aiDotEngineer Code Summit on how Software Engineering is becoming Factory Engineering.
The core idea is this: using coding agents is an engineering problem, not a prompting problem. Every company should strive to define the system in code, measure what it produces on your codebase, and keep iterating.
Everyone has agents. The hard (and fun) part is engineering the factory around them so they reliably ship better software.
ALT Zach presents at the Code Summit, with a large audience and slides discussing software factory concepts and engineering innovations.
ALT Zach presents "The Factories Stack" diagram at the Code Summit, highlighting various stages of engineering and data management.
ALT Zach presents at the AI Engineer Code Summit, discussing software engineering’s evolution with a focus on coding agents.
ALT Zach presents at the Code Summit, showcasing a blueprint interface for engineering workflows on a large screen.
Warp retweeted
I recorded a session on addressing code review bottlenecks! Especially relevant if you're building a software factory. We cover:
- Building your own code review agent
- Verification with e2e computer use checks
- Risk assessment using /merge-risk
youtu.be/uPCH2axUpbc
Warp retweeted
I started having agents run a /merge-risk assessment skill as a last-mile step to catch risky changes before it makes a PR:
- Strip out unaligned changes to avoid drive-by fixes
- Add error monitoring on new paths for better tracking
- Break up high-impact changes into stacked PRs for simpler review
- Avoid code sensitive paths if it can achieve the same outcome
- And of course, verify work end-to-end with screenshots and recordings
Warp retweeted
I was talking to an engineer the other day about this question:
How do you make a codebase easy-to-navigate by both agents and humans, when everything from the hello world is agent-built?
The answer is system architecture. Agents (like humans) copy bad patterns like a virus. It's a ball of clay that needs constant reshaping.
You can write down best practices into your factory's skills, yes, but keeping the codebase itself on the right path is the real goal.
It's too expensive to have an agent review the universe constantly, spitting out architecture change PRs no one wants to look at. But, say, a weekly automation with a frontier reasoning model (Fable 5.2 is a good one to try) that pulls out one improvement to your system architecture is a start. I like @steveruizok's prompt to "look for implicit state machines."
Will experiment with a factory automation like this and see how it goes.
Warp retweeted
Get code review agents off of GitHub! I don't want to see the noise in my PR description.
Agents should run through their own internal code review process before a PR ever reaches a teammate.
You can bring your ChatGPT subscription to the Warp terminal, now with support for business plans too!
Now, you can use your ChatGPT Business plan in tools like @NotionHQ, @v0, @ampcode, @kilocode, @warpdotdev, Hermes from @NousResearch, @openclaw, @opencode, and more.
Connect with Sign in with ChatGPT to get started.
Warp retweeted
Lots of discussion about what a "software factory" is.
Also useful to call out what a software factory is not:
1/ A loose collection of automations around cloud agents
2/ A team of "AI Engineers" that developers manage interactively
3/ A set of skills and MCPs that individual engineers all adopt across a team
4/ A personal agent swarm
These are all useful things, but they aren't the right primitives for engineering scalable agent systems across an organization.
The right primitives allow you to define all your agentic workflows in code, run them entirely in the cloud, and, crucially, measure, test and improve those workflows over time to maximize agent productivity while minimizing cost.
If you start thinking from first principles "what is the system we should build as a company to get the most out of coding agents" you will end up with something like this.
Warp retweeted
I'll be live in just a moment to talk code review bottlenecks! We'll break down:
- Code review loops and how to benchmark their usefulness
- End-to-end verification skills
- A /merge-risk skill to mitigate risk ahead of PR
RSVP to join: warp.dev/events/solving-the-…
Great resource for those building their software factory:
My @aiDotEngineer NYC talk is now on YouTube. I explain what a software factory really is and how teams should think about building them:
youtube.com/watch?v=nKjPBAw_…
Fable 5.2 is a great systems thinker.
Try using it to find implicit state machines in your apps to make your codebase easier for agents to navigate.
Live in Warp Factories + the Warp terminal
If you have 100 agent PRs waiting for your review, this should help you get through them:
We built a skill for addressing the code review bottleneck: /merge-risk
This assesses the risk of any PR across 5 criteria: alignment, impact, verification, rollback cost, and error monitoring.
1. Alignment: Does the code follow the issue and PR description, or are there drive-by fixes and unintended side effects?
2. Impact: What is the blast radius of the change, and is it touching sensitive code paths like auth, permissions, or billing?
3. Verification: Is the code well verified, both with test coverage and end-to-end computer use artifacts?
4. Rollback cost: How easily can this be rolled back, and what is the rollback procedure in the case of an incident? Think one-way vs. two-way doors.
5. Error detection: If something goes wrong in production, how much visibility will we have? Are errors swallowed, logged, or reported to an error monitoring service?
You can run it as a PR comment automation, or make it a pre-PR check for the agent to derisk its own work before tagging in a human.
Warp retweeted
Inspired by @poteto I benchmarked Warp's factory to see if adversarial code review was worth it.
On the task set I checked it increased cost by 140% while moving quality up by 7.5%. So, probably not, although maybe for really impt changes (many caveats here, you should watch the video if curious).
I still believe though that benchmarking is 100% the right approach and everyone should try to set up a factory where they can measure rather than go on vibes.
Warp retweeted
It's worth building your own verification tools for agents: code review loops, e2e simulations, and merge risk checks.
I'll be hosting a live session next week to build these together!
Tue, Nov 10th at 2pm ET
RSVP: luma.com/warp-daxp
If you're buried under 1000 agent code review requests, read this:
Warp retweeted
Your team should have a shared configuration for your agents that lives in code. The same way you use Terraform to deploy cloud infrastructure, you use a factory configuration to deploy your software factory.
This gives you the typical advantages of version controlled systems: check in changes with code owners, test changes with rollback support, and review updates as diffs.