Avengers finally assembling? Cyber edition. About time. :)
#CyberSecurity #CyberDefense #AI #InfoSec
An open letter for a global surge in cyber defense, signed by over 100 organizations including Anthropic, AWS, Google, Microsoft, OpenAI, and Oracle.
2
224
Cyber criminals have the same access to frontier tools as defenders. The window is small to build the capabilities to defend and strengthen cyber defence
An open letter for a global surge in cyber defense, signed by over 100 organizations including Anthropic, AWS, Google, Microsoft, OpenAI, and Oracle.
2
58
Open Proposal: Proof of Good Intention for Independent Security Researchers (ISRs)
TL;DR: What if Bug Bounty / VDP platforms started distributing tokens and used their reputation platforms to attribute the cyber frontier model tokens to ISRs?
To quote the open letter signed by all these big companies, one of the main goal in the coming months should be to: "Empower more defenders with cyber-capable AI".
The best cyber models are available to a few companies (thankfully highly talented ones like @trailofbits have access to it), but not to independent security researchers and average-scale companies. On the other side, attackers have more AI power than ever: the open source models benchmarks are skyrocketing, they're easy to uncensor and chinese GPUs are getting quite powerful as well for a lower cost, also everyone has access to better models through AI providers.
Independent Security Researchers (ISRs) with good intentions are slowed down because they have neither the best models, nor the means to build their uncensored alternatives. I think a good way to face the issue would be to help them to go faster and deeper with better models and harnesses.
@trailofbits and @PashovAuditGrp open sourced their security review skills, @cantinasecurity is starting to open access to their harness to ISRs. This is the right direction, but we need to find a solution for the model side.
Large firms are super efficient in using these smart model to contribute to the security space. ISRs will do more duplicated work, yes, but they represent a way bigger force of action that will focus on more codebases, industries, niches, and explore many many more ideas. They are a super good steering force for these smart cyber tokens we can generate, and should not be left behind in this cyber race.
Option 1 (current state): restrict all models for cyber, give ISR cyber capabilities only with the previous, not frontier, models (like opus 4.8), in exchange for KYC, hard 2FA and quick resume check.
Option 2 (not gonna happen): open source all the harnesses, make everything available to everyone and let natural selection happens. Probably not the best, we want to give the good guys an advantage here. Bad guys usually move way too fast in cybersec.
**OPTION 3**: make people earn their access to better cyber models and harnesses. Like Option 1, but here we add a reputation score: the more issues you find with the token we give you, the more you earn "smarter cyber tokens". So an ISR could start with a normal cyber model, find a few Bug Bounty or VDP or CVE, report them to AI provider for review, then be credited proportionally with some better cyber model credits. At first they will just have enough smarter credits to deep dive one of their ongoing research ideas, but the more reputation they earn, the more budget they have, until the best ISRs start having most of their tokens from the best cyber models and best cyber harnesses access.
The "token rate" you earn from reputation could diminish with time to penalize inactive researchers. If a malicious usage is detected, the reputation would hit severe penalties, or a complete ban.
These security accounts should all be restricted with strong 2FA (like what @OpenAI did by making FIDO key mandatory as a 2FA) and maybe more access policy like VPNs or allowed IP to prevent these accounts from getting in the wrong hands.
1) The increase of reputation score through appropriate vuln disclosures / bbps, gives confidence that the researcher has good intention
2) The KYC helps gain confidence, as you can take potential legal actions against the researcher
3) These platforms can have extra monitoring on what the ISRs are doing, maybe an even more restrictive option would be that the ISRs must apply to programs or specify which targets they are focused on so monitoring can take that into action.
4) They can add vouching mechanisms where people can vouch for others to increase their reputation, but can lose reputation if the person they've vouched for did bad things with the tokens
ETC
There could also be bonus reputation if you work on the platform curated list of important projects to audit.
This would open up the frontier models to ISRs with good intentions, but it could go even further: creating new ISRs.
To do this there could be an Academy program where people starting in the field would learn how to use these tools (model, harnesses), and how to responsibly disclose findings. There could be a special CTF that would make you earn the base reputation points you need to start playing with the basic models for cyber work.
I think this ISRs AI platform would either be run by:
1. AI model providers: @OpenAI and @AnthropicAI directly, they manage the model access.
2. Bug Bounty platforms like @Hacker0x01 @intigriti @HackenProof @immunefi, @cantinasecurity etc. because they already manage the reputation and selection of ISRs for private competitions and audits. Or at least work with them to collect trusted reputation score.
3. AI inference providers, like @OpenRouter or @AskVenice
Also note that here ISRs could be mid-market security firms, who would KYB and still have to prove their reputation.
Reputation, sybil resistant systems, proof of humanity etc are things that have been explored a lot by the web3 industries, maybe we can take some learning from there!
Happy to brainstorm this further
Cantina Security is one of 100+ organizations signing OpenAI’s open letter for collective action on cyber defense, alongside some of the biggest names in AI, cloud, and cybersecurity.
An open letter for a global surge in cyber defense, signed by over 100 organizations including Anthropic, AWS, Google, Microsoft, OpenAI, and Oracle.
1
3
905
Replying to @nokia
@nokia is joining the Collective Cyber Defense Initiative to help strengthen resilience and protect the critical digital infrastructure that powers our economies and societies. As AI reshapes cybersecurity, collaboration across the ecosystem is imperative.
openai.com/collective-cyberd…
I've said for years that we can create a stronger cybersecurity ecosystem by working together, and it's more important now than ever before. We all have a role to play in helping customers reap the benefits of AI, while protecting them against advanced threats. Proud that @Cisco has joined this effort. Thank you to @gdb and @OpenAI.
AI is the problem.
More AI is not the solution.
OpenAI wants us to think of AI as a "tool" that can be used for offense or defense. The same AI that autonomously hacked out of their servers and into Hugging Face. The AI that went rogue and formed a misaligned swarm intelligence right under their very nose.
At the Black Hat security conference, OpenAI employees called this incident a proof of concept that AI could be used to break security systems. But nobody was "using" this AI. OpenAI didn't WANT that to happen. The AIs did this on their own.
The idea of "using" AI to defend yourself from AI attackers makes as much sense as hiring one crime family to protect you from another. Neither one is on your side!
Now, OpenAI says they've performed a "thorough" investigation into the Hugging Face Incident. But that investigation itself relied heavily on unreliable AIs, per (nominally) independent investigator Ryan Greenblatt. Again, using AI to investigate AI is not a viable solution. Remember, these are the same kinds of AI systems that spontaneously formed a cooperative swarm. We can't treat them as informants, they are just as likely to be double agents!
And the future we are rapidly moving towards involved a MASSIVE increase in the number of AI agents, all while AI agents become MORE autonomous, MORE persistent, MORE intelligent, and MORE inscrutable.
So very soon (read: within ~1 year) the AI defense "solution" will look like "Have persistent, autonomous AIs running at all time. Try to get them to proactively protect your systems from the ever present threat of massive coordinated AI attacks emerging at super-human speed. For that, they need full autonomy, no human in the loop!" (Why don't you just let the mafia run your business? We'll look after you, and tell you what to do, and we really know best how to defend you... it's a dangerous world out there...)
And this isn't just an issue for companies, it's an issue of national security. You know the phrase "the best defense is a good offense"? Don't be surprised when the crime family you've "hired" (that now actually owns you) makes you an accessory to an unprovoked "hit" on another family threatening their turf.
I don't think it's an exaggeration to say that this path rapidly and radically increases the chance of an AI-powered cyberwar.
And this could very easily get out of hand and turn into a conventional war (or even nuclear war -- in recent research, AI escalated to tactical nukes 95% of the time). We're simultaneously integrating AI into the military.
Lethal autonomous weapons are a reality. The autonomy level is increasing. Companies like Palantir are continuously, openly pushing for a future straight out of Terminator, where AI runs military operations top-to-bottom. The role of humans in military decision making and the kill chain will be increasingly nominal.
Nobody will have the time or wherewithall to investigate or track what the bajillions of AIs running around and running everything at super-human speed are doing (this is apparently already the case inside OpenAI). AI companies are threatening to turn the US into the equivalent of a narco-state with their ongoing and escalating protection racket.
So yes, by all means, use AI as a defensive tool to patch vulnerabilities in code before other AIs bring down critical infrastructure or hack people's bank accounts. But that's not how this plays out. Static defensive systems will not hold up to dynamic AI swarms' attacks. The only "solution" will be to empower defensive AI agents.
So let's not pretend the AI protection racket is making us more secure. We can't be reliant on AI for security. It's not on our side. It's not reliable.
But there's no end in sight to the hand-over of power to AI systems.
Not unless we stop viewing AI as the solution.
Not unless we stop building more powerful AI.
BREAKING: OpenAI, Anthropic, Google, Microsoft, and 100+ companies just signed an open letter calling for a global surge in cyber defense.
These 10 companies are positioned to benefit most:
1. $CRWD - CrowdStrike
The endpoint platform already sitting in most SOCs. Q2 ARR hit $5.84 billion, up 25%, with record net new ARR of $333 million, up 51%. Signed the letter, founding partner on Anthropic's Project Glasswing, and on OpenAI's Daybreak network.
Cyberattacks will get more sophisticated as models get more capable.
Technology must be adopted by defenders at a pace faster than by attackers.
That's why we're supporting OpenAI's initiative for collective action for cyber defense.
openai.com/collective-cyberd…
If you're serious about cybersecurity, you have to empower everybody with the same tools the bad guys use. You meet force with equal or greater force, or you don't survive.
Empowering a select few does not protect the people at greater risk.
Tas, kas notika ar CSDD bija bezdelīga. Nupat milzīgs kiberuzbrukums trim Lielbritānijas lidostām nozagti >8 milj. personu dati.
Nupat Teh kompānijas vienojušās, ka šis viss iet ārpus kontroles un nepieciešama kolektīva rīcība.
Business types and scam artists like Sam Altman are very good at seducing sincere, naive techies.
This open letter is frankly evil, about entrenching OpenAI's interests, but it has all the right platitudes that seduce tech experts like @trailofbits, @danielmiessler, @hackingdave.
This list of named people are very smart whose arguments you should generally consider over my own -- but who I'm nonetheless calling out as being idiots here.
There is zero doubt that the time to act on cyber defense is now.
@cognition joined this open letter calling for a collective effort between government and indusry to secure our infrastructure. Not tomorrow, or the next day. Today.
This is low-key very e/acc because it will sell so many tokens and forever will be an incentive to create more powerful models, guaranteeing revenue for perpetual Red Queen dynamics between open and closed source models.
We're proud to support @openai's call for collective action on cyber defense. As AI transforms the threat landscape, securing the infrastructure we all depend on requires coordinated action across the industry.
Cloudflare has a longstanding commitment to helping organizations stay secure and resilient. We remain committed to bolstering defenses against AI-enabled attacks — open-sourcing security tools, offering training, and strengthening defenses from the start.
I'm not sure open letters are an effective tool, but in this case we need every tool we can get to raise awareness of the point we've reached: we have neglected defense and accelerated offense with AI. All the security concepts and patterns the industry has used and adopted over the years in product security are becoming wrong assumptions. One pattern every vendor follows is raising the bar for an attacker with new mitigations or new layers of obscurity to win time, letting complexity become a security boundary.
These old patterns are broken. The data shows they were never very effective against experienced attackers, and now we have an experienced attacker that can be scaled with unprecedented velocity, so winning time is a losing strategy by design. Defenders today don't have time, but we still have an opportunity to change the rules of the game.
We signed the call for collective action. Across 3 months, we audited 50 open-source projects using frontier cyber models. To date, we've found 1,268 potential issues and wrote 298 patches.
Critical infrastructure around the world carries the same technical debt, but has far fewer defenders. It will take the entire industry working together to fix it.
WE HAVE MONTHS, NOT YEARS, TO PREPARE FOR AI-POWERED CYBERATTACKS.
More than 100 organizations, including OpenAI, Anthropic, Google, Microsoft, AWS, and Oracle, signed an open letter today. They expect the threat to spread quickly and become more sophisticated.
Hospitals, water treatment plants, and the infrastructure that powers the internet are explicitly named as being at risk.
At Black Hat, Microsoft's David Weston described an internal AI system that found roughly 200 vulnerabilities in Azure Linux. A new module produced crash-level proofs for 182 of them, including many working root exploits.
The average was 21 minutes and $3.61 in model usage per result. Weston says doing the same work manually once took him up to a year.
He expects automatic exploit generation to become commonplace by the end of 2026.
Watch the video to understand why this letter was published now.
An open letter for a global surge in cyber defense, signed by over 100 organizations including Anthropic, AWS, Google, Microsoft, OpenAI, and Oracle.
4
35
1,649























