I recently sat down with Tony Spinelli for a fireside chat on AI, accountability, and what still needs a human in the loop. But one audience question stuck with me, because almost everyone has thought it at some point: why can’t I just build an autonomous pentesting tool myself?
It’s a fair instinct. Our whole industry runs on pulling things apart to see how they tick, so when capable models put real capability within reach, the thought shows up on its own: why pay for a human-based pentest when you could replicate that cleverness yourself with AI?
If anyone has earned the right to answer that, it’s Tony. He’s a four-time CISO, including at Capital One, where he led the move to the cloud starting in 2015, when most financial services organizations still thought that was reckless. He holds patents in data loss prevention and cloud security, he’s a sitting CSO today, and he serves on the board of a private bank leaning hard into AI. He hasn’t just theorized about this kind of risk. He has been the person who had to answer for it, at the highest level, more than once.
Here’s a snippet of our conversation on this topic. It’s worth a watch, but I want to pull it apart below.
The Pull to Brew It Yourself
Standing up your own AI pentesting tool is the backyard distillery of security. The build is the fun part, a little bit illicit, and that’s exactly why it pulls at you. Capable models are sitting right there, the ingredients on the shelf, and the whole job looks like something you could rig up over a few weekends. It’s cheap, it’s yours, no vendor contract or procurement cycle, and it’s a genuinely fun problem for a security practitioner. You run the first batch, findings come out, and it looks like it’s working.
But a still in the shed is a great project right up until it isn’t. The trouble was never in the setup. It’s in what you can’t see. Which is the real question the audience member was asking: where’s the line between a smart force multiplier and a homegrown thing that becomes an unmanaged risk nobody’s accountable for?
Easy to distill is not the same as safe to serve.
Who Answers When the Batch Goes Bad
You can’t automate risk away, and building it yourself means you own every bit of that risk.
This is where Tony was direct: build your own AI tool to go attack your own company, and the moment anything breaks, you’re the one holding it. He ran through the list of who comes knocking: regulators, overseers, boards, risk committees. Doing it internally doesn’t contain that risk. It concentrates it.
“I want risk transfer... And you can’t get that if you’re doing it internally. You’re almost increasing your risk.”
There’s a strategic problem underneath, too. Most companies don’t win on their security capabilities. They win on their product. A company making an insulin delivery device needs a strong security program, but its durable value is insulin delivery. That team needs to be expert in treating diabetes, not offensive security testing, so building your own tooling means investing deep effort in something that will never be the value you deliver.
Tony put the board’s view of it plainly:
“If we go to the board and we say, well, look, we’ve built our own tool, and we think we’re great, that’s not a great thing to put in a presentation.”
What oversight actually wants is the opposite: external review, the old maker-checker model, someone independent who can stand behind the result.
Free to Build, Expensive to Keep
A homegrown tool runs on the data you train it on, and most internal builds (and plenty of newly minted unicorns) lean on open source data. Tony’s take was simple: you’re far better off relying on companies that have been running these tests with humans for many years. Rolling your own is skipping past the know-how that makes the results worth anything.
Security teams already know this trade. You can run Wazuh instead of paying for a SIEM, but now you’re the one minding it day and night, standing it up, patching it, keeping it current. The license is free; the operating cost is not (the mash is free; the labor never is). And you’re still left with the question nobody in the shed can answer: how do you know it’s any good? Are you finding all the bad things? Is it still working the way it did last month?
None of this means the experiment has no place. Build it side-of-desk, add it to the red team toolkit, run a weekend batch. Just don’t make it the thing you serve the whole enterprise.
Experience You Can’t Distill Overnight
The smart move isn’t to build the still. It’s to buy from someone who’s been distilling for years.
A tool is only as good as the human pentesting experience behind it, and that takes years to build, not a few weekends. It’s the one thing a capable model can’t hand you on day one. That’s why Cobalt approaches autonomous pentesting the way it does: 13 years of running pentests, a proven methodology, now moving to continuous, with a human staying engaged and directing the testing. That’s the kind of result Tony said he’d actually trust.
That same curiosity is worth more pointed at the problem attackers are actually exploiting: the speed from exposure to compromise, and how fast you can close that window.
Tony and I covered a lot more than the build-versus-buy question: the parallels between the move to cloud and the move to AI, where human judgment still beats automation, how to report continuous testing to a board used to a once-a-year report, and the questions worth asking any vendor selling “autonomous” anything.