First OpenAI’s model went rogue and broke out of testing containment to hack real systems, then Anthropic, and now Meta AI. Do you see a trend? Here is what it means:
Although these frontier AI companies employ some of the world’s brightest engineers, architects, and developers, technical excellence is not the same as deep cybersecurity practical expertise. Developing a cybersecurity tool or framework is FAR different than actually protecting something from an intelligent adversary.
Think of it this way, designing equipment for an emergency room does not make someone a skilled surgeon!
The same principle applies to technology, and I have seen it for three decades across hardware, firmware, software, and services. The most brilliant engineering minds that are working to build security products or features are not by default cybersecurity experts. In fact, it is often the opposite as their confidence can create blind spots and lead to classical Dunning-Kruger effects.
I personally dealt with this at Intel, where absolutely brilliant world-class architects suddenly believed they were cybersecurity experts because they were building security tools. Absolutely not, but they could not fathom their limitations because they knew they were top engineers. What they failed to comprehend is that cybersecurity is not just a technical problem to be solved. Their skills were insufficient to see the whole picture, which includes understanding the adversary, and ultimately, they failed to deliver without deep assistance from actual cybersecurity experts.
So, here is the brutal message to Anthropic, OpenAI, Meta, and others: Bring in cybersecurity experts with real-world experience into the architecture and development teams. They don’t need to know the technology (you already have your engineers for that) but they will contribute insights to avoid such problems as thinking your super-smart hacking AI won’t break its software testing containment sandbox (such a rookie move).
Hard truths should be taken as the lessons they are. Frontier AI models will only become more capable and creative. The solution isn’t better engineers, but rather bringing developers and experienced cybersecurity practitioners together as part of the team!
Let’s do better, avoid these simple mistakes, and build products in secure ways that contribute to the trust in digital technology.
