How to stop conflating cybersecurity with AI security
My wife and I honeymooned in the Balkans, one of my favorite regions in Europe. We bounced around Croatia, Bosnia, and Montenegro seeing sites and experiencing some wonderful food from a mix of cultures. One thing we saw a lot of: walled cities.
At first, walled cities like Dubrovnik were a novelty compared to cities we see in the US. Inside the walls, there were narrow streets designed for single riders on horseback and storefronts that had probably been there longer than America has been a country. Dubrovnik’s sloped walls fell right into the stunningly blue Adriatic Sea. From one part of the wall, one could jump directly into the sea, which I absolutely did.
The history of walled cities and castles is fascinating but also good metaphor for how we think in cybersecurity. Walls around cities is a good analogy for traditional cybersecurity measures, but that analogy fails against the multifaceted risks that need to be mitigated in AI security. AI safety and security is about testing and securing what’s already inside the walls, which is a more difficult concept. If traditional cybersecurity represents the walls protecting the valuable data and systems we rely on, what is protecting the users of AI systems from failures inside the AI? This is equivalent to walls protecting from invaders but not helping protect people from those already inside. That takes a different technological approach.
Modern Walls
There was a time when walls represented existential threats to societies. If the walls were so large and so thick that an invading army could not inflict damage upon them, there would be a fundamental shift in the balance of power. For instance, the Theodosian Walls around Constantinople were originally built in 417 AD and protected the city for over 1,000 years until they were breached by the Ottoman Turks in 1453.
Theodosian Walls of Constantinople
But here’s the thing about walls: You will always have to build them higher and thicker because technology progresses.
When Emperor Theodosius II began construction of the walls in the 5th century, there was no such thing as a cannon, much less one like Basillica that would bring so much damage upon the walls.
Today, we face the same challenge with cybersecurity. We continue to build “higher and thicker walls” around our digital keeps in hopes that malicious cyber actors will not be able to scale them. But invaders have always found new siege engines. Twas ever thus.
Back in Constantinople, the walls repelled attacks for over 1,000 years keeping the malicious actors out of the safety of the city. But the walls did nothing at all to protect an individual city dweller from things already inside the walls. If a crop had a blight or livestock had a disease, the walls did exactly nothing and people got hurt.
Cybersecurity and learning to fly
Today, people often conflate cybersecurity with AI security. Cybersecurity as a field has existed for decades and we’ve professionalized the field with advanced degrees and billion-dollar companies. Cybersecurity is a comfortable place, and we are happy in places we understand, like sitting behind a wall
Walls worked just fine for keeping out armies on foot, but then they invented catapults. Wall builders responded by building walls higher and thicker. As offensive innovations continued, like the trebuchet, wall technology advanced as well in terms of strength but also in terms of things like arrow slits and moats. This is what cybersecurity has been for the last two decades at least. But we no longer protect our cities with walls. Why? Because now we can fly.
The introduction of AI into online systems and SaaS is the equivalent of the opening of a new domain in defense. AI safety and security is not about building walls to keep people (or other AI) out. It is about figuring out how AI fails and testing for those failures in novel ways. Yes, we need to protect the system from outside interference, but AI demands we understand the potential problems that are already inside the walls, like how AI acts under stress or when presented with a socio-linguistic variant. Those things are happening inside the walls of systems all over the world right now and we don’t have a defense against it.
AI Inside the Walls
When the very first computers were networked together, it became clear that security threats were real. As the technology scaled, the threat landscape scaled with it and in response, we built an entire academic and professional discipline that focused only on the security of virtual systems. This means that there has been a rough parallel between integration of cyber technologies into our lives and the development of associated security. No such parallel exists for AI integration.
The integration of cyber technologies into our personal and economic lives has been more gradual and taken place over several decades. The integration of AI is taking place right now, but it on an undisputedly shorter timeline. As people use generative AI and advanced analytics to drive everything from business decisions to protecting our national security, the innovations around how to make AI safer and more accurate are not running in parallel. This gap is akin to building higher and thicker walls around our city but neglecting anything that’s happening inside.
A “vulnerability” in an AI system is not defined the same way as a vulnerability in a mobile application or a network. An AI vulnerability is about how humans interact with the model in question and how the model will perform under different circumstances.
Ordinary AI Users
In many cases, it is not necessary to approach an AI model with malicious intent to get the model to perform outside of its intended purposes. That means that no walls are necessary because there’s no attacker. A simple case of a bilingual person typing a prompt that includes both languages might be enough to get a harmful result. There are cases of people engaging in self-harm after interactions with AI models and some believe it is because the context window was too long, and the model’s guardrails were not designed to work in such long conversations.
This is not the work of a genius, state-sponsored hacker. It’s a fundamental issue with how AI is built. Its non-deterministic nature means that it responds differently under different conditions. We’ve seen this issue after the fact. We need to detect these potential failures before they fail. It’s like waiting until after the castle is invaded to build the walls.
Thinking of Security Differently
A mistake of many AI buyers and implementors is looking at AI through the lens of traditional IT. AI cannot be effectively governed or managed the same way as IT or cyber systems. AI needs walls to protect it from outside interference, but it also needs measures to protect it from things like model drift and hallucinations. The answer is not to build walls inside the walls. The answer is to look at security differently.
The security of an AI system is partially its security from outside interference, but it is also ensuring that its performance stays within the defined boundaries of the model’s intent. Whether you are thinking about compliance issues or how a child might use a model for a science project, we should be thinking about how to ensure the security of the system and the safety of the users. We can’t deploy “set it and forget it” solutions because models are constantly being trained and retrained and continually learning from interactions with their users. The constant movement of the model’s status means that a single test only provides a view of how that model exists at that moment in time. After it is trained with new data, the model is fundamentally different from a test that predates the training invalidating it. The answer is constant testing.
Adaptive AI Testing
To secure an AI system in the sense that we have assurance that the model will perform according to its established guardrails, we must conduct constant testing under multiple different conditions to identify where vulnerabilities lie. Perhaps a model fails when someone enters a prompt in Leet Speak or when someone uses Saudi Arabic instead of Egyptian Arabic. These edge cases are unknown until a regular user, or a malicious actor finds them. From the perspective of the model builder or operator, this constitutes a significant risk to their investment and their reputation. Some might be left wondering how their cyber walls failed them. They didn’t. Walls don’t protect you from what’s already inside.
Traditional cybersecurity is the place where many organizational leaders are comfortable. They understand they need cloud security, multi-factor authentication, and SOC2 certifications and they understand why. Those are walls and walls keep out bad guys. Securing AI is all together a different matter. When an AI system is implemented, organizations are letting the AI system inside their walls. The system is there, and it is protected from the outside. Providing assurance that the AI system is not only streamlining processes and introducing efficiencies, but doing so accurately is a gap that is not covered by traditional cybersecurity measures.
Conclusion
The longer that we conflate cybersecurity measures with AI security measures, the more AI failures we are likely to see. AI security means taking care of what’s inside the walls in addition to what’s coming at us from outside the walls. The trick is that the tools are different. Proactive and adaptive testing to find myriad stressors for the AI system is how we solve this problem. The challenge of higher and thicker walls has been a constant throughout history and plays out every day in the cyber domain. For AI, we need to hit a moving target and that requires different technology. And a different mindset.