As artificial intelligence (AI) continues to evolve, so do the ways in which it can be exploited. One of the big concerns these days is how AI can fall victim to ‘adversarial attacks,’ where malicious actors manage to breach systems. Yaron Singer, once a professor at Harvard and now Cisco’s VP of AI and security, co-founded Robust Intelligence to tackle these very issues. Cisco acquired the company in 2024, and it’s been instrumental in evaluating AI models to safeguard against potential misuse.
In the early days, AI safety practices weren’t exactly commonplace. Singer’s insights reveal a shift from traditional machine learning (ML) to generative AI, marked by breakthroughs like OpenAI’s ChatGPT. Unlike traditional ML models that predict based on data, generative AI creates new content, which opens up both opportunities and risks. Take, for instance, a spam filter that might misclassify emails. Generative AI, on the other hand, could churn out misleading content. Even small changes in input data can significantly alter AI’s output, presenting challenges.
Back in 2023, researchers discovered vulnerabilities in Nvidia’s AI guardrails that allowed unauthorized access to sensitive information. With AI being integrated into products via APIs, hidden vulnerabilities can have widespread consequences. Companies find themselves in a bit of a bind. On one hand, leveraging data provides advantages, but privacy concerns are a looming threat. AI bias only adds to these challenges, as poorly calibrated models might produce discriminatory content. That’s why rigorous testing and validation are so crucial.
Businesses are increasingly relying on APIs. A 2024 Gartner survey showed that 71% of digital enterprises depend on third-party APIs. Singer emphasizes the importance of education on model safety and data. Measures like real-time AI validation and software ‘bouncers’ ensure input safety. Singer’s team was at the forefront of many validation techniques, which were initially uncommon in AI security.
Data-related vulnerabilities like prompt engineering and data poisoning can be exploited by malicious actors to manipulate AI. Similarly, AI jailbreaking and adversarial testing expose model weaknesses. In 2023, Robust Intelligence tested OpenAI’s GPT-4 and found some vulnerabilities, including its ability to generate phishing messages. Although these were addressed, new vulnerabilities continue to emerge as AI develops. Innovations like AI firewalls are helping to protect models from threats.
Despite these advancements, AI vulnerabilities persist. It’s a bit of a ‘cat-and-mouse game,’ as Singer puts it, highlighting the ongoing challenges in securing AI systems. Quantum cryptography offers promising solutions, but unforeseen human errors still introduce risks. Protecting AI requires collaboration among developers, businesses, and regulators. At Cisco, Singer aims to ensure a secure future for AI, acknowledging the trade-offs and risks involved. As he notes, ‘In society, we accept trade-offs. AI will introduce risks, but we’ll mitigate what we can.’
