The speedy evolution and enterprise adoption of AI has motivated unhealthy actors to focus on these methods with higher frequency and class. Many safety leaders acknowledge the significance and urgency of AI safety, however don’t but have processes in place to successfully handle and mitigate rising AI dangers with complete protection of your complete adversarial AI risk panorama.
Strong Intelligence (now part of Cisco) and the UK AI Safety Institute partnered with the Nationwide Institute of Requirements and Expertise (NIST) to launch the most recent replace to the Adversarial Machine Studying Taxonomy. This transatlantic partnership aimed to fill this want for a complete adversarial AI risk panorama, whereas creating alignment throughout areas in standardizing an strategy to understanding and mitigating adversarial AI.
Survey outcomes from the World Cybersecurity Outlook 2025 revealed by the World Financial Discussion board spotlight the hole between AI adoption and preparedness: “Whereas 66% of organizations count on AI to have essentially the most vital impression on cybersecurity within the yr to come back, solely 37% report having processes in place to evaluate the safety of AI instruments earlier than deployment.”
As a way to efficiently mitigate these assaults, it’s crucial that AI and cybersecurity communities are properly knowledgeable about as we speak’s AI safety challenges. To that finish, we’ve co-authored the 2025 replace to NIST’s taxonomy and terminology of adversarial machine studying.
Let’s take a look at what’s new on this newest replace to the publication, stroll by means of the taxonomies of assaults and mitigations at a excessive stage, after which briefly mirror on the aim of taxonomies themselves—what are they for, and why are they so helpful?
What’s new?
The earlier iteration of the NIST Adversarial Machine Studying Taxonomy centered on predictive AI, fashions designed to make correct predictions primarily based on historic knowledge patterns. Particular person adversarial strategies had been grouped into three major attacker aims: availability breakdown, integrity violations, and privateness compromise. It additionally included a preliminary AI attacker approach panorama for generative AI, fashions that generate new content material primarily based on present knowledge. Generative AI adopted all three adversarial approach teams and added misuse violations as a further class.
Within the newest replace of the taxonomy, we increase on the generative AI adversarial strategies and violations part, whereas additionally making certain the predictive AI part stays correct and related to as we speak’s adversarial AI panorama. One of many main updates to the most recent model is the addition of an index of strategies and violations firstly of the doc. Not solely does this make the taxonomy simpler to navigate, nevertheless it permits for a better strategy to reference strategies and violations in exterior references to the taxonomy. This makes the taxonomy a extra sensible useful resource to AI safety practitioners.
Clarifying assaults on Predictive AI fashions
The three attacker aims constant throughout predictive and generative AI sections, are as follows:
- Availability breakdown assaults degrade the efficiency and availability of a mannequin for its customers.
- Integrity violations try and undermine mannequin integrity and generate incorrect outputs.
- Privateness compromises unintended leakage of restricted or proprietary info equivalent to details about the underlying mannequin and coaching knowledge.
Classifying assaults on Generative AI fashions
The generative AI taxonomy inherits the identical three attacker aims as predictive AI—availability, integrity, and privateness—and encapsulates extra particular person strategies. There’s a fourth attacker goal distinctive to generative AI: misuse violations. The up to date model of the taxonomy expanded on generative AI adversarial strategies to account for essentially the most up-to-date panorama of attacker strategies.
Misuse violations repurpose the capabilities of generative AI to additional an adversary’s malicious aims by creating dangerous content material that helps cyber-attack initiatives.
Harms related to misuse violations are meant to supply outputs that would trigger hurt to others. For instance, attackers may use direct prompting assaults to bypass mannequin defenses and produce dangerous or undesirable output.
To attain one or a number of of those targets, adversaries can leverage numerous strategies. The growth of the generative AI part highlights attacker strategies distinctive to generative AI, equivalent to direct immediate injection, knowledge extraction, and oblique immediate injection. As well as, there’s a wholly new arsenal of provide chain assaults. Provide chain assaults should not a violation particular to a mannequin, and due to this fact should not included within the above taxonomy diagram.
Provide chain assaults are rooted within the complexity and inherited danger of the AI provide chain. Each part—open-source fashions and third-party knowledge, for instance—can introduce safety points into your complete system.
These could be mitigated with provide chain assurance practices equivalent to vulnerability scanning and validation of datasets.
Direct immediate injection alters the habits of a mannequin by means of direct enter from an adversary. This may be finished to create deliberately malicious content material or for delicate knowledge extraction.
Mitigation measures embrace coaching for alignment and deploying a real-time immediate injection detection answer for added safety.
Oblique immediate injection differs in that adversarial inputs are delivered through a third-party channel. This method might help additional a number of aims: manipulation of knowledge, knowledge extraction, unauthorized disclosure, fraud, malware distribution, and extra.
Proposed mitigations assist decrease danger by means of reinforcement studying from human suggestions, enter filtering, and the usage of an LLM moderator or interpretability-based answer.
What are taxonomies for, anyhow?
Co-author and Cisco Director of AI & Safety, Hyrum Anderson, put it finest when he mentioned that “taxonomies are most clearly essential to arrange our understanding of assault strategies, capabilities, and aims. Additionally they have an extended tail impact in bettering communication and collaboration in a area that’s shifting in a short time.”
It’s why Cisco strives to help within the creation and steady enchancment of shared requirements, collaborating with main organizations like NIST and the UK AI Safety Institute.
These assets give us higher psychological fashions for classifying and discussing new strategies and capabilities. Consciousness and schooling of those vulnerabilities facilitate the event of extra resilient AI methods and extra knowledgeable requirements and insurance policies.
You possibly can evaluate your complete NIST Adversarial Machine Studying Taxonomy and be taught extra with an entire glossary of key terminology within the full paper.
We’d love to listen to what you suppose. Ask a Query, Remark Under, and Keep Related with Cisco Safe on social!
Cisco Safety Social Channels
Share:
