AI as a Weapon · 4.3
🧬 Attacks on the Models Themselves
Data poisoning, evasion, and stealing a model's secrets⏱ ~2 min
Beyond tricking an AI with words, attackers can target the machine learning system itself — corrupting how it learned, fooling what it sees, or stealing what it knows. These are studied formally in MITRE ATLAS, the AI equivalent of the famous ATT&CK framework.
The Main Categories of Attacks on ML
| Attack | What Happens | Real-World Worry |
|---|---|---|
| Data poisoning | Attacker corrupts the training data so the model learns wrong things | A spam filter taught to let the attacker's spam through |
| Evasion (adversarial examples) | Tiny, invisible tweaks to an input fool the model at inference time | A stop sign with subtle stickers read as 'speed limit 45' by a car |
| Model inversion | Attacker reconstructs private training data from model outputs | Extracting faces or records the model was trained on |
| Model stealing | Attacker copies a proprietary model by querying it a lot | Cloning a company's expensive model for free |
| Membership inference | Attacker figures out if specific data was in the training set | Proving someone's private record was used to train a model |
Adversarial Examples — The Spooky One
★ FactResearchers showed that changing a handful of pixels — imperceptible to humans — can make an image classifier confidently mislabel a panda as a gibbon. The same idea can fool facial recognition, medical imaging AI, and self-driving car vision. The model saw a pattern humans can't even perceive, and it was the wrong one.
Why This Matters More Every Year
🔒 SecurityAs AI makes more real decisions — approving loans, screening resumes, driving cars, flagging tumors — attacks on those models become attacks on real life. Securing the model is now as important as securing the server it runs on. This is a fast-growing career field with far more open jobs than qualified people.
🔗MITRE ATLAS
MITRE's knowledge base of real-world adversarial attacks against AI systems — the ATT&CK framework for machine learning.
https://atlas.mitre.org
🧠Quick Checkfirst try = +5 XP
Changing a few pixels invisibly to fool an image classifier is a(n)…
🎮 Practice what you learned
⭐ 0 XP🔥 0 days