AI as a Shield · 5.2

🔒 Securing AI Systems

Defending the AI itself — the newest security discipline⏱ ~2 min

When your company deploys an AI system, that system becomes something you have to defend — a new kind of attack surface. This discipline is often called 'AI security' or 'ML security,' and it borrows from traditional security while adding brand-new concerns.

The Layers You Have to Protect

  1. 1.The training data — poisoned or leaked data corrupts or exposes the model (guard the supply chain)
  2. 2.The model itself — protect against theft, inversion, and adversarial inputs
  3. 3.The application around it — prompt injection, insecure output handling, over-broad permissions
  4. 4.The infrastructure — the servers, APIs, and pipelines (classic security still fully applies)
  5. 5.The humans — the people who use and trust the AI's output (social layer)

Practical Defenses for LLM Applications

  • Never fully trust model output — validate and sanitize it before using it in code, queries, or commands
  • Least privilege for AI agents — an AI connected to tools should only access what it truly needs
  • Keep a human in the loop — for consequential actions (payments, deletions), require human approval
  • Filter inputs and outputs — screen for injection attempts and sensitive data leakage
  • Don't put secrets in prompts — anything in the context window can potentially leak out
  • Log and monitor — treat AI interactions as security-relevant events worth recording
🔒 SecurityA core principle: an LLM is untrusted-by-default, like user input. Never let an AI's raw output directly run code, execute a database query, or trigger a real-world action without validation and, for anything important, human review. Treat it as a brilliant but naive assistant that a stranger might be whispering to.
🔗
OWASP GenAI Security Project
Community-driven, vendor-neutral guidance for securing generative AI and LLM applications — cheat sheets, the LLM Top 10, and more.
https://genai.owasp.org
🧠Quick Checkfirst try = +5 XP

How should you treat an LLM's raw output in an application?

0 XP🔥 0 days