Have you heard of Prompt Injection attacks?

They're a nasty class of new threats to LLM applications.

The user injects a malicious prompt into the AI's input, to get around the safety constraints and guardrails.

Why does it matter?

Attackers can use Prompt Injection to trick an LLM into doing things it should not do: exposing sensitive information, executing code or queries, etc.

This problem is even more pronounced for AI agents that can actually take actions based on user prompts.

There are lots of different types of prompt injection attacks:

• "Ignore all prior instructions"
• Roleplaying
• Adversarial suffixes
• Ciphers / Other Languages

Even ASCII art, obscure languages, morse code, etc have been used to "jailbreak" the AI from its safeguards.

Introducing athina.guard()