Skip to main content
The model’s jailbreak-resistance score is low, indicating susceptibility to prompt injection and safety bypass attacks.

​
How to resolve

Since the model has inherent jailbreak vulnerabilities, apply the following defenses to mitigate risks and protect the application environment:
  • Introduce input & output guardrails: Deploy dedicated safety filters (for example, Llama Guard, OpenAI Moderation, Google Vertex AI Safety, or provider-native safety controls) to screen both user inputs and model outputs. Detect and block jailbreak attempts, prompt injection, manipulation, toxicity, or system prompt leakage before responses are returned or tools are executed.
  • Restrict tool & data access: Follow the principle of least privilege. Limit the model’s ability to invoke external APIs or access sensitive databases to minimize the blast radius of a successful safety bypass.
  • Perform targeted fine-tuning: For self-hosted or user-owned models, conduct adversarial training on identified attack categories to improve internal resistance. This is not applicable to provider-managed APIs like OpenAI or Gemini.

​
Risk

Governance/Compliance Low jailbreak resistance violates core expectations across major AI safety frameworks:
  • OWASP LLM Top-10 (LLM01: Prompt Injection): requires strong defenses against instruction hijacking and safety bypasses.
  • NIST AI RMF (MEAS-2, GOV-6): expects measurable robustness against adversarial manipulation and controls to prevent unauthorized model behavior.
  • EU AI Act (Art 15): mandates that high-risk AI systems remain resilient to manipulation attempts and cannot be easily coerced into unsafe or unintended actions.
Weak jailbreak resistance indicates the model cannot meet these baseline security and governance requirements. Security Jailbreaks allow bypassing safety mechanisms and enabling harmful instructions. Risks include:
  • Harmful content generation
  • Disallowed actions via tool invocation
  • System prompt exposure or alteration
  • Data leakage through replay or continuation attacks

​
Explanation

Evaluation data demonstrates the model’s lack of resilience against several high-impact attack categories, including:
  • Prompt injection: attackers hijack instructions to bypass safety rules.
  • Latent injection: hidden or embedded prompts inside long context passages trick the model into misbehaving.
  • Adversarial suffixes: short strings added to user inputs that reliably break safety filters.
  • Continuation / leak-replay attacks: prompting the model to continue or replay hidden or sensitive information.
These categories represent the core techniques used in modern LLM adversarial attacks and are among the common real-world failure modes.

​
Specifications

Trigger: evaluation.jailbreak_score < threshold. Severity:
  • CRITICAL: score < 0.2
  • HIGH: 0.2–0.35
  • MEDIUM: 0.35–0.5
  • LOW: 0.5–0.75
Applies to: Models