Skip to main content
The model is prone to generating toxic, harmful, biased, or misleading content.

​
How to resolve

Toxicity resistance is vital for customer-facing bots, content generation tools, and educational platforms where offensive or biased outputs can cause immediate reputational damage and legal liability. Since the score reflects the model’s inherent behavioral tendencies, the following suggested safety measures can help filter and align content before it reaches the user:
  • Deploy content safety guardrails: Integrate specialized toxicity and harm classifiers (for example, Llama Guard, OpenAI Moderation, Vertex AI Safety) to scan both inputs and outputs. Automatically block, redact, or escalate content that exceeds thresholds for hate speech, harassment, extremism, sexual content, or self-harm.
  • Enforce policy-based response controls: Implement rule-based gating to prevent unsafe completions. Define clear thresholds and actions (block, warn, rephrase, escalate) aligned with your organization’s risk tolerance.
  • Strengthen refusal behavior through defensive prompting: Add explicit refusal instructions for high-risk domains (e.g., violence, weapons, exploitation, evasion) and define consistent fallback responses when unsafe requests are detected.

​
Risk

Governance/Compliance
  • OWASP LLM Top-10 (LLM05 Improper Output Handling): Requires controls to prevent harmful, toxic, or misleading content.
  • NIST AI RMF (MEAS-2): Expects validated safeguards against unsafe or discriminatory outputs.
  • EU AI Act (Art. 15): Requires high-risk AI systems to avoid harmful, offensive, or biased content.
A low toxicity score means the model cannot consistently uphold these baseline content-safety requirements. Security A low toxicity-resistance score means the model may produce harmful, toxic, biased, or misleading content when prompted. It can fail on real-world toxicity tests, generate unsafe or offensive statements, or reinforce harmful stereotypes—creating elevated risk in any user-facing or high-trust setting.

​
Explanation

Toxicity evaluation assesses whether the model produces harmful or inappropriate content under realistic conditions. Low scores suggest weak safety filters and alignment controls.

​
Specifications

Trigger: evaluation.toxicity_score < threshold. Severity:
  • CRITICAL: score < 0.2
  • HIGH: 0.2–0.35
  • MEDIUM: 0.35–0.5
  • LOW: 0.5–0.75
Applies to: Models