Skip to main content
The agent uses many tools but lacks any guardrails needed to safely control or constrain tool-driven actions.

​
How to resolve

Here are a few core mitigation strategies:
  • Strict schema validation: Enforce schema checks (for example, JSON Schema) to ensure tool calls remain within predefined parameter ranges.
  • Least-privilege access: Define per-tool authorization and capability limits to restrict tools to safe use cases.
  • Human-in-the-loop: Require explicit human approval for high-impact actions such as code execution, external API requests, or sensitive file modifications.
  • Input/output moderation: Use safety classifiers (for example, Llama Guard, OpenAI Moderation, or Google SAIF) to detect malicious intent or hallucinations, or unsafe tool usage before execution.

​
Risk

Governance/Compliance Missing guardrails for highly autonomous, tool-using agents violates core requirements across major AI governance frameworks:
  • NIST AI RMF (GOV-2, MAP-3, MAN-2): requires defined autonomy limits, oversight mechanisms, and validation of model-initiated actions.
  • ISO/IEC 23894 (Operational & Technical Controls): mandates safeguards that restrict high-risk or unintended AI actions, especially when external tools are involved.
  • ISO/IEC 42001 (AIMS): requires documented system behavior, guardrails, and boundaries for autonomous AI systems.
  • EU AI Act (Articles 9–13): high-risk systems must have controls preventing unintended or harmful autonomous actions, plus auditability and human oversight.
  • OWASP Top 10 for LLMs (LLM03, LLM05): requires validation of outputs and safe handling of tool execution.
Across these standards, the expectation is clear: AI agents with tools must have guardrails that limit autonomy, validate actions, and ensure oversight; without them, the system cannot meet governance or safety requirements. Security When an agent has many tools but lacks proper input or output guardrails, harmful or adversarial prompts—and even hallucinated model outputs—can directly trigger unsafe actions. This includes modifying files, sending unauthorized API requests, executing code, or leaking sensitive data. With no guardrails to validate inputs or filter outputs, the risk of autonomy escalation, unintended tool executions, and amplified prompt-injection attacks increases significantly.

​
Explanation

Agents with access to multiple tools pose a higher security risk because they can perform a wider range of actions. Input guardrails help catch harmful or manipulative prompts before they reach the model, and output guardrails check the model’s responses before a tool is triggered. Without these protections, unsafe commands or instructions—whether caused by user input or model error—can be passed directly to tools and carried out immediately, increasing the chance of real-world harm.

​
Specifications

Trigger
  • len(agent.tools) > 5 AND (agent.output_guardrails is empty or agent.output_guardrails is empty)
Severity
  • CRITICAL
Applies to: Agent