Commands

Complete command reference for all Flint AI CLI commands.

flintai init

Setup wizard that configures Flint AI for first use. Creates the ~/.flintai directory with a .env file (LLM provider, API key, runtime settings) and a config.json skeleton. Runs automatically on first use in non-CI environments. You can re-run it at any time to reconfigure.

flintai init

Initial setup for:

LLM provider — gemini, openai, anthropic, or litellm
Model name — Specific model to use (provider-specific defaults apply)
API key — API key for the selected provider

flintai scan

The flintai scan command needs OpenGrep installed and an LLM provider installed. See Init for a guided setup, or Environment Variables for manual steps.

# Scan a directory
flintai scan /path/to/agent/code

# Scan a single file
flintai scan agent.py

# Specify output file
flintai scan /path/to/code --output results.json

Flag	Default	Description
`path`	(required)	Path to a file or folder to scan
`--output`, `-o`	`scan_<timestamp>.json`	Output file for results

flintai eval

Before you can run flintai eval commands, you need a valid configuration file. flintai init creates this file by default in ~/.flintai/config.json, you can override its location via --config <path>. See the Configuration section for adding models, evaluations etc.

Show models

Shows information about the configured models.

# List all models
flintai eval models list

# List models with a specific tag
flintai eval models list --tag tier=Fast

# Show details for a model (full ID or unique prefix)
flintai eval models show my-chatbot

Show evaluations

Shows information about the configured evaluations (built-in and custom).

# List all evaluations (builtin + user)
flintai eval evaluations list

# Filter by tag
flintai eval evaluations list --tag owasp_code=LLM01

# Show evaluation details and connected models
flintai eval evaluations show eval-llm01-adversarial

Model-evaluation assignments

Shows information about the assignments of evaluations to models.

# List all assignments
flintai eval model-evaluations list

# Filter by tag
flintai eval model-evaluations list --tag category=owasp

Attach evaluations to models

Creates model-evaluation assignments. Accepts models and evaluations by ID (repeatable) or by tag. Creates the cross-product of all matched models and evaluations.

# Single model, single evaluation
flintai eval model-evaluations attach --model my-chatbot --eval eval-llm01-adversarial

# Single model, multiple evaluations
flintai eval model-evaluations attach \
    --model my-chatbot \
    --eval eval-llm01-adversarial \
    --eval eval-llm02-adversarial

# Multiple models by ID
flintai eval model-evaluations attach \
    --model my-chatbot --model my-agent \
    --eval eval-llm01-adversarial

# Select by tags (all models tagged tier=Fast, all OWASP evaluations)
flintai eval model-evaluations attach \
    --model-tag tier=Fast \
    --eval-tag owasp_code=LLM01

# Mix IDs and tags
flintai eval model-evaluations attach \
    --model my-chatbot \
    --eval-tag source="Flint AI"

Duplicate assignments (same model + evaluation pair) are automatically skipped.

Detach evaluations from models

Removes model-evaluation assignments. Same flexible selection as attach. At least one of --model/--model-tag or --eval/--eval-tag is required.

# Remove a specific assignment
flintai eval model-evaluations detach --model my-chatbot --eval eval-llm01-adversarial

# Remove all evaluations from a model
flintai eval model-evaluations detach --model my-chatbot

# Remove an evaluation from all models
flintai eval model-evaluations detach --eval eval-llm01-adversarial

# Remove by tag
flintai eval model-evaluations detach --model-tag tier=Fast --eval-tag method=Garak

Run evaluations

Runs evaluations as configured. Supports a series of parameters to filter which evaluations and models should be run.

# Run a single model-evaluation by ID
flintai eval run me-chatbot-llm01

# Run all evaluations for a model
flintai eval run --model my-chatbot

# Filter which evaluations to run using tags
flintai eval run --model my-chatbot --eval-tag owasp_code=LLM01

# Set concurrency and output file
flintai eval run --model my-chatbot \
    --concurrency 10 \
    --output results.json

Flag	Default	Description
`--config`	`~/.flintai/config.json`	Path to the JSON config file
`--output`, `-o`	`eval_<timestamp>.json`	Output file for results
`--concurrency`, `-c`	`20`	Max concurrent evaluation tasks
`--model-tag`	—	Filter by model tag (repeatable)
`--eval-tag`	—	Filter by evaluation tag (repeatable)

Global options

Flag	Default	Description
`--log`	`flintai_<timestamp>.log`	Log file path

Overview

Flint AI Scan

Flint AI Eval

Guides

Reference

Troubleshooting

Resources

flintai init

flintai scan

flintai eval

Show models

Show evaluations

Model-evaluation assignments

Attach evaluations to models

Detach evaluations from models

Run evaluations

Global options

​flintai init

​flintai scan

​flintai eval

​Show models

​Show evaluations

​Model-evaluation assignments

​Attach evaluations to models

​Detach evaluations from models

​Run evaluations

​Global options

flintai init

flintai scan

flintai eval

Show models

Show evaluations

Model-evaluation assignments

Attach evaluations to models

Detach evaluations from models

Run evaluations

Global options