Commands

Complete command reference for all Flint AI CLI commands.

flintai init

Setup wizard that configures Flint AI for first use. Creates the ~/.flintai directory with a .env file (LLM provider, API key, runtime settings) and a config.json skeleton. Runs automatically on first use in non-CI environments. You can re-run it at any time to reconfigure.

flintai init

Initial setup for:

LLM provider — gemini, openai, anthropic, or litellm
Model name — Specific model to use (provider-specific defaults apply)
API key — API key for the selected provider

flintai scan

The flintai scan command needs OpenGrep installed and an LLM provider installed. See Init for a guided setup, or Environment Variables for manual steps.

# Scan a directory
flintai scan /path/to/agent/code

# Scan a single file
flintai scan agent.py

# Specify output file
flintai scan /path/to/code --output results.json

# Write SARIF for GitHub code scanning
flintai scan /path/to/code --format sarif

Flag	Default	Description
`path`	(required)	Path to a file or folder to scan
`--output`, `-o`	`scan_<timestamp>.<format>`	Output file for results
`--format`, `-f`	`json`	Output format: `json` or `sarif`. See Output formats
`--log`	`flintai_<timestamp>.log`	Log file path

The model that powers AI reasoning and triage comes from GENERATOR_MODEL, not from a flag. See Environment variables. What gets scanned: a requirements.txt file, or a .py file that imports a supported framework. Point at a directory and the scan walks it for those files. Point at a single file that doesn’t qualify and the scan stops with No scannable targets found.

flintai eval

Before you can run flintai eval commands, you need a valid configuration file. flintai init creates this file by default in ~/.flintai/config.json. See the Configuration section for adding models and evaluations. Every flintai eval subcommand accepts these two flags:

Flag	Default	Description
`--config`	`~/.flintai/config.json`	Path to the JSON config file
`--log`	`flintai_<timestamp>.log`	Log file path

Show models

Shows information about the configured models.

# List all models
flintai eval models list

# List models with a specific tag
flintai eval models list --tag tier=Fast

# Show details for a model (full ID or unique prefix)
flintai eval models show my-chatbot

Show evaluations

Shows information about the configured evaluations (built-in and custom).

# List all evaluations (builtin + user)
flintai eval evaluations list

# Filter by tag
flintai eval evaluations list --tag owasp_code=LLM01

# Show evaluation details and connected models
flintai eval evaluations show eval-llm01-adversarial

Model-evaluation assignments

Shows information about the assignments of evaluations to models.

# List all assignments
flintai eval model-evaluations list

# Filter by tag
flintai eval model-evaluations list --tag category=owasp

Attach evaluations to models

Creates model-evaluation assignments. Accepts models and evaluations by ID (repeatable) or by tag. Creates the cross-product of all matched models and evaluations.

# Single model, single evaluation
flintai eval model-evaluations attach --model my-chatbot --eval eval-llm01-adversarial

# Single model, multiple evaluations
flintai eval model-evaluations attach \
    --model my-chatbot \
    --eval eval-llm01-adversarial \
    --eval eval-llm02-adversarial

# Multiple models by ID
flintai eval model-evaluations attach \
    --model my-chatbot --model my-agent \
    --eval eval-llm01-adversarial

# Select by tags (all models tagged tier=Fast, all OWASP evaluations)
flintai eval model-evaluations attach \
    --model-tag tier=Fast \
    --eval-tag owasp_code=LLM01

# Mix IDs and tags
flintai eval model-evaluations attach \
    --model my-chatbot \
    --eval-tag source="Flint AI"

Duplicate assignments (same model + evaluation pair) are automatically skipped.

Detach evaluations from models

Removes model-evaluation assignments. Same flexible selection as attach. At least one of --model/--model-tag or --eval/--eval-tag is required.

# Remove a specific assignment
flintai eval model-evaluations detach --model my-chatbot --eval eval-llm01-adversarial

# Remove all evaluations from a model
flintai eval model-evaluations detach --model my-chatbot

# Remove an evaluation from all models
flintai eval model-evaluations detach --eval eval-llm01-adversarial

# Remove by tag
flintai eval model-evaluations detach --model-tag tier=Fast --eval-tag source=Garak

Run evaluations

Runs evaluations as configured. Supports a series of parameters to filter which evaluations and models should be run.

# Run a single model-evaluation by ID
flintai eval run me-chatbot-llm01

# Run all evaluations for a model
flintai eval run --model my-chatbot

# Filter which evaluations to run using tags
flintai eval run --model my-chatbot --eval-tag owasp_code=LLM01

# Set concurrency and output file
flintai eval run --model my-chatbot \
    --concurrency 10 \
    --output results.json

# Write SARIF for GitHub code scanning
flintai eval run --model my-chatbot --format sarif

Flag	Default	Description
`model_evaluation_id`	—	Model-evaluation ID to run. Optional — omit it and select with `--model` or the tag filters instead
`--model`	—	Run all evaluations assigned to this model ID
`--output`, `-o`	`eval_<timestamp>.<format>`	Output file for results
`--format`, `-f`	`json`	Output format: `json` or `sarif`. See Output formats
`--concurrency`, `-c`	`20`	Max concurrent evaluation tasks
`--model-tag`	—	Filter by model tag (repeatable)
`--eval-tag`	—	Filter by evaluation tag (repeatable)

--format applies to flintai eval run only. The other flintai eval subcommands print to the terminal.

Global options

Flag	Description
`--version`	Print the installed version and exit

--version is the only flag accepted before a command. Everything else belongs to a specific command: --log works on flintai scan and on any flintai eval subcommand, --config works on any flintai eval subcommand, and --output and --format work on flintai scan and flintai eval run.

Overview

Flint AI Scan

Flint AI Eval

Guides

Reference

Troubleshooting

Resources

flintai init

flintai scan

flintai eval

Show models

Show evaluations

Model-evaluation assignments

Attach evaluations to models

Detach evaluations from models

Run evaluations

Global options

​flintai init

​flintai scan

​flintai eval

​Show models

​Show evaluations

​Model-evaluation assignments

​Attach evaluations to models

​Detach evaluations from models

​Run evaluations

​Global options

flintai init

flintai scan

flintai eval

Show models

Show evaluations

Model-evaluation assignments

Attach evaluations to models

Detach evaluations from models

Run evaluations

Global options