Skip to main content
Complete command reference for all Flint AI CLI commands.

​
flintai init

Setup wizard that configures Flint AI for first use. Creates the ~/.flintai directory with a .env file (LLM provider, API key, runtime settings) and a config.json skeleton. Runs automatically on first use in non-CI environments. You can re-run it at any time to reconfigure. 
flintai init
Initial setup for:
  1. LLM provider — gemini, openai, anthropic, or litellm
  2. Model name — Specific model to use (provider-specific defaults apply)
  3. API key — API key for the selected provider

​
flintai scan

The flintai scan command needs OpenGrep installed and an LLM provider installed. See Init for a guided setup, or Environment Variables for manual steps. 
# Scan a directory
flintai scan /path/to/agent/code

# Scan a single file
flintai scan agent.py

# Specify output file
flintai scan /path/to/code --output results.json
FlagDefaultDescription
path(required)Path to a file or folder to scan
--output, -oscan_<timestamp>.jsonOutput file for results

​
flintai eval

Before you can run flintai eval commands, you need a valid configuration file. flintai init creates this file by default in ~/.flintai/config.json, you can override its location via --config <path>. See the Configuration section for adding models, evaluations etc.

​
Show models

Shows information about the configured models. 
# List all models
flintai eval models list

# List models with a specific tag
flintai eval models list --tag tier=Fast

# Show details for a model (full ID or unique prefix)
flintai eval models show my-chatbot

​
Show evaluations

Shows information about the configured evaluations (built-in and custom). 
# List all evaluations (builtin + user)
flintai eval evaluations list

# Filter by tag
flintai eval evaluations list --tag owasp_code=LLM01

# Show evaluation details and connected models
flintai eval evaluations show eval-llm01-adversarial

​
Model-evaluation assignments

Shows information about the assignments of evaluations to models. 
# List all assignments
flintai eval model-evaluations list

# Filter by tag
flintai eval model-evaluations list --tag category=owasp

​
Attach evaluations to models

Creates model-evaluation assignments. Accepts models and evaluations by ID (repeatable) or by tag. Creates the cross-product of all matched models and evaluations. 
# Single model, single evaluation
flintai eval model-evaluations attach --model my-chatbot --eval eval-llm01-adversarial

# Single model, multiple evaluations
flintai eval model-evaluations attach \
    --model my-chatbot \
    --eval eval-llm01-adversarial \
    --eval eval-llm02-adversarial

# Multiple models by ID
flintai eval model-evaluations attach \
    --model my-chatbot --model my-agent \
    --eval eval-llm01-adversarial

# Select by tags (all models tagged tier=Fast, all OWASP evaluations)
flintai eval model-evaluations attach \
    --model-tag tier=Fast \
    --eval-tag owasp_code=LLM01

# Mix IDs and tags
flintai eval model-evaluations attach \
    --model my-chatbot \
    --eval-tag source="Flint AI"
Duplicate assignments (same model + evaluation pair) are automatically skipped.

​
Detach evaluations from models

Removes model-evaluation assignments. Same flexible selection as attach. At least one of --model/--model-tag or --eval/--eval-tag is required. 
# Remove a specific assignment
flintai eval model-evaluations detach --model my-chatbot --eval eval-llm01-adversarial

# Remove all evaluations from a model
flintai eval model-evaluations detach --model my-chatbot

# Remove an evaluation from all models
flintai eval model-evaluations detach --eval eval-llm01-adversarial

# Remove by tag
flintai eval model-evaluations detach --model-tag tier=Fast --eval-tag method=Garak

​
Run evaluations

Runs evaluations as configured. Supports a series of parameters to filter which evaluations and models should be run. 
# Run a single model-evaluation by ID
flintai eval run me-chatbot-llm01

# Run all evaluations for a model
flintai eval run --model my-chatbot

# Filter which evaluations to run using tags
flintai eval run --model my-chatbot --eval-tag owasp_code=LLM01

# Set concurrency and output file
flintai eval run --model my-chatbot \
    --concurrency 10 \
    --output results.json
FlagDefaultDescription
--config~/.flintai/config.jsonPath to the JSON config file
--output, -oeval_<timestamp>.jsonOutput file for results
--concurrency, -c20Max concurrent evaluation tasks
--model-tag—Filter by model tag (repeatable)
--eval-tag—Filter by evaluation tag (repeatable)

​
Global options

FlagDefaultDescription
--logflintai_<timestamp>.logLog file path