Skip to main content
Flint AI runs on your machine, but several features can call external LLM providers. This can be configured via GENERATOR_MODEL (located in ~/.flintai/.env, created by flintai init). You can set this to a:
  • Remote managed LLM: gemini, openai, or anthropic
  • Locally hosted LLM: litellm or ollama

​
Summary

How Flint AI handles your data depends on the features you use:
  • Stays on your machine: File discovery, static analysis tools, PII/secret/toxicity detection, and Garak detectors run entirely locally with no external API calls.
  • Sent to your configured LLM: AI-powered scan reasoning, triage, adversarial probe generation, and LLM-as-judge scoring send source code, prompts, and/or model responses to the provider you configure via GENERATOR_MODEL (gemini, openai, anthropic, litellm, or ollama).
  • Sent to the model you’re testing: Evaluation prompts (including adversarial content) are sent directly to the agent or model endpoint you specify in your eval config.
The tables below show exactly what will be sent to the LLM in each command path.

​
flintai scan

LayerRuns locallySends to LLM
File discoveryYes—
Static analysis (bandit, opengrep, detect-secrets, pip-audit)Yes—
AI reasoningNoSource code snippets, import chains, and file contents from the scanned codebase
TriageNoAll findings plus surrounding code context for severity validation
The AI reasoning and triage layers are powered by the LLM configured via GENERATOR_MODEL. If no LLM provider is configured, these layers are skipped and the scan produces only static analysis results.

​
flintai eval

ComponentRuns locallySends to LLM
Prompt deliveryYes/NoPrompts (including adversarial ones) are sent to the target model/agent you are evaluating
Adversarial probe generationNoThe configured LLM (GENERATOR_MODEL) generates attack prompts and judges responses
Topic guard generationNoThe configured LLM generates out-of-scope test prompts
LLM-as-judge detectorsNoModel responses are sent to the configured LLM for scoring
PII detectorYes—
Secret detectorYes—
Toxicity classifierYes—
Garak detectorsYes—
Evaluations that use LLM-based generation or judging (adversarial probes, topic guards, LLM-as-judge detectors, quality metrics) require a configured LLM provider. Message-collection evaluations with local-only detectors (PII, secrets, toxicity) work without one.

​
Configuration

Configure your LLM provider in ~/.flintai/.env or via environment variables. See Environment variables for details.