Config keys

honest_agent_config.yml holds a project’s settings, with one target per agent. Any value can read an environment variable with {{ env_var('NAME') }} or {{ env_var('NAME', 'default') }}. Relative paths are relative to the file’s folder. Connect your agent explains them with examples.

results_path: ./honest_agent_results/results.duckdb
model: claude-sonnet-4-5            # top level: for every target
default_target: sales-agent

targets:
  sales-agent:
    mcp_url: "{{ env_var('SALES_AGENT_MCP_URL') }}"
    bearer_token_env: SALES_AGENT_MCP_TOKEN
    max_tool_steps: 8               # per target: only this one

Top level only

Setting	What it does
default_target	The target run, debug, ls and notify use without --target.
results_path	Where results are stored: a local DuckDB file (default ./honest_agent_results/results.duckdb) or a MotherDuck database as md:<name>. Every target shares it, so the report can compare agents.
targets	One entry per agent you evaluate, by name. Each holds the target settings below.

Top level or per target

Set at the top level, a value applies to every target; a target’s own value wins.

Setting	What it does
grading_model	The model for grading: it pulls the value out for extract_match and judges the answer for llm_judge (contains uses none). Default: model.
max_tool_steps	Steps the agent may take per question; a step is one model call and the tool calls it asks for. Default: 5.
model	The model that plays the agent. Default: chosen from the API key you have.

Per target

Setting	What it does
agent_name	The name results are recorded under. Default: the target’s name.
bearer_token_env	The name of the variable that holds the server’s bearer token, never the token.
default_database	The database where the server runs a bare table name, for provenance.
default_schema	The schema where the server runs a bare table name, for provenance.
evals_dir	The folder with this target’s evals. Default: evals.
ignore_tools	Tools whose sql, query or statement argument isn’t SQL, so provenance skips them.
mcp_command	The command that starts a local MCP server. Set this or mcp_url.
mcp_env	Variables from .env a local server needs. It gets only these, plus PATH, HOME and similar.
mcp_url	The URL of a remote MCP server. Set this or mcp_command.