b77c8b0044
Design foundation for POC. Monorepo: FastAPI+Postgres backend, Vue 3 frontend, Python packages (llm-gateway, artifacts, connectors). Approval gate and token budgets are architectural constraints per measured prototype findings.
1.8 KiB
1.8 KiB
ADR-0001: LLM at decision points; state machine owns the flow
Status: accepted (2026-07-30)
Context
We prototyped this workflow as an autonomous agent setup (Hermes + LLM subagents + cron). Three measured findings:
- Free-roaming agents burned ~10x expected token budget in 2 days; silent credential inheritance billed cheap-model work to the paid model (~45M tokens).
- Unapproved external sends caused real harm (personnummer emailed without consent).
- Fully AI-drafted cover letters scored 5-7.5/10 in owner review vs 9/10 for human-drafted + AI-reviewed.
Decision
- A deterministic state machine (
ApplicationStatetransition table) owns pipeline flow. The LLM never chooses the next step. - The LLM is invoked only inside named steps with typed inputs/outputs (JSON schema validated), capped iterations (no agent loops in v1), and per-task token budgets.
- Model routing per task class: extraction/scoring -> cheap model (GLM-5.2 via ollama-cloud), prose review/critique -> strong model. Every call logs tokens and cost. Fallback chains must never route to a paid provider for cheap-task classes.
- Approval enforcement: server requires a confirmed
Approvalrow whoseartifact_hashmatches the exact bytes of what will be sent. Approvals expire after 24h or on any artifact mutation. - Draft policy: cover letters default to user-drafted, AI-reviewed. Full-AI drafting exists as opt-in per application.
Consequences
- Positive: predictable cost, auditable behavior, no unapproved sends possible, quality aligned with measured reality.
- Negative: fewer "magical" autonomous behaviors; some flows need explicit user input.
- Mitigation: an opt-in per-step "agent mode" may be added later, sandboxed inside a single step with iteration+token caps.