b77c8b0044
Design foundation for POC. Monorepo: FastAPI+Postgres backend, Vue 3 frontend, Python packages (llm-gateway, artifacts, connectors). Approval gate and token budgets are architectural constraints per measured prototype findings.
25 lines
1.8 KiB
Markdown
25 lines
1.8 KiB
Markdown
# ADR-0001: LLM at decision points; state machine owns the flow
|
|
|
|
Status: accepted (2026-07-30)
|
|
|
|
## Context
|
|
|
|
We prototyped this workflow as an autonomous agent setup (Hermes + LLM subagents + cron). Three measured findings:
|
|
|
|
1. Free-roaming agents burned ~10x expected token budget in 2 days; silent credential inheritance billed cheap-model work to the paid model (~45M tokens).
|
|
2. Unapproved external sends caused real harm (personnummer emailed without consent).
|
|
3. Fully AI-drafted cover letters scored 5-7.5/10 in owner review vs 9/10 for human-drafted + AI-reviewed.
|
|
|
|
## Decision
|
|
|
|
- A deterministic **state machine** (`ApplicationState` transition table) owns pipeline flow. The LLM never chooses the next step.
|
|
- The LLM is invoked only inside named steps with typed inputs/outputs (JSON schema validated), capped iterations (no agent loops in v1), and per-task token budgets.
|
|
- **Model routing per task class**: extraction/scoring -> cheap model (GLM-5.2 via ollama-cloud), prose review/critique -> strong model. Every call logs tokens and cost. Fallback chains must never route to a paid provider for cheap-task classes.
|
|
- **Approval enforcement**: server requires a confirmed `Approval` row whose `artifact_hash` matches the exact bytes of what will be sent. Approvals expire after 24h or on any artifact mutation.
|
|
- **Draft policy**: cover letters default to user-drafted, AI-reviewed. Full-AI drafting exists as opt-in per application.
|
|
|
|
## Consequences
|
|
|
|
- Positive: predictable cost, auditable behavior, no unapproved sends possible, quality aligned with measured reality.
|
|
- Negative: fewer "magical" autonomous behaviors; some flows need explicit user input.
|
|
- Mitigation: an opt-in per-step "agent mode" may be added later, sandboxed inside a single step with iteration+token caps.
|