mirror of
https://github.com/AmruthPillai/Reactive-Resume.git
synced 2026-08-24 15:22:20 +10:00
Amruth Pillai
c8081ac2fe
* docs(adr): propose agent AI SDK v7 adoption plan * fix(ai): bind analyzeResume through aiService in service test The test destructured analyzeResume as a named export that does not exist; main was red. * test(agent): keep pure ai helpers real via spread-actual mock factory * feat(agent): add run guards, patch version guard, run wall-clock timeout * feat(agent): validate UI messages at the send boundary * feat(agent): crash-safe draft-row persistence and server-side cancellation * feat(agent): reap stale run claims at boot, on send, and on thread open * feat(agent): fresh-document patch output and tiered context pruning * feat(ai): shared agent tool contracts and message metadata schema * feat(agent): add per-thread review-patches setting with update endpoint * feat(agent): gate resume patches behind hmac-signed tool approval * feat(agent): merge question answers and approval decisions before run claim * feat(agent): approval ui with composed auto-send and fixture-driven tests * feat(agent): usage metadata, tool activity cards, smoother streaming * feat(agent): tool-call repair, input examples, structured step logging * chore(i18n): translate new agent workspace strings across all locales * fix(agent): gate stale-run draft cancellation on winning the claim clear Snapshot streaming drafts before the conditional clear and skip the flip entirely when another reaper or a replacement run already cleared the claim. Also address review nits in eleven locale catalogs. * fix(agent): flip reaped drafts only when their snapshotted state is unchanged * fix(agent): address review findings across run lifecycle, context budget, and approval flow - bind patches to the revision the model read via signed baseUpdatedAt - claim the run before consuming a continuation; recorded-but-unexecuted approvals retry as pending continuations - keep run ownership on stop() until cancellation persists; preserve the claim for the reaper when final persistence fails - estimate tokens without serializing binary attachments (tokenx) and enforce the budget by dropping oldest whole turns - mark crash-recovered patch results as snapshot boundaries; strip /data prefixes at execution time - retry failed continuations without regenerate; mount a single AgentChat; disable response controls on read-only threads; freeze review toggle during runs (client+server) - accumulate usage across continuations and match the SDK's nested usage shape; label-form token strings; reorderable source label; accessible note field; state-neutral web-search label * chore(i18n): translate revised agent strings across all locales * fix(agent): harden baseUpdatedAt validation and address review follow-ups - bundle tokenx in the server runtime dependencies (e2e boot failure) - strict ISO schema for baseUpdatedAt plus loud executor rejection of unparseable values - it-IT source label consistency (Fonte) - prove penultimate-turn retention in the context pruning test * chore(deps): exempt tokenx from knip for the externalized server bundle
112 lines
17 KiB
Markdown
112 lines
17 KiB
Markdown
# ADR 0002: Agent AI SDK Adoption
|
|
|
|
## Status
|
|
|
|
Proposed
|
|
|
|
## Context
|
|
|
|
The `/agent` workspace (and the builder's in-resume assistant) is a single `ToolLoopAgent` loop in `packages/api/src/features/agent/service.ts` with four tools, resumable Redis streams, Postgres-persisted `UIMessage` history, and snapshot-based undo.
|
|
|
|
LangGraph was evaluated and rejected: everything it would add (loop control, persistence, resumability, interrupts) is already built here on the AI SDK. This work instead adopts more of the AI SDK v7 surface (`ai@7.0.66`, already installed) to fix real defects and add the features it makes cheap, plus shadcn's `@shadcn/helpers` ai-sdk fixture library (a deterministic scripted `ChatTransport` for `useChat` — a testing harness, not a runtime HITL implementation) for backend-free UI tests.
|
|
|
|
Defects driving this:
|
|
|
|
1. Unbounded context growth — every turn replays the full thread, including every full-resume `read_resume` dump; long threads will exceed model context and permanently break.
|
|
2. Stale-resume editing — `apply_resume_patch` returns only metadata; subsequent JSON Patch array indexes can silently target wrong items after removes/moves.
|
|
3. Crash-unsafe runs — patches commit immediately but the assistant message persists only in `onFinish`; process death orphans applied edits; a stuck `activeRunId` (no reaper; `activeRunStartedAt` written, never read) permanently CONFLICTs the thread.
|
|
4. Client-trusted cancellation — `messages.stop` persists a client-authored `partialMessage`.
|
|
5. Weak validation — `isUiMessage` checks three fields; arbitrary `parts` get persisted and replayed.
|
|
6. No run guards — no timeout, `maxRetries`, or `maxOutputTokens` on the run.
|
|
7. Invisible tool activity — `tool-read_resume`/`read_attachment`/`web_search`/`dynamic-tool` parts render as nothing; the patch card is a raw `JSON.stringify` dump; zero token-usage visibility.
|
|
8. Latent bug found during design: on an `ask_user_question` continuation, `toUIMessageStream({originalMessages})` continues the existing assistant message (same `uiMessage.id`) but `persistMessage` is insert-only, producing duplicate rows sharing one message id. Fixed by the Phase 1 upsert; regression-tested.
|
|
|
|
Verified API surface from published `ai@7.0.66` and `@ai-sdk/react@4.0.69` typings and docs: `prepareStep` and `pruneMessages`; tool-level `needsApproval`, call-level `toolApproval`, `experimental_toolApprovalSecret` (HMAC-signs approval requests); UIMessage tool-part states `approval-requested`/`approval-responded`/`output-denied`; `lastAssistantMessageIsCompleteWithApprovalResponses`; `useChat().addToolApprovalResponse`; `validateUIMessages`/`safeValidateUIMessages`; `repairToolCall`; tool `inputExamples` plus `addToolInputExamplesMiddleware`; `messageMetadata` on `toUIMessageStream` (plus `messageMetadataSchema` on `useChat`); `onStepEnd`; `ToolLoopAgentSettings` extends `LanguageModelCallOptions` so `timeout`/`maxOutputTokens`/`maxRetries` are valid in the constructor; `smoothStream` passed as `agent.stream({ experimental_transform })`; `useChat` `throttle`; `isStepCount`; `InferAgentUIMessage`; `LanguageModelUsage` with cache read/write detail; `totalUsage` on finish events.
|
|
|
|
Constraints the implementation must respect:
|
|
|
|
- `getAgentModel` silently switches to the OpenAI Responses API for direct-OpenAI web-search models — every model-facing change is QA'd on both a Responses and a chat-completions provider.
|
|
- Abort reasons must be `DOMException(label, "AbortError")`; `AbortSignal.timeout()`/`AbortSignal.any()` produce `TimeoutError` and would crash the resumable-stream pump, so the run timeout uses `setTimeout` plus `controller.abort(abortReason("RUN_TIMEOUT"))`.
|
|
- The `activeRunControllers` map is single-process; multi-process abort (Redis pub/sub) is explicitly out of scope.
|
|
- `service.test.ts` mocks the whole `"ai"` module with a closed factory and scripts DB queries positionally; the test harness is migrated first (Phase 1.0).
|
|
- Shared contracts live in `packages/ai` (runtime-universal per ADR 0001): zod-only runtime, `import type` from `"ai"` only, with `ai` added to devDependencies.
|
|
- No new environment variables anywhere in this plan (the approval secret derives from `ENCRYPTION_SECRET`), so `turbo.json` `globalEnv` stays untouched.
|
|
- `resumeService.patchInTransaction` already accepts `expectedUpdatedAt` and throws `RESUME_VERSION_CONFLICT`; the agent tool just passes it.
|
|
- Legacy persisted rows must never hard-fail: validation happens only at the network boundary (schema-less), and the metadata schema is loose and optional.
|
|
- Lingui macros for all new user-facing copy; Base UI `render` prop (no Radix `asChild`); `cn` from `@reactive-resume/utils/style`; toasts via `toast.add`.
|
|
|
|
## Decision
|
|
|
|
Adopt the AI SDK v7 surface in three phases on top of the existing loop, rather than introducing a graph framework.
|
|
|
|
### Phase 1 — correctness
|
|
|
|
Order: 1.0 → 1.1 → 1.2 → 1.3 → 1.4 → 1.5 → 1.6 (spine: 1.3 → 1.4 → 1.5; 1.6 last because it changes model-visible behavior).
|
|
|
|
- **1.0 Test-harness prep (no behavior change).** Replace the closed `vi.mock("ai")` factory with a spread-actual factory so pure helpers (`isStepCount`, `safeValidateUIMessages`, `pruneMessages`, `JsonToSseTransformStream`) stay real and only the scripted seams (`convertToModelMessages`, `ToolLoopAgent`) stay mocked. Adopt the rule that new DB queries on the send path go into separate injectable modules (the `runs.ts` pattern) and are module-mocked wholesale. Gate: suite green with zero source changes.
|
|
- **1.1 Run guards, snapshot consistency, misc.** `createAgent` adds `maxOutputTokens: 8_192`, `maxRetries: 2`, `timeout: 120_000` (per provider request). Whole-run wall clock: after run claim, `setTimeout(() => controller.abort(abortReason("RUN_TIMEOUT")), 600_000)`, handle stored in a module map keyed by runId and cleared in `cleanupActiveRun`. `applyResumePatch` passes `expectedUpdatedAt` to `patchInTransaction` and rethrows `RESUME_VERSION_CONFLICT` as a recoverable tool error. `readAttachment` uses a named `MAX_ATTACHMENT_TEXT_CHARS` constant.
|
|
- **1.2 Real message validation at the send boundary.** Keep `isUiMessage` as the sync oRPC gate; inside `messages.send`, run `safeValidateUIMessages({ messages: [input.message] })` before `claimActiveAgentRun` and return BAD_REQUEST on failure. Deliberately schema-less so provider-echoed parts pass and replayed history is never re-validated.
|
|
- **1.3 Crash-safe incremental persistence (draft row).** New `messages-persistence.ts` (injectable DB): `applyStepToUiMessage` (pure fold), `insertDraftAssistantMessage`, `upsertAssistantUiMessage` (by row id, then by `uiMessage->>'id'`, then insert), `deleteDraftIfEmpty`. The service pre-generates the response message id, inserts a draft before `agent.stream`, folds and upserts on `onStepEnd` (≤30 writes/run), and upserts the SDK's authoritative message with `completed|canceled` in `onFinish`. `apply_resume_patch` threads `toolCallId` through and sets `agentAction.messageId` at INSERT time. The upsert also fixes the duplicate-row continuation bug (defect 8).
|
|
- **1.4 Server-side cancellation.** `messages.stop` drops `partialMessage` persistence; the body aborts with `abortReason("USER_STOPPED")`, clears the timeout handle, and clears the run claim. Partial content persists server-side via `onFinish({isAborted: true})`. The router keeps `partialMessage` in the input schema for one release (deprecated, ignored).
|
|
- **1.5 Stale-run reaper.** `STALE_AGENT_RUN_TTL_MS = 15 * 60_000` (greater than the run timeout, so live runs always die by their own timeout first; deliberately TTL-only and multi-replica-safe). Wired at server boot, lazily in `messages.send` before the CONFLICT throw, and at the top of `threads.get`. Reap conditionally clears run columns and flips `streaming` draft rows to `canceled`, appending synthetic `tool-apply_resume_patch` parts rebuilt from action rows so replayed history stays provider-valid. Applied actions stay applied — they are real committed edits, individually revertable.
|
|
- **1.6 Context growth: fresh-document patch output plus pruning.** `applyResumePatch` returns the full post-patch document plus `changedPaths`, with a tool description telling the model to base further patches on it. New pure `context.ts`: `AGENT_CONTEXT_TOKEN_BUDGET = 40_000`, `estimateTokenCount` (chars/4), `pruneAgentModelContext`. `createAgent` gains `prepareStep` wiring so pruning runs every loop step.
|
|
|
|
Pruning tiers (all pure, wired via `prepareStep`):
|
|
|
|
- Tier 0, always: supersede resume snapshots — every `read_resume` result and every patch result's embedded `resume` except the last in the conversation becomes a stub note. A stale snapshot is actively harmful (shifted indexes), so this runs even in short threads; exactly one full snapshot survives, positioned where the model last acted.
|
|
- Tier 1, over budget: strip reasoning from all but the last assistant message.
|
|
- Tier 2, still over: collapse oldest tool call/result pairs (never orphan one side — several BYOK gateways reject unpaired tool messages) into one-line stubs, excluding the last assistant message, unresolved question/approval parts, and the surviving snapshot.
|
|
- Tier 3, last resort: attachment parts on non-latest user messages become stubs teaching the `read_attachment` recovery path.
|
|
- Never pruned: instructions, latest user message, last assistant message, unresolved interactive parts.
|
|
|
|
Full-document patch output was chosen over a `prepareStep`-injected "current resume" reminder because a reminder re-sends a snapshot every step, defeats provider prompt caching, and splits authority into two places; a summarized view invites hallucination and forces a `read_resume` round-trip per edit against a 30-step cap. Output plus Tier 0 bounds cost: N patches do not produce N surviving snapshots.
|
|
|
|
### Phase 2 — human-in-the-loop approvals
|
|
|
|
Order: 2.1 contracts → 2.2 migration/endpoint → 2.3 tool+secret → 2.4 merge/continuation → 2.5 UI → 2.6 tests.
|
|
|
|
- **2.1 Shared typed tool contracts in `packages/ai`.** New `agent-tool-contracts.ts` with zod schemas for tool inputs/outputs and a loose all-optional message metadata schema, plus `AgentTools`/`AgentUIMessage` types. `"ai"` goes in devDependencies, `import type` only; zod stays the only runtime import.
|
|
- **2.2 Migration plus thread setting.** `reviewPatches: boolean, default false` on `agentThread` (the plan's only migration), a `threads.update` procedure, and the flag in `toThreadSummary`. Auto-apply stays the default.
|
|
- **2.3 `needsApproval` plus HMAC secret.** `getAgentToolApprovalSecret()` derives `sha256(ENCRYPTION_SECRET + ":agent-tool-approval")` — domain-separated from the AES key, no new env var, and deterministic so a signature minted at halt verifies at continuation even across a restart. `buildAgentTools` gains `requirePatchApproval`; `createAgent` reads `thread.reviewPatches` and passes `experimental_toolApprovalSecret`.
|
|
- **2.4 Merge generalization plus continuation.** Extract the tool-response merge into `messages-merge.ts` as `mergeClientToolResponses`, handling both `ask_user_question` outputs and `approval-responded` parts in one pass. `{0,0}` maps to BAD_REQUEST, `{0,>0}` to CONFLICT("already handled") before claiming a run.
|
|
- **2.5 Client: approval UI, composed auto-send, `useConfirm`.** `sendAutomaticallyWhen` composes `lastAssistantMessageIsCompleteWithToolCalls` with `lastAssistantMessageIsCompleteWithApprovalResponses`. New `patch-approval-card.tsx` (web feature, not `packages/ui`) renders `approval-requested` with Approve/Deny plus optional reason, and `output-denied` as a muted declined card. A "Review edits" toggle lands in the thread menu. `window.confirm` is replaced with the existing `useConfirm()`.
|
|
- **2.6 Approval-flow tests with `@shadcn/helpers`.** Component tests for the card, plus a small harness wiring `useChat` to the scripted transport to exercise the composed auto-send and approval state machine with no backend. `AgentChat` is not refactored to accept a transport prop just for tests.
|
|
|
|
Approval flow end-to-end: the halt leaves no active run (identical lifecycle to today's `ask_user_question` halt); the client resubmits the last assistant message byte-for-byte through the existing transport; the merge copies approval-response fields onto the stored, signed request part matched by `toolCallId` plus approval id, so a client cannot substitute a forged request; the continuation run replays history, the fresh agent derives the same secret, the signature verifies, and the SDK executes (or denies) the call, streaming into the same message id via the 1.3 upsert — which is load-bearing, so 1.3 ships before 2.4. Idempotency ladder: UI disables buttons on state flip → concurrent sends race the atomic `claimActiveAgentRun` → post-completion resubmits match `alreadyResolved` and CONFLICT before any run is claimed → conflicting approved values are BAD_REQUEST.
|
|
|
|
### Phase 3 — visibility and polish
|
|
|
|
- **3.1 Usage metadata.** `messageMetadata` on `toUIMessageStream` records `{usage, model}` on finish; it round-trips in the `uiMessage` jsonb with no migration. The client renders a muted per-message token footer and a per-thread aggregate; legacy rows without metadata render fine.
|
|
- **3.2 Render the invisible parts plus a real patch card.** New generic `tool-part-card.tsx` (collapsed icon/label/state card with expandable input/output) for `read_resume`/`read_attachment`/`web_search`/`dynamic-tool`; the patch card gains human-readable operation rows with raw JSON demoted to a nested `details` block; consecutive `source-url` parts group into one sources block.
|
|
- **3.3 Streaming performance.** `useChat({throttle: 50})`, memoized markdown/message components, and `smoothStream({chunking: "word"})` via `experimental_transform`.
|
|
- **3.4 Tool-input robustness.** `inputExamples` on `apply_resume_patch` plus `addToolInputExamplesMiddleware`; a `repairToolCall` callback (`repair.ts`) that runs `jsonrepair`, strips `/data` prefixes and section shortcuts, and re-validates against the shared schema, falling back to the SDK re-ask on `null`. `normalizeAgentResumePatchOperations` stays as last-line defense; the instructions blob shrinks accordingly.
|
|
- **3.5 Structured logging plus provider options.** One JSON line per step (`agent.step`) and per tool execution (`agent.tool`) — greppable structured console, no OpenTelemetry dependency. Optionally `anthropic.cacheControl: ephemeral` on the instructions block.
|
|
|
|
### Crash-safety design: draft-row upsert over reconciliation-on-read
|
|
|
|
The failure that matters is a patch committed plus process death before `onFinish`. Reconciliation would synthesize an assistant message from action rows, but a synthesized message has no model-authored text and no tool call/result pairing that replays validly. The draft row records the real transcript step-by-step (≤30 single-row jsonb upserts per run); a patch executes within a step, so the orphan window shrinks from "entire run" to milliseconds. It also subsumes server-side cancellation partials, `agentAction.messageId` at INSERT time, and the duplicate-row continuation bug. The reaper covers the remaining sliver by appending synthetic tool parts from action rows during reap.
|
|
|
|
## Consequences
|
|
|
|
- Long threads stop growing without bound; exactly one resume snapshot survives in model context, and patch results carry the fresh document so array indexes never go stale.
|
|
- Runs are bounded (per-request timeout, whole-run wall clock, output-token cap, retry cap) and crash-safe (draft-row persistence, boot/lazy/read-path reaping); cancellation is server-authored.
|
|
- Users can opt threads into edit review; approvals are HMAC-signed server-side and survive restarts.
|
|
- Tool activity, token usage, and patch contents become visible in the UI; streaming is smoother and cheaper to render.
|
|
- The send path gains real message validation without breaking legacy rows.
|
|
- Test coverage moves toward pure, mock-free modules; the `"ai"` mock keeps real helpers.
|
|
- An approval continuation costs a second `send` against the in-process rate limit (20/min) — acceptable, noted for tuning.
|
|
|
|
Out of scope, documented as follow-ups: multi-process cancellation via Redis pub/sub; OpenTelemetry; removing the deprecated `partialMessage` input (next release); the legacy non-agent `chat()` in `ai/service.ts`; thread-title generation via a cheap model call.
|
|
|
|
## Rejected Alternatives
|
|
|
|
LangGraph: rejected — loop control, persistence, resumability, and interrupts are already built here on the AI SDK; a graph framework would add a dependency and a second orchestration model without removing any existing code.
|
|
|
|
`prepareStep`-injected "current resume" reminder instead of full-document patch output: rejected — re-sends a snapshot every step, defeats provider prompt caching, and splits document authority.
|
|
|
|
Reconciliation-on-read instead of draft-row persistence: rejected — synthesized messages replay invalidly against providers that require call/result pairing.
|
|
|
|
Random per-boot approval secret: rejected — it would strand pending approvals across restarts; the deterministic derivation from `ENCRYPTION_SECRET` keeps signatures verifiable.
|
|
|
|
New environment variable for the approval secret: rejected — derivation avoids env, `turbo.json`, and deployment churn.
|