Ryan/documenso
1
0
Fork 1
mirror of https://github.com/documenso/documenso.git synced 2026-07-24 00:43:40 +10:00
Code Issues Packages Projects Releases Wiki Activity

chore: merge main, resolve biome formatting conflicts

Browse Source 
Merge origin/main into feat/external-2fa-codes. Resolve formatting
conflicts caused by biome rollout; preserve both feature streams:
PR's external 2FA token + signing-session 2FA proof additions plus
main's RateLimit/RecipientExpired/signingReminders/date-auto-insert.

In complete-document-with-token.ts, drop the duplicate early
field-fetching block introduced when main moved that logic later
with date auto-insert support; keep the EXTERNAL_TWO_FACTOR_AUTH
check using derivedRecipientActionAuth.
This commit is contained in:
ephraimduncan
2026-05-12 11:46:11 +00:00
parent 9194884fbe
commit 138d663c25
1959 changed files with 93488 additions and 47038 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/plans/bright-emerald-flower-bullmq-background-jobs.md
+239
View File
@@ -0,0 +1,239 @@
---
date: 2026-03-26
title: Bullmq Background Jobs
---
## Context
The codebase has a well-designed background job provider abstraction (`BaseJobProvider`) with two existing implementations:
- **InngestJobProvider** — cloud/SaaS provider, externally hosted
- **LocalJobProvider** — database-backed (Postgres via Prisma), uses HTTP self-calls to dispatch
The goal is to add a third provider backed by a proper job queue library for self-hosted deployments that need more reliability than the Local provider offers.
### Current Architecture
All code lives in `packages/lib/jobs/`:
- `client/base.ts` — Abstract `BaseJobProvider` with 4 methods: `defineJob()`, `triggerJob()`, `getApiHandler()`, `startCron()`
- `client/client.ts` — `JobClient` facade, selects provider via `NEXT_PRIVATE_JOBS_PROVIDER` env var
- `client/inngest.ts` — Inngest implementation
- `client/local.ts` — Local/Postgres implementation
- `client/_internal/job.ts` — Core types: `JobDefinition`, `JobRunIO`, `SimpleTriggerJobOptions`
- `definitions/` — 19 job definitions (15 event-triggered, 4 cron)
The `JobRunIO` interface provided to handlers includes:
- `runTask(cacheKey, callback)` — idempotent task execution (cached via `BackgroundJobTask` table)
- `triggerJob(cacheKey, options)` — chain jobs from within handlers
- `wait(cacheKey, ms)` — delay/sleep (not implemented in Local provider)
- `logger` — structured logging
### Local Provider Limitations
The current Local provider has several issues that motivate this work:
1. `io.wait()` throws "Not implemented"
2. HTTP self-call with 150ms fire-and-forget `Promise.race` is fragile
3. No concurrency control — jobs run in the web server process
4. No real retry backoff (immediate re-dispatch)
5. No monitoring/visibility into job status
6. Jobs compete for resources with HTTP request handling
---
## Provider Evaluation
Three alternatives were evaluated against the existing provider interface and project requirements.
### BullMQ (Redis-backed) — Recommended
| Attribute | Detail |
| ------------------- | -------------------------- |
| Backend | Redis 7.x |
| npm downloads/month | ~15M |
| TypeScript | Native |
| Delayed jobs | Yes (ms precision) |
| Cron/repeatable | Yes (`upsertJobScheduler`) |
| Retries + backoff | Yes (exponential, custom) |
| Concurrency control | Yes (per-worker) |
| Rate limiting | Yes (per-queue, per-group) |
| Dashboard | Bull Board (mature) |
| New infrastructure | Yes — Redis required |
**Why BullMQ**: Most mature and widely-adopted Node.js queue. Native delayed jobs solve the `io.wait()` gap. Redis is purpose-built for queue workloads and keeps Postgres clean for application data. Bull Board gives immediate operational visibility. The provider abstraction already exists so wrapping BullMQ is straightforward.
**Trade-off**: Requires Redis, which is additional infrastructure. However, Redis is a single Docker Compose service or a free Upstash tier, and the operational benefit is significant.
### pg-boss (PostgreSQL-backed) — Strong Alternative
| Attribute | Detail |
| ------------------- | ----------------------------- |
| Backend | PostgreSQL (existing) |
| npm downloads/month | ~1.4M |
| TypeScript | Native |
| Delayed jobs | Yes (`startAfter`) |
| Cron/repeatable | Yes (`schedule()`) |
| New infrastructure | No — reuses existing Postgres |
**Why it could work**: Zero new infrastructure since the project already uses Postgres. API maps well to existing patterns.
**Why it's second choice**: Polling-based (no LISTEN/NOTIFY), adds write amplification to the primary database, smaller ecosystem, no dashboard. At scale, queue operations on the primary database become a concern.
### Graphile Worker (PostgreSQL-backed) — Less Suitable
Uses LISTEN/NOTIFY for instant pickup but has a file-based task convention and separate schema that don't mesh well with the existing Prisma-centric architecture. Would require more adapter work.
### Improving the Local Provider — Not Recommended
Fixing the Local provider's issues (wait support, replacing HTTP self-calls, adding concurrency control, backoff) essentially means rebuilding a queue library from scratch with less robustness and no community maintenance.
---
## Recommendation
**Proceed with BullMQ.** It's the most capable option, maps cleanly to the existing provider interface, and is the standard choice for production Node.js applications. Redis is lightweight infrastructure with managed options available at every cloud provider.
**If Redis is a hard blocker**, pg-boss is the clear fallback — but the plan below assumes BullMQ.
---
## Implementation Plan
### Phase 1: BullMQ Provider Core
**File: `packages/lib/jobs/client/bullmq.ts`**
Create `BullMQJobProvider extends BaseJobProvider` with singleton pattern matching the existing providers.
Key implementation details:
1. **Constructor / `getInstance()`**
- Initialize a Redis `IORedis` connection using new env var: `NEXT_PRIVATE_REDIS_URL`
- Create a single `Queue` instance for dispatching jobs, using `NEXT_PRIVATE_REDIS_PREFIX` as the BullMQ `prefix` option (defaults to `documenso` if unset). This namespaces all Redis keys so multiple environments (worktrees, branches, developers) sharing the same Redis instance don't collide.
- Create a single `Worker` instance for processing jobs (in-process, same prefix)
- Store job definitions in a `_jobDefinitions` record (same pattern as Local provider)
2. **`defineJob()`**
- Store definition in `_jobDefinitions` keyed by ID
- If the definition has a `trigger.cron`, register it via `queue.upsertJobScheduler()` with the cron expression
3. **`triggerJob(options)`**
- Find eligible definitions by `trigger.name` (same lookup as Local provider)
- For each, call `queue.add(jobDefinitionId, payload)` with appropriate options
- Support `options.id` for deduplication via BullMQ's `jobId` option
4. **`getApiHandler()`**
- Return a minimal health-check / queue-status handler. Unlike the Local provider, BullMQ workers don't need an HTTP endpoint to receive jobs — they pull from Redis directly. The API handler can return queue metrics for monitoring.
5. **`startCron()`**
- No-op — cron is handled by BullMQ's `upsertJobScheduler` registered during `defineJob()`
6. **Worker setup**
- Single worker processes all job types by dispatching to the correct handler from `_jobDefinitions`
- Configure concurrency with a default of 10 (overridable via `NEXT_PRIVATE_BULLMQ_CONCURRENCY` env var for those who need to tune it)
- Configure retry with exponential backoff: `backoff: { type: 'exponential', delay: 1000 }`
- Default 3 retries (matching current Local provider behavior)
7. **`createJobRunIO(jobId)`** — Implement `JobRunIO`:
- `runTask()`: Reuse the existing `BackgroundJobTask` Prisma table for idempotent task tracking (same pattern as Local provider)
- `triggerJob()`: Delegate to `this.triggerJob()`
- `wait()`: Throw "Not implemented" (same as Local provider). No handler uses `io.wait()` so this has zero impact
- `logger`: Same console-based logger pattern as Local provider
### Phase 2: Provider Registration
**File: `packages/lib/jobs/client/client.ts`**
Add `'bullmq'` case to the provider match:
```typescript
this._provider = match(env('NEXT_PRIVATE_JOBS_PROVIDER'))
.with('inngest', () => InngestJobProvider.getInstance())
.with('bullmq', () => BullMQJobProvider.getInstance())
.otherwise(() => LocalJobProvider.getInstance());
```
**File: `packages/tsconfig/process-env.d.ts`**
Add `'bullmq'` to the `NEXT_PRIVATE_JOBS_PROVIDER` type union and add Redis env var types:
```typescript
NEXT_PRIVATE_JOBS_PROVIDER?: 'inngest' | 'local' | 'bullmq';
NEXT_PRIVATE_REDIS_URL?: string;
NEXT_PRIVATE_REDIS_PREFIX?: string;
NEXT_PRIVATE_BULLMQ_CONCURRENCY?: string;
```
**File: `.env.example`**
Add Redis configuration examples:
```env
NEXT_PRIVATE_JOBS_PROVIDER="local" # Options: local, inngest, bullmq
NEXT_PRIVATE_REDIS_URL="redis://localhost:63790"
NEXT_PRIVATE_REDIS_PREFIX="documenso" # Namespace for Redis keys (useful when sharing a Redis instance)
```
**File: `turbo.json`**
Add `NEXT_PRIVATE_REDIS_URL`, `NEXT_PRIVATE_REDIS_PREFIX`, and `NEXT_PRIVATE_BULLMQ_CONCURRENCY` to the env vars list for cache invalidation.
### Phase 3: Infrastructure & Dependencies
**File: `packages/lib/package.json`**
Add dependencies:
- `bullmq` — the queue library
- `ioredis` — Redis client (peer dependency of BullMQ, but explicit is better)
**File: `docker-compose.yml` (or equivalent)**
Add Redis service for local development:
```yaml
redis:
image: redis:7-alpine
ports:
- '6379:6379'
```
### Phase 4: Optional Enhancements
These are not required for the initial implementation but worth considering for follow-up:
1. **Bull Board integration** — Add a `/api/jobs/dashboard` route that serves Bull Board UI for monitoring. Gate behind an admin auth check.
2. **Separate worker process** — Add an `apps/worker` entry point that runs BullMQ workers without the web server, for deployments that want to isolate job processing from request handling.
3. **Graceful shutdown** — Register `SIGTERM`/`SIGINT` handlers to call `worker.close()` and `queue.close()` for clean shutdown.
4. **BackgroundJob table integration** — Optionally continue writing to the `BackgroundJob` Prisma table for audit/history, using BullMQ events (`completed`, `failed`) to update status. This preserves the existing database-level visibility.
---
## Files to Create/Modify
| File | Action | Description |
| ------------------------------------ | ---------- | ---------------------------------------- |
| `packages/lib/jobs/client/bullmq.ts` | **Create** | BullMQ provider implementation |
| `packages/lib/jobs/client/client.ts` | Modify | Add `'bullmq'` provider case |
| `packages/tsconfig/process-env.d.ts` | Modify | Add type for `'bullmq'` + Redis env vars |
| `.env.example` | Modify | Add Redis config example |
| `turbo.json` | Modify | Add Redis env var to cache keys |
| `packages/lib/package.json` | Modify | Add `bullmq` + `ioredis` dependencies |
| `docker-compose.yml` | Modify | Add Redis service |
---
## Open Questions
1. **Should the BullMQ provider also write to the `BackgroundJob` Prisma table?** This would maintain audit history and allow existing admin tooling to query job status. Trade-off is dual-write complexity.
2. **Redis connection resilience**: Should the provider gracefully degrade if Redis is unavailable (e.g., fall back to Local provider), or fail hard? Failing hard is simpler and more predictable.
## Resolved Questions
- **`io.wait()`**: Not a concern. Only Inngest implements it (via `step.sleep`), the Local provider throws "Not implemented", and no job handler calls `io.wait()`. The BullMQ provider can throw "Not implemented" identically to the Local provider.
.agents/plans/calm-violet-tide-envelope-expiration.md
+519
View File
@@ -0,0 +1,519 @@
---
date: 2026-02-10
title: Envelope Expiration
---
## Summary
Envelopes (documents sent for signing) should automatically expire after a configurable period, preventing recipients from completing stale documents. Expiration is tracked **per-recipient** — when a recipient's signing window lapses, the document owner is notified and can resend (extending the deadline) or cancel. The document itself stays PENDING so other recipients can continue signing.
**Settings cascade**: Organisation → Team → Document (each level can override the prior).
**Default**: 1 month from when the envelope is sent (transitions to PENDING).
---
## 1. Database Schema Changes
### 1.1 Expiration period data shape
Store expiration as a structured JSON object rather than an enum or raw milliseconds. This avoids the enum treadmill (adding `FOUR_MONTHS` later requires a migration) while keeping values validated and meaningful.
**Zod schema** (defined in `packages/lib/constants/envelope-expiration.ts`):
```typescript
export const ZEnvelopeExpirationPeriod = z.union([
z.object({ unit: z.enum(['day', 'week', 'month', 'year']), amount: z.number().int().min(1) }),
z.object({ disabled: z.literal(true) }),
]);
export type TEnvelopeExpirationPeriod = z.infer<typeof ZEnvelopeExpirationPeriod>;
```
Semantics:
- `null` on `DocumentMeta` / `TeamGlobalSettings` = inherit from parent
- `{ disabled: true }` = explicitly never expires
- `{ unit: 'month', amount: 1 }` = expires in 1 month
No Prisma enum is needed — the period is stored as `Json?` on the relevant models (see sections 1.3 and 1.4).
### 1.2 Add expiration fields to `Recipient`
```prisma
model Recipient {
// ... existing fields
expiresAt DateTime?
expirationNotifiedAt DateTime? // null = not yet notified; set when owner notification sent
@@index([expiresAt])
}
```
`expiresAt` is a computed timestamp set when the envelope transitions to PENDING (at send time). It is calculated from the effective expiration period. Storing the concrete timestamp rather than a relative duration means:
- Sweep queries are simple (`WHERE expiresAt <= NOW() AND expirationNotifiedAt IS NULL`)
- No need to re-resolve the settings cascade at query time
- The sender can see the exact deadline in the UI
- The index on `expiresAt` ensures the expiration sweep query is efficient
`expirationNotifiedAt` tracks whether the owner has already been notified about this recipient's expiration, making the notification job idempotent.
### 1.3 Add expiration period to settings models
**OrganisationGlobalSettings** (JSON, application-level default):
```prisma
model OrganisationGlobalSettings {
// ... existing fields
envelopeExpirationPeriod Json?
}
```
Prisma `@default` doesn't work for `Json` columns, so the application-level default (`{ unit: 'month', amount: 1 }`) is applied in `extractDerivedTeamSettings` / `extractDerivedDocumentMeta` when the value is null. The migration should backfill existing rows with `{ "unit": "month", "amount": 1 }`.
**TeamGlobalSettings** (nullable, null = inherit from org):
```prisma
model TeamGlobalSettings {
// ... existing fields
envelopeExpirationPeriod Json?
}
```
### 1.4 Add expiration period to DocumentMeta
This allows per-document override during the document editing flow:
```prisma
model DocumentMeta {
// ... existing fields
envelopeExpirationPeriod Json?
}
```
When null on DocumentMeta, the resolved team/org setting is used at send time. Validated at write time using `ZEnvelopeExpirationPeriod.nullable()`.
**Important**: `envelopeExpirationPeriod` on `DocumentMeta` is a user-facing preference that may be set during the draft editing flow. It does NOT determine the final expiration — that is resolved at send time (see section 2.3). The value stored here is just the user's selection in the document editor.
---
## 2. Expiration Period Resolution
### 2.1 Duration mapping
Add to `packages/lib/constants/envelope-expiration.ts` alongside the Zod schema:
```typescript
import { Duration } from 'luxon';
const UNIT_TO_LUXON_KEY: Record<TEnvelopeExpirationPeriod['unit'], string> = {
day: 'days',
week: 'weeks',
month: 'months',
year: 'years',
};
export const DEFAULT_ENVELOPE_EXPIRATION_PERIOD: TEnvelopeExpirationPeriod = {
unit: 'month',
amount: 1,
};
export const getEnvelopeExpirationDuration = (period: TEnvelopeExpirationPeriod): Duration => {
return Duration.fromObject({ [UNIT_TO_LUXON_KEY[period.unit]]: period.amount });
};
```
### 2.2 Settings cascade integration
`extractDerivedTeamSettings()` in `packages/lib/utils/teams.ts` needs **no code changes** — it iterates `Object.keys(derivedSettings)` and overrides with non-null team values at runtime. The new `envelopeExpirationPeriod` field on both `OrganisationGlobalSettings` and `TeamGlobalSettings` will be automatically picked up.
Update `extractDerivedDocumentMeta()` in `packages/lib/utils/document.ts` to include the new field:
```typescript
envelopeExpirationPeriod: meta.envelopeExpirationPeriod ?? settings.envelopeExpirationPeriod,
```
### 2.3 Compute `expiresAt` at send time
The expiration period is **locked at send time** — when the envelope transitions to PENDING. The concrete `expiresAt` timestamp is computed for each recipient when the document is actually sent.
In `packages/lib/server-only/document/send-document.ts`:
```typescript
// Resolve effective period: document meta -> team/org settings -> default
const rawPeriod =
envelope.documentMeta?.envelopeExpirationPeriod ?? settings.envelopeExpirationPeriod;
const expiresAt = resolveExpiresAt(rawPeriod);
// Inside the $transaction, for each recipient:
await tx.recipient.updateMany({
where: { envelopeId: envelope.id },
data: { expiresAt },
});
```
### 2.4 Compute `expiresAt` in the direct template flow
`create-document-from-direct-template.ts` creates envelopes directly as PENDING and then calls `sendDocument` afterward. Since `sendDocument` handles setting `expiresAt` on recipients, the direct template flow doesn't need to set it directly — `sendDocument` handles it.
---
## 3. Cron Job Infrastructure (New)
The current job system is purely event-triggered. Inngest natively supports cron-triggered functions, but the local provider (used in dev and by self-hosters who don't want a third-party dependency) has no scheduling capability. This section adds cron support to the local provider to maintain feature parity.
### 3.1 Extend `JobDefinition` with cron support
Add an optional `cron` field to the trigger type in `packages/lib/jobs/client/_internal/job.ts`:
```typescript
export type JobDefinition<Name extends string = string, Schema = any> = {
id: string;
name: string;
version: string;
enabled?: boolean;
optimizeParallelism?: boolean;
trigger: {
name: Name;
schema?: z.ZodType<Schema>;
/** Cron expression (e.g. "* * * * *"). When set, the job runs on a schedule. */
cron?: string;
};
handler: (options: { payload: Schema; io: JobRunIO }) => Promise<Json | void>;
};
```
### 3.2 Inngest provider: wire up native cron
In `packages/lib/jobs/client/inngest.ts`, when defining a function, check for `cron`:
```typescript
defineJob(job) {
if (job.trigger.cron) {
this._functions.push(
this._client.createFunction(
{ id: job.id, name: job.name },
{ cron: job.trigger.cron },
async ({ step, logger }) => {
const io = convertInngestIoToJobRunIo(step, logger, this);
await job.handler({ payload: {} as any, io });
},
),
);
} else {
// Existing event-triggered logic (unchanged)
}
}
```
### 3.3 Local provider: poller + deterministic `BackgroundJob` IDs
Use the existing `BackgroundJob` table for multi-instance dedupe instead of advisory locks. This approach keeps implementation Prisma-only (no raw SQL), works for single-instance and multi-instance deployments, and preserves existing retry/visibility behavior.
**On `defineJob()`**: If the job has a `cron` field, register an in-process scheduler entry and start a lightweight poller (every 30s with jitter).
**Each poll tick**:
1. Evaluate whether the cron schedule has one or more due run slots since the last tick (use a real cron parser, e.g. `cron-parser`)
2. For each due slot, build a deterministic run ID from job ID + scheduled slot time
3. Create a `BackgroundJob` row with that deterministic ID using Prisma
4. If insert succeeds → enqueue via the existing local job pipeline
5. If insert fails with Prisma `P2002` (unique violation) → another node already enqueued that run, skip
### 3.4 Summary of changes to the job system
| File | Change |
| ------------------------------------------- | ---------------------------------------------------------------- |
| `packages/lib/jobs/client/_internal/job.ts` | Add optional `cron` field to `trigger` type |
| `packages/lib/jobs/client/local.ts` | Add cron poller + deterministic `BackgroundJob.id` dedupe |
| `packages/lib/jobs/client/inngest.ts` | Wire up `{ cron: ... }` in `createFunction` for cron jobs |
| `packages/lib/jobs/client/_internal/*` | Add cron helper utilities (`getDueCronSlots`, run ID generation) |
---
## 4. Expiration Processing
### 4.1 Two-job architecture
Expiration uses two jobs: a **sweep dispatcher** that runs on a cron schedule and finds expired recipients, and an **individual notification job** that handles the audit log, owner notification email, and webhook for a single recipient. This separation means:
- The sweep is lightweight and fast (just a query + N job triggers)
- Each recipient's expiration notification is independently retryable
- The individual jobs are idempotent — they check `expirationNotifiedAt IS NULL` before processing
### 4.2 Sweep job: `EXPIRE_RECIPIENTS_SWEEP_JOB`
A cron-triggered job that runs every minute to find and dispatch notifications for expired recipients.
**Definition:** `packages/lib/jobs/definitions/internal/expire-recipients-sweep.ts`
**Handler:** `packages/lib/jobs/definitions/internal/expire-recipients-sweep.handler.ts`
```typescript
const expiredRecipients = await prisma.recipient.findMany({
where: {
expiresAt: { lte: new Date() },
expirationNotifiedAt: null,
signingStatus: { notIn: [SigningStatus.SIGNED, SigningStatus.REJECTED] },
envelope: { status: DocumentStatus.PENDING },
},
select: { id: true },
take: 100,
});
for (const recipient of expiredRecipients) {
await jobs.triggerJob({
name: 'internal.notify-recipient-expired',
payload: { recipientId: recipient.id },
});
}
```
### 4.3 Individual notification job: `NOTIFY_RECIPIENT_EXPIRED_JOB`
An event-triggered job that handles a single recipient's expiration.
**Definition:** `packages/lib/jobs/definitions/internal/notify-recipient-expired.ts`
**Handler:** `packages/lib/jobs/definitions/internal/notify-recipient-expired.handler.ts`
The handler:
1. Fetches the recipient (with guard: `expirationNotifiedAt IS NULL` + not signed/rejected)
2. Sets `recipient.expirationNotifiedAt = now()` (idempotency)
3. Creates audit log entry with `DOCUMENT_RECIPIENT_EXPIRED` type
4. Sends email notification to the **document owner** (inline — no separate email job)
5. The document stays PENDING — the owner decides whether to resend or cancel
### 4.4 Register in job client
Add `EXPIRE_RECIPIENTS_SWEEP_JOB_DEFINITION` and `NOTIFY_RECIPIENT_EXPIRED_JOB_DEFINITION` to the job registry in `packages/lib/jobs/client.ts`.
### 4.5 Email template: Recipient Expired
Target the **document owner**:
- Subject: `Signing window expired for "{recipientName}" on "{documentTitle}"`
- Body: "The signing window for {recipientName} ({recipientEmail}) on document {title} has expired. You can resend the document to extend their deadline or cancel the document."
- Include a "View Document" link to the document page in the app
Template files:
- `packages/email/templates/recipient-expired.tsx` — wrapper
- `packages/email/template-components/template-recipient-expired.tsx` — body
### 4.6 Recipient signing guard
In the signing flow, check `recipient.expiresAt` before allowing any signing action. Note that the document stays PENDING even after recipient expiration, so the existing `status !== PENDING` guard does not block expired recipients — an explicit expiration check is required:
```typescript
if (recipient.expiresAt && recipient.expiresAt <= new Date()) {
throw new AppError(AppErrorCode.RECIPIENT_EXPIRED, {
message: 'Recipient signing window has expired',
});
}
```
**Files to update:**
- `packages/lib/server-only/document/complete-document-with-token.ts`
- `packages/lib/server-only/field/sign-field-with-token.ts`
- `packages/lib/server-only/field/remove-signed-field-with-token.ts`
- `packages/lib/server-only/document/reject-document-with-token.ts`
---
## 5. UI Design
### 5.1 Expiration Period Selector Component
Use a number input + unit selector combo. This gives organisations full flexibility to configure any duration without needing schema changes for new options.
**Layout**: A horizontal group with:
- A number `<Input>` (min 1, integer)
- A `<Select>` for the unit (`day`, `week`, `month`, `year`)
- A "Never expires" toggle/checkbox that disables the duration inputs and sets the value to `{ disabled: true }`
At the team level, include an "Inherit from organisation" option that clears the value to `null`.
**Validation**: Use `ZEnvelopeExpirationPeriod` for form validation.
### 5.2 Organisation Settings → Document Preferences
Add a "Default Envelope Expiration" field to the `DocumentPreferencesForm` component. At the org level, there is no "Inherit" option — it must have a concrete value (default: `{ unit: 'month', amount: 1 }`).
### 5.3 Team Settings → Document Preferences
Same field as org, but with the additional "Inherit from organisation" option (stored as `null`).
### 5.4 Document Editor → Settings Step
Add the expiration selector to `packages/ui/primitives/document-flow/add-settings.tsx` inside the "Advanced Options" accordion.
Label: **"Expiration"**
Description: _"How long recipients have to complete this document after it is sent."_
### 5.5 Recipient Signing Page — Expired State
When a recipient visits a signing link for an expired recipient:
- Redirect to `/sign/{token}/expired`
- Show a clear, non-alarming message: "Your signing window has expired. Please contact the sender for a new invitation."
- Do not show the signing form or fields
- The `isExpired` flag in `get-envelope-for-recipient-signing.ts` is derived from `recipient.expiresAt`
### 5.6 Embed Signing — Expired State
Embed signing routes handle recipient expiration by throwing `embed-recipient-expired`:
- `apps/remix/app/routes/embed+/_v0+/sign.$token.tsx` — both V1 and V2 loaders check expiration
- The embed error boundary renders an `EmbedRecipientExpired` component
- Direct templates (`direct.$token.tsx`) create fresh recipients so `isExpired` is always `false`
---
## 6. API / TRPC Changes
### 6.1 Update settings mutation schemas
- `packages/trpc/server/organisation-router/update-organisation-settings.types.ts` — add `envelopeExpirationPeriod: ZEnvelopeExpirationPeriod` (non-nullable at org level)
- `packages/trpc/server/team-router/update-team-settings.types.ts` — add `envelopeExpirationPeriod: ZEnvelopeExpirationPeriod.nullable()` (null = inherit from org)
### 6.2 Update document mutation schemas
- `packages/lib/types/document-meta.ts` — add `envelopeExpirationPeriod: ZEnvelopeExpirationPeriod.nullable()` to the meta schema
- `packages/trpc/server/document-router/create-document.types.ts` — include in meta
- `packages/trpc/server/document-router/update-document.types.ts` — include in meta
- `packages/trpc/server/document-router/distribute-document.types.ts` — include in meta
### 6.3 Expose `expiresAt` in recipient responses
Ensure `expiresAt` and `expirationNotifiedAt` are returned when fetching recipients/documents so the UI can display expiration status.
### 6.4 Webhook / API schema updates
- Recipient schema includes `expiresAt` and `expirationNotifiedAt` fields (replacing the old `expired` field)
- Update `packages/api/v1/schema.ts`, webhook payload types, zapier integration, and sample data generators
---
## 7. Edge Cases & Considerations
### 7.1 Already-sent documents
The migration should NOT retroactively expire existing recipients. `expiresAt` will be null for all existing recipients, meaning they never expire (backward-compatible).
### 7.2 Re-sending / redistributing
When `redistribute` is called on a PENDING document, `expiresAt` should be refreshed on all eligible recipients. Redistributing signals active intent, so the clock should restart.
**Implementation**: `resendDocument` refreshes `recipient.expiresAt` for all recipients that haven't signed/rejected yet.
### 7.3 Multi-recipient partial expiration
If some recipients have signed and others expire, the document stays PENDING. This is the key advantage over document-level expiration — the owner can resend to extend the expired recipients' deadlines without affecting those who've already signed.
### 7.4 Partial completion
Partial signatures are preserved. The document is not sealed/completed until all required recipients have signed (or the owner cancels).
### 7.5 Timezone handling
`expiresAt` is stored as UTC. Display in the sender's configured timezone.
### 7.6 Race condition: signing at expiration time
The signing guard checks `recipient.expiresAt` in application code before the signing operation. The notification job's guard (`expirationNotifiedAt IS NULL` + `signingStatus NOT IN (SIGNED, REJECTED)`) prevents double-notifications. If a recipient signs just before expiration, the sweep's `signingStatus` filter skips them.
### 7.7 Direct template flow
`create-document-from-direct-template.ts` creates envelopes directly as PENDING then calls `sendDocument`. Since `sendDocument` sets `recipient.expiresAt`, no special handling is needed in the direct template flow.
---
## 8. Migration Plan
1. Add Prisma schema changes (`expiresAt` + `expirationNotifiedAt` on Recipient, `Json?` fields on settings models, index)
2. Generate and run migration
3. Backfill: set `envelopeExpirationPeriod` to `{ "unit": "month", "amount": 1 }` on all existing `OrganisationGlobalSettings` rows
4. No backfill on `Recipient.expiresAt` — existing recipients keep null (never expire)
5. Deploy backend changes (jobs, guards, email template)
6. Deploy frontend changes (settings UI, document editor, signing page, embeds)
---
## 9. Files to Create or Modify
### New Files
- `packages/lib/constants/envelope-expiration.ts` — `ZEnvelopeExpirationPeriod` schema, types, `DEFAULT_ENVELOPE_EXPIRATION_PERIOD`, `getEnvelopeExpirationDuration()`, `resolveExpiresAt()` helper
- `packages/lib/jobs/definitions/internal/expire-recipients-sweep.ts` — cron sweep job definition
- `packages/lib/jobs/definitions/internal/expire-recipients-sweep.handler.ts` — cron sweep handler
- `packages/lib/jobs/definitions/internal/notify-recipient-expired.ts` — individual notification job definition
- `packages/lib/jobs/definitions/internal/notify-recipient-expired.handler.ts` — notification handler (includes inline email sending)
- `packages/email/templates/recipient-expired.tsx` — email template wrapper
- `packages/email/template-components/template-recipient-expired.tsx` — email template body
- `apps/remix/app/components/embed/embed-recipient-expired.tsx` — embed expired component
### Modified Files
**Job system (cron infrastructure):**
- `packages/lib/jobs/client/_internal/job.ts` — add optional `cron` field to `trigger` type
- `packages/lib/jobs/client/local.ts` — add cron poller + deterministic `BackgroundJob.id` dedupe
- `packages/lib/jobs/client/inngest.ts` — wire up `{ cron: ... }` in `createFunction`
- `packages/lib/jobs/client/_internal/*` — add cron helper utilities (slot calc + run ID)
- `packages/lib/jobs/client.ts` — register new jobs
**Schema & data layer:**
- `packages/prisma/schema.prisma` — model changes + index
- `packages/lib/utils/document.ts` — `extractDerivedDocumentMeta` (add `envelopeExpirationPeriod`)
- `packages/lib/server-only/document/send-document.ts` — resolve settings + compute and set `recipient.expiresAt`
- `packages/lib/server-only/template/create-document-from-direct-template.ts` — no changes (sendDocument handles it)
- `packages/lib/server-only/document/resend-document.ts` — refresh `recipient.expiresAt` on redistribute
- `packages/lib/server-only/document/complete-document-with-token.ts` — recipient expiration guard
- `packages/lib/server-only/field/sign-field-with-token.ts` — recipient expiration guard
- `packages/lib/server-only/field/remove-signed-field-with-token.ts` — recipient expiration guard
- `packages/lib/server-only/document/reject-document-with-token.ts` — recipient expiration guard
**Error handling:**
- `packages/lib/errors/app-error.ts` — add `RECIPIENT_EXPIRED` error code
**Audit logs:**
- `packages/lib/types/document-audit-logs.ts` — add `DOCUMENT_RECIPIENT_EXPIRED` type with `recipientEmail`/`recipientName` data fields
- `packages/lib/utils/document-audit-logs.ts` — add human-readable rendering for `DOCUMENT_RECIPIENT_EXPIRED`
**Signing page:**
- `packages/lib/server-only/envelope/get-envelope-for-recipient-signing.ts` — derive `isExpired` from `recipient.expiresAt`
- `apps/remix/app/routes/_recipient+/sign.$token+/_index.tsx` — keep redirect to expired page using `isExpired`
**Embeds:**
- `apps/remix/app/routes/embed+/_v0+/sign.$token.tsx` — check recipient expiration in V1/V2 loaders
- `apps/remix/app/routes/embed+/_v0+/_layout.tsx` — handle `embed-recipient-expired` in error boundary
**Webhook / API:**
- `packages/lib/types/recipient.ts` — add `expiresAt`/`expirationNotifiedAt` to recipient type
- `packages/lib/types/webhook-payload.ts` — add `expiresAt`/`expirationNotifiedAt` to webhook recipient
- `packages/lib/server-only/webhooks/trigger/generate-sample-data.ts` — update sample data
- `packages/lib/server-only/webhooks/zapier/list-documents.ts` — update zapier recipient shape
- `packages/api/v1/schema.ts` — add `expiresAt` to API recipient schema
**TRPC / settings:**
- `packages/trpc/server/organisation-router/update-organisation-settings.types.ts`
- `packages/trpc/server/team-router/update-team-settings.types.ts`
- `packages/lib/types/document-meta.ts`
**UI:**
- `apps/remix/app/components/forms/document-preferences-form.tsx` — add expiration period picker
- `packages/ui/primitives/document-flow/add-settings.tsx` — add expiration field
- `packages/ui/primitives/document-flow/add-settings.types.ts` — add to schema
.agents/plans/dark-hazel-beam-swap-subscription-between-orgs.md
+151
View File
@@ -0,0 +1,151 @@
---
date: 2026-03-04
title: Swap Subscription Between Orgs
---
## Overview
Add the ability for admins to move a subscription (and its associated Stripe customerId) from one organisation to another, when viewing a user in the admin panel. The target org must be owned by the same user and must be on the free plan (no existing active subscription).
## Context & Data Model
- `Organisation` has a 1:1 optional `Subscription` and a `customerId` (Stripe customer ID, `@unique`)
- `Organisation` has a 1:1 `OrganisationClaim` that tracks entitlements (team count, member count, feature flags)
- `Subscription` also stores a redundant `customerId` and has `organisationId` (`@unique`)
- When a subscription is removed from an org, its `OrganisationClaim` should be reset to the FREE claim
- Relationship chain: `User --owns--> Organisation --has--> Subscription + OrganisationClaim`
## Constraints
- **paid → free only**: The target org must NOT have an active subscription (status ACTIVE or PAST_DUE). It must be on the free plan.
- **same owner**: Both source and target orgs must be owned by the same user (the user being viewed).
- The `customerId` must move with the subscription to the target org (cleared from source, set on target).
- The Stripe subscription object itself is NOT modified — only the DB-level mapping changes. The Stripe customer stays the same; we just reassociate it to a different org.
## Implementation Plan
### 1. Backend: TRPC Admin Route
**Files to create:**
- `packages/trpc/server/admin-router/swap-organisation-subscription.types.ts`
- `packages/trpc/server/admin-router/swap-organisation-subscription.ts`
**Request schema (`ZSwapOrganisationSubscriptionRequestSchema`):**
```ts
z.object({
sourceOrganisationId: z.string(),
targetOrganisationId: z.string(),
});
```
**Response schema:** `z.void()`
**Route logic (in a single `prisma.$transaction`):**
1. Fetch source org with `subscription` + `organisationClaim`
2. Fetch target org with `subscription` + `organisationClaim`
3. Validate:
- Source org has an active subscription (status `ACTIVE` or `PAST_DUE`)
- Target org does NOT have an active subscription (no subscription record, or status `INACTIVE`)
- Both orgs have the same `ownerUserId`
4. In a transaction:
a. Clear `customerId` on source org (set to `null`)
b. Set `customerId` on target org to the source's `customerId`
c. Move the `Subscription` record: update `organisationId` to target org ID
d. Copy the source org's `OrganisationClaim` entitlements to the target org's `OrganisationClaim` (`originalSubscriptionClaimId`, `teamCount`, `memberCount`, `envelopeItemCount`, `flags`)
e. Reset the source org's `OrganisationClaim` to the FREE claim (using `createOrganisationClaimUpsertData(internalClaims[INTERNAL_CLAIM_ID.FREE])` pattern from `on-subscription-deleted.ts`)
**Note on ordering:** Because `Organisation.customerId` is `@unique`, we must clear the source first, then set the target — or do both in a transaction that handles the constraint. Prisma transactions handle this correctly as they apply all writes atomically.
**Register the route:**
- Import in `packages/trpc/server/admin-router/router.ts`
- Add under `organisation` as `swapSubscription`
- Call path: `trpc.admin.organisation.swapSubscription`
### 2. Frontend: Dialog Component
**File to create:**
- `apps/remix/app/components/dialogs/admin-swap-subscription-dialog.tsx`
**Props:**
```ts
type AdminSwapSubscriptionDialogProps = {
trigger?: React.ReactNode;
sourceOrganisationId: string;
sourceOrganisationName: string;
userId: number;
} & Omit<DialogPrimitive.DialogProps, 'children'>;
```
**Dialog behavior:**
1. Opens when the trigger is clicked (from the organisations table actions dropdown)
2. Fetches the user's owned orgs via `trpc.admin.organisation.find.useQuery({ ownerUserId: userId })`
3. Filters to only show orgs that are on the free plan (no active subscription) and excludes the source org
4. Displays a select dropdown to pick the target org
5. Shows a warning alert: "This will move the subscription from {source} to {target}. The source organisation will be reset to the free plan."
6. On submit, calls `trpc.admin.organisation.swapSubscription.useMutation()`
7. On success, shows a toast, invalidates relevant queries, and closes the dialog
**UI layout (following existing dialog patterns like `admin-organisation-create-dialog.tsx`):**
- `DialogHeader` with title "Move Subscription" and description
- A select dropdown listing eligible target orgs (name + url)
- An `Alert` explaining what will happen
- `DialogFooter` with Cancel + "Move Subscription" buttons (submit button uses `loading` prop)
### 3. Frontend: Wire into the Organisations Table
**File to modify:**
- `apps/remix/app/components/tables/admin-organisations-table.tsx`
**Changes:**
- Import the `AdminSwapSubscriptionDialog`
- Add a new prop `ownerUserId?: number` to `AdminOrganisationsTableOptions` (needed so the dialog can query other owned orgs)
- Add a new dropdown menu item in the actions column: "Move Subscription" with `ArrowRightLeftIcon` from lucide
- Only render this item when the org row has an active subscription (`subscription?.status === 'ACTIVE' || subscription?.status === 'PAST_DUE'`)
- The menu item renders inside `AdminSwapSubscriptionDialog` with `trigger` prop as the menu item
### 4. Frontend: Pass userId from User Detail Page
**File to modify:**
- `apps/remix/app/routes/_authenticated+/admin+/users.$id.tsx`
**Changes:**
- Pass `ownerUserId={user.id}` to `<AdminOrganisationsTable>` so it can forward this to the swap dialog
## File Change Summary
| File | Action | Description |
| --------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------- |
| `packages/trpc/server/admin-router/swap-organisation-subscription.types.ts` | **Create** | Request/response Zod schemas + TS types |
| `packages/trpc/server/admin-router/swap-organisation-subscription.ts` | **Create** | Admin mutation with prisma transaction |
| `packages/trpc/server/admin-router/router.ts` | **Modify** | Register route at `organisation.swapSubscription` |
| `apps/remix/app/components/dialogs/admin-swap-subscription-dialog.tsx` | **Create** | Dialog for selecting target org |
| `apps/remix/app/components/tables/admin-organisations-table.tsx` | **Modify** | Add "Move Subscription" action + accept `ownerUserId` prop |
| `apps/remix/app/routes/_authenticated+/admin+/users.$id.tsx` | **Modify** | Pass `ownerUserId={user.id}` to table |
## Edge Cases & Considerations
1. **Stripe customer stays the same**: The Stripe subscription is tied to a Stripe customer. We move the `customerId` to the target org, so webhook lookups (`findFirst where customerId`) will correctly resolve to the target org going forward.
2. **`@unique` constraint on `Organisation.customerId`**: Must clear source before setting target within the transaction. Prisma interactive transactions handle this correctly.
3. **`@unique` constraint on `Subscription.organisationId`**: Since the target org should not have a subscription record, updating the existing subscription's `organisationId` to the target should work. If the target has an INACTIVE subscription record, we need to delete it first.
4. **Target org has INACTIVE subscription**: The target org might have a stale INACTIVE subscription from a previous cancellation. In this case, delete the target's old subscription record before moving the source's subscription over.
5. **Seat-based plans**: If the subscription is seat-based, the Stripe quantity may not match the target org's member count. Consider calling `syncMemberCountWithStripeSeatPlan` after the swap as a post-transaction step.
6. **OrganisationClaim transfer**: Copy `originalSubscriptionClaimId`, `teamCount`, `memberCount`, `envelopeItemCount`, and `flags` from source claim to target claim. Reset source claim to FREE.
7. **No Stripe API calls needed**: This is purely a DB-level reassociation. The Stripe subscription, customer, and payment method all remain unchanged.
.agents/plans/fresh-gold-rock-cert-page-width-mismatch.md
+168
View File
@@ -0,0 +1,168 @@
---
date: 2026-02-11
title: Cert Page Width Mismatch
---
## Problem
Certificate and audit log pages are generated with hardcoded A4 dimensions (`PDF_SIZE_A4_72PPI`: 595×842) regardless of the actual document page sizes. When the source document uses a different page size (e.g., Letter, Legal, or custom dimensions), the certificate/audit log pages end up with a different width than the document pages. This causes problems with courts that expect uniform page dimensions throughout a PDF.
**Both width and height must match** the last page of the document so the entire PDF prints uniformly.
**Root cause**: In `seal-document.handler.ts` (lines 186-187), the certificate payload always uses:
```ts
pageWidth: PDF_SIZE_A4_72PPI.width, // 595
pageHeight: PDF_SIZE_A4_72PPI.height, // 842
```
These hardcoded values flow into `generateCertificatePdf`, `generateAuditLogPdf`, `renderCertificate`, and `renderAuditLogs` — all of which use `pageWidth`/`pageHeight` to set Konva stage dimensions and layout content.
## Key Files
| File | Role |
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| `packages/lib/jobs/definitions/internal/seal-document.handler.ts` | Orchestrates sealing; passes page dimensions to cert/audit generators |
| `packages/lib/constants/pdf.ts` | Defines `PDF_SIZE_A4_72PPI` (595×842) |
| `packages/lib/server-only/pdf/generate-certificate-pdf.ts` | Generates certificate PDF; accepts `pageWidth`/`pageHeight` |
| `packages/lib/server-only/pdf/generate-audit-log-pdf.ts` | Generates audit log PDF; accepts `pageWidth`/`pageHeight` |
| `packages/lib/server-only/pdf/render-certificate.ts` | Renders certificate pages via Konva; uses `pageWidth`/`pageHeight` for stage + layout |
| `packages/lib/server-only/pdf/render-audit-logs.ts` | Renders audit log pages via Konva; uses `pageWidth`/`pageHeight` for stage + layout |
| `packages/lib/server-only/pdf/get-page-size.ts` | Existing utility — extend with `@libpdf/core` version |
| `packages/trpc/server/document-router/download-document-certificate.ts` | Standalone certificate download (also hardcodes A4) |
| `packages/trpc/server/document-router/download-document-audit-logs.ts` | Standalone audit log download (also hardcodes A4) |
## Architecture
### Current Flow
1. **One cert PDF + one audit log PDF** generated per envelope with hardcoded A4 dims
2. Both appended to **every** envelope item (document) via `decorateAndSignPdf` → `pdfDoc.copyPagesFrom()`
3. The audit log is envelope-level (all recipients, all events across all docs) — one per envelope, not per document
### Multi-Document Envelopes
- V1 envelopes: single document only
- V2 envelopes: support multiple documents (envelope items)
- Each envelope item gets both cert + audit log pages appended to it
- If documents have different page sizes → need size-matched cert/audit for each
### Reading Page Dimensions (`@libpdf/core` only)
Use `@libpdf/core`'s `PDF` class — NOT `@cantoo/pdf-lib`:
```ts
const pdfDoc = await PDF.load(pdfData);
const lastPage = pdfDoc.getPage(pdfDoc.getPageCount() - 1);
const { width, height } = lastPage; // e.g. 612, 792 for Letter
```
Already used this way in `seal-document.handler.ts` lines 403-410 for V2 field insertion.
"Last page" = last page of the original document, before cert/audit pages are appended.
### Content Layout Adaptation
Both renderers already handle variable dimensions gracefully:
- **Width**: `render-certificate.ts:713` / `render-audit-logs.ts:588` — `Math.min(pageWidth - minimumMargin * 2, contentMaxWidth)` with `contentMaxWidth = 768`. Wider pages get more margin, narrower pages tighter margins.
- **Height**: Both renderers paginate content into pages using `groupRowsIntoPages()` which respects `pageHeight` via `maxTableHeight = pageHeight - pageTopMargin - pageBottomMargin`. Shorter pages just mean more pages; taller pages fit more rows per page.
### Playwright PDF Path — Out of Scope
The `NEXT_PRIVATE_USE_PLAYWRIGHT_PDF` toggle enables a deprecated Playwright-based PDF generation path (`get-certificate-pdf.ts`, `get-audit-logs-pdf.ts`) that also hardcodes `format: 'A4'` in `page.pdf()`. This path is **not being updated** as part of this fix:
- Both files are marked `@deprecated`
- The Konva-based path is the default and recommended path
- The Playwright path is behind a feature flag and will be removed
No changes needed. Add a code comment noting the A4 limitation if the Playwright path is ever re-enabled.
## Plan
### 1. Extend `get-page-size.ts` with `@libpdf/core` utility
Add a `getLastPageDimensions` function to the existing `packages/lib/server-only/pdf/get-page-size.ts` file. This consolidates page-size logic in one place (the file already has the legacy `@cantoo/pdf-lib` version).
```ts
export const getLastPageDimensions = (pdfDoc: PDF): { width: number; height: number } => {
const lastPage = pdfDoc.getPage(pdfDoc.getPageCount() - 1);
const width = Math.round(lastPage.width);
const height = Math.round(lastPage.height);
if (width < MIN_CERT_PAGE_WIDTH || height < MIN_CERT_PAGE_HEIGHT) {
return { width: PDF_SIZE_A4_72PPI.width, height: PDF_SIZE_A4_72PPI.height };
}
return { width, height };
};
```
**Dimension rounding**: `Math.round()` both width and height. PDF points at 72ppi are typically whole numbers; rounding avoids spurious float-precision mismatches (e.g., 612.0 vs 612.00001) that would cause unnecessary duplicate cert/audit PDF generation.
**Minimum page dimensions**: Enforce a minimum threshold (e.g., 300pt for both width and height). If either dimension falls below the minimum, fall back to A4 (595×842). The certificate and audit log renderers have headers, table rows, margins, and QR codes that require a minimum viable area.
### 2. Read last page dimensions from each envelope item's PDF
In `seal-document.handler.ts`, before generating cert/audit PDFs:
- For each `envelopeItem`, load the PDF and read the **last page's width and height** using `getLastPageDimensions`
- Use `PDF.load()` then pass the loaded doc to the utility
**Resealing consideration**: When `isResealing` is true, envelope items are remapped to use `initialData` (lines 152-158) before this point. Page-size extraction must operate on the same data source that `decorateAndSignPdf` will use. Since the `envelopeItems` array is already remapped by the time we read dimensions, reading from `envelopeItem.documentData` will naturally give the correct (initial) data. No special handling needed beyond ensuring the dimension read happens **after** the resealing remap.
### 3. Generate cert/audit PDFs per unique page size
Current flow generates one cert + one audit log doc per envelope. Change to:
1. Collect `{ width, height }` of the last page for each envelope item
2. Deduplicate by `"${width}x${height}"` key (using the already-rounded integers)
3. For each unique size, generate cert PDF and audit log PDF with those dimensions
4. Store in a `Map<string, { certificateDoc, auditLogDoc }>` keyed by `"${width}x${height}"`
For the common single-document case, this is one generation — same perf as today.
### 4. Thread the correct docs into `decorateAndSignPdf`
In the envelope item loop, look up the item's last-page dimensions in the map and pass the matching cert/audit docs. Signature of `decorateAndSignPdf` doesn't change — it still receives a single `certificateDoc` and `auditLogDoc`, just the right ones per item.
### 5. Update standalone download routes
`download-document-certificate.ts` and `download-document-audit-logs.ts` also hardcode A4:
- Both routes have `documentId` which maps to a specific envelope item
- Fetch **that specific document's** PDF data, load it, read last page width + height via `getLastPageDimensions`
- Pass `{ pageWidth, pageHeight }` to the generator
- This ensures the standalone download matches the dimensions the user would see in the sealed PDF for that document
### 6. Edge cases
| Scenario | Behavior |
| --------------------------------------- | ------------------------------------------------------------------------------------------- |
| Mixed page sizes within one PDF | Use last page's dimensions (per spec) |
| Page dimensions below minimum threshold | Fall back to A4 (595×842) |
| Landscape pages | width/height just swap roles; renderers adapt via `Math.min()` capping. No special handling |
| Fallback if page dims unreadable | Default to A4 (595×842) |
| Resealing | Dimensions read after `initialData` remap — correct source automatically |
| Playwright PDF path enabled | Remains A4 — out of scope, deprecated |
| Single-doc envelope (most common) | One generation, same perf as today |
| Multi-doc envelope, same page sizes | Dedup key matches → one generation |
| Multi-doc envelope, different sizes | One generation per unique size |
### 7. Tests
- Add assertion-based E2E test (no visual regression / reference images needed)
- Seal a Letter-size (612×792) PDF through the full flow
- Load the sealed output and assert all pages (document + cert + audit) have matching width/height
- Can be added to `envelope-alignment.spec.ts` or as a new focused test
## Implementation Steps
1. **Extend `get-page-size.ts`** — add `getLastPageDimensions(pdfDoc: PDF): { width: number; height: number }` using `@libpdf/core`, with `Math.round()` and minimum dimension enforcement
2. **In `seal-document.handler.ts`**:
a. After the resealing remap (line ~159), load each envelope item's PDF via `PDF.load()` and collect last page `{ width, height }` using `getLastPageDimensions`
b. Deduplicate by `"${width}x${height}"` key
c. Generate cert/audit PDFs per unique size (parallel via `Promise.all`)
d. In envelope item loop, look up matching cert/audit doc by size key
3. **Fix `download-document-certificate.ts`** — load the specific document's PDF, read last page dims via `getLastPageDimensions`, pass to generator
4. **Fix `download-document-audit-logs.ts`** — same as above, using the specific `documentId`'s PDF
5. **Add E2E test** — assertion-based test with a Letter-size document verifying all page dimensions match after sealing
.agents/plans/happy-teal-tree-platform-signing-page-branding.md
+138
View File
@@ -0,0 +1,138 @@
---
date: 2026-05-06
title: Platform Signing Page Branding
---
## What
Platform-plan organisations (and their teams) can customise the **non-embed
signing pages** (`/sign/:token`, `/d/:token`, and the sibling
complete/expired/rejected/waiting pages) with:
- Six brand colour tokens (background, foreground, primary, primary-foreground,
border, ring) plus a border-radius length.
- A free-text custom CSS block (up to 256 KB).
Settings live on `OrganisationGlobalSettings` and `TeamGlobalSettings`. Teams
inherit from the org via the existing `brandingEnabled === null` mechanism.
## Why
- Embed customers already have white-label CSS; Platform customers want the
same coverage on direct signing URLs that they iframe or link to.
- Persisting on org/team (not per envelope) means it's set-and-forget.
- Sanitising **on save** lets us inline the verbatim string at SSR — no
per-render parsing cost, no `<style>.innerHTML` injection on the client.
- Reusing the existing `embedSigningWhiteLabel` claim flag keeps "if you can
white-label an embed, you can white-label this" as one decision.
## How
### Storage (`packages/prisma/schema.prisma`)
Two new fields on each settings model. No new tables.
| Field | Org type | Team type |
| ---------------- | ------------------ | ------------------ |
| `brandingColors` | `Json?` (nullable) | `Json?` (nullable) |
| `brandingCss` | `String @default("")` | `String?` |
Colours are validated against `ZCssVarsSchema`. The team's `null` means
"inherit"; an empty colour object is collapsed to `null` server-side so a
team toggling `brandingEnabled = true` without filling in colours doesn't
silently override the org's defaults with nothing.
### Sanitiser (`packages/lib/utils/sanitize-branding-css.ts`)
PostCSS + `postcss-selector-parser`. Runs on save only.
- Drops selectors containing `::before`/`::after`/`::backdrop`/`::marker` or
the universal `*`.
- Drops integrity-breaking properties (`display`, `position`, `transform`,
layout-affecting dimensions, text-hiding properties).
- Drops declaration values containing `url(`, `expression(`, `@import`,
`javascript:`.
- Strips `!important`.
- Allows `@media` only; drops other at-rules.
- **Does not** rewrite selectors. Scoping happens at render time via native
CSS nesting under `.documenso-branded { ... }`.
- Final-pass tripwire: if a literal `</style` somehow survives serialization,
reject the entire output. PostCSS already escapes `<` to `\3c` whenever it
would form `</...`; the explicit check is belt-and-braces in case a future
serializer regresses.
- Returns `{ css, warnings[] }`. Warnings are surfaced in the UI.
Border-radius is the only token interpolated raw into a `<style>` block; it
is regex-validated (`CSS_LENGTH_REGEX`) at both the Zod schema and the
runtime `toNativeCssVars` call. Belt-and-braces against schema drift.
### Render (`apps/remix/app/components/general/recipient-branding.tsx`)
Each recipient loader calls `loadRecipientBrandingByTeamId` and threads the
payload through to `<RecipientBranding>`, which emits a single
nonce-attributed `<style>`:
```
.documenso-branded {
--background: ...; ...
<user css>
}
```
Native CSS nesting expands user rules under the wrapper. The body class is
applied unconditionally to recipient routes in `root.tsx` via `useMatches()`
so portaled Radix content (dialogs, popovers, tooltips, dropdowns) inherits
the scope.
CSP for recipient routes already supports `<style nonce>`; no policy
changes needed.
### Plan gate
`organisationClaim.flags.embedSigningWhiteLabel || !IS_BILLING_ENABLED()`.
Self-hosted instances always allow. The outer paywall for logo/URL/details
stays on `allowCustomBranding` (Team plan and up); only the new
colour/CSS section is Platform-only.
### UI (`apps/remix/app/components/forms/branding-preferences-form.tsx`)
Extends the existing branding form. Six `<ColorPicker showHex>` (rewritten
to use the native `<input type="color">` instead of `react-colorful`, which
was removed) in a 2-col grid, plus a free-text radius input and an
`<Accordion>` revealing a mono `<Textarea>`. Defaults come from
`packages/lib/constants/theme.ts` (light-mode hex mirror of `theme.css`).
Warnings from the sanitiser are surfaced in an `<Alert variant="warning">`
after save, and the `brandingCss` textarea is re-synced from the persisted
value so the user sees exactly what was stored. Other fields are
deliberately NOT reset on settings refetch — that would clobber in-flight
edits.
### TRPC
`update-organisation-settings` and `update-team-settings` accept the new
fields, run them through `sanitizeBrandingCss` + `normalizeBrandingColors`,
and return any sanitiser warnings to the client. The team route treats
`null` as "inherit"; an empty post-sanitisation string is collapsed to
`null` (team) so an empty override doesn't mask the org's CSS.
## Known accepted limitations
- The sanitiser does not prevent hostile-but-syntactically-valid CSS
(`color: transparent`, low-contrast values, etc.). The customer is
branding **their own** signing pages — we focus on integrity (no
overlay/hide/exfiltrate), not aesthetic policing.
- User rules targeting `body`/`html`/`:root` no-op once nested under the
wrapper class. Documented for users.
- CSS nesting baseline is Chrome 120+ / Firefox 117+ / Safari 16.5+.
Acceptable for the Platform-tier audience.
- No automated `theme.css` ↔ `theme.ts` sync check; fat comment in
`theme.ts` reminds devs to update both.
- Per-section team inherit is coarse — `brandingEnabled = null` inherits
everything from the org. Per-field inherit toggles are deferred.
## Out of scope
Live preview, embed-route sanitiser unification, email/PDF certificate
branding, custom font upload, the full ~30 colour tokens in the picker UI,
wiring `hidePoweredBy` through to the actual footer.
.agents/plans/pale-lemon-rock-pnpm-migration.md
+312
View File
@@ -0,0 +1,312 @@
---
date: 2026-02-26
title: pnpm Migration
---
## Overview
Migrate from npm to pnpm to eliminate dependency resolution duplication issues that cause bundler problems. The current npm workspace setup results in nested `node_modules` copies that don't deduplicate reliably, requiring manual hoisting and `npm dedupe` cycles. pnpm's content-addressable store and strict symlink structure eliminates this class of problem entirely.
## Current State
- **Package manager:** npm@10.7.0 with `legacy-peer-deps=true` and `prefer-dedupe=true`
- **Workspaces:** 18 total (3 apps, 15 packages) declared in root `package.json` `workspaces` field
- **Lockfile:** `package-lock.json`
- **Patches:** `patch-package` with one patch (`@ai-sdk+google-vertex+3.0.81`)
- **Overrides:** `lodash`, `pdfjs-dist`, `typescript`, `zod` in root `package.json`
- **Syncpack:** installed but unconfigured (no `.syncpackrc`)
- **Heavy duplication:** `zod` in 7 workspaces, `ts-pattern` in 9, `luxon` in 8, `react` in 6, etc.
- **Docker:** `turbo prune` → `npm ci` → `npm ci --only=production` multi-stage build
- **Existing Dockerfiles:** `docker/Dockerfile` (primary, npm), `apps/remix/Dockerfile.pnpm` (already exists, needs review)
## Migration Steps
### Phase 1: Core Migration
#### Step 1: Enable pnpm via corepack
```bash
corepack enable pnpm
corepack use pnpm@latest
```
This adds a `"packageManager"` field to root `package.json` (e.g. `"packageManager": "pnpm@10.x.x"`). Remove the existing `"engines"` npm constraint if present.
#### Step 2: Create `pnpm-workspace.yaml`
```yaml
packages:
- apps/*
- packages/*
```
Remove the `"workspaces"` field from root `package.json` — pnpm uses `pnpm-workspace.yaml` instead.
#### Step 3: Convert lockfile
```bash
pnpm import
```
This reads `package-lock.json` and generates `pnpm-lock.yaml`. After verifying, delete `package-lock.json`.
#### Step 4: Create `.npmrc` for pnpm
Replace the current `.npmrc` contents. The existing settings (`legacy-peer-deps=true`, `prefer-dedupe=true`) are npm-specific.
```ini
# Hoist packages that expect to be resolvable from any workspace.
# Start strict, add patterns here only as needed.
shamefully-hoist=true
```
> **Note:** `shamefully-hoist=true` is the pragmatic starting point. It mimics npm's flat `node_modules` layout. Once the migration is stable, this can be tightened to `hoist-pattern[]` entries for specific packages that need it, moving toward pnpm's strict isolation model.
#### Step 5: Clean install
```bash
rm -rf node_modules apps/*/node_modules packages/*/node_modules
pnpm install
```
Verify the install completes without errors. Fix any peer dependency warnings — pnpm is stricter than npm with `legacy-peer-deps=true`.
#### Step 6: Convert `overrides` to `pnpm.overrides`
In root `package.json`, move the `overrides` block under `pnpm`:
```json
{
"pnpm": {
"overrides": {
"lodash": "4.17.23",
"pdfjs-dist": "5.4.296",
"typescript": "5.6.2",
"zod": "^3.25.76"
}
}
}
```
Remove the top-level `overrides` field (that's npm-specific).
#### Step 7: Convert patch-package to pnpm patches
pnpm has native patching. Convert the existing `@ai-sdk+google-vertex+3.0.81` patch:
```bash
# Remove patch-package dependency and postinstall script
# Then use pnpm's native patching:
pnpm patch @ai-sdk/google-vertex@3.0.81
# Apply the same changes from patches/@ai-sdk+google-vertex+3.0.81.patch
pnpm patch-commit <temp-dir>
```
This adds a `pnpm.patchedDependencies` entry to root `package.json` and stores the patch in a `patches/` directory (pnpm's own format). Remove `patch-package` from dependencies and the `postinstall` script.
### Phase 2: Catalogs
#### Step 8: Identify catalog candidates
Packages duplicated across 3+ workspaces are prime candidates:
| Package | Workspaces | Catalog? |
| ----------------------------------------------- | ---------- | ---------------------- |
| `zod` | 7 | Yes |
| `ts-pattern` | 9 | Yes |
| `luxon` | 8 | Yes |
| `react` / `react-dom` | 6 / 3 | Yes |
| `typescript` | 6 | Yes |
| `nanoid` | 4 | Yes |
| `@lingui/core` / `macro` / `react` | 2-3 | Yes |
| `@simplewebauthn/server` | 3 | Yes |
| `@documenso/*` (internal) | varies | No (use `workspace:*`) |
| `@aws-sdk/*` | 2 | Yes |
| `hono` | 2 | Yes |
| `posthog-node` / `posthog-js` | 2 | Yes |
| `remeda` | 3 | Yes |
| `@tanstack/react-query` | 2 | Yes |
| `@trpc/*` | 2 | Yes |
| `superjson` | 2 | Yes |
| `kysely` | 2 | Yes |
| `@types/react` / `@types/node` / `@types/luxon` | 3-4 | Yes |
#### Step 9: Define catalogs in `pnpm-workspace.yaml`
```yaml
packages:
- apps/*
- packages/*
catalog:
# Core
react: ^18
react-dom: ^18
typescript: 5.6.2
zod: ^3.25.76
# Shared utilities
ts-pattern: <current-version>
luxon: ^3.7.2
nanoid: ^5.1.6
remeda: <current-version>
superjson: ^2.2.5
# AWS
'@aws-sdk/client-s3': ^3.998.0
'@aws-sdk/client-sesv2': ^3.998.0
'@aws-sdk/cloudfront-signer': ^3.998.0
'@aws-sdk/s3-request-presigner': ^3.998.0
'@aws-sdk/signature-v4-crt': ^3.998.0
# Framework
hono: 4.12.2
'@tanstack/react-query': <current-version>
'@trpc/client': 11.8.1
'@trpc/react-query': 11.8.1
'@trpc/server': 11.8.1
# i18n
'@lingui/core': ^5.6.0
'@lingui/macro': ^5.6.0
'@lingui/react': ^5.6.0
# Auth
'@simplewebauthn/server': <current-version>
# Observability
posthog-node: 4.18.0
posthog-js: <current-version>
# Database
kysely: <current-version>
'@prisma/client': ^6.19.0
prisma: ^6.19.0
# Types
'@types/react': <current-version>
'@types/react-dom': <current-version>
'@types/node': ^20
'@types/luxon': <current-version>
```
#### Step 10: Update workspace `package.json` files
Replace pinned versions with `catalog:` protocol for all cataloged packages:
```json
{
"dependencies": {
"zod": "catalog:",
"ts-pattern": "catalog:",
"luxon": "catalog:"
}
}
```
This is a mechanical find-and-replace across all workspace `package.json` files.
### Phase 3: Internal Workspace References
#### Step 11: Convert internal references to `workspace:*`
All `@documenso/*` internal package references currently use `"*"`. Convert to pnpm's `workspace:*` protocol:
```json
{
"dependencies": {
"@documenso/lib": "workspace:*",
"@documenso/prisma": "workspace:*"
}
}
```
This makes the workspace resolution explicit and prevents accidental resolution to a published version.
### Phase 4: Docker & CI
#### Step 12: Update primary Dockerfile (`docker/Dockerfile`)
The existing multi-stage build needs to change:
1. **base:** Add pnpm — `corepack enable pnpm` or install via `npm i -g pnpm`
2. **builder:** `turbo prune` still works with pnpm. Output structure is the same.
3. **installer:**
- Replace `npm ci` with `pnpm install --frozen-lockfile`
- Copy `pnpm-lock.yaml` and `pnpm-workspace.yaml` instead of `package-lock.json`
- Remove `patch-package` from postinstall (pnpm patches are applied natively)
4. **runner:**
- Replace `npm ci --only=production` with `pnpm install --frozen-lockfile --prod`
- Or use `pnpm deploy` for standalone output (copies only production deps to a flat directory)
Review `apps/remix/Dockerfile.pnpm` — it already exists and may have most of this solved. Reconcile with the primary `docker/Dockerfile`.
#### Step 13: Update CI workflows
Search for all `npm ci`, `npm install`, `npm run` in CI config files (`.github/workflows/`, etc.) and replace with `pnpm install --frozen-lockfile`, `pnpm run`, etc.
Ensure corepack is enabled in CI runners:
```yaml
- run: corepack enable pnpm
```
#### Step 14: Update turborepo config
Turbo works with pnpm out of the box. The `turbo.json` should not need changes. Verify `turbo prune` generates correct output with pnpm lockfile.
### Phase 5: Cleanup & Tighten
#### Step 15: Remove npm-specific tooling
- Remove `syncpack` (catalogs replace its purpose)
- Remove `patch-package` (pnpm native patches replace it)
- Remove `"workspaces"` from root `package.json` if not already done
- Delete `package-lock.json`
- Update `.gitignore` if needed (pnpm store is outside the repo by default)
#### Step 16: Tighten hoisting (optional, future)
Once stable, replace `shamefully-hoist=true` with targeted hoist patterns:
```ini
shamefully-hoist=false
hoist-pattern[]=*eslint*
hoist-pattern[]=*prettier*
# Add others as discovered
```
This moves toward strict isolation where each package can only import what it declares. Catches phantom dependency issues. Do this incrementally — let the bundler tell you what breaks.
#### Step 17: Remove root-level dependency hoisting
With catalogs and strict resolution, dependencies currently hoisted to root `package.json` for deduplication purposes can be moved back to the workspaces that actually use them. The root `package.json` should only contain tooling deps (`turbo`, `prettier`, `eslint`, etc.) and `pnpm.overrides`.
## Risks and Mitigations
1. **Phantom dependencies surface:** pnpm's strict isolation will expose imports that work today only because npm hoisted them. `shamefully-hoist=true` defers this, but tightening later will reveal them.
- **Mitigation:** Start with `shamefully-hoist=true`. Tighten incrementally after the migration is stable.
2. **Peer dependency strictness:** pnpm enforces peer deps by default. The current `.npmrc` has `legacy-peer-deps=true` which suppresses all peer dep errors.
- **Mitigation:** Run `pnpm install` and address peer dep warnings. Most will be resolvable by adding missing peer deps to the relevant workspace.
3. **Docker build breakage:** The `turbo prune` + `npm ci` pipeline is battle-tested. Switching to pnpm changes the install semantics.
- **Mitigation:** The existing `Dockerfile.pnpm` in `apps/remix/` provides a reference. Test the Docker build in CI before merging.
4. **CI cache invalidation:** Switching lockfiles invalidates all CI dependency caches.
- **Mitigation:** Update cache keys to use `pnpm-lock.yaml` hash. First CI run will be slower, subsequent runs will cache normally.
5. **Turbo + pnpm compatibility:** Turbo has first-class pnpm support, but `turbo prune` output format may differ slightly.
- **Mitigation:** Test `turbo prune --scope=@documenso/remix --docker` and verify output structure before updating Dockerfile.
## Verification Checklist
- [ ] `pnpm install` succeeds with no errors
- [ ] `pnpm run build` succeeds (all workspaces)
- [ ] `pnpm run lint` passes
- [ ] `pnpm run dev` starts correctly
- [ ] Docker build produces a working image
- [ ] E2E tests pass (`pnpm run test:e2e`)
- [ ] No duplicate package copies in `node_modules` for key deps (`zod`, `react`, `typescript`)
- [ ] `pnpm audit` shows same or better results than current npm audit
- [ ] CI pipeline passes end-to-end
.agents/plans/smooth-coral-earth-database-rate-limiting.md
+551
View File
@@ -0,0 +1,551 @@
---
date: 2026-02-19
title: Database Rate Limiting
---
## Summary
Replace the in-memory `hono-rate-limiter` with a database-backed rate limiting system using Prisma and PostgreSQL. The current in-memory approach is ineffective in multi-instance deployments since there are no sticky sessions. The new system uses **bucketed counters** (one row per key/action/time-bucket with atomic increment) to efficiently handle both high-throughput API rate limiting and granular auth/email rate limiting.
### Design Decisions
- **Bucketed counters** over row-per-request: high-throughput consumers would create thousands of rows per minute; bucketed counters reduce this to one row per key per time bucket
- **Fixed time windows**: simpler than sliding windows, the 2x burst-at-boundary scenario is acceptable for rate limiting purposes
- **Dual-key rate limiting**: per-identifier (`max`) and per-IP (`globalMax`) checked independently via separate rows with a `key` prefix (`id:` / `ip:`)
- **Accept slight over-count**: the upsert is atomic (increment + return count in one operation) but concurrent requests near the limit may both see a count just under the threshold before either commits, allowing a slight overshoot
- **Fail-open on errors**: if the rate limit DB query fails, allow the request through rather than blocking legitimate users
- **Prisma upsert** with `{ increment: 1 }` for atomic counter updates, returns the updated row so count check is a single operation
- **Application cron job** for cleanup of expired bucket rows
### Rate Limit Check Flow
```
check({ ip, identifier }) ->
1. Upsert IP row (ip:{ip} / action / bucket) with count + 1, RETURNING count
-> if globalMax is set and count >= globalMax, return { isLimited: true }
2. Upsert identifier row (id:{identifier} / action / bucket) with count + 1, RETURNING count
-> if count >= max, return { isLimited: true }
3. Neither limited -> return { isLimited: false }
```
Each upsert atomically increments and returns the new count in a single operation. Both counters always increment on every check — there's no conditional logic to skip one based on the other. This keeps the implementation simple and avoids read-then-write race conditions. If only IP is provided (API rate limiting), only step 1 runs.
---
## 1. Database Schema
### 1.1 Prisma model
Add to `packages/prisma/schema.prisma` after the `Counter` model:
```prisma
model RateLimit {
key String
action String
bucket DateTime
count Int @default(1)
createdAt DateTime @default(now())
@@id([key, action, bucket])
@@index([createdAt])
}
```
- **Composite primary key** `(key, action, bucket)` serves as both the unique constraint for upserts and the lookup index
- **`key`** is prefixed: `ip:1.2.3.4` or `id:user@example.com`
- **`action`** is the rate limit action name: `auth.forgot-password`, `api.v1`, etc.
- **`bucket`** is the start of the time window, truncated to the window size (e.g., `2026-02-19T10:05:00Z` for a 5-minute bucket)
- **`createdAt` index** is for the cleanup job to efficiently delete old rows
- **`count`** starts at 1 (set by the create side of the upsert)
### 1.2 Migration
Generate with `npx prisma migrate dev --name add-rate-limits`.
---
## 2. Rate Limit Library
### 2.1 Core module
Create `packages/lib/server-only/rate-limit/rate-limit.ts`:
```typescript
type WindowUnit = 's' | 'm' | 'h' | 'd';
type WindowStr = `${number}${WindowUnit}`;
type RateLimitConfig = {
action: string;
max: number;
globalMax?: number;
window: WindowStr;
};
type CheckParams = {
ip: string;
identifier?: string;
};
export const rateLimit = (config: RateLimitConfig) => {
return {
async check(params: CheckParams): Promise<{
isLimited: boolean;
remaining: number;
limit: number;
reset: Date;
}> { ... }
};
};
```
### 2.2 Window parsing and bucket computation
```typescript
const parseWindow = (window: WindowStr): number => {
const value = parseInt(window.slice(0, -1), 10);
const unit = window.slice(-1) as WindowUnit;
const multipliers: Record<WindowUnit, number> = {
s: 1000,
m: 60 * 1000,
h: 60 * 60 * 1000,
d: 24 * 60 * 60 * 1000,
};
return value * multipliers[unit];
};
const getBucket = (windowMs: number): Date => {
const now = Date.now();
return new Date(now - (now % windowMs));
};
```
### 2.3 Check implementation
The `check()` method:
1. Compute the current bucket from the window
2. Compute `reset` as `bucket + windowMs` (the start of the next window)
3. If `globalMax` is set, upsert the IP row and check count
4. If `identifier` is provided, upsert the identifier row and check count
5. Wrap in try/catch — **fail-open** on any database error (log the error, return `{ isLimited: false }`)
Each upsert uses Prisma's `upsert` with `{ increment: 1 }`:
```typescript
const result = await prisma.rateLimit.upsert({
where: {
key_action_bucket: {
key: `ip:${params.ip}`,
action: config.action,
bucket,
},
},
create: {
key: `ip:${params.ip}`,
action: config.action,
bucket,
count: 1,
},
update: {
count: { increment: 1 },
},
});
if (config.globalMax && result.count >= config.globalMax) {
return { isLimited: true, remaining: 0, limit: config.globalMax };
}
```
### 2.4 Rate limit definitions
Create `packages/lib/server-only/rate-limit/rate-limits.ts` with all rate limit instances:
```typescript
// ---- Auth (Tier 1 - Critical, sends emails) ----
export const signupRateLimit = rateLimit({
action: 'auth.signup',
max: 5,
globalMax: 10,
window: '1h',
});
export const forgotPasswordRateLimit = rateLimit({
action: 'auth.forgot-password',
max: 3,
globalMax: 20,
window: '1h',
});
export const resendVerifyEmailRateLimit = rateLimit({
action: 'auth.resend-verify-email',
max: 3,
globalMax: 20,
window: '1h',
});
export const request2FAEmailRateLimit = rateLimit({
action: 'auth.request-2fa-email',
max: 5,
globalMax: 20,
window: '15m',
});
// ---- Auth (Tier 2 - Unauthenticated) ----
export const loginRateLimit = rateLimit({
action: 'auth.login',
max: 10,
globalMax: 50,
window: '15m',
});
export const resetPasswordRateLimit = rateLimit({
action: 'auth.reset-password',
max: 5,
globalMax: 20,
window: '1h',
});
export const verifyEmailRateLimit = rateLimit({
action: 'auth.verify-email',
max: 5,
globalMax: 20,
window: '15m',
});
export const passkeyRateLimit = rateLimit({
action: 'auth.passkey',
max: 10,
globalMax: 50,
window: '15m',
});
export const oauthRateLimit = rateLimit({
action: 'auth.oauth',
max: 10,
globalMax: 50,
window: '15m',
});
export const linkOrgAccountRateLimit = rateLimit({
action: 'auth.link-org-account',
max: 5,
globalMax: 20,
window: '1h',
});
// ---- API (Tier 4 - Standard) ----
export const apiV1RateLimit = rateLimit({
action: 'api.v1',
max: 100,
window: '1m',
});
export const apiV2RateLimit = rateLimit({
action: 'api.v2',
max: 100,
window: '1m',
});
export const apiTrpcRateLimit = rateLimit({
action: 'api.trpc',
max: 100,
window: '1m',
});
export const aiRateLimit = rateLimit({
action: 'api.ai',
max: 3,
window: '1m',
});
export const fileUploadRateLimit = rateLimit({
action: 'api.file-upload',
max: 20,
window: '1m',
});
```
Exact limits are initial values — tune based on observed traffic patterns. These should be easy to adjust.
---
## 3. Integration Points
### 3.1 Hono middleware for API routes
Create a reusable Hono middleware factory in `packages/lib/server-only/rate-limit/rate-limit-middleware.ts` that wraps the `rateLimit` checker into Hono middleware:
```typescript
import { type MiddlewareHandler } from 'hono';
import { getIpAddress } from '@documenso/lib/universal/get-ip-address';
export const createRateLimitMiddleware = (
limiter: ReturnType<typeof rateLimit>,
options?: { identifierFn?: (c: Context) => string | undefined },
): MiddlewareHandler => {
return async (c, next) => {
let ip: string;
try {
ip = getIpAddress(c.req.raw);
} catch {
ip = 'unknown';
}
const identifier = options?.identifierFn?.(c);
const result = await limiter.check({ ip, identifier });
c.header('X-RateLimit-Limit', String(result.limit));
c.header('X-RateLimit-Remaining', String(result.remaining));
c.header('X-RateLimit-Reset', String(Math.ceil(result.reset.getTime() / 1000)));
if (result.isLimited) {
c.header('Retry-After', String(Math.ceil((result.reset.getTime() - Date.now()) / 1000)));
return c.json({ error: 'Too many requests, please try again later.' }, 429);
}
await next();
};
};
```
### 3.2 Replace existing Hono rate limiters
In `apps/remix/server/router.ts`:
- Remove `hono-rate-limiter` import and both `rateLimiter()` instances
- Replace with `createRateLimitMiddleware()` calls using the defined rate limits
- API routes use IP-only limiting (no identifier)
- AI route uses IP-only limiting with the stricter 3/min limit
```typescript
// Before
import { rateLimiter } from 'hono-rate-limiter';
const rateLimitMiddleware = rateLimiter({ ... });
// After
import { createRateLimitMiddleware } from '@documenso/lib/server-only/rate-limit/rate-limit-middleware';
import { apiV1RateLimit, apiV2RateLimit, aiRateLimit } from '@documenso/lib/server-only/rate-limit/rate-limits';
const apiV1RateLimitMiddleware = createRateLimitMiddleware(apiV1RateLimit);
const apiV2RateLimitMiddleware = createRateLimitMiddleware(apiV2RateLimit);
const aiRateLimitMiddleware = createRateLimitMiddleware(aiRateLimit);
```
### 3.3 Response helpers for inline checks
For auth routes (Hono handlers) and tRPC routes where rate limiting is applied inline rather than via middleware, provide helpers that handle the response formatting and headers consistently.
**Hono helper** — returns a 429 `Response` with headers if limited, or `null` if allowed:
```typescript
export const rateLimitResponse = (c: Context, result: RateLimitCheckResult): Response | null => {
c.header('X-RateLimit-Limit', String(result.limit));
c.header('X-RateLimit-Remaining', String(result.remaining));
c.header('X-RateLimit-Reset', String(Math.ceil(result.reset.getTime() / 1000)));
if (result.isLimited) {
c.header('Retry-After', String(Math.ceil((result.reset.getTime() - Date.now()) / 1000)));
return c.json({ error: 'Too many requests, please try again later.' }, 429);
}
return null;
};
```
Usage in auth routes:
```typescript
const result = await loginRateLimit.check({
ip: requestMetadata.ipAddress ?? 'unknown',
identifier: input.email,
});
const limited = rateLimitResponse(c, result);
if (limited) return limited;
```
**tRPC helper** — throws a `TRPCError` with rate limit headers if limited:
```typescript
export const assertRateLimit = (result: RateLimitCheckResult): void => {
if (result.isLimited) {
throw new TRPCError({
code: 'TOO_MANY_REQUESTS',
});
}
};
```
Usage in tRPC routes:
```typescript
const result = await request2FAEmailRateLimit.check({
ip: ctx.requestMetadata.ipAddress ?? 'unknown',
identifier: input.recipientId,
});
assertRateLimit(result);
```
Both helpers live in `packages/lib/server-only/rate-limit/rate-limit-middleware.ts` alongside the Hono middleware.
### 3.4 Auth endpoint rate limiting
In `packages/auth/server/routes/email-password.ts`, add rate limit checks at the start of each handler using the `rateLimitResponse` helper.
Apply to each endpoint per the tier list:
| Endpoint | Rate Limit |
| --------------------------- | ----------------------------------------------------- |
| `POST /signup` | `signupRateLimit` with `identifier: email` |
| `POST /authorize` (login) | `loginRateLimit` with `identifier: email` |
| `POST /forgot-password` | `forgotPasswordRateLimit` with `identifier: email` |
| `POST /resend-verify-email` | `resendVerifyEmailRateLimit` with `identifier: email` |
| `POST /verify-email` | `verifyEmailRateLimit` with `identifier: token` |
| `POST /reset-password` | `resetPasswordRateLimit` with `identifier: token` |
| `POST /passkey/authorize` | `passkeyRateLimit` (IP only, no identifier) |
| `POST /oauth/authorize/*` | `oauthRateLimit` (IP only) |
### 3.4 tRPC unauthenticated route rate limiting
For unauthenticated tRPC routes that send emails, add rate limit checks at the start of the route handler:
| Route | Rate Limit | Identifier |
| ---------------------------------------------------------- | ------------------------------------ | ---------------------- |
| `document.accessAuth.request2FAEmail` | `request2FAEmailRateLimit` | `recipientId` or token |
| `enterprise.organisation.authenticationPortal.linkAccount` | `linkOrgAccountRateLimit` | email |
| `template.createDocumentFromDirectTemplate` | Dedicated direct template rate limit | IP only |
Access `requestMetadata` from the tRPC context (`ctx.requestMetadata.ipAddress`).
### 3.5 tRPC and file routes — general API rate limiting
Add rate limit middleware for currently unprotected routes:
- `/api/trpc/*` — apply `apiTrpcRateLimit` middleware
- `/api/files/*` — apply `fileUploadRateLimit` middleware
---
## 4. Cleanup Job
### 4.1 Job definition
Create `packages/lib/jobs/definitions/internal/cleanup-rate-limits.ts`:
```typescript
export const CLEANUP_RATE_LIMITS_JOB_DEFINITION = {
id: 'internal.cleanup-rate-limits',
name: 'Cleanup Rate Limits',
version: '1.0.0',
trigger: {
name: 'internal.cleanup-rate-limits',
schema: z.object({}),
cron: '*/15 * * * *', // Every 15 minutes
},
handler: async ({ payload, io }) => {
const handler = await import('./cleanup-rate-limits.handler');
await handler.run({ payload, io });
},
} as const satisfies JobDefinition<...>;
```
### 4.2 Job handler
Create `packages/lib/jobs/definitions/internal/cleanup-rate-limits.handler.ts`:
- Delete all `RateLimit` rows where `createdAt` is older than 24 hours (covers all possible windows with margin)
- Use batched deletes to avoid long-running transactions
- Batch in chunks of 10,000 rows
```typescript
export const run = async () => {
const cutoff = new Date(Date.now() - 24 * 60 * 60 * 1000);
let deleted = 0;
do {
// Prisma doesn't support DELETE with LIMIT, so use raw SQL for batching
deleted = await prisma.$executeRaw`
DELETE FROM "RateLimit"
WHERE "createdAt" < ${cutoff}
AND ctid IN (
SELECT ctid FROM "RateLimit"
WHERE "createdAt" < ${cutoff}
LIMIT 10000
)
`;
} while (deleted > 0);
};
```
### 4.3 Register in job client
Add `CLEANUP_RATE_LIMITS_JOB_DEFINITION` to the job registry in `packages/lib/jobs/client.ts`.
---
## 5. Remove hono-rate-limiter Dependency
After the migration is complete:
- Remove `hono-rate-limiter` from `apps/remix/package.json`
- Run `npm install` to clean up
---
## 6. Files to Create or Modify
### New Files
| File | Purpose |
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `packages/lib/server-only/rate-limit/rate-limit.ts` | Core rate limit factory (`rateLimit()`) with window parsing, bucket computation, Prisma upsert, fail-open |
| `packages/lib/server-only/rate-limit/rate-limits.ts` | All rate limit instances (auth, API, AI, file upload) |
| `packages/lib/server-only/rate-limit/rate-limit-middleware.ts` | Hono middleware factory, `rateLimitResponse` helper for Hono handlers, `assertRateLimit` helper for tRPC routes |
| `packages/lib/jobs/definitions/internal/cleanup-rate-limits.ts` | Cleanup cron job definition |
| `packages/lib/jobs/definitions/internal/cleanup-rate-limits.handler.ts` | Cleanup handler (batched deletes) |
### Modified Files
| File | Change |
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `packages/prisma/schema.prisma` | Add `RateLimit` model |
| `apps/remix/server/router.ts` | Replace `hono-rate-limiter` with DB-backed middleware, add rate limits for `/api/trpc/*` and `/api/files/*` |
| `apps/remix/package.json` | Remove `hono-rate-limiter` dependency |
| `packages/auth/server/routes/email-password.ts` | Add rate limit checks to signup, login, forgot-password, resend-verify-email, verify-email, reset-password |
| `packages/auth/server/routes/passkey.ts` | Add rate limit check to passkey authorize |
| `packages/auth/server/routes/oauth.ts` | Add rate limit check to OAuth authorize endpoints |
| `packages/trpc/server/document-router/access-auth-request-2fa-email.ts` | Add rate limit check (sends email, unauthenticated) |
| `packages/trpc/server/enterprise-router/link-organisation-account.ts` | Add rate limit check (sends email, unauthenticated) |
| `packages/lib/jobs/client.ts` | Register cleanup-rate-limits job definition |
---
## 7. Considerations
### 7.1 Fail-open
All rate limit checks must be wrapped in try/catch. On any DB error, log the error and allow the request through. Rate limiting should never block legitimate traffic due to infrastructure issues.
### 7.2 Performance
- Each API request adds 1 upsert query (~1ms)
- Auth requests add 2 upsert queries (~2ms total)
- The composite primary key ensures all lookups and upserts are index-only operations
- No `COUNT(*)` queries — the count is stored directly in the row
### 7.3 Monitoring
Log rate limit hits at `warn` level with the action, key type (IP/identifier), and count. This provides visibility into traffic patterns and helps tune limits.
### 7.4 Testing
The rate limit module should be mockable in tests. Consider exporting the bucket computation and window parsing as standalone functions for unit testing. Integration tests can verify the upsert + count logic against a test database.
### 7.5 Future improvements
- **Redis backend**: if DB pressure from rate limiting becomes measurable, swap the Prisma upsert for Redis `INCR` + `EXPIRE` with no API changes
- **System-wide circuit breaker**: add a `systemMax` config option that counts all requests for an action regardless of key
.agents/plans/smooth-ivory-sky-partial-signed-pdf-download.md
+94
View File
@@ -0,0 +1,94 @@
---
date: 2026-04-22
title: Partial Signed Pdf Download
---
## Summary
Let team members fetch a PDF with all currently-inserted fields burned in while the envelope is still in `PENDING` status. Today the only available bytes for a pending envelope are the original (no fields) - the sealed PDF only materialises after the last recipient signs and the `seal-document` job runs.
Exposed in two places:
- v2 API: `GET /api/v2/envelope/item/{envelopeItemId}/download?version=pending` (API-token auth)
- UI: a `Partial` button in the existing `EnvelopeDownloadDialog`, alongside `Original`. Replaces the `Signed` slot when the envelope is `PENDING`. Backed by the existing session-authed file route `GET /api/files/envelope/{envelopeId}/envelopeItem/{id}/download/pending`.
## Scope
- v2 API only (no v1).
- `internalVersion === 2` envelopes only. Legacy v1 returns 400 `ENVELOPE_LEGACY`.
- Team-side / owner only. No recipient-token download path - recipients have the in-app overlay viewer for verification, and a downloadable half-signed PDF is a leak vector for partially-executed contracts. Enforced both at the server (the recipient-token file route does not accept `pending`) and at the UI (the dialog hides the Partial button when a recipient token is set).
- No PKI signature, no certificate page, no audit log appendix - the response is explicitly not a final executed document.
- No watermark or banner text. The filename suffix (`_pending.pdf`), the `Cache-Control: no-store, private` header, and the absence of a PKI signature are sufficient to signal draft status.
## Behaviour
API response matrix (both `/api/v2/envelope/item/{id}/download?version=pending` and the UI-facing `/api/files/envelope/{envelopeId}/envelopeItem/{id}/download/pending`):
| Envelope status | Response |
|---|---|
| `PENDING` (v2) | 200, PDF with currently-inserted fields burned in |
| `PENDING` (v1) | 400 `ENVELOPE_LEGACY` |
| `DRAFT` | 400 `ENVELOPE_DRAFT` |
| `COMPLETED` | 400 `ENVELOPE_COMPLETED` |
| `REJECTED` | 400 `ENVELOPE_REJECTED` |
All v1-vs-v2 / status-mismatch errors are 4xx so callers can cleanly separate them from real server failures (5xx). Specifically v1 PENDING returns 400 not 501: 5xx is reserved for actual server problems, while "this envelope can't satisfy this request shape" is a client-addressable condition.
Filename: `{title}_pending.pdf`.
ETag is content-addressed over `sha256(envelope.status + sorted((field.id, field.customText, field.signature?.id, field.signature?.created) for inserted===true fields))`. Returns 304 on `If-None-Match` match.
No persistent caching. Generated on-demand per request when ETag misses.
Error response shape (envelope item v2 download route and the team-side file route): preserves the existing `{ error: <message> }` field for backwards compatibility and adds `code: <APP_ERROR_CODE>` as a new field for callers that want to branch on it. The document download route (`/document/{documentId}/download`) is untouched.
## UI
`apps/remix/app/components/dialogs/envelope-download-dialog.tsx`:
- The dialog shows `Original` plus one of:
- `Signed` when status is `COMPLETED` (existing behaviour)
- `Partial` when status is `PENDING`, there is no recipient token, and the envelope is not legacy (`!isLegacy`)
- nothing otherwise
- New optional prop `isLegacy?: boolean`. Only consulted to gate the `Partial` button, so callers whose status can never be `PENDING` (DRAFT/COMPLETED/REJECTED hardcoded, or `isComplete: true` matchers) and callers that always set a recipient token can omit it. Three call sites pass it (`isLegacy={envelope.internalVersion === 1}`): `documents-table-action-dropdown.tsx`, `envelope-editor.tsx`, `document-page-view-dropdown.tsx`. The other eight callers were left alone.
Trade-off: a future team-side dialog usage where status could be PENDING but the dev forgets `isLegacy` will silently not render the Partial button. The status gate prevents an actively broken click; missing button is discoverable in testing. Required-prop alternative was rejected because eight of eleven call sites would carry a meaningless value.
## Files
Server:
- `apps/remix/server/api/download/download.types.ts` - added `'pending'` to the `version` enum; split the validator into `param` (envelopeItemId) + `query` (version). The original wiring as a path-param validator was a pre-existing bug: requests like `?version=original` were silently returning the signed PDF since `version` actually arrives as a query string. Fixed as a side effect.
- `packages/trpc/server/envelope-router/download-envelope-item.types.ts` - mirrored the enum change in the OpenAPI schema.
- `apps/remix/server/api/download/download.ts` - the envelope item v2 route now fetches envelope recipients alongside the envelope, branches on `version` when calling the helper, and emits AppError responses as `{ error, code }` consistently across all status codes.
- `apps/remix/server/api/files/files.types.ts` - added `'pending'` to the team-side download schema only. The recipient-token download schema is untouched, so `/api/files/token/.../download/pending` is rejected by the schema validator.
- `apps/remix/server/api/files/files.ts` - the team-side download handler fetches envelope recipients and dispatches the `pending` branch through the same `handleEnvelopeItemFileRequest` helper. Wrapped in a try/catch that returns `{ error, code }` for AppErrors.
- `apps/remix/server/api/files/files.helpers.ts` - `handleEnvelopeItemFileRequest` is now a single entry point taking a discriminated-union options type. The static-file flow (`signed`/`original`) and the on-demand pending flow are private helpers in the same module.
- `packages/lib/server-only/pdf/generate-partial-signed-pdf.ts` (new) - small orchestrator that loads the original PDF, groups inserted fields by page, calls the existing `insertFieldInPDFV2` overlay helper for each page, flattens, and saves.
- `packages/lib/errors/app-error.ts` - added `ENVELOPE_DRAFT`, `ENVELOPE_COMPLETED`, `ENVELOPE_REJECTED`, `ENVELOPE_LEGACY` codes, all mapped to 400. The legacy-envelope case deliberately returns 4xx rather than 501 to keep "this resource can't satisfy this operation" distinct from real 5xx server failures in caller logs/metrics.
Client:
- `packages/lib/utils/envelope-download.ts` - `EnvelopeItemPdfUrlOptions` download variant now allows `'pending'` as a version. The recipient-token URL builder will produce a URL the server rejects, but the dialog gates on no-token at the call site.
- `packages/lib/client-only/download-pdf.ts` - `DocumentVersion` extended; filename suffix logic moved into a small switch (`_signed.pdf`, `_pending.pdf`, `.pdf`).
- `apps/remix/app/components/dialogs/envelope-download-dialog.tsx` - secondary download derivation with the new `Partial` branch, optional `isLegacy` prop.
- `apps/remix/app/components/tables/documents-table-action-dropdown.tsx`, `apps/remix/app/components/general/envelope-editor/envelope-editor.tsx`, `apps/remix/app/components/general/document/document-page-view-dropdown.tsx` - pass `isLegacy={envelope.internalVersion === 1}` (or `row.internalVersion === 1`) to the dialog.
## Verification
1. E2E (`packages/app-tests/e2e/api/v2/partial-signed-pdf-download.spec.ts`):
- Pending envelope, recipient 1 signs, API token download with `?version=pending` returns 200 + PDF; subsequent call with `If-None-Match: <etag>` returns 304; after recipient 2 completes the envelope flips to `COMPLETED` and the same call returns 400 `ENVELOPE_COMPLETED`; `?version=signed` then succeeds.
- Draft envelope returns 400 `ENVELOPE_DRAFT`.
- `internalVersion === 1` pending envelope returns 400 `ENVELOPE_LEGACY`.
2. `npx tsc --noEmit -p apps/remix/tsconfig.json` and `npm run lint`.
3. Manual: open the Documents table or envelope editor on a PENDING envelope (v2), open the download dialog, confirm `Partial` appears alongside `Original` and produces a `_pending.pdf` with current fields burned in. Same dialog on a COMPLETED envelope shows `Signed`. Same dialog on a v1 PENDING envelope shows neither (status gate would show Partial, but the `isLegacy` flag filters it out).
## Out of Scope / Follow-ups
- Recipient-token download path (API and UI) - decided against. Revisit if there is concrete demand and a story for limiting the leak vector.
- v1 API parity / v1 partial rendering - not building. Implementing partial for v1 would require porting `legacy_insertFieldInPDF` / `insertFieldInPDFV1` into a partial-only flow, which is code with no long-term home as v1 is being phased out.
- Document download route (`/document/{documentId}/download`) - untouched. Same error shape and validator wiring as before. Consider normalising to the same `{ error, code }` shape in a follow-up if any caller wants to branch on `code` from that route.
- Persistent caching layer / job-queue generation - revisit if p95 latency on large PDFs becomes an issue.
- Specific toast for `ENVELOPE_LEGACY` in the dialog - currently the catch-all "Something went wrong" handles it. Worth a polish if v1 PENDING envelopes are common in your data and we see complaints. (Note: with the `isLegacy` gate at the UI, the error is unreachable from the dialog itself; the API can still surface it for direct callers.)
.agents/plans/warm-purple-flower-custom-email-domain-sync-and-recovery.md
+263
View File
@@ -0,0 +1,263 @@
---
date: 2026-02-24
title: Custom Email Domain Sync And Recovery
---
## Problem Statement
Custom email domains configured via AWS SES can get stuck in a `PENDING` state or fail validation silently. Currently, there is **no automated verification** -- users must manually click "Sync" in the UI to check domain status. If a domain fails to validate, the only option is to delete it and recreate it, which generates new DKIM keys and requires the user to update their DNS records.
### Current Pain Points
1. **No background sync** -- Domain verification status is never checked automatically; users must manually click "Sync"
2. **Stuck domains** -- Domains can remain in `PENDING` state indefinitely with no alerting or auto-recovery
3. **Failed recovery requires DNS changes** -- Deleting and recreating a domain generates new keys, forcing the user to update DNS records
4. **No visibility into failure duration** -- There's no tracking of how long a domain has been pending
## Proposed Solution
### 1. Hourly Background Sync Job
Create a new cron job (`internal.sync-email-domains`) that runs every hour to automatically verify all `PENDING` email domains.
**Job Definition:** `packages/lib/jobs/definitions/internal/sync-email-domains.ts`
**Job Handler:** `packages/lib/jobs/definitions/internal/sync-email-domains.handler.ts`
**Pattern:** Follow the existing `cleanup-rate-limits` cron job pattern:
- `cron: '0 * * * *'` (every hour, on the hour)
- Empty `z.object({})` schema (no payload needed)
- Register in `packages/lib/jobs/client.ts`
**Handler Logic:**
1. Query all `EmailDomain` records with `status: 'PENDING'`
2. For each domain, call `verifyEmailDomain(emailDomainId)` which:
- Calls AWS SES `GetEmailIdentityCommand` to check current verification status
- Updates DB status to `ACTIVE` if verified, keeps `PENDING` otherwise
3. Log results via `io.logger` (how many checked, how many transitioned to ACTIVE)
4. Process domains in batches to avoid overwhelming SES API rate limits
5. Add error handling per-domain so one failure doesn't stop the entire sweep
### 2. Schema Changes -- Track Pending Duration
Add a `lastVerifiedAt` column to the `EmailDomain` model to track when verification was last attempted, enabling "stale domain" detection.
**File:** `packages/prisma/schema.prisma`
```prisma
model EmailDomain {
// ... existing fields ...
lastVerifiedAt DateTime? // Last time verification was checked against SES
}
```
**Migration:** Create a new Prisma migration for this column addition.
**Updates needed:**
- `verify-email-domain.ts` -- Update `lastVerifiedAt` when verification is checked
- The sync job handler -- Use `lastVerifiedAt` to avoid re-checking domains that were just verified
### 3. Domain Re-registration (Recovery) -- Delete & Recreate in SES Without Changing Keys
Add a new "Re-register" action that deletes the SES identity and recreates it using the **same** DKIM key pair stored in the database, so the user's DNS records remain valid.
#### 3a. New Service Function
**File:** `packages/ee/server-only/lib/reregister-email-domain.ts`
```typescript
export const reregisterEmailDomain = async (options: { emailDomainId: string }) => {
// 1. Fetch the EmailDomain record (including encrypted privateKey)
// 2. Decrypt the private key using DOCUMENSO_ENCRYPTION_KEY
// 3. Call DeleteEmailIdentityCommand on SES (ignore NotFoundException)
// 4. Call CreateEmailIdentityCommand with BYODKIM using the SAME selector + private key
// 5. Update EmailDomain status back to PENDING, update lastVerifiedAt
// 6. Return the updated domain
};
```
Key points:
- Uses the existing encrypted `privateKey` from the DB -- no new key generation
- Uses the existing `selector` -- DNS records stay the same
- Deletes first, then recreates -- handles cases where SES state is corrupted
- Resets status to `PENDING` since verification will need to re-occur
- Uses `verifyDomainWithDKIM()` from `create-email-domain.ts` (may need to extract/export this helper)
#### 3b. Admin TRPC Routes (Find, Get, Re-register)
All email domain admin routes use `adminProcedure` -- requires system-level `Role.ADMIN`.
**Find (list) route:**
**File:** `packages/trpc/server/admin-router/find-email-domains.ts`
**Types:** `packages/trpc/server/admin-router/find-email-domains.types.ts`
- Query route: `admin.emailDomain.find`
- Input: `{ query?: string, page?: number, perPage?: number, status?: EmailDomainStatus }`
- Extends `ZFindSearchParamsSchema` with optional `status` filter
- Returns standard `ZFindResultResponse` with email domain data including: id, domain, status, selector, createdAt, lastVerifiedAt, organisation name, email count
- Prisma query filters by domain name (LIKE search on `query`), optional status, joins organisation for name, counts emails
**Get (detail) route:**
**File:** `packages/trpc/server/admin-router/get-email-domain.ts`
**Types:** `packages/trpc/server/admin-router/get-email-domain.types.ts`
- Query route: `admin.emailDomain.get`
- Input: `{ emailDomainId: string }`
- Returns full email domain detail: all fields (except privateKey), organisation info, list of associated emails, DNS records (generated from publicKey + selector)
- Omits `privateKey` from response
**Re-register (mutation) route:**
**File:** `packages/trpc/server/admin-router/reregister-email-domain.ts`
**Types:** `packages/trpc/server/admin-router/reregister-email-domain.types.ts`
- Mutation route: `admin.emailDomain.reregister`
- Input: `{ emailDomainId: string }`
- Calls `reregisterEmailDomain()`
- Rationale: Re-registration is a recovery/operational action that deletes and recreates an SES identity. This is a privileged operation that should only be performed by platform operators, not self-service by org admins.
#### 3c. Register in Admin Router
**File:** `packages/trpc/server/admin-router/router.ts`
Add a new `emailDomain` namespace to the admin router:
```typescript
emailDomain: {
find: findEmailDomainsRoute,
get: getEmailDomainRoute,
reregister: reregisterEmailDomainRoute,
},
```
#### 3d. Admin Panel UI -- Email Domains Section
**List page:** `apps/remix/app/routes/_authenticated+/admin+/email-domains._index.tsx`
- New admin panel page at `/admin/email-domains`
- Follow the existing admin documents list pattern (client-side TRPC data fetching)
- Search input (debounced) filtering by domain name
- Status filter dropdown (All / Pending / Active)
- DataTable with columns: Domain, Organisation, Status (badge), Email Count, Created, Last Verified, Actions
- Actions dropdown per row: View details, Re-register
- Pagination via `DataTablePagination`
**Detail page:** `apps/remix/app/routes/_authenticated+/admin+/email-domains.$id.tsx`
- Shows full domain details: domain, selector, status, organisation, created date, last verified date
- Shows DNS records (DKIM + SPF) with copy buttons (reuse `organisation-email-domain-records-dialog` pattern)
- Table of associated organisation emails
- "Re-register" button with confirmation dialog explaining the action (SES identity will be deleted and recreated with the same keys)
- "Verify Now" button to manually trigger a verification check
- Shows how long the domain has been pending (using `lastVerifiedAt` or `createdAt`)
**Navigation:** Add menu item to admin sidebar in `_layout.tsx`:
```tsx
<Button
variant="ghost"
className={cn(
'justify-start md:w-full',
pathname?.startsWith('/admin/email-domains') && 'bg-secondary',
)}
asChild
>
<Link to="/admin/email-domains">
<MailIcon className="mr-2 h-5 w-5" />
<Trans>Email Domains</Trans>
</Link>
</Button>
```
**Table component:** `apps/remix/app/components/tables/admin-email-domains-table.tsx` (optional -- can be inline in the route file like the documents page)
#### 3e. Automatic Re-registration in Sync Job (Optional Enhancement)
In the hourly sync job, after checking verification status, if a domain has been `PENDING` for more than 48 hours:
- Automatically call `reregisterEmailDomain()` to attempt recovery
- Log the auto-recovery attempt
- This provides a self-healing mechanism without user intervention
## Implementation Plan
### Phase 1: Background Sync Job (Core)
1. Create `sync-email-domains.ts` job definition with hourly cron
2. Create `sync-email-domains.handler.ts` with batch verification logic
3. Register job in `packages/lib/jobs/client.ts`
4. Add error handling and logging
### Phase 2: Schema Enhancement
5. Add `lastVerifiedAt` column to `EmailDomain` model
6. Create Prisma migration
7. Update `verifyEmailDomain()` to set `lastVerifiedAt` on each check
8. Update sync job to use `lastVerifiedAt` for intelligent scheduling
### Phase 3: Admin Email Domains Panel
9. Create `find-email-domains` admin TRPC route + types (list/search with pagination and status filter)
10. Create `get-email-domain` admin TRPC route + types (detail view with org info, emails, DNS records)
11. Register find + get routes in admin router under `emailDomain` namespace
12. Create admin list page (`admin+/email-domains._index.tsx`) with search, status filter, DataTable
13. Create admin detail page (`admin+/email-domains.$id.tsx`) with domain info, emails table, DNS records
14. Add "Email Domains" menu item to admin sidebar (`_layout.tsx`)
### Phase 4: Re-registration Feature
15. Extract `verifyDomainWithDKIM()` as a shared helper (if not already exported)
16. Create `reregisterEmailDomain()` service function
17. Create `reregister-email-domain` admin TRPC mutation route + types
18. Register reregister route in admin router under `emailDomain.reregister`
19. Add "Re-register" button + confirmation dialog on admin detail page
### Phase 5: Auto-Recovery (Optional)
20. Add 48-hour stale detection logic to sync job
21. Auto-trigger re-registration for stale domains
22. Add logging/notifications for auto-recovery events
## Files to Create/Modify
### New Files
- `packages/lib/jobs/definitions/internal/sync-email-domains.ts`
- `packages/lib/jobs/definitions/internal/sync-email-domains.handler.ts`
- `packages/ee/server-only/lib/reregister-email-domain.ts`
- `packages/trpc/server/admin-router/find-email-domains.ts`
- `packages/trpc/server/admin-router/find-email-domains.types.ts`
- `packages/trpc/server/admin-router/get-email-domain.ts`
- `packages/trpc/server/admin-router/get-email-domain.types.ts`
- `packages/trpc/server/admin-router/reregister-email-domain.ts`
- `packages/trpc/server/admin-router/reregister-email-domain.types.ts`
- `apps/remix/app/routes/_authenticated+/admin+/email-domains._index.tsx`
- `apps/remix/app/routes/_authenticated+/admin+/email-domains.$id.tsx`
### Modified Files
- `packages/prisma/schema.prisma` -- Add `lastVerifiedAt` field
- `packages/lib/jobs/client.ts` -- Register new sync job
- `packages/ee/server-only/lib/verify-email-domain.ts` -- Update `lastVerifiedAt`
- `packages/ee/server-only/lib/create-email-domain.ts` -- Export `verifyDomainWithDKIM` helper
- `packages/trpc/server/admin-router/router.ts` -- Add `emailDomain.{find, get, reregister}` routes
- `apps/remix/app/routes/_authenticated+/admin+/_layout.tsx` -- Add "Email Domains" nav item to sidebar
- New Prisma migration file
## Technical Considerations
1. **SES API Rate Limits** -- AWS SES has rate limits on `GetEmailIdentityCommand`. The sync job should process domains in batches with small delays between calls (e.g., 5-10 per batch with 1s delay).
2. **Concurrency** -- The local job provider has deterministic deduplication via SHA-256 IDs, so multiple app instances won't run the same cron tick twice.
3. **Error Isolation** -- Each domain verification in the sync job should be wrapped in try/catch so one failing domain doesn't prevent others from being checked.
4. **Re-registration Safety** -- The re-register function should be idempotent. Deleting a non-existent SES identity should be handled gracefully (already done in `deleteEmailDomain`).
5. **Private Key Security** -- The private key is encrypted at rest and should only be decrypted transiently during re-registration. It should never be logged or exposed in API responses.
6. **Feature Gating** -- The sync job should only process domains belonging to organisations with active `emailDomains` claim flags. This prevents processing domains for orgs that have downgraded.
7. **Observability** -- Add structured logging to the sync job so operations teams can monitor domain verification health across all tenants.
.agents/plans/wild-teal-wind-add-folder-support-to-v1-api.md
+210
View File
@@ -0,0 +1,210 @@
---
date: 2026-02-10
title: Add Folder Support To V1 Api
status: ready
---
## Problem
The `GET /api/v1/documents` endpoint does not return documents inside folders. The underlying `findDocuments()` function defaults to `folderId: null` when no `folderId` is provided, meaning only root-level documents are returned. The V1 API never passes `folderId`, so folder documents are invisible to API consumers.
Additionally, neither the list endpoint nor the single-document endpoint exposes `folderId` in the response, so consumers cannot know which folder a document belongs to.
## Root Cause
In `packages/lib/server-only/document/find-documents.ts` (line 222-226):
```ts
if (folderId !== undefined) {
whereClause.folderId = folderId;
} else {
whereClause.folderId = null; // Only root documents returned
}
```
The V1 `getDocuments` handler in `packages/api/v1/implementation.ts` (line 61-70) only passes `page` and `perPage` to `findDocuments` — it never extracts or forwards a `folderId` from the query string.
## Decisions
These decisions were made during the spec interview:
1. **Fix V1 directly** — The V1 API is deprecated but still actively used. This is a quick, low-risk fix. No need to defer to a newer API.
2. **Breaking change accepted** — Returning ALL documents by default (instead of root-only) is intentional. The current root-only behavior is a bug, not a feature.
3. **No root-only query option needed** — Not all documents are in folders, so consumers can filter client-side using the `folderId` field in the response if needed.
4. **No folder existence validation** — `?folderId=nonexistent` returns empty array, not 404. Consistent with V1 list endpoint patterns.
5. **Add `folderId` to both endpoints** — Both `GET /api/v1/documents` (list) and `GET /api/v1/documents/:id` (single) will include `folderId` in the response.
6. **Top-level `skipFolderFilter` is sufficient** — The inner helper filters (`findDocumentsFilter`, `findTeamDocumentsFilter`) receive `folderId: undefined` when skip is active. Prisma ignores `undefined` values in WHERE clauses, so these inner filters will not constrain by folder. No propagation needed.
7. **Scope is minimal** — Only `folderId` support. No other filters (status, period, query, senderIds) added in this change.
## Scope
Three files need changes. No new files.
| File | Change |
| ----------------------------------------------------- | ------------------------------------------------------------------------ |
| `packages/api/v1/schema.ts` | Add `folderId` to query schema + both response schemas |
| `packages/api/v1/implementation.ts` | Pass `folderId` through in `getDocuments`, add to `getDocument` response |
| `packages/lib/server-only/document/find-documents.ts` | Add `skipFolderFilter` option |
## Changes
### 1. `packages/api/v1/schema.ts` — Add `folderId` to query + response schemas
**Query schema** (`ZGetDocumentsQuerySchema`, line 35-38):
```ts
export const ZGetDocumentsQuerySchema = z.object({
page: z.coerce.number().min(1).optional().default(1),
perPage: z.coerce.number().min(1).optional().default(10),
folderId: z
.string()
.describe(
'Filter documents by folder ID. When omitted, returns all documents regardless of folder.',
)
.optional(),
});
```
**List response schema** (`ZSuccessfulDocumentResponseSchema`, line 46-56):
Add `folderId: z.string().nullish()` so consumers can see which folder each document belongs to.
**Single document response schema** (`ZSuccessfulGetDocumentResponseSchema`, line 58-79):
Add `folderId: z.string().nullish()` to the extended schema as well.
### 2. `packages/api/v1/implementation.ts` — Pass `folderId` through + add to responses
**`getDocuments` handler** (line 61-70):
```ts
getDocuments: authenticatedMiddleware(async (args, user, team) => {
const page = Number(args.query.page) || 1;
const perPage = Number(args.query.perPage) || 10;
const { data: documents, totalPages } = await findDocuments({
page,
perPage,
userId: user.id,
teamId: team.id,
folderId: args.query.folderId,
skipFolderFilter: args.query.folderId === undefined,
});
return {
status: 200,
body: {
documents: documents.map((document) => ({
id: mapSecondaryIdToDocumentId(document.secondaryId),
externalId: document.externalId,
userId: document.userId,
teamId: document.teamId,
folderId: document.folderId,
title: document.title,
status: document.status,
createdAt: document.createdAt,
updatedAt: document.updatedAt,
completedAt: document.completedAt,
})),
totalPages,
},
};
}),
```
**`getDocument` handler** (line 91-197):
Add `folderId: envelope.folderId` to the response body mapping (alongside `id`, `externalId`, etc.).
### 3. `packages/lib/server-only/document/find-documents.ts` — Handle "return all" semantics
Add `skipFolderFilter` to the options type and modify the WHERE clause logic:
```ts
export type FindDocumentsOptions = {
// ... existing fields ...
folderId?: string;
skipFolderFilter?: boolean;
};
```
Modify the folderId logic (line 222-226):
```ts
if (!skipFolderFilter) {
if (folderId !== undefined) {
whereClause.folderId = folderId;
} else {
whereClause.folderId = null;
}
}
```
When `skipFolderFilter` is true:
- The top-level `whereClause.folderId` is never set — no folder constraint at the top level.
- The inner helpers (`findDocumentsFilter`, `findTeamDocumentsFilter`) receive `folderId: undefined`, which Prisma ignores in WHERE objects — no folder constraint at the inner level either.
- Result: all documents returned regardless of folder.
When `skipFolderFilter` is false (default, used by UI/tRPC callers):
- Existing behavior is completely unchanged. `folderId: undefined` still defaults to root-only.
## Why `skipFolderFilter` (Option B)
Two approaches were considered:
**Option A: Change `folderId: undefined` semantics to mean "all documents"**
- Risky: would affect all callers of `findDocuments` (UI, tRPC) unless every caller is audited.
- The UI intentionally shows root-only when no folder is selected.
**Option B (chosen): Add `skipFolderFilter` boolean**
- Additive — no existing callers pass this flag, so they're unaffected.
- Explicit — the intent is clear in the code.
- Safe — zero risk to UI/tRPC behavior.
## Behavior Matrix
| Request | Current Behavior | New Behavior |
| -------------------------------------------- | ------------------- | ------------------------- |
| `GET /api/v1/documents` | Root docs only | ALL docs (root + folders) |
| `GET /api/v1/documents?folderId=abc` | Not supported | Docs in folder `abc` only |
| `GET /api/v1/documents?folderId=nonexistent` | Not supported | Empty array, 200 OK |
| `GET /api/v1/documents/:id` response | No `folderId` field | Includes `folderId` |
## Implementation Notes
- `folderId` is a `String?` on the `Envelope` model in Prisma, not a number.
- The `findDocuments` function already accepts `folderId` in its options type — it just needs the `skipFolderFilter` escape hatch.
- No need to propagate `skipFolderFilter` into `findDocumentsFilter` or `findTeamDocumentsFilter`. When `folderId` is `undefined`, those helpers embed `folderId: undefined` in their Prisma WHERE objects. Prisma strips `undefined` keys, so no folder constraint is applied. This is well-documented Prisma behavior.
- The `createDocument` endpoint already supports `folderId` in the request body (line 139-144 of schema.ts), confirming the pattern.
- The `getDocument` handler fetches from `prisma.envelope.findFirstOrThrow` which already includes `folderId` on the envelope — just needs to be added to the response mapping.
## Testing
Manual and automated test cases:
1. `GET /api/v1/documents` returns docs from root AND subfolders.
2. `GET /api/v1/documents?folderId=<valid-id>` returns only docs in that folder.
3. `GET /api/v1/documents?folderId=<nonexistent-id>` returns empty array with 200 status.
4. List response includes `folderId` field on each document (null for root docs, string for folder docs).
5. `GET /api/v1/documents/:id` response includes `folderId` field.
6. Existing UI/tRPC callers of `findDocuments` are unaffected (they don't pass `skipFolderFilter`).
7. Pagination: verify `totalPages` correctly reflects the larger result set when all docs are returned.
## Breaking Change Notice
This is a **breaking change** for existing V1 API consumers:
- **Before**: `GET /api/v1/documents` returned only root-level documents (those not in any folder).
- **After**: `GET /api/v1/documents` returns all documents regardless of folder placement.
Impact:
- Consumers paginating through results will see more documents in the total count.
- Consumers building UIs will now display folder documents they previously didn't see.
- The new `folderId` field is additive and won't break existing response parsing.
This is considered a **bug fix**, not a feature removal. The previous behavior silently hid documents from API consumers.
.agents/scratches/search-query-optimization.md
+113
View File
@@ -0,0 +1,113 @@
---
date: 2026-03-07
title: Search Query Optimization
---
## Problem
The `searchDocumentsWithKeyword` and `searchTemplatesWithKeyword` functions generate a single massive Prisma `findMany` with 7 OR branches. This produces a SQL query that:
- Joins `Team` twice (aliased j3 and j8) for the two team-access branches
- Embeds 4-level deep EXISTS subqueries (`TeamGroup -> OrganisationGroup -> OrganisationGroupMember -> OrganisationMember`) for each team branch
- Uses `ILIKE` across multiple columns with no way for Postgres to use indexes effectively across the OR
- Includes `recipients: true` on the result even though only a small subset of fields are needed
- Fetches all matching rows then filters visibility **in application code**
With 1,000 documents seeded under `medium-account@documenso.com`, this query is noticeably slow.
---
## Option A: Pre-resolve team IDs, keep Prisma
**Change:** Before the envelope query, resolve the user's accessible team IDs in a single query:
```ts
const teamIds = await prisma.teamGroup
.findMany({
where: {
organisationGroup: {
organisationGroupMembers: {
some: { organisationMember: { userId } },
},
},
},
select: { teamId: true },
})
.then((rows) => [...new Set(rows.map((r) => r.teamId))]);
```
Then replace `team: buildTeamWhereQuery(...)` with `teamId: { in: teamIds }` in the envelope query.
**Benefits:**
- Eliminates the duplicated 4-level deep join chain from the envelope query
- The team ID resolution is a simple indexed lookup (runs once, not twice)
- Minimal code change -- still Prisma, same structure
- Can also pre-resolve team roles to move visibility filtering into the WHERE clause
**Drawbacks:**
- Still a single large OR query with ILIKE branches
- Prisma still generates suboptimal SQL for the remaining OR conditions
---
## Option B: Kysely rewrite with pre-resolved teams
**Change:** Rewrite using Kysely (already set up in codebase as `kyselyPrisma.$kysely`). Follow the pattern in `find-envelopes.ts` -- use Kysely for filtering/ID fetching, then Prisma for hydration.
Structure as a UNION of targeted queries instead of a single OR:
```
Query 1: owned docs matching title/externalId (simple indexed lookup)
Query 2: docs where user is recipient matching title (EXISTS on Recipient)
Query 3: team docs matching title/externalId (using pre-resolved teamIds)
UNION ALL -> deduplicate -> ORDER BY createdAt DESC -> LIMIT 20
```
Then hydrate the 20 IDs with Prisma for the include data.
**Benefits:**
- Each sub-query is simple and independently optimizable by Postgres
- UNION eliminates the massive OR which forces bad query plans
- Kysely gives control over exact SQL structure
- Only hydrate the final 20 results (not all matches)
- Follows existing `find-envelopes.ts` pattern -- not a new paradigm
**Drawbacks:**
- More code than Option A
- Two query layers (Kysely for IDs, Prisma for hydration)
---
## Option C: Hybrid -- pre-resolve teams + simplify Prisma OR
**Change:** Pre-resolve team IDs (like Option A), but also restructure the Prisma query to reduce OR branches:
- Merge "owned + title" and "owned + externalId" and "owned + recipient email" into a single owned-docs branch with nested OR
- Merge "team + title" and "team + externalId" into a single team-docs branch
- Keep "recipient inbox" branches separate
This reduces from 7 OR branches to ~3-4, with simpler conditions in each.
**Benefits:**
- Simpler than Kysely rewrite
- Fewer OR branches = better query plan
- Pre-resolved team IDs eliminate the deep joins
- Still pure Prisma
**Drawbacks:**
- Postgres still has to handle OR across different access patterns in one query
- Less control over SQL than Kysely
---
## Recommendation
**Option B (Kysely)** is the strongest choice. The codebase already uses this exact pattern for `find-envelopes.ts` which solves the same class of problem. The UNION approach gives Postgres the best chance at using indexes per sub-query. Pre-resolving team IDs is a prerequisite for all options and is trivially cheap.
The template search query has the same structure and should get the same treatment.
.agents/skills/agent-browser/SKILL.md
+207
View File
@@ -0,0 +1,207 @@
---
name: agent-browser
description: Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction.
allowed-tools: Bash(agent-browser:*)
---
# Browser Automation with agent-browser
## Core Workflow
Every browser automation follows this pattern:
1. **Navigate**: `agent-browser open <url>`
2. **Snapshot**: `agent-browser snapshot -i` (get element refs like `@e1`, `@e2`)
3. **Interact**: Use refs to click, fill, select
4. **Re-snapshot**: After navigation or DOM changes, get fresh refs
```bash
agent-browser open https://example.com/form
agent-browser snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"
agent-browser fill @e1 "user@example.com"
agent-browser fill @e2 "password123"
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser snapshot -i # Check result
```
## Essential Commands
```bash
# Navigation
agent-browser open <url> # Navigate (aliases: goto, navigate)
agent-browser close # Close browser
# Snapshot
agent-browser snapshot -i # Interactive elements with refs (recommended)
agent-browser snapshot -s "#selector" # Scope to CSS selector
# Interaction (use @refs from snapshot)
agent-browser click @e1 # Click element
agent-browser fill @e2 "text" # Clear and type text
agent-browser type @e2 "text" # Type without clearing
agent-browser select @e1 "option" # Select dropdown option
agent-browser check @e1 # Check checkbox
agent-browser press Enter # Press key
agent-browser scroll down 500 # Scroll page
# Get information
agent-browser get text @e1 # Get element text
agent-browser get url # Get current URL
agent-browser get title # Get page title
# Wait
agent-browser wait @e1 # Wait for element
agent-browser wait --load networkidle # Wait for network idle
agent-browser wait --url "**/page" # Wait for URL pattern
agent-browser wait 2000 # Wait milliseconds
# Capture
agent-browser screenshot # Screenshot to temp dir
agent-browser screenshot --full # Full page screenshot
agent-browser pdf output.pdf # Save as PDF
```
## Common Patterns
### Form Submission
```bash
agent-browser open https://example.com/signup
agent-browser snapshot -i
agent-browser fill @e1 "Jane Doe"
agent-browser fill @e2 "jane@example.com"
agent-browser select @e3 "California"
agent-browser check @e4
agent-browser click @e5
agent-browser wait --load networkidle
```
### Authentication with State Persistence
```bash
# Login once and save state
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "$USERNAME"
agent-browser fill @e2 "$PASSWORD"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
agent-browser state save auth.json
# Reuse in future sessions
agent-browser state load auth.json
agent-browser open https://app.example.com/dashboard
```
### Data Extraction
```bash
agent-browser open https://example.com/products
agent-browser snapshot -i
agent-browser get text @e5 # Get specific element text
agent-browser get text body > page.txt # Get all page text
# JSON output for parsing
agent-browser snapshot -i --json
agent-browser get text @e1 --json
```
### Parallel Sessions
```bash
agent-browser --session site1 open https://site-a.com
agent-browser --session site2 open https://site-b.com
agent-browser --session site1 snapshot -i
agent-browser --session site2 snapshot -i
agent-browser session list
```
### Visual Browser (Debugging)
```bash
agent-browser --headed open https://example.com
agent-browser highlight @e1 # Highlight element
agent-browser record start demo.webm # Record session
```
### iOS Simulator (Mobile Safari)
```bash
# List available iOS simulators
agent-browser device list
# Launch Safari on a specific device
agent-browser -p ios --device "iPhone 16 Pro" open https://example.com
# Same workflow as desktop - snapshot, interact, re-snapshot
agent-browser -p ios snapshot -i
agent-browser -p ios tap @e1 # Tap (alias for click)
agent-browser -p ios fill @e2 "text"
agent-browser -p ios swipe up # Mobile-specific gesture
# Take screenshot
agent-browser -p ios screenshot mobile.png
# Close session (shuts down simulator)
agent-browser -p ios close
```
**Requirements:** macOS with Xcode, Appium (`npm install -g appium && appium driver install xcuitest`)
**Real devices:** Works with physical iOS devices if pre-configured. Use `--device "<UDID>"` where UDID is from `xcrun xctrace list devices`.
## Ref Lifecycle (Important)
Refs (`@e1`, `@e2`, etc.) are invalidated when the page changes. Always re-snapshot after:
- Clicking links or buttons that navigate
- Form submissions
- Dynamic content loading (dropdowns, modals)
```bash
agent-browser click @e5 # Navigates to new page
agent-browser snapshot -i # MUST re-snapshot
agent-browser click @e1 # Use new refs
```
## Semantic Locators (Alternative to Refs)
When refs are unavailable or unreliable, use semantic locators:
```bash
agent-browser find text "Sign In" click
agent-browser find label "Email" fill "user@test.com"
agent-browser find role button click --name "Submit"
agent-browser find placeholder "Search" type "query"
agent-browser find testid "submit-btn" click
```
## Deep-Dive Documentation
| Reference | When to Use |
|-----------|-------------|
| [references/commands.md](references/commands.md) | Full command reference with all options |
| [references/snapshot-refs.md](references/snapshot-refs.md) | Ref lifecycle, invalidation rules, troubleshooting |
| [references/session-management.md](references/session-management.md) | Parallel sessions, state persistence, concurrent scraping |
| [references/authentication.md](references/authentication.md) | Login flows, OAuth, 2FA handling, state reuse |
| [references/video-recording.md](references/video-recording.md) | Recording workflows for debugging and documentation |
| [references/proxy-support.md](references/proxy-support.md) | Proxy configuration, geo-testing, rotating proxies |
## Ready-to-Use Templates
| Template | Description |
|----------|-------------|
| [templates/form-automation.sh](templates/form-automation.sh) | Form filling with validation |
| [templates/authenticated-session.sh](templates/authenticated-session.sh) | Login once, reuse state |
| [templates/capture-workflow.sh](templates/capture-workflow.sh) | Content extraction with screenshots |
```bash
./templates/form-automation.sh https://example.com/form
./templates/authenticated-session.sh https://app.example.com/login
./templates/capture-workflow.sh https://example.com ./output
```
.agents/skills/agent-browser/references/authentication.md
+202
View File
@@ -0,0 +1,202 @@
# Authentication Patterns
Login flows, session persistence, OAuth, 2FA, and authenticated browsing.
**Related**: [session-management.md](session-management.md) for state persistence details, [SKILL.md](../SKILL.md) for quick start.
## Contents
- [Basic Login Flow](#basic-login-flow)
- [Saving Authentication State](#saving-authentication-state)
- [Restoring Authentication](#restoring-authentication)
- [OAuth / SSO Flows](#oauth--sso-flows)
- [Two-Factor Authentication](#two-factor-authentication)
- [HTTP Basic Auth](#http-basic-auth)
- [Cookie-Based Auth](#cookie-based-auth)
- [Token Refresh Handling](#token-refresh-handling)
- [Security Best Practices](#security-best-practices)
## Basic Login Flow
```bash
# Navigate to login page
agent-browser open https://app.example.com/login
agent-browser wait --load networkidle
# Get form elements
agent-browser snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Sign In"
# Fill credentials
agent-browser fill @e1 "user@example.com"
agent-browser fill @e2 "password123"
# Submit
agent-browser click @e3
agent-browser wait --load networkidle
# Verify login succeeded
agent-browser get url # Should be dashboard, not login
```
## Saving Authentication State
After logging in, save state for reuse:
```bash
# Login first (see above)
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "user@example.com"
agent-browser fill @e2 "password123"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
# Save authenticated state
agent-browser state save ./auth-state.json
```
## Restoring Authentication
Skip login by loading saved state:
```bash
# Load saved auth state
agent-browser state load ./auth-state.json
# Navigate directly to protected page
agent-browser open https://app.example.com/dashboard
# Verify authenticated
agent-browser snapshot -i
```
## OAuth / SSO Flows
For OAuth redirects:
```bash
# Start OAuth flow
agent-browser open https://app.example.com/auth/google
# Handle redirects automatically
agent-browser wait --url "**/accounts.google.com**"
agent-browser snapshot -i
# Fill Google credentials
agent-browser fill @e1 "user@gmail.com"
agent-browser click @e2 # Next button
agent-browser wait 2000
agent-browser snapshot -i
agent-browser fill @e3 "password"
agent-browser click @e4 # Sign in
# Wait for redirect back
agent-browser wait --url "**/app.example.com**"
agent-browser state save ./oauth-state.json
```
## Two-Factor Authentication
Handle 2FA with manual intervention:
```bash
# Login with credentials
agent-browser open https://app.example.com/login --headed # Show browser
agent-browser snapshot -i
agent-browser fill @e1 "user@example.com"
agent-browser fill @e2 "password123"
agent-browser click @e3
# Wait for user to complete 2FA manually
echo "Complete 2FA in the browser window..."
agent-browser wait --url "**/dashboard" --timeout 120000
# Save state after 2FA
agent-browser state save ./2fa-state.json
```
## HTTP Basic Auth
For sites using HTTP Basic Authentication:
```bash
# Set credentials before navigation
agent-browser set credentials username password
# Navigate to protected resource
agent-browser open https://protected.example.com/api
```
## Cookie-Based Auth
Manually set authentication cookies:
```bash
# Set auth cookie
agent-browser cookies set session_token "abc123xyz"
# Navigate to protected page
agent-browser open https://app.example.com/dashboard
```
## Token Refresh Handling
For sessions with expiring tokens:
```bash
#!/bin/bash
# Wrapper that handles token refresh
STATE_FILE="./auth-state.json"
# Try loading existing state
if [[ -f "$STATE_FILE" ]]; then
agent-browser state load "$STATE_FILE"
agent-browser open https://app.example.com/dashboard
# Check if session is still valid
URL=$(agent-browser get url)
if [[ "$URL" == *"/login"* ]]; then
echo "Session expired, re-authenticating..."
# Perform fresh login
agent-browser snapshot -i
agent-browser fill @e1 "$USERNAME"
agent-browser fill @e2 "$PASSWORD"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
agent-browser state save "$STATE_FILE"
fi
else
# First-time login
agent-browser open https://app.example.com/login
# ... login flow ...
fi
```
## Security Best Practices
1. **Never commit state files** - They contain session tokens
```bash
echo "*.auth-state.json" >> .gitignore
```
2. **Use environment variables for credentials**
```bash
agent-browser fill @e1 "$APP_USERNAME"
agent-browser fill @e2 "$APP_PASSWORD"
```
3. **Clean up after automation**
```bash
agent-browser cookies clear
rm -f ./auth-state.json
```
4. **Use short-lived sessions for CI/CD**
```bash
# Don't persist state in CI
agent-browser open https://app.example.com/login
# ... login and perform actions ...
agent-browser close # Session ends, nothing persisted
```
.agents/skills/agent-browser/references/commands.md
+259
View File
@@ -0,0 +1,259 @@
# Command Reference
Complete reference for all agent-browser commands. For quick start and common patterns, see SKILL.md.
## Navigation
```bash
agent-browser open <url> # Navigate to URL (aliases: goto, navigate)
# Supports: https://, http://, file://, about:, data://
# Auto-prepends https:// if no protocol given
agent-browser back # Go back
agent-browser forward # Go forward
agent-browser reload # Reload page
agent-browser close # Close browser (aliases: quit, exit)
agent-browser connect 9222 # Connect to browser via CDP port
```
## Snapshot (page analysis)
```bash
agent-browser snapshot # Full accessibility tree
agent-browser snapshot -i # Interactive elements only (recommended)
agent-browser snapshot -c # Compact output
agent-browser snapshot -d 3 # Limit depth to 3
agent-browser snapshot -s "#main" # Scope to CSS selector
```
## Interactions (use @refs from snapshot)
```bash
agent-browser click @e1 # Click
agent-browser dblclick @e1 # Double-click
agent-browser focus @e1 # Focus element
agent-browser fill @e2 "text" # Clear and type
agent-browser type @e2 "text" # Type without clearing
agent-browser press Enter # Press key (alias: key)
agent-browser press Control+a # Key combination
agent-browser keydown Shift # Hold key down
agent-browser keyup Shift # Release key
agent-browser hover @e1 # Hover
agent-browser check @e1 # Check checkbox
agent-browser uncheck @e1 # Uncheck checkbox
agent-browser select @e1 "value" # Select dropdown option
agent-browser select @e1 "a" "b" # Select multiple options
agent-browser scroll down 500 # Scroll page (default: down 300px)
agent-browser scrollintoview @e1 # Scroll element into view (alias: scrollinto)
agent-browser drag @e1 @e2 # Drag and drop
agent-browser upload @e1 file.pdf # Upload files
```
## Get Information
```bash
agent-browser get text @e1 # Get element text
agent-browser get html @e1 # Get innerHTML
agent-browser get value @e1 # Get input value
agent-browser get attr @e1 href # Get attribute
agent-browser get title # Get page title
agent-browser get url # Get current URL
agent-browser get count ".item" # Count matching elements
agent-browser get box @e1 # Get bounding box
agent-browser get styles @e1 # Get computed styles (font, color, bg, etc.)
```
## Check State
```bash
agent-browser is visible @e1 # Check if visible
agent-browser is enabled @e1 # Check if enabled
agent-browser is checked @e1 # Check if checked
```
## Screenshots and PDF
```bash
agent-browser screenshot # Save to temporary directory
agent-browser screenshot path.png # Save to specific path
agent-browser screenshot --full # Full page
agent-browser pdf output.pdf # Save as PDF
```
## Video Recording
```bash
agent-browser record start ./demo.webm # Start recording
agent-browser click @e1 # Perform actions
agent-browser record stop # Stop and save video
agent-browser record restart ./take2.webm # Stop current + start new
```
## Wait
```bash
agent-browser wait @e1 # Wait for element
agent-browser wait 2000 # Wait milliseconds
agent-browser wait --text "Success" # Wait for text (or -t)
agent-browser wait --url "**/dashboard" # Wait for URL pattern (or -u)
agent-browser wait --load networkidle # Wait for network idle (or -l)
agent-browser wait --fn "window.ready" # Wait for JS condition (or -f)
```
## Mouse Control
```bash
agent-browser mouse move 100 200 # Move mouse
agent-browser mouse down left # Press button
agent-browser mouse up left # Release button
agent-browser mouse wheel 100 # Scroll wheel
```
## Semantic Locators (alternative to refs)
```bash
agent-browser find role button click --name "Submit"
agent-browser find text "Sign In" click
agent-browser find text "Sign In" click --exact # Exact match only
agent-browser find label "Email" fill "user@test.com"
agent-browser find placeholder "Search" type "query"
agent-browser find alt "Logo" click
agent-browser find title "Close" click
agent-browser find testid "submit-btn" click
agent-browser find first ".item" click
agent-browser find last ".item" click
agent-browser find nth 2 "a" hover
```
## Browser Settings
```bash
agent-browser set viewport 1920 1080 # Set viewport size
agent-browser set device "iPhone 14" # Emulate device
agent-browser set geo 37.7749 -122.4194 # Set geolocation (alias: geolocation)
agent-browser set offline on # Toggle offline mode
agent-browser set headers '{"X-Key":"v"}' # Extra HTTP headers
agent-browser set credentials user pass # HTTP basic auth (alias: auth)
agent-browser set media dark # Emulate color scheme
agent-browser set media light reduced-motion # Light mode + reduced motion
```
## Cookies and Storage
```bash
agent-browser cookies # Get all cookies
agent-browser cookies set name value # Set cookie
agent-browser cookies clear # Clear cookies
agent-browser storage local # Get all localStorage
agent-browser storage local key # Get specific key
agent-browser storage local set k v # Set value
agent-browser storage local clear # Clear all
```
## Network
```bash
agent-browser network route <url> # Intercept requests
agent-browser network route <url> --abort # Block requests
agent-browser network route <url> --body '{}' # Mock response
agent-browser network unroute [url] # Remove routes
agent-browser network requests # View tracked requests
agent-browser network requests --filter api # Filter requests
```
## Tabs and Windows
```bash
agent-browser tab # List tabs
agent-browser tab new [url] # New tab
agent-browser tab 2 # Switch to tab by index
agent-browser tab close # Close current tab
agent-browser tab close 2 # Close tab by index
agent-browser window new # New window
```
## Frames
```bash
agent-browser frame "#iframe" # Switch to iframe
agent-browser frame main # Back to main frame
```
## Dialogs
```bash
agent-browser dialog accept [text] # Accept dialog
agent-browser dialog dismiss # Dismiss dialog
```
## JavaScript
```bash
agent-browser eval "document.title" # Simple expressions only
agent-browser eval -b "<base64>" # Any JavaScript (base64 encoded)
agent-browser eval --stdin # Read script from stdin
```
Use `-b`/`--base64` or `--stdin` for reliable execution. Shell escaping with nested quotes and special characters is error-prone.
```bash
# Base64 encode your script, then:
agent-browser eval -b "ZG9jdW1lbnQucXVlcnlTZWxlY3RvcignW3NyYyo9Il9uZXh0Il0nKQ=="
# Or use stdin with heredoc for multiline scripts:
cat <<'EOF' | agent-browser eval --stdin
const links = document.querySelectorAll('a');
Array.from(links).map(a => a.href);
EOF
```
## State Management
```bash
agent-browser state save auth.json # Save cookies, storage, auth state
agent-browser state load auth.json # Restore saved state
```
## Global Options
```bash
agent-browser --session <name> ... # Isolated browser session
agent-browser --json ... # JSON output for parsing
agent-browser --headed ... # Show browser window (not headless)
agent-browser --full ... # Full page screenshot (-f)
agent-browser --cdp <port> ... # Connect via Chrome DevTools Protocol
agent-browser -p <provider> ... # Cloud browser provider (--provider)
agent-browser --proxy <url> ... # Use proxy server
agent-browser --headers <json> ... # HTTP headers scoped to URL's origin
agent-browser --executable-path <p> # Custom browser executable
agent-browser --extension <path> ... # Load browser extension (repeatable)
agent-browser --ignore-https-errors # Ignore SSL certificate errors
agent-browser --help # Show help (-h)
agent-browser --version # Show version (-V)
agent-browser <command> --help # Show detailed help for a command
```
## Debugging
```bash
agent-browser --headed open example.com # Show browser window
agent-browser --cdp 9222 snapshot # Connect via CDP port
agent-browser connect 9222 # Alternative: connect command
agent-browser console # View console messages
agent-browser console --clear # Clear console
agent-browser errors # View page errors
agent-browser errors --clear # Clear errors
agent-browser highlight @e1 # Highlight element
agent-browser trace start # Start recording trace
agent-browser trace stop trace.zip # Stop and save trace
```
## Environment Variables
```bash
AGENT_BROWSER_SESSION="mysession" # Default session name
AGENT_BROWSER_EXECUTABLE_PATH="/path/chrome" # Custom browser path
AGENT_BROWSER_EXTENSIONS="/ext1,/ext2" # Comma-separated extension paths
AGENT_BROWSER_PROVIDER="browserbase" # Cloud browser provider
AGENT_BROWSER_STREAM_PORT="9223" # WebSocket streaming port
AGENT_BROWSER_HOME="/path/to/agent-browser" # Custom install location
```
.agents/skills/agent-browser/references/proxy-support.md
+188
View File
@@ -0,0 +1,188 @@
# Proxy Support
Proxy configuration for geo-testing, rate limiting avoidance, and corporate environments.
**Related**: [commands.md](commands.md) for global options, [SKILL.md](../SKILL.md) for quick start.
## Contents
- [Basic Proxy Configuration](#basic-proxy-configuration)
- [Authenticated Proxy](#authenticated-proxy)
- [SOCKS Proxy](#socks-proxy)
- [Proxy Bypass](#proxy-bypass)
- [Common Use Cases](#common-use-cases)
- [Verifying Proxy Connection](#verifying-proxy-connection)
- [Troubleshooting](#troubleshooting)
- [Best Practices](#best-practices)
## Basic Proxy Configuration
Set proxy via environment variable before starting:
```bash
# HTTP proxy
export HTTP_PROXY="http://proxy.example.com:8080"
agent-browser open https://example.com
# HTTPS proxy
export HTTPS_PROXY="https://proxy.example.com:8080"
agent-browser open https://example.com
# Both
export HTTP_PROXY="http://proxy.example.com:8080"
export HTTPS_PROXY="http://proxy.example.com:8080"
agent-browser open https://example.com
```
## Authenticated Proxy
For proxies requiring authentication:
```bash
# Include credentials in URL
export HTTP_PROXY="http://username:password@proxy.example.com:8080"
agent-browser open https://example.com
```
## SOCKS Proxy
```bash
# SOCKS5 proxy
export ALL_PROXY="socks5://proxy.example.com:1080"
agent-browser open https://example.com
# SOCKS5 with auth
export ALL_PROXY="socks5://user:pass@proxy.example.com:1080"
agent-browser open https://example.com
```
## Proxy Bypass
Skip proxy for specific domains:
```bash
# Bypass proxy for local addresses
export NO_PROXY="localhost,127.0.0.1,.internal.company.com"
agent-browser open https://internal.company.com # Direct connection
agent-browser open https://external.com # Via proxy
```
## Common Use Cases
### Geo-Location Testing
```bash
#!/bin/bash
# Test site from different regions using geo-located proxies
PROXIES=(
"http://us-proxy.example.com:8080"
"http://eu-proxy.example.com:8080"
"http://asia-proxy.example.com:8080"
)
for proxy in "${PROXIES[@]}"; do
export HTTP_PROXY="$proxy"
export HTTPS_PROXY="$proxy"
region=$(echo "$proxy" | grep -oP '^\w+-\w+')
echo "Testing from: $region"
agent-browser --session "$region" open https://example.com
agent-browser --session "$region" screenshot "./screenshots/$region.png"
agent-browser --session "$region" close
done
```
### Rotating Proxies for Scraping
```bash
#!/bin/bash
# Rotate through proxy list to avoid rate limiting
PROXY_LIST=(
"http://proxy1.example.com:8080"
"http://proxy2.example.com:8080"
"http://proxy3.example.com:8080"
)
URLS=(
"https://site.com/page1"
"https://site.com/page2"
"https://site.com/page3"
)
for i in "${!URLS[@]}"; do
proxy_index=$((i % ${#PROXY_LIST[@]}))
export HTTP_PROXY="${PROXY_LIST[$proxy_index]}"
export HTTPS_PROXY="${PROXY_LIST[$proxy_index]}"
agent-browser open "${URLS[$i]}"
agent-browser get text body > "output-$i.txt"
agent-browser close
sleep 1 # Polite delay
done
```
### Corporate Network Access
```bash
#!/bin/bash
# Access internal sites via corporate proxy
export HTTP_PROXY="http://corpproxy.company.com:8080"
export HTTPS_PROXY="http://corpproxy.company.com:8080"
export NO_PROXY="localhost,127.0.0.1,.company.com"
# External sites go through proxy
agent-browser open https://external-vendor.com
# Internal sites bypass proxy
agent-browser open https://intranet.company.com
```
## Verifying Proxy Connection
```bash
# Check your apparent IP
agent-browser open https://httpbin.org/ip
agent-browser get text body
# Should show proxy's IP, not your real IP
```
## Troubleshooting
### Proxy Connection Failed
```bash
# Test proxy connectivity first
curl -x http://proxy.example.com:8080 https://httpbin.org/ip
# Check if proxy requires auth
export HTTP_PROXY="http://user:pass@proxy.example.com:8080"
```
### SSL/TLS Errors Through Proxy
Some proxies perform SSL inspection. If you encounter certificate errors:
```bash
# For testing only - not recommended for production
agent-browser open https://example.com --ignore-https-errors
```
### Slow Performance
```bash
# Use proxy only when necessary
export NO_PROXY="*.cdn.com,*.static.com" # Direct CDN access
```
## Best Practices
1. **Use environment variables** - Don't hardcode proxy credentials
2. **Set NO_PROXY appropriately** - Avoid routing local traffic through proxy
3. **Test proxy before automation** - Verify connectivity with simple requests
4. **Handle proxy failures gracefully** - Implement retry logic for unstable proxies
5. **Rotate proxies for large scraping jobs** - Distribute load and avoid bans
.agents/skills/agent-browser/references/session-management.md
+193
View File
@@ -0,0 +1,193 @@
# Session Management
Multiple isolated browser sessions with state persistence and concurrent browsing.
**Related**: [authentication.md](authentication.md) for login patterns, [SKILL.md](../SKILL.md) for quick start.
## Contents
- [Named Sessions](#named-sessions)
- [Session Isolation Properties](#session-isolation-properties)
- [Session State Persistence](#session-state-persistence)
- [Common Patterns](#common-patterns)
- [Default Session](#default-session)
- [Session Cleanup](#session-cleanup)
- [Best Practices](#best-practices)
## Named Sessions
Use `--session` flag to isolate browser contexts:
```bash
# Session 1: Authentication flow
agent-browser --session auth open https://app.example.com/login
# Session 2: Public browsing (separate cookies, storage)
agent-browser --session public open https://example.com
# Commands are isolated by session
agent-browser --session auth fill @e1 "user@example.com"
agent-browser --session public get text body
```
## Session Isolation Properties
Each session has independent:
- Cookies
- LocalStorage / SessionStorage
- IndexedDB
- Cache
- Browsing history
- Open tabs
## Session State Persistence
### Save Session State
```bash
# Save cookies, storage, and auth state
agent-browser state save /path/to/auth-state.json
```
### Load Session State
```bash
# Restore saved state
agent-browser state load /path/to/auth-state.json
# Continue with authenticated session
agent-browser open https://app.example.com/dashboard
```
### State File Contents
```json
{
"cookies": [...],
"localStorage": {...},
"sessionStorage": {...},
"origins": [...]
}
```
## Common Patterns
### Authenticated Session Reuse
```bash
#!/bin/bash
# Save login state once, reuse many times
STATE_FILE="/tmp/auth-state.json"
# Check if we have saved state
if [[ -f "$STATE_FILE" ]]; then
agent-browser state load "$STATE_FILE"
agent-browser open https://app.example.com/dashboard
else
# Perform login
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "$USERNAME"
agent-browser fill @e2 "$PASSWORD"
agent-browser click @e3
agent-browser wait --load networkidle
# Save for future use
agent-browser state save "$STATE_FILE"
fi
```
### Concurrent Scraping
```bash
#!/bin/bash
# Scrape multiple sites concurrently
# Start all sessions
agent-browser --session site1 open https://site1.com &
agent-browser --session site2 open https://site2.com &
agent-browser --session site3 open https://site3.com &
wait
# Extract from each
agent-browser --session site1 get text body > site1.txt
agent-browser --session site2 get text body > site2.txt
agent-browser --session site3 get text body > site3.txt
# Cleanup
agent-browser --session site1 close
agent-browser --session site2 close
agent-browser --session site3 close
```
### A/B Testing Sessions
```bash
# Test different user experiences
agent-browser --session variant-a open "https://app.com?variant=a"
agent-browser --session variant-b open "https://app.com?variant=b"
# Compare
agent-browser --session variant-a screenshot /tmp/variant-a.png
agent-browser --session variant-b screenshot /tmp/variant-b.png
```
## Default Session
When `--session` is omitted, commands use the default session:
```bash
# These use the same default session
agent-browser open https://example.com
agent-browser snapshot -i
agent-browser close # Closes default session
```
## Session Cleanup
```bash
# Close specific session
agent-browser --session auth close
# List active sessions
agent-browser session list
```
## Best Practices
### 1. Name Sessions Semantically
```bash
# GOOD: Clear purpose
agent-browser --session github-auth open https://github.com
agent-browser --session docs-scrape open https://docs.example.com
# AVOID: Generic names
agent-browser --session s1 open https://github.com
```
### 2. Always Clean Up
```bash
# Close sessions when done
agent-browser --session auth close
agent-browser --session scrape close
```
### 3. Handle State Files Securely
```bash
# Don't commit state files (contain auth tokens!)
echo "*.auth-state.json" >> .gitignore
# Delete after use
rm /tmp/auth-state.json
```
### 4. Timeout Long Sessions
```bash
# Set timeout for automated scripts
timeout 60 agent-browser --session long-task get text body
```
.agents/skills/agent-browser/references/snapshot-refs.md
+194
View File
@@ -0,0 +1,194 @@
# Snapshot and Refs
Compact element references that reduce context usage dramatically for AI agents.
**Related**: [commands.md](commands.md) for full command reference, [SKILL.md](../SKILL.md) for quick start.
## Contents
- [How Refs Work](#how-refs-work)
- [Snapshot Command](#the-snapshot-command)
- [Using Refs](#using-refs)
- [Ref Lifecycle](#ref-lifecycle)
- [Best Practices](#best-practices)
- [Ref Notation Details](#ref-notation-details)
- [Troubleshooting](#troubleshooting)
## How Refs Work
Traditional approach:
```
Full DOM/HTML → AI parses → CSS selector → Action (~3000-5000 tokens)
```
agent-browser approach:
```
Compact snapshot → @refs assigned → Direct interaction (~200-400 tokens)
```
## The Snapshot Command
```bash
# Basic snapshot (shows page structure)
agent-browser snapshot
# Interactive snapshot (-i flag) - RECOMMENDED
agent-browser snapshot -i
```
### Snapshot Output Format
```
Page: Example Site - Home
URL: https://example.com
@e1 [header]
@e2 [nav]
@e3 [a] "Home"
@e4 [a] "Products"
@e5 [a] "About"
@e6 [button] "Sign In"
@e7 [main]
@e8 [h1] "Welcome"
@e9 [form]
@e10 [input type="email"] placeholder="Email"
@e11 [input type="password"] placeholder="Password"
@e12 [button type="submit"] "Log In"
@e13 [footer]
@e14 [a] "Privacy Policy"
```
## Using Refs
Once you have refs, interact directly:
```bash
# Click the "Sign In" button
agent-browser click @e6
# Fill email input
agent-browser fill @e10 "user@example.com"
# Fill password
agent-browser fill @e11 "password123"
# Submit the form
agent-browser click @e12
```
## Ref Lifecycle
**IMPORTANT**: Refs are invalidated when the page changes!
```bash
# Get initial snapshot
agent-browser snapshot -i
# @e1 [button] "Next"
# Click triggers page change
agent-browser click @e1
# MUST re-snapshot to get new refs!
agent-browser snapshot -i
# @e1 [h1] "Page 2" ← Different element now!
```
## Best Practices
### 1. Always Snapshot Before Interacting
```bash
# CORRECT
agent-browser open https://example.com
agent-browser snapshot -i # Get refs first
agent-browser click @e1 # Use ref
# WRONG
agent-browser open https://example.com
agent-browser click @e1 # Ref doesn't exist yet!
```
### 2. Re-Snapshot After Navigation
```bash
agent-browser click @e5 # Navigates to new page
agent-browser snapshot -i # Get new refs
agent-browser click @e1 # Use new refs
```
### 3. Re-Snapshot After Dynamic Changes
```bash
agent-browser click @e1 # Opens dropdown
agent-browser snapshot -i # See dropdown items
agent-browser click @e7 # Select item
```
### 4. Snapshot Specific Regions
For complex pages, snapshot specific areas:
```bash
# Snapshot just the form
agent-browser snapshot @e9
```
## Ref Notation Details
```
@e1 [tag type="value"] "text content" placeholder="hint"
│ │ │ │ │
│ │ │ │ └─ Additional attributes
│ │ │ └─ Visible text
│ │ └─ Key attributes shown
│ └─ HTML tag name
└─ Unique ref ID
```
### Common Patterns
```
@e1 [button] "Submit" # Button with text
@e2 [input type="email"] # Email input
@e3 [input type="password"] # Password input
@e4 [a href="/page"] "Link Text" # Anchor link
@e5 [select] # Dropdown
@e6 [textarea] placeholder="Message" # Text area
@e7 [div class="modal"] # Container (when relevant)
@e8 [img alt="Logo"] # Image
@e9 [checkbox] checked # Checked checkbox
@e10 [radio] selected # Selected radio
```
## Troubleshooting
### "Ref not found" Error
```bash
# Ref may have changed - re-snapshot
agent-browser snapshot -i
```
### Element Not Visible in Snapshot
```bash
# Scroll to reveal element
agent-browser scroll --bottom
agent-browser snapshot -i
# Or wait for dynamic content
agent-browser wait 1000
agent-browser snapshot -i
```
### Too Many Elements
```bash
# Snapshot specific container
agent-browser snapshot @e5
# Or use get text for content-only extraction
agent-browser get text @e5
```
.agents/skills/agent-browser/references/video-recording.md
+173
View File
@@ -0,0 +1,173 @@
# Video Recording
Capture browser automation as video for debugging, documentation, or verification.
**Related**: [commands.md](commands.md) for full command reference, [SKILL.md](../SKILL.md) for quick start.
## Contents
- [Basic Recording](#basic-recording)
- [Recording Commands](#recording-commands)
- [Use Cases](#use-cases)
- [Best Practices](#best-practices)
- [Output Format](#output-format)
- [Limitations](#limitations)
## Basic Recording
```bash
# Start recording
agent-browser record start ./demo.webm
# Perform actions
agent-browser open https://example.com
agent-browser snapshot -i
agent-browser click @e1
agent-browser fill @e2 "test input"
# Stop and save
agent-browser record stop
```
## Recording Commands
```bash
# Start recording to file
agent-browser record start ./output.webm
# Stop current recording
agent-browser record stop
# Restart with new file (stops current + starts new)
agent-browser record restart ./take2.webm
```
## Use Cases
### Debugging Failed Automation
```bash
#!/bin/bash
# Record automation for debugging
agent-browser record start ./debug-$(date +%Y%m%d-%H%M%S).webm
# Run your automation
agent-browser open https://app.example.com
agent-browser snapshot -i
agent-browser click @e1 || {
echo "Click failed - check recording"
agent-browser record stop
exit 1
}
agent-browser record stop
```
### Documentation Generation
```bash
#!/bin/bash
# Record workflow for documentation
agent-browser record start ./docs/how-to-login.webm
agent-browser open https://app.example.com/login
agent-browser wait 1000 # Pause for visibility
agent-browser snapshot -i
agent-browser fill @e1 "demo@example.com"
agent-browser wait 500
agent-browser fill @e2 "password"
agent-browser wait 500
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser wait 1000 # Show result
agent-browser record stop
```
### CI/CD Test Evidence
```bash
#!/bin/bash
# Record E2E test runs for CI artifacts
TEST_NAME="${1:-e2e-test}"
RECORDING_DIR="./test-recordings"
mkdir -p "$RECORDING_DIR"
agent-browser record start "$RECORDING_DIR/$TEST_NAME-$(date +%s).webm"
# Run test
if run_e2e_test; then
echo "Test passed"
else
echo "Test failed - recording saved"
fi
agent-browser record stop
```
## Best Practices
### 1. Add Pauses for Clarity
```bash
# Slow down for human viewing
agent-browser click @e1
agent-browser wait 500 # Let viewer see result
```
### 2. Use Descriptive Filenames
```bash
# Include context in filename
agent-browser record start ./recordings/login-flow-2024-01-15.webm
agent-browser record start ./recordings/checkout-test-run-42.webm
```
### 3. Handle Recording in Error Cases
```bash
#!/bin/bash
set -e
cleanup() {
agent-browser record stop 2>/dev/null || true
agent-browser close 2>/dev/null || true
}
trap cleanup EXIT
agent-browser record start ./automation.webm
# ... automation steps ...
```
### 4. Combine with Screenshots
```bash
# Record video AND capture key frames
agent-browser record start ./flow.webm
agent-browser open https://example.com
agent-browser screenshot ./screenshots/step1-homepage.png
agent-browser click @e1
agent-browser screenshot ./screenshots/step2-after-click.png
agent-browser record stop
```
## Output Format
- Default format: WebM (VP8/VP9 codec)
- Compatible with all modern browsers and video players
- Compressed but high quality
## Limitations
- Recording adds slight overhead to automation
- Large recordings can consume significant disk space
- Some headless environments may have codec limitations
.agents/skills/agent-browser/templates/authenticated-session.sh
Executable
+97
View File
@@ -0,0 +1,97 @@
#!/bin/bash
# Template: Authenticated Session Workflow
# Purpose: Login once, save state, reuse for subsequent runs
# Usage: ./authenticated-session.sh <login-url> [state-file]
#
# Environment variables:
# APP_USERNAME - Login username/email
# APP_PASSWORD - Login password
#
# Two modes:
# 1. Discovery mode (default): Shows form structure so you can identify refs
# 2. Login mode: Performs actual login after you update the refs
#
# Setup steps:
# 1. Run once to see form structure (discovery mode)
# 2. Update refs in LOGIN FLOW section below
# 3. Set APP_USERNAME and APP_PASSWORD
# 4. Delete the DISCOVERY section
set -euo pipefail
LOGIN_URL="${1:?Usage: $0 <login-url> [state-file]}"
STATE_FILE="${2:-./auth-state.json}"
echo "Authentication workflow: $LOGIN_URL"
# ================================================================
# SAVED STATE: Skip login if valid saved state exists
# ================================================================
if [[ -f "$STATE_FILE" ]]; then
echo "Loading saved state from $STATE_FILE..."
agent-browser state load "$STATE_FILE"
agent-browser open "$LOGIN_URL"
agent-browser wait --load networkidle
CURRENT_URL=$(agent-browser get url)
if [[ "$CURRENT_URL" != *"login"* ]] && [[ "$CURRENT_URL" != *"signin"* ]]; then
echo "Session restored successfully"
agent-browser snapshot -i
exit 0
fi
echo "Session expired, performing fresh login..."
rm -f "$STATE_FILE"
fi
# ================================================================
# DISCOVERY MODE: Shows form structure (delete after setup)
# ================================================================
echo "Opening login page..."
agent-browser open "$LOGIN_URL"
agent-browser wait --load networkidle
echo ""
echo "Login form structure:"
echo "---"
agent-browser snapshot -i
echo "---"
echo ""
echo "Next steps:"
echo " 1. Note the refs: username=@e?, password=@e?, submit=@e?"
echo " 2. Update the LOGIN FLOW section below with your refs"
echo " 3. Set: export APP_USERNAME='...' APP_PASSWORD='...'"
echo " 4. Delete this DISCOVERY MODE section"
echo ""
agent-browser close
exit 0
# ================================================================
# LOGIN FLOW: Uncomment and customize after discovery
# ================================================================
# : "${APP_USERNAME:?Set APP_USERNAME environment variable}"
# : "${APP_PASSWORD:?Set APP_PASSWORD environment variable}"
#
# agent-browser open "$LOGIN_URL"
# agent-browser wait --load networkidle
# agent-browser snapshot -i
#
# # Fill credentials (update refs to match your form)
# agent-browser fill @e1 "$APP_USERNAME"
# agent-browser fill @e2 "$APP_PASSWORD"
# agent-browser click @e3
# agent-browser wait --load networkidle
#
# # Verify login succeeded
# FINAL_URL=$(agent-browser get url)
# if [[ "$FINAL_URL" == *"login"* ]] || [[ "$FINAL_URL" == *"signin"* ]]; then
# echo "Login failed - still on login page"
# agent-browser screenshot /tmp/login-failed.png
# agent-browser close
# exit 1
# fi
#
# # Save state for future runs
# echo "Saving state to $STATE_FILE"
# agent-browser state save "$STATE_FILE"
# echo "Login successful"
# agent-browser snapshot -i
.agents/skills/agent-browser/templates/capture-workflow.sh
Executable
+69
View File
@@ -0,0 +1,69 @@
#!/bin/bash
# Template: Content Capture Workflow
# Purpose: Extract content from web pages (text, screenshots, PDF)
# Usage: ./capture-workflow.sh <url> [output-dir]
#
# Outputs:
# - page-full.png: Full page screenshot
# - page-structure.txt: Page element structure with refs
# - page-text.txt: All text content
# - page.pdf: PDF version
#
# Optional: Load auth state for protected pages
set -euo pipefail
TARGET_URL="${1:?Usage: $0 <url> [output-dir]}"
OUTPUT_DIR="${2:-.}"
echo "Capturing: $TARGET_URL"
mkdir -p "$OUTPUT_DIR"
# Optional: Load authentication state
# if [[ -f "./auth-state.json" ]]; then
# echo "Loading authentication state..."
# agent-browser state load "./auth-state.json"
# fi
# Navigate to target
agent-browser open "$TARGET_URL"
agent-browser wait --load networkidle
# Get metadata
TITLE=$(agent-browser get title)
URL=$(agent-browser get url)
echo "Title: $TITLE"
echo "URL: $URL"
# Capture full page screenshot
agent-browser screenshot --full "$OUTPUT_DIR/page-full.png"
echo "Saved: $OUTPUT_DIR/page-full.png"
# Get page structure with refs
agent-browser snapshot -i > "$OUTPUT_DIR/page-structure.txt"
echo "Saved: $OUTPUT_DIR/page-structure.txt"
# Extract all text content
agent-browser get text body > "$OUTPUT_DIR/page-text.txt"
echo "Saved: $OUTPUT_DIR/page-text.txt"
# Save as PDF
agent-browser pdf "$OUTPUT_DIR/page.pdf"
echo "Saved: $OUTPUT_DIR/page.pdf"
# Optional: Extract specific elements using refs from structure
# agent-browser get text @e5 > "$OUTPUT_DIR/main-content.txt"
# Optional: Handle infinite scroll pages
# for i in {1..5}; do
# agent-browser scroll down 1000
# agent-browser wait 1000
# done
# agent-browser screenshot --full "$OUTPUT_DIR/page-scrolled.png"
# Cleanup
agent-browser close
echo ""
echo "Capture complete:"
ls -la "$OUTPUT_DIR"
.agents/skills/agent-browser/templates/form-automation.sh
Executable
+62
View File
@@ -0,0 +1,62 @@
#!/bin/bash
# Template: Form Automation Workflow
# Purpose: Fill and submit web forms with validation
# Usage: ./form-automation.sh <form-url>
#
# This template demonstrates the snapshot-interact-verify pattern:
# 1. Navigate to form
# 2. Snapshot to get element refs
# 3. Fill fields using refs
# 4. Submit and verify result
#
# Customize: Update the refs (@e1, @e2, etc.) based on your form's snapshot output
set -euo pipefail
FORM_URL="${1:?Usage: $0 <form-url>}"
echo "Form automation: $FORM_URL"
# Step 1: Navigate to form
agent-browser open "$FORM_URL"
agent-browser wait --load networkidle
# Step 2: Snapshot to discover form elements
echo ""
echo "Form structure:"
agent-browser snapshot -i
# Step 3: Fill form fields (customize these refs based on snapshot output)
#
# Common field types:
# agent-browser fill @e1 "John Doe" # Text input
# agent-browser fill @e2 "user@example.com" # Email input
# agent-browser fill @e3 "SecureP@ss123" # Password input
# agent-browser select @e4 "Option Value" # Dropdown
# agent-browser check @e5 # Checkbox
# agent-browser click @e6 # Radio button
# agent-browser fill @e7 "Multi-line text" # Textarea
# agent-browser upload @e8 /path/to/file.pdf # File upload
#
# Uncomment and modify:
# agent-browser fill @e1 "Test User"
# agent-browser fill @e2 "test@example.com"
# agent-browser click @e3 # Submit button
# Step 4: Wait for submission
# agent-browser wait --load networkidle
# agent-browser wait --url "**/success" # Or wait for redirect
# Step 5: Verify result
echo ""
echo "Result:"
agent-browser get url
agent-browser snapshot -i
# Optional: Capture evidence
agent-browser screenshot /tmp/form-result.png
echo "Screenshot saved: /tmp/form-result.png"
# Cleanup
agent-browser close
echo "Done"
.agents/skills/create-documentation/SKILL.md
+337
View File
@@ -0,0 +1,337 @@
---
name: create-documentation
description: Generate markdown documentation for a module or feature
---
You are creating proper markdown documentation for a feature or guide in the Documenso documentation site.
**Read [WRITING_STYLE.md](../../../WRITING_STYLE.md) first** for tone, formatting conventions, and anti-patterns to avoid.
## Your Task
1. **Identify the scope** - Based on the conversation context, determine what feature or topic needs documentation. Ask the user if unclear.
2. **Identify the audience** - Is this for Users, Developers, or Self-Hosters?
3. **Read the source code** - Understand the feature, API, or configuration being documented.
4. **Read existing docs** - Check `apps/docs/content/docs/` for documentation to update.
5. **Write comprehensive documentation** - Create or update MDX docs following the patterns below.
6. **Update navigation** - Add to the relevant `meta.json` if creating a new page.
## Documentation Framework
This project uses [Fumadocs](https://fumadocs.dev). All documentation lives in `apps/docs/content/docs/` as MDX files. The docs app is a Next.js app at `apps/docs/`.
## Documentation Structure
```
apps/docs/content/docs/
├── index.mdx # Landing page with audience navigation
├── meta.json # Root navigation: guides + resources
├── users/ # Application usage guides
│ ├── meta.json # { "root": true, "pages": [...] }
│ ├── getting-started/ # Account creation, first document
│ ├── documents/ # Upload, recipients, fields, send
│ │ └── advanced/ # AI detection, visibility, placeholders
│ ├── templates/ # Create and use templates
│ ├── organisations/ # Overview, members, groups, SSO, billing
│ │ ├── single-sign-on/
│ │ └── preferences/
│ └── settings/ # Profile, security, API tokens
├── developers/ # API and integration docs
│ ├── meta.json # { "root": true, "pages": [...] }
│ ├── getting-started/ # Authentication, first API call
│ ├── api/ # Documents, recipients, fields, templates, teams
│ ├── webhooks/ # Setup, events, verification
│ ├── embedding/ # Authoring, direct links, CSS vars, SDKs
│ │ └── sdks/ # React, Vue, Svelte, Solid, Preact, Angular
│ ├── examples/ # Common workflows
│ ├── local-development/ # Quickstart, manual, translations
│ └── contributing/ # Contributing translations
├── self-hosting/ # Self-hosting documentation
│ ├── meta.json # { "root": true, "pages": [...] }
│ ├── getting-started/ # Quick start, requirements, tips
│ ├── deployment/ # Docker, docker-compose, Kubernetes, Railway
│ ├── configuration/ # Environment, database, email, storage
│ │ ├── signing-certificate/ # Local, Google Cloud HSM, timestamp
│ │ └── advanced/ # OAuth providers, AI features
│ └── maintenance/ # Upgrades, backups, troubleshooting
├── concepts/ # Shared across audiences
│ └── ... # Document lifecycle, field types, signing
├── compliance/ # eSign, GDPR, standards, certifications
└── policies/ # Terms, privacy, security, licenses
```
### Where to Put Documentation
| Type | Location | When to use |
| ------------------- | ------------------------------------------------ | -------------------------------------------------- |
| **User Guide** | `apps/docs/content/docs/users/<section>/` | UI workflows for using the Documenso web app |
| **Developer Guide** | `apps/docs/content/docs/developers/<section>/` | API reference, SDK guides, webhooks, embedding |
| **Self-Hosting** | `apps/docs/content/docs/self-hosting/<section>/` | Deployment, configuration, environment variables |
| **Concept** | `apps/docs/content/docs/concepts/` | Cross-audience concepts (document lifecycle, etc.) |
| **Compliance** | `apps/docs/content/docs/compliance/` | Legal and regulatory documentation |
### Navigation (meta.json)
Each directory has a `meta.json` controlling navigation order:
```json
{
"title": "Section Title",
"pages": ["getting-started", "documents", "templates"]
}
```
Top-level audience sections use `"root": true`:
```json
{
"title": "Users",
"description": "Send and sign documents",
"root": true,
"pages": ["getting-started", "documents", "templates", "organisations", "settings"]
}
```
Root `meta.json` uses `---Label---` for section dividers:
```json
{
"title": "Documentation",
"pages": [
"---Guides---",
"users",
"developers",
"self-hosting",
"---Resources---",
"concepts",
"compliance",
"policies"
]
}
```
## MDX File Format
### Frontmatter
Every page needs frontmatter:
```yaml
---
title: Upload Documents
description: Upload documents to Documenso to prepare them for signing. Covers supported formats, file size limits, and upload methods.
---
```
### Fumadocs Components
Import components at the top of the file (after frontmatter):
```mdx
import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';
import { Callout } from 'fumadocs-ui/components/callout';
import { Step, Steps } from 'fumadocs-ui/components/steps';
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';
;
```
Callouts (use sparingly for warnings, beta features, security):
```mdx
<Callout type="info">Informational note about behavior.</Callout>
<Callout type="warn">Warning about potential issues or breaking changes.</Callout>
<Callout type="error">Critical warning about data loss or security.</Callout>
```
Steps (for sequential UI instructions):
```mdx
{/* prettier-ignore */}
<Steps>
<Step>
### Step title
Step description.
</Step>
<Step>
### Next step
Next description.
</Step>
</Steps>
```
Tabs (for multiple approaches or platforms):
````mdx
<Tabs items={['cURL', 'JavaScript', 'Python']}>
<Tab value="cURL">```bash curl -X POST ... ```</Tab>
<Tab value="JavaScript">```typescript const response = await fetch(...) ```</Tab>
</Tabs>
````
## Page Structure by Audience
### User Documentation
```mdx
---
title: Feature Name
description: Brief description for SEO and previews.
---
import { Callout } from 'fumadocs-ui/components/callout';
import { Step, Steps } from 'fumadocs-ui/components/steps';
## Limitations
| Limitation | Value |
| ----------------- | -------- |
| Supported format | PDF only |
| Maximum file size | 50MB |
## How to Do the Thing
{/* prettier-ignore */}
<Steps>
<Step>
### Navigate to the page
Open **Settings > Feature**.
</Step>
<Step>
### Configure the setting
Fill in the required fields and click **Save**.
</Step>
</Steps>
---
## See Also
- [Related Guide](/docs/users/related)
```
### Developer Documentation
````mdx
---
title: Documents API
description: Create, manage, and send documents for signing via the API.
---
import { Callout } from 'fumadocs-ui/components/callout';
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';
<Callout type="warn">
This guide may not reflect the latest endpoints. For an always up-to-date reference, see the
[OpenAPI Reference](https://openapi.documenso.com).
</Callout>
## Overview
Brief description of the resource and what you can do with it.
## Resource Object
| Property | Type | Description |
| -------- | ------ | ----------------- |
| `id` | string | Unique identifier |
| `status` | string | Current status |
## Create a Resource
```typescript
const response = await fetch('https://app.documenso.com/api/v2/document', {
method: 'POST',
headers: {
Authorization: 'Bearer YOUR_API_TOKEN',
'Content-Type': 'application/json',
},
body: JSON.stringify({
title: 'Service Agreement',
}),
});
```
````
---
## See Also
- [Related Guide](/docs/developers/related)
````
### Self-Hosting Documentation
```mdx
---
title: Environment Variables
description: Complete reference for all environment variables used to configure Documenso.
---
## Required Variables
| Variable | Description |
| ------------------ | ------------------------------------------------ |
| `NEXTAUTH_SECRET` | Secret key for session encryption (min 32 chars) |
| `DATABASE_URL` | PostgreSQL connection URL |
---
## Optional Variables
| Variable | Default | Description |
| -------------- | ------- | ---------------------- |
| `PORT` | `3000` | Port the server runs on |
---
## See Also
- [Database Configuration](/docs/self-hosting/configuration/database)
````
## Documentation Audiences
Tailor content to the audience:
- **User docs**: Focus on UI workflows, bold UI elements (**Settings**, **Save**), use `>` for navigation paths (**Settings > Team > Members**), number sequential steps, no code required
- **Developer docs**: API/SDK examples, authentication, webhooks, code samples in TypeScript, link to OpenAPI reference
- **Self-hosting docs**: Deployment guides, environment variables, Docker/non-Docker approaches, system requirements, troubleshooting
## Guidelines
See [WRITING_STYLE.md](../../../WRITING_STYLE.md) for complete guidelines. Key points:
- **Tone**: Direct, second-person, no emojis, no excessive personality
- **Examples**: Progressive complexity, all must be valid TypeScript
- **Tables**: Use Sharp-style nested parameter tables for API docs
- **Callouts**: Use sparingly for warnings, beta features, security
- **Cross-references**: Link related docs, add "See Also" sections
- **Navigation**: Update `meta.json` when adding new pages
- **Limitations**: Explicitly list what is NOT supported
- **Images**: Use `.webp` format, store in `apps/docs/public/`
## Process
1. **Identify the audience** - Users, Developers, or Self-Hosters?
2. **Explore the code** - Read source files to understand the feature
3. **Check existing docs** - Look in `apps/docs/content/docs/` for related pages
4. **Draft the structure** - Outline sections before writing
5. **Write content** - Fill in each section following audience-specific patterns
6. **Update navigation** - Add to relevant `meta.json` if creating a new page
7. **Add cross-references** - Link from related docs, add "See Also" section
## Begin
Analyze the conversation context to determine the documentation scope, read the relevant source code, and create comprehensive MDX documentation in `apps/docs/content/docs/`.
.agents/skills/create-justification/SKILL.md
+56
View File
@@ -0,0 +1,56 @@
---
name: create-justification
description: Create a new justification file in .agents/justifications/ with a unique three-word ID, frontmatter, and formatted title
license: MIT
compatibility: opencode
metadata:
audience: agents
workflow: decision-making
---
## What I do
I help you create new justification files in the `.agents/justifications/` directory. Each justification file gets:
- A unique three-word identifier (e.g., `swift-emerald-river`)
- Frontmatter with the current date and formatted title
- Content you provide
## How to use
Run the script with a slug and content:
```bash
npx tsx scripts/create-justification.ts "decision-name" "Justification content here"
```
Or use heredoc for multi-line content:
```bash
npx tsx scripts/create-justification.ts "decision-name" << HEREDOC
Multi-line
justification content
goes here
HEREDOC
```
## File format
Files are created as: `{three-word-id}-{slug}.md`
Example: `swift-emerald-river-decision-name.md`
The file includes frontmatter:
```markdown
---
date: 2026-01-13
title: Decision Name
---
Your content here
```
## When to use me
Use this skill when you need to document the reasoning or justification for a decision, approach, or architectural choice. The unique ID ensures no filename conflicts, and the frontmatter provides metadata for organization.
.agents/skills/create-plan/SKILL.md
+56
View File
@@ -0,0 +1,56 @@
---
name: create-plan
description: Create a new plan file in .agents/plans/ with a unique three-word ID, frontmatter, and formatted title
license: MIT
compatibility: opencode
metadata:
audience: agents
workflow: planning
---
## What I do
I help you create new plan files in the `.agents/plans/` directory. Each plan file gets:
- A unique three-word identifier (e.g., `happy-blue-moon`)
- Frontmatter with the current date and formatted title
- Content you provide
## How to use
Run the script with a slug and content:
```bash
npx tsx scripts/create-plan.ts "feature-name" "Plan content here"
```
Or use heredoc for multi-line content:
```bash
npx tsx scripts/create-plan.ts "feature-name" << HEREDOC
Multi-line
plan content
goes here
HEREDOC
```
## File format
Files are created as: `{three-word-id}-{slug}.md`
Example: `happy-blue-moon-feature-name.md`
The file includes frontmatter:
```markdown
---
date: 2026-01-13
title: Feature Name
---
Your content here
```
## When to use me
Use this skill when you need to create a new plan document for a feature, task, or project. The unique ID ensures no filename conflicts, and the frontmatter provides metadata for organization.
.agents/skills/create-scratch/SKILL.md
+56
View File
@@ -0,0 +1,56 @@
---
name: create-scratch
description: Create a new scratch file in .agents/scratches/ with a unique three-word ID, frontmatter, and formatted title
license: MIT
compatibility: opencode
metadata:
audience: agents
workflow: exploration
---
## What I do
I help you create new scratch files in the `.agents/scratches/` directory. Each scratch file gets:
- A unique three-word identifier (e.g., `calm-teal-cloud`)
- Frontmatter with the current date and formatted title
- Content you provide
## How to use
Run the script with a slug and content:
```bash
npx tsx scripts/create-scratch.ts "note-name" "Scratch content here"
```
Or use heredoc for multi-line content:
```bash
npx tsx scripts/create-scratch.ts "note-name" << HEREDOC
Multi-line
scratch content
goes here
HEREDOC
```
## File format
Files are created as: `{three-word-id}-{slug}.md`
Example: `calm-teal-cloud-note-name.md`
The file includes frontmatter:
```markdown
---
date: 2026-01-13
title: Note Name
---
Your content here
```
## When to use me
Use this skill when you need to create a temporary note, exploration document, or scratch pad for ideas. The unique ID ensures no filename conflicts, and the frontmatter provides metadata for organization.
.agents/skills/envelope-editor-v2-e2e/SKILL.md
+371
View File
@@ -0,0 +1,371 @@
---
name: envelope-editor-v2-e2e
description: Writing and maintaining Playwright E2E tests for the Envelope Editor V2. Use when the user needs to create, modify, debug, or extend E2E tests in packages/app-tests/e2e/envelope-editor-v2/. Triggers include requests to "write an e2e test", "add a test for the envelope editor", "test envelope settings/recipients/fields/items/attachments", "fix a failing envelope test", or any task involving Playwright tests for the envelope editor feature.
---
# Envelope Editor V2 E2E Tests
## Overview
The Envelope Editor V2 E2E test suite lives in `packages/app-tests/e2e/envelope-editor-v2/`. Each test file covers a distinct feature area of the envelope editor and follows a strict architectural pattern that tests the **same flow** across four surfaces:
1. **Document** (`documents/<id>`) - Native document editor
2. **Template** (`templates/<id>`) - Native template editor
3. **Embedded Create** (`/embed/v2/authoring/envelope/create`) - Embedded editor creating a new envelope
4. **Embedded Edit** (`/embed/v2/authoring/envelope/edit/<id>`) - Embedded editor updating an existing envelope
## Project Structure
```
packages/app-tests/
e2e/
envelope-editor-v2/
envelope-attachments.spec.ts # Attachment CRUD
envelope-fields.spec.ts # Field placement on PDF canvas
envelope-items.spec.ts # PDF document item CRUD
envelope-recipients.spec.ts # Recipient management
envelope-settings.spec.ts # Settings dialog
fixtures/
authentication.ts # apiSignin, apiSignout
documents.ts # Document tab helpers
envelope-editor.ts # Core fixture: surface openers + locator/action helpers
generic.ts # Toast assertions, text visibility
signature.ts # Signature pad helpers
playwright.config.ts # Test configuration
```
## Core Abstraction: `TEnvelopeEditorSurface`
Every test revolves around the `TEnvelopeEditorSurface` type from `fixtures/envelope-editor.ts`. This is the central abstraction that normalizes differences between the four surfaces:
```typescript
type TEnvelopeEditorSurface = {
root: Page; // The Playwright page
isEmbedded: boolean; // true for embed surfaces
envelopeId?: string; // Set for document/template/embed-edit, undefined for embed-create
envelopeType: 'DOCUMENT' | 'TEMPLATE';
userId: number; // Seeded user ID
userEmail: string; // Seeded user email
userName: string; // Seeded user name
teamId: number; // Seeded team ID
};
```
### Surface Openers (from `fixtures/envelope-editor.ts`)
```typescript
// Native surfaces - seed user + document/template, sign in, navigate
const surface = await openDocumentEnvelopeEditor(page);
const surface = await openTemplateEnvelopeEditor(page);
// Embedded surfaces - seed user, create API token, get presign token, navigate
const surface = await openEmbeddedEnvelopeEditor(page, {
envelopeType: 'DOCUMENT' | 'TEMPLATE',
mode?: 'create' | 'edit', // default: 'create'
tokenNamePrefix?: string, // for unique API token names
externalId?: string, // optional external ID in hash
features?: EmbeddedEditorConfig, // feature flags
});
```
## Test Architecture Pattern
Every test file follows this structure, with four `test.describe` blocks grouping tests by editor surface:
### 1. Imports
```typescript
import { type Page, expect, test } from '@playwright/test';
// Prisma enums if needed for DB assertions
import { SomePrismaEnum } from '@prisma/client';
import { nanoid } from '@documenso/lib/universal/id';
import { prisma } from '@documenso/prisma';
import {
type TEnvelopeEditorSurface, // Import needed helpers from the fixture
openDocumentEnvelopeEditor,
openEmbeddedEnvelopeEditor,
openTemplateEnvelopeEditor,
persistEmbeddedEnvelope, // ... other helpers
} from '../fixtures/envelope-editor';
import { expectToastTextToBeVisible } from '../fixtures/generic';
```
### 2. Type definitions and constants
```typescript
type FlowResult = {
externalId: string;
// ... other data needed for DB assertions
};
const TEST_VALUES = {
// Centralized test data constants
};
```
### 3. Local helper functions
```typescript
// Common: open settings and set external ID for DB lookup
const openSettingsDialog = async (root: Page) => {
await getEnvelopeEditorSettingsTrigger(root).click();
await expect(root.getByRole('heading', { name: 'Document Settings' })).toBeVisible();
};
const updateExternalId = async (surface: TEnvelopeEditorSurface, externalId: string) => {
await openSettingsDialog(surface.root);
await surface.root.locator('input[name="externalId"]').fill(externalId);
await surface.root.getByRole('button', { name: 'Update' }).click();
if (!surface.isEmbedded) {
await expectToastTextToBeVisible(surface.root, 'Envelope updated');
}
};
```
### 4. The flow function
A single `runXxxFlow` function that works across ALL surfaces. It handles embedded vs non-embedded differences internally:
```typescript
const runMyFeatureFlow = async (surface: TEnvelopeEditorSurface): Promise<FlowResult> => {
const externalId = `e2e-feature-${nanoid()}`;
// For embedded create, may need to add a PDF first
if (surface.isEmbedded && !surface.envelopeId) {
await addEnvelopeItemPdf(surface.root, 'embedded-feature.pdf');
}
await updateExternalId(surface, externalId);
// Handle embedded vs native differences
if (surface.isEmbedded) {
// No "Add Myself" button in embedded mode
await setRecipientEmail(surface.root, 0, 'embedded@example.com');
} else {
await clickAddMyselfButton(surface.root);
}
// ... perform feature-specific actions ...
// Navigate away and back to verify UI persistence
await clickEnvelopeEditorStep(surface.root, 'addFields');
await clickEnvelopeEditorStep(surface.root, 'upload');
// ... assert UI state after navigation ...
return { externalId /* ... */ };
};
```
### 5. Database assertion function
Uses Prisma directly to verify data was persisted correctly:
```typescript
const assertFeaturePersistedInDatabase = async ({
surface,
externalId,
// ... expected values
}: {
surface: TEnvelopeEditorSurface;
externalId: string;
// ...
}) => {
const envelope = await prisma.envelope.findFirstOrThrow({
where: {
externalId,
userId: surface.userId,
teamId: surface.teamId,
type: surface.envelopeType,
},
include: {
// Include related data as needed
documentMeta: true,
recipients: true,
fields: true,
envelopeAttachments: true,
},
orderBy: { createdAt: 'desc' },
});
// Assert expected values
expect(envelope.someField).toBe(expectedValue);
};
```
### 6. The four `test.describe` blocks
Tests are organized into four `test.describe` blocks, one per editor surface. Each describe block contains the tests relevant to that surface. This structure allows adding multiple tests per surface while keeping them grouped:
```typescript
test.describe('document editor', () => {
test('description of what is tested', async ({ page }) => {
const surface = await openDocumentEnvelopeEditor(page);
const result = await runMyFeatureFlow(surface);
await assertFeaturePersistedInDatabase({
surface,
...result,
});
});
// Additional document-editor-specific tests here...
});
test.describe('template editor', () => {
test('description of what is tested', async ({ page }) => {
const surface = await openTemplateEnvelopeEditor(page);
const result = await runMyFeatureFlow(surface);
await assertFeaturePersistedInDatabase({
surface,
...result,
});
});
// Additional template-editor-specific tests here...
});
test.describe('embedded create', () => {
test('description of what is tested', async ({ page }) => {
const surface = await openEmbeddedEnvelopeEditor(page, {
envelopeType: 'DOCUMENT',
tokenNamePrefix: 'e2e-embed-feature',
});
const result = await runMyFeatureFlow(surface);
// IMPORTANT: Must persist before DB assertions for embedded
await persistEmbeddedEnvelope(surface);
await assertFeaturePersistedInDatabase({
surface,
...result,
});
});
// Additional embedded-create-specific tests here...
});
test.describe('embedded edit', () => {
test('description of what is tested', async ({ page }) => {
const surface = await openEmbeddedEnvelopeEditor(page, {
envelopeType: 'TEMPLATE',
mode: 'edit',
tokenNamePrefix: 'e2e-embed-feature',
});
const result = await runMyFeatureFlow(surface);
// IMPORTANT: Must persist before DB assertions for embedded
await persistEmbeddedEnvelope(surface);
await assertFeaturePersistedInDatabase({
surface,
...result,
});
});
// Additional embedded-edit-specific tests here...
});
```
When a test only applies to specific surfaces (e.g., a document-only action like "send document"), only include it in the relevant describe block(s). Not every describe block needs the same tests -- the structure groups tests by surface, not by requiring symmetry.
## Key Differences Between Surfaces
| Behavior | Document/Template | Embedded Create | Embedded Edit |
| -------------------------- | -------------------------- | ----------------------------------------- | ----------------------------------------- |
| User seeding | Seed + sign in | Seed + API token | Seed + API token + seed envelope |
| "Add Myself" button | Available | Not available | Not available |
| Toast on settings update | Yes (`'Envelope updated'`) | No | No |
| PDF already attached | Yes (1 item) | No (0 items, must upload) | Yes (1 item) |
| Delete confirmation dialog | Yes (`'Delete'` button) | No (immediate) | No (immediate) |
| DB persistence timing | Immediate (autosaved) | After `persistEmbeddedEnvelope()` | After `persistEmbeddedEnvelope()` |
| Persist button label | N/A | `'Create Document'` / `'Create Template'` | `'Update Document'` / `'Update Template'` |
## Available Fixture Helpers
### From `fixtures/envelope-editor.ts`
**Locator helpers** (return Playwright Locators):
- `getEnvelopeEditorSettingsTrigger(root)` - Settings gear button
- `getEnvelopeItemTitleInputs(root)` - Title inputs for envelope items
- `getEnvelopeItemDragHandles(root)` - Drag handles for reordering items
- `getEnvelopeItemRemoveButtons(root)` - Remove buttons for items
- `getEnvelopeItemDropzoneInput(root)` - File input for PDF upload
- `getRecipientEmailInputs(root)` - Email inputs for recipients
- `getRecipientNameInputs(root)` - Name inputs for recipients
- `getRecipientRows(root)` - Full recipient row fieldsets
- `getRecipientRemoveButtons(root)` - Remove buttons for recipients
- `getSigningOrderInputs(root)` - Signing order number inputs
**Action helpers**:
- `addEnvelopeItemPdf(root, fileName?)` - Upload a PDF to the dropzone
- `clickEnvelopeEditorStep(root, stepId)` - Navigate to a step: `'upload'`, `'addFields'`, `'preview'`
- `clickAddMyselfButton(root)` - Click "Add Myself" (native only)
- `clickAddSignerButton(root)` - Click "Add Signer"
- `setRecipientEmail(root, index, email)` - Fill recipient email
- `setRecipientName(root, index, name)` - Fill recipient name
- `setRecipientRole(root, index, roleLabel)` - Set role via combobox
- `assertRecipientRole(root, index, roleLabel)` - Assert role value
- `toggleSigningOrder(root, enabled)` - Toggle signing order switch
- `toggleAllowDictateSigners(root, enabled)` - Toggle dictate signers switch
- `setSigningOrderValue(root, index, value)` - Set signing order number
- `persistEmbeddedEnvelope(surface)` - Click Create/Update button for embedded flows
### From `fixtures/generic.ts`
- `expectTextToBeVisible(page, text)` - Assert text visible on page
- `expectTextToNotBeVisible(page, text)` - Assert text not visible
- `expectToastTextToBeVisible(page, text)` - Assert toast message visible
## External ID Pattern
Every test uses an `externalId` (e.g., `e2e-feature-${nanoid()}`) set via the settings dialog. This unique ID is then used in Prisma queries to reliably locate the envelope in the database for assertions. This is critical because multiple tests run in parallel.
## Running Tests
```bash
# Run all envelope editor tests
npm run test:dev -w @documenso/app-tests -- --grep "Envelope Editor V2"
# Run a specific test file
npm run test:dev -w @documenso/app-tests -- e2e/envelope-editor-v2/envelope-recipients.spec.ts
# Run with UI
npm run test-ui:dev -w @documenso/app-tests -- e2e/envelope-editor-v2/
# Run specific test by name
npm run test:dev -w @documenso/app-tests -- --grep "documents/<id>: add myself"
```
## Checklist When Writing a New Test
1. Create the spec file in `packages/app-tests/e2e/envelope-editor-v2/`
2. Import `TEnvelopeEditorSurface` and the three opener functions
3. Import `persistEmbeddedEnvelope` if you need DB assertions for embedded flows
4. Define a `FlowResult` type for data passed between flow and assertion
5. Define `TEST_VALUES` constants for test data
6. Write `updateExternalId` helper (or reuse the pattern)
7. Write the `runXxxFlow` function handling embedded vs native differences
8. Write the `assertXxxPersistedInDatabase` function using Prisma
9. Create four `test.describe` blocks: `'document editor'`, `'template editor'`, `'embedded create'`, `'embedded edit'`
10. Place tests inside the appropriate describe block for each surface
11. For embedded create tests, add a PDF via `addEnvelopeItemPdf` before the flow
12. For embedded tests, call `persistEmbeddedEnvelope(surface)` before DB assertions
13. Use `surface.isEmbedded` to branch on behavioral differences (toasts, "Add Myself", etc.)
## Common Pitfalls
- **Missing `persistEmbeddedEnvelope`**: Embedded flows don't autosave. You MUST call this before any DB assertions.
- **PDF required for embedded create**: Embedded create starts with 0 items. Upload a PDF before navigating to fields.
- **Toast assertions in embedded**: Don't assert toasts for settings updates in embedded mode (they don't appear).
- **Parallel test isolation**: Always use a unique `externalId` via `nanoid()` so parallel tests don't collide.
- **Navigation verification**: Navigate away from and back to the current step to verify UI state persistence (the editor may re-render).
- **Delete confirmation**: Native surfaces show a confirmation dialog for item deletion; embedded surfaces delete immediately.