mirror of
https://github.com/documenso/documenso.git
synced 2026-07-25 01:15:49 +10:00
chore: merge main, resolve biome formatting conflicts
Merge origin/main into feat/external-2fa-codes. Resolve formatting conflicts caused by biome rollout; preserve both feature streams: PR's external 2FA token + signing-session 2FA proof additions plus main's RateLimit/RecipientExpired/signingReminders/date-auto-insert. In complete-document-with-token.ts, drop the duplicate early field-fetching block introduced when main moved that logic later with date auto-insert support; keep the EXTERNAL_TWO_FACTOR_AUTH check using derivedRecipientActionAuth.
This commit is contained in:
ephraimduncan
@@ -0,0 +1,239 @@
|
||||
---
|
||||
date: 2026-03-26
|
||||
title: Bullmq Background Jobs
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
The codebase has a well-designed background job provider abstraction (`BaseJobProvider`) with two existing implementations:
|
||||
|
||||
- **InngestJobProvider** — cloud/SaaS provider, externally hosted
|
||||
- **LocalJobProvider** — database-backed (Postgres via Prisma), uses HTTP self-calls to dispatch
|
||||
|
||||
The goal is to add a third provider backed by a proper job queue library for self-hosted deployments that need more reliability than the Local provider offers.
|
||||
|
||||
### Current Architecture
|
||||
|
||||
All code lives in `packages/lib/jobs/`:
|
||||
|
||||
- `client/base.ts` — Abstract `BaseJobProvider` with 4 methods: `defineJob()`, `triggerJob()`, `getApiHandler()`, `startCron()`
|
||||
- `client/client.ts` — `JobClient` facade, selects provider via `NEXT_PRIVATE_JOBS_PROVIDER` env var
|
||||
- `client/inngest.ts` — Inngest implementation
|
||||
- `client/local.ts` — Local/Postgres implementation
|
||||
- `client/_internal/job.ts` — Core types: `JobDefinition`, `JobRunIO`, `SimpleTriggerJobOptions`
|
||||
- `definitions/` — 19 job definitions (15 event-triggered, 4 cron)
|
||||
|
||||
The `JobRunIO` interface provided to handlers includes:
|
||||
|
||||
- `runTask(cacheKey, callback)` — idempotent task execution (cached via `BackgroundJobTask` table)
|
||||
- `triggerJob(cacheKey, options)` — chain jobs from within handlers
|
||||
- `wait(cacheKey, ms)` — delay/sleep (not implemented in Local provider)
|
||||
- `logger` — structured logging
|
||||
|
||||
### Local Provider Limitations
|
||||
|
||||
The current Local provider has several issues that motivate this work:
|
||||
|
||||
1. `io.wait()` throws "Not implemented"
|
||||
2. HTTP self-call with 150ms fire-and-forget `Promise.race` is fragile
|
||||
3. No concurrency control — jobs run in the web server process
|
||||
4. No real retry backoff (immediate re-dispatch)
|
||||
5. No monitoring/visibility into job status
|
||||
6. Jobs compete for resources with HTTP request handling
|
||||
|
||||
---
|
||||
|
||||
## Provider Evaluation
|
||||
|
||||
Three alternatives were evaluated against the existing provider interface and project requirements.
|
||||
|
||||
### BullMQ (Redis-backed) — Recommended
|
||||
|
||||
| Attribute | Detail |
|
||||
| ------------------- | -------------------------- |
|
||||
| Backend | Redis 7.x |
|
||||
| npm downloads/month | ~15M |
|
||||
| TypeScript | Native |
|
||||
| Delayed jobs | Yes (ms precision) |
|
||||
| Cron/repeatable | Yes (`upsertJobScheduler`) |
|
||||
| Retries + backoff | Yes (exponential, custom) |
|
||||
| Concurrency control | Yes (per-worker) |
|
||||
| Rate limiting | Yes (per-queue, per-group) |
|
||||
| Dashboard | Bull Board (mature) |
|
||||
| New infrastructure | Yes — Redis required |
|
||||
|
||||
**Why BullMQ**: Most mature and widely-adopted Node.js queue. Native delayed jobs solve the `io.wait()` gap. Redis is purpose-built for queue workloads and keeps Postgres clean for application data. Bull Board gives immediate operational visibility. The provider abstraction already exists so wrapping BullMQ is straightforward.
|
||||
|
||||
**Trade-off**: Requires Redis, which is additional infrastructure. However, Redis is a single Docker Compose service or a free Upstash tier, and the operational benefit is significant.
|
||||
|
||||
### pg-boss (PostgreSQL-backed) — Strong Alternative
|
||||
|
||||
| Attribute | Detail |
|
||||
| ------------------- | ----------------------------- |
|
||||
| Backend | PostgreSQL (existing) |
|
||||
| npm downloads/month | ~1.4M |
|
||||
| TypeScript | Native |
|
||||
| Delayed jobs | Yes (`startAfter`) |
|
||||
| Cron/repeatable | Yes (`schedule()`) |
|
||||
| New infrastructure | No — reuses existing Postgres |
|
||||
|
||||
**Why it could work**: Zero new infrastructure since the project already uses Postgres. API maps well to existing patterns.
|
||||
|
||||
**Why it's second choice**: Polling-based (no LISTEN/NOTIFY), adds write amplification to the primary database, smaller ecosystem, no dashboard. At scale, queue operations on the primary database become a concern.
|
||||
|
||||
### Graphile Worker (PostgreSQL-backed) — Less Suitable
|
||||
|
||||
Uses LISTEN/NOTIFY for instant pickup but has a file-based task convention and separate schema that don't mesh well with the existing Prisma-centric architecture. Would require more adapter work.
|
||||
|
||||
### Improving the Local Provider — Not Recommended
|
||||
|
||||
Fixing the Local provider's issues (wait support, replacing HTTP self-calls, adding concurrency control, backoff) essentially means rebuilding a queue library from scratch with less robustness and no community maintenance.
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Proceed with BullMQ.** It's the most capable option, maps cleanly to the existing provider interface, and is the standard choice for production Node.js applications. Redis is lightweight infrastructure with managed options available at every cloud provider.
|
||||
|
||||
**If Redis is a hard blocker**, pg-boss is the clear fallback — but the plan below assumes BullMQ.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: BullMQ Provider Core
|
||||
|
||||
**File: `packages/lib/jobs/client/bullmq.ts`**
|
||||
|
||||
Create `BullMQJobProvider extends BaseJobProvider` with singleton pattern matching the existing providers.
|
||||
|
||||
Key implementation details:
|
||||
|
||||
1. **Constructor / `getInstance()`**
|
||||
- Initialize a Redis `IORedis` connection using new env var: `NEXT_PRIVATE_REDIS_URL`
|
||||
- Create a single `Queue` instance for dispatching jobs, using `NEXT_PRIVATE_REDIS_PREFIX` as the BullMQ `prefix` option (defaults to `documenso` if unset). This namespaces all Redis keys so multiple environments (worktrees, branches, developers) sharing the same Redis instance don't collide.
|
||||
- Create a single `Worker` instance for processing jobs (in-process, same prefix)
|
||||
- Store job definitions in a `_jobDefinitions` record (same pattern as Local provider)
|
||||
|
||||
2. **`defineJob()`**
|
||||
- Store definition in `_jobDefinitions` keyed by ID
|
||||
- If the definition has a `trigger.cron`, register it via `queue.upsertJobScheduler()` with the cron expression
|
||||
|
||||
3. **`triggerJob(options)`**
|
||||
- Find eligible definitions by `trigger.name` (same lookup as Local provider)
|
||||
- For each, call `queue.add(jobDefinitionId, payload)` with appropriate options
|
||||
- Support `options.id` for deduplication via BullMQ's `jobId` option
|
||||
|
||||
4. **`getApiHandler()`**
|
||||
- Return a minimal health-check / queue-status handler. Unlike the Local provider, BullMQ workers don't need an HTTP endpoint to receive jobs — they pull from Redis directly. The API handler can return queue metrics for monitoring.
|
||||
|
||||
5. **`startCron()`**
|
||||
- No-op — cron is handled by BullMQ's `upsertJobScheduler` registered during `defineJob()`
|
||||
|
||||
6. **Worker setup**
|
||||
- Single worker processes all job types by dispatching to the correct handler from `_jobDefinitions`
|
||||
- Configure concurrency with a default of 10 (overridable via `NEXT_PRIVATE_BULLMQ_CONCURRENCY` env var for those who need to tune it)
|
||||
- Configure retry with exponential backoff: `backoff: { type: 'exponential', delay: 1000 }`
|
||||
- Default 3 retries (matching current Local provider behavior)
|
||||
|
||||
7. **`createJobRunIO(jobId)`** — Implement `JobRunIO`:
|
||||
- `runTask()`: Reuse the existing `BackgroundJobTask` Prisma table for idempotent task tracking (same pattern as Local provider)
|
||||
- `triggerJob()`: Delegate to `this.triggerJob()`
|
||||
- `wait()`: Throw "Not implemented" (same as Local provider). No handler uses `io.wait()` so this has zero impact
|
||||
- `logger`: Same console-based logger pattern as Local provider
|
||||
|
||||
### Phase 2: Provider Registration
|
||||
|
||||
**File: `packages/lib/jobs/client/client.ts`**
|
||||
|
||||
Add `'bullmq'` case to the provider match:
|
||||
|
||||
```typescript
|
||||
this._provider = match(env('NEXT_PRIVATE_JOBS_PROVIDER'))
|
||||
.with('inngest', () => InngestJobProvider.getInstance())
|
||||
.with('bullmq', () => BullMQJobProvider.getInstance())
|
||||
.otherwise(() => LocalJobProvider.getInstance());
|
||||
```
|
||||
|
||||
**File: `packages/tsconfig/process-env.d.ts`**
|
||||
|
||||
Add `'bullmq'` to the `NEXT_PRIVATE_JOBS_PROVIDER` type union and add Redis env var types:
|
||||
|
||||
```typescript
|
||||
NEXT_PRIVATE_JOBS_PROVIDER?: 'inngest' | 'local' | 'bullmq';
|
||||
NEXT_PRIVATE_REDIS_URL?: string;
|
||||
NEXT_PRIVATE_REDIS_PREFIX?: string;
|
||||
NEXT_PRIVATE_BULLMQ_CONCURRENCY?: string;
|
||||
```
|
||||
|
||||
**File: `.env.example`**
|
||||
|
||||
Add Redis configuration examples:
|
||||
|
||||
```env
|
||||
NEXT_PRIVATE_JOBS_PROVIDER="local" # Options: local, inngest, bullmq
|
||||
NEXT_PRIVATE_REDIS_URL="redis://localhost:63790"
|
||||
NEXT_PRIVATE_REDIS_PREFIX="documenso" # Namespace for Redis keys (useful when sharing a Redis instance)
|
||||
```
|
||||
|
||||
**File: `turbo.json`**
|
||||
|
||||
Add `NEXT_PRIVATE_REDIS_URL`, `NEXT_PRIVATE_REDIS_PREFIX`, and `NEXT_PRIVATE_BULLMQ_CONCURRENCY` to the env vars list for cache invalidation.
|
||||
|
||||
### Phase 3: Infrastructure & Dependencies
|
||||
|
||||
**File: `packages/lib/package.json`**
|
||||
|
||||
Add dependencies:
|
||||
|
||||
- `bullmq` — the queue library
|
||||
- `ioredis` — Redis client (peer dependency of BullMQ, but explicit is better)
|
||||
|
||||
**File: `docker-compose.yml` (or equivalent)**
|
||||
|
||||
Add Redis service for local development:
|
||||
|
||||
```yaml
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
ports:
|
||||
- '6379:6379'
|
||||
```
|
||||
|
||||
### Phase 4: Optional Enhancements
|
||||
|
||||
These are not required for the initial implementation but worth considering for follow-up:
|
||||
|
||||
1. **Bull Board integration** — Add a `/api/jobs/dashboard` route that serves Bull Board UI for monitoring. Gate behind an admin auth check.
|
||||
|
||||
2. **Separate worker process** — Add an `apps/worker` entry point that runs BullMQ workers without the web server, for deployments that want to isolate job processing from request handling.
|
||||
|
||||
3. **Graceful shutdown** — Register `SIGTERM`/`SIGINT` handlers to call `worker.close()` and `queue.close()` for clean shutdown.
|
||||
|
||||
4. **BackgroundJob table integration** — Optionally continue writing to the `BackgroundJob` Prisma table for audit/history, using BullMQ events (`completed`, `failed`) to update status. This preserves the existing database-level visibility.
|
||||
|
||||
---
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
| ------------------------------------ | ---------- | ---------------------------------------- |
|
||||
| `packages/lib/jobs/client/bullmq.ts` | **Create** | BullMQ provider implementation |
|
||||
| `packages/lib/jobs/client/client.ts` | Modify | Add `'bullmq'` provider case |
|
||||
| `packages/tsconfig/process-env.d.ts` | Modify | Add type for `'bullmq'` + Redis env vars |
|
||||
| `.env.example` | Modify | Add Redis config example |
|
||||
| `turbo.json` | Modify | Add Redis env var to cache keys |
|
||||
| `packages/lib/package.json` | Modify | Add `bullmq` + `ioredis` dependencies |
|
||||
| `docker-compose.yml` | Modify | Add Redis service |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should the BullMQ provider also write to the `BackgroundJob` Prisma table?** This would maintain audit history and allow existing admin tooling to query job status. Trade-off is dual-write complexity.
|
||||
|
||||
2. **Redis connection resilience**: Should the provider gracefully degrade if Redis is unavailable (e.g., fall back to Local provider), or fail hard? Failing hard is simpler and more predictable.
|
||||
|
||||
## Resolved Questions
|
||||
|
||||
- **`io.wait()`**: Not a concern. Only Inngest implements it (via `step.sleep`), the Local provider throws "Not implemented", and no job handler calls `io.wait()`. The BullMQ provider can throw "Not implemented" identically to the Local provider.
|
||||
@@ -0,0 +1,519 @@
|
||||
---
|
||||
date: 2026-02-10
|
||||
title: Envelope Expiration
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Envelopes (documents sent for signing) should automatically expire after a configurable period, preventing recipients from completing stale documents. Expiration is tracked **per-recipient** — when a recipient's signing window lapses, the document owner is notified and can resend (extending the deadline) or cancel. The document itself stays PENDING so other recipients can continue signing.
|
||||
|
||||
**Settings cascade**: Organisation → Team → Document (each level can override the prior).
|
||||
**Default**: 1 month from when the envelope is sent (transitions to PENDING).
|
||||
|
||||
---
|
||||
|
||||
## 1. Database Schema Changes
|
||||
|
||||
### 1.1 Expiration period data shape
|
||||
|
||||
Store expiration as a structured JSON object rather than an enum or raw milliseconds. This avoids the enum treadmill (adding `FOUR_MONTHS` later requires a migration) while keeping values validated and meaningful.
|
||||
|
||||
**Zod schema** (defined in `packages/lib/constants/envelope-expiration.ts`):
|
||||
|
||||
```typescript
|
||||
export const ZEnvelopeExpirationPeriod = z.union([
|
||||
z.object({ unit: z.enum(['day', 'week', 'month', 'year']), amount: z.number().int().min(1) }),
|
||||
z.object({ disabled: z.literal(true) }),
|
||||
]);
|
||||
|
||||
export type TEnvelopeExpirationPeriod = z.infer<typeof ZEnvelopeExpirationPeriod>;
|
||||
```
|
||||
|
||||
Semantics:
|
||||
|
||||
- `null` on `DocumentMeta` / `TeamGlobalSettings` = inherit from parent
|
||||
- `{ disabled: true }` = explicitly never expires
|
||||
- `{ unit: 'month', amount: 1 }` = expires in 1 month
|
||||
|
||||
No Prisma enum is needed — the period is stored as `Json?` on the relevant models (see sections 1.3 and 1.4).
|
||||
|
||||
### 1.2 Add expiration fields to `Recipient`
|
||||
|
||||
```prisma
|
||||
model Recipient {
|
||||
// ... existing fields
|
||||
expiresAt DateTime?
|
||||
expirationNotifiedAt DateTime? // null = not yet notified; set when owner notification sent
|
||||
|
||||
@@index([expiresAt])
|
||||
}
|
||||
```
|
||||
|
||||
`expiresAt` is a computed timestamp set when the envelope transitions to PENDING (at send time). It is calculated from the effective expiration period. Storing the concrete timestamp rather than a relative duration means:
|
||||
|
||||
- Sweep queries are simple (`WHERE expiresAt <= NOW() AND expirationNotifiedAt IS NULL`)
|
||||
- No need to re-resolve the settings cascade at query time
|
||||
- The sender can see the exact deadline in the UI
|
||||
- The index on `expiresAt` ensures the expiration sweep query is efficient
|
||||
|
||||
`expirationNotifiedAt` tracks whether the owner has already been notified about this recipient's expiration, making the notification job idempotent.
|
||||
|
||||
### 1.3 Add expiration period to settings models
|
||||
|
||||
**OrganisationGlobalSettings** (JSON, application-level default):
|
||||
|
||||
```prisma
|
||||
model OrganisationGlobalSettings {
|
||||
// ... existing fields
|
||||
envelopeExpirationPeriod Json?
|
||||
}
|
||||
```
|
||||
|
||||
Prisma `@default` doesn't work for `Json` columns, so the application-level default (`{ unit: 'month', amount: 1 }`) is applied in `extractDerivedTeamSettings` / `extractDerivedDocumentMeta` when the value is null. The migration should backfill existing rows with `{ "unit": "month", "amount": 1 }`.
|
||||
|
||||
**TeamGlobalSettings** (nullable, null = inherit from org):
|
||||
|
||||
```prisma
|
||||
model TeamGlobalSettings {
|
||||
// ... existing fields
|
||||
envelopeExpirationPeriod Json?
|
||||
}
|
||||
```
|
||||
|
||||
### 1.4 Add expiration period to DocumentMeta
|
||||
|
||||
This allows per-document override during the document editing flow:
|
||||
|
||||
```prisma
|
||||
model DocumentMeta {
|
||||
// ... existing fields
|
||||
envelopeExpirationPeriod Json?
|
||||
}
|
||||
```
|
||||
|
||||
When null on DocumentMeta, the resolved team/org setting is used at send time. Validated at write time using `ZEnvelopeExpirationPeriod.nullable()`.
|
||||
|
||||
**Important**: `envelopeExpirationPeriod` on `DocumentMeta` is a user-facing preference that may be set during the draft editing flow. It does NOT determine the final expiration — that is resolved at send time (see section 2.3). The value stored here is just the user's selection in the document editor.
|
||||
|
||||
---
|
||||
|
||||
## 2. Expiration Period Resolution
|
||||
|
||||
### 2.1 Duration mapping
|
||||
|
||||
Add to `packages/lib/constants/envelope-expiration.ts` alongside the Zod schema:
|
||||
|
||||
```typescript
|
||||
import { Duration } from 'luxon';
|
||||
|
||||
const UNIT_TO_LUXON_KEY: Record<TEnvelopeExpirationPeriod['unit'], string> = {
|
||||
day: 'days',
|
||||
week: 'weeks',
|
||||
month: 'months',
|
||||
year: 'years',
|
||||
};
|
||||
|
||||
export const DEFAULT_ENVELOPE_EXPIRATION_PERIOD: TEnvelopeExpirationPeriod = {
|
||||
unit: 'month',
|
||||
amount: 1,
|
||||
};
|
||||
|
||||
export const getEnvelopeExpirationDuration = (period: TEnvelopeExpirationPeriod): Duration => {
|
||||
return Duration.fromObject({ [UNIT_TO_LUXON_KEY[period.unit]]: period.amount });
|
||||
};
|
||||
```
|
||||
|
||||
### 2.2 Settings cascade integration
|
||||
|
||||
`extractDerivedTeamSettings()` in `packages/lib/utils/teams.ts` needs **no code changes** — it iterates `Object.keys(derivedSettings)` and overrides with non-null team values at runtime. The new `envelopeExpirationPeriod` field on both `OrganisationGlobalSettings` and `TeamGlobalSettings` will be automatically picked up.
|
||||
|
||||
Update `extractDerivedDocumentMeta()` in `packages/lib/utils/document.ts` to include the new field:
|
||||
|
||||
```typescript
|
||||
envelopeExpirationPeriod: meta.envelopeExpirationPeriod ?? settings.envelopeExpirationPeriod,
|
||||
```
|
||||
|
||||
### 2.3 Compute `expiresAt` at send time
|
||||
|
||||
The expiration period is **locked at send time** — when the envelope transitions to PENDING. The concrete `expiresAt` timestamp is computed for each recipient when the document is actually sent.
|
||||
|
||||
In `packages/lib/server-only/document/send-document.ts`:
|
||||
|
||||
```typescript
|
||||
// Resolve effective period: document meta -> team/org settings -> default
|
||||
const rawPeriod =
|
||||
envelope.documentMeta?.envelopeExpirationPeriod ?? settings.envelopeExpirationPeriod;
|
||||
|
||||
const expiresAt = resolveExpiresAt(rawPeriod);
|
||||
|
||||
// Inside the $transaction, for each recipient:
|
||||
await tx.recipient.updateMany({
|
||||
where: { envelopeId: envelope.id },
|
||||
data: { expiresAt },
|
||||
});
|
||||
```
|
||||
|
||||
### 2.4 Compute `expiresAt` in the direct template flow
|
||||
|
||||
`create-document-from-direct-template.ts` creates envelopes directly as PENDING and then calls `sendDocument` afterward. Since `sendDocument` handles setting `expiresAt` on recipients, the direct template flow doesn't need to set it directly — `sendDocument` handles it.
|
||||
|
||||
---
|
||||
|
||||
## 3. Cron Job Infrastructure (New)
|
||||
|
||||
The current job system is purely event-triggered. Inngest natively supports cron-triggered functions, but the local provider (used in dev and by self-hosters who don't want a third-party dependency) has no scheduling capability. This section adds cron support to the local provider to maintain feature parity.
|
||||
|
||||
### 3.1 Extend `JobDefinition` with cron support
|
||||
|
||||
Add an optional `cron` field to the trigger type in `packages/lib/jobs/client/_internal/job.ts`:
|
||||
|
||||
```typescript
|
||||
export type JobDefinition<Name extends string = string, Schema = any> = {
|
||||
id: string;
|
||||
name: string;
|
||||
version: string;
|
||||
enabled?: boolean;
|
||||
optimizeParallelism?: boolean;
|
||||
trigger: {
|
||||
name: Name;
|
||||
schema?: z.ZodType<Schema>;
|
||||
/** Cron expression (e.g. "* * * * *"). When set, the job runs on a schedule. */
|
||||
cron?: string;
|
||||
};
|
||||
handler: (options: { payload: Schema; io: JobRunIO }) => Promise<Json | void>;
|
||||
};
|
||||
```
|
||||
|
||||
### 3.2 Inngest provider: wire up native cron
|
||||
|
||||
In `packages/lib/jobs/client/inngest.ts`, when defining a function, check for `cron`:
|
||||
|
||||
```typescript
|
||||
defineJob(job) {
|
||||
if (job.trigger.cron) {
|
||||
this._functions.push(
|
||||
this._client.createFunction(
|
||||
{ id: job.id, name: job.name },
|
||||
{ cron: job.trigger.cron },
|
||||
async ({ step, logger }) => {
|
||||
const io = convertInngestIoToJobRunIo(step, logger, this);
|
||||
await job.handler({ payload: {} as any, io });
|
||||
},
|
||||
),
|
||||
);
|
||||
} else {
|
||||
// Existing event-triggered logic (unchanged)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Local provider: poller + deterministic `BackgroundJob` IDs
|
||||
|
||||
Use the existing `BackgroundJob` table for multi-instance dedupe instead of advisory locks. This approach keeps implementation Prisma-only (no raw SQL), works for single-instance and multi-instance deployments, and preserves existing retry/visibility behavior.
|
||||
|
||||
**On `defineJob()`**: If the job has a `cron` field, register an in-process scheduler entry and start a lightweight poller (every 30s with jitter).
|
||||
|
||||
**Each poll tick**:
|
||||
|
||||
1. Evaluate whether the cron schedule has one or more due run slots since the last tick (use a real cron parser, e.g. `cron-parser`)
|
||||
2. For each due slot, build a deterministic run ID from job ID + scheduled slot time
|
||||
3. Create a `BackgroundJob` row with that deterministic ID using Prisma
|
||||
4. If insert succeeds → enqueue via the existing local job pipeline
|
||||
5. If insert fails with Prisma `P2002` (unique violation) → another node already enqueued that run, skip
|
||||
|
||||
### 3.4 Summary of changes to the job system
|
||||
|
||||
| File | Change |
|
||||
| ------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| `packages/lib/jobs/client/_internal/job.ts` | Add optional `cron` field to `trigger` type |
|
||||
| `packages/lib/jobs/client/local.ts` | Add cron poller + deterministic `BackgroundJob.id` dedupe |
|
||||
| `packages/lib/jobs/client/inngest.ts` | Wire up `{ cron: ... }` in `createFunction` for cron jobs |
|
||||
| `packages/lib/jobs/client/_internal/*` | Add cron helper utilities (`getDueCronSlots`, run ID generation) |
|
||||
|
||||
---
|
||||
|
||||
## 4. Expiration Processing
|
||||
|
||||
### 4.1 Two-job architecture
|
||||
|
||||
Expiration uses two jobs: a **sweep dispatcher** that runs on a cron schedule and finds expired recipients, and an **individual notification job** that handles the audit log, owner notification email, and webhook for a single recipient. This separation means:
|
||||
|
||||
- The sweep is lightweight and fast (just a query + N job triggers)
|
||||
- Each recipient's expiration notification is independently retryable
|
||||
- The individual jobs are idempotent — they check `expirationNotifiedAt IS NULL` before processing
|
||||
|
||||
### 4.2 Sweep job: `EXPIRE_RECIPIENTS_SWEEP_JOB`
|
||||
|
||||
A cron-triggered job that runs every minute to find and dispatch notifications for expired recipients.
|
||||
|
||||
**Definition:** `packages/lib/jobs/definitions/internal/expire-recipients-sweep.ts`
|
||||
|
||||
**Handler:** `packages/lib/jobs/definitions/internal/expire-recipients-sweep.handler.ts`
|
||||
|
||||
```typescript
|
||||
const expiredRecipients = await prisma.recipient.findMany({
|
||||
where: {
|
||||
expiresAt: { lte: new Date() },
|
||||
expirationNotifiedAt: null,
|
||||
signingStatus: { notIn: [SigningStatus.SIGNED, SigningStatus.REJECTED] },
|
||||
envelope: { status: DocumentStatus.PENDING },
|
||||
},
|
||||
select: { id: true },
|
||||
take: 100,
|
||||
});
|
||||
|
||||
for (const recipient of expiredRecipients) {
|
||||
await jobs.triggerJob({
|
||||
name: 'internal.notify-recipient-expired',
|
||||
payload: { recipientId: recipient.id },
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 Individual notification job: `NOTIFY_RECIPIENT_EXPIRED_JOB`
|
||||
|
||||
An event-triggered job that handles a single recipient's expiration.
|
||||
|
||||
**Definition:** `packages/lib/jobs/definitions/internal/notify-recipient-expired.ts`
|
||||
|
||||
**Handler:** `packages/lib/jobs/definitions/internal/notify-recipient-expired.handler.ts`
|
||||
|
||||
The handler:
|
||||
|
||||
1. Fetches the recipient (with guard: `expirationNotifiedAt IS NULL` + not signed/rejected)
|
||||
2. Sets `recipient.expirationNotifiedAt = now()` (idempotency)
|
||||
3. Creates audit log entry with `DOCUMENT_RECIPIENT_EXPIRED` type
|
||||
4. Sends email notification to the **document owner** (inline — no separate email job)
|
||||
5. The document stays PENDING — the owner decides whether to resend or cancel
|
||||
|
||||
### 4.4 Register in job client
|
||||
|
||||
Add `EXPIRE_RECIPIENTS_SWEEP_JOB_DEFINITION` and `NOTIFY_RECIPIENT_EXPIRED_JOB_DEFINITION` to the job registry in `packages/lib/jobs/client.ts`.
|
||||
|
||||
### 4.5 Email template: Recipient Expired
|
||||
|
||||
Target the **document owner**:
|
||||
|
||||
- Subject: `Signing window expired for "{recipientName}" on "{documentTitle}"`
|
||||
- Body: "The signing window for {recipientName} ({recipientEmail}) on document {title} has expired. You can resend the document to extend their deadline or cancel the document."
|
||||
- Include a "View Document" link to the document page in the app
|
||||
|
||||
Template files:
|
||||
|
||||
- `packages/email/templates/recipient-expired.tsx` — wrapper
|
||||
- `packages/email/template-components/template-recipient-expired.tsx` — body
|
||||
|
||||
### 4.6 Recipient signing guard
|
||||
|
||||
In the signing flow, check `recipient.expiresAt` before allowing any signing action. Note that the document stays PENDING even after recipient expiration, so the existing `status !== PENDING` guard does not block expired recipients — an explicit expiration check is required:
|
||||
|
||||
```typescript
|
||||
if (recipient.expiresAt && recipient.expiresAt <= new Date()) {
|
||||
throw new AppError(AppErrorCode.RECIPIENT_EXPIRED, {
|
||||
message: 'Recipient signing window has expired',
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Files to update:**
|
||||
|
||||
- `packages/lib/server-only/document/complete-document-with-token.ts`
|
||||
- `packages/lib/server-only/field/sign-field-with-token.ts`
|
||||
- `packages/lib/server-only/field/remove-signed-field-with-token.ts`
|
||||
- `packages/lib/server-only/document/reject-document-with-token.ts`
|
||||
|
||||
---
|
||||
|
||||
## 5. UI Design
|
||||
|
||||
### 5.1 Expiration Period Selector Component
|
||||
|
||||
Use a number input + unit selector combo. This gives organisations full flexibility to configure any duration without needing schema changes for new options.
|
||||
|
||||
**Layout**: A horizontal group with:
|
||||
|
||||
- A number `<Input>` (min 1, integer)
|
||||
- A `<Select>` for the unit (`day`, `week`, `month`, `year`)
|
||||
- A "Never expires" toggle/checkbox that disables the duration inputs and sets the value to `{ disabled: true }`
|
||||
|
||||
At the team level, include an "Inherit from organisation" option that clears the value to `null`.
|
||||
|
||||
**Validation**: Use `ZEnvelopeExpirationPeriod` for form validation.
|
||||
|
||||
### 5.2 Organisation Settings → Document Preferences
|
||||
|
||||
Add a "Default Envelope Expiration" field to the `DocumentPreferencesForm` component. At the org level, there is no "Inherit" option — it must have a concrete value (default: `{ unit: 'month', amount: 1 }`).
|
||||
|
||||
### 5.3 Team Settings → Document Preferences
|
||||
|
||||
Same field as org, but with the additional "Inherit from organisation" option (stored as `null`).
|
||||
|
||||
### 5.4 Document Editor → Settings Step
|
||||
|
||||
Add the expiration selector to `packages/ui/primitives/document-flow/add-settings.tsx` inside the "Advanced Options" accordion.
|
||||
|
||||
Label: **"Expiration"**
|
||||
Description: _"How long recipients have to complete this document after it is sent."_
|
||||
|
||||
### 5.5 Recipient Signing Page — Expired State
|
||||
|
||||
When a recipient visits a signing link for an expired recipient:
|
||||
|
||||
- Redirect to `/sign/{token}/expired`
|
||||
- Show a clear, non-alarming message: "Your signing window has expired. Please contact the sender for a new invitation."
|
||||
- Do not show the signing form or fields
|
||||
- The `isExpired` flag in `get-envelope-for-recipient-signing.ts` is derived from `recipient.expiresAt`
|
||||
|
||||
### 5.6 Embed Signing — Expired State
|
||||
|
||||
Embed signing routes handle recipient expiration by throwing `embed-recipient-expired`:
|
||||
|
||||
- `apps/remix/app/routes/embed+/_v0+/sign.$token.tsx` — both V1 and V2 loaders check expiration
|
||||
- The embed error boundary renders an `EmbedRecipientExpired` component
|
||||
- Direct templates (`direct.$token.tsx`) create fresh recipients so `isExpired` is always `false`
|
||||
|
||||
---
|
||||
|
||||
## 6. API / TRPC Changes
|
||||
|
||||
### 6.1 Update settings mutation schemas
|
||||
|
||||
- `packages/trpc/server/organisation-router/update-organisation-settings.types.ts` — add `envelopeExpirationPeriod: ZEnvelopeExpirationPeriod` (non-nullable at org level)
|
||||
- `packages/trpc/server/team-router/update-team-settings.types.ts` — add `envelopeExpirationPeriod: ZEnvelopeExpirationPeriod.nullable()` (null = inherit from org)
|
||||
|
||||
### 6.2 Update document mutation schemas
|
||||
|
||||
- `packages/lib/types/document-meta.ts` — add `envelopeExpirationPeriod: ZEnvelopeExpirationPeriod.nullable()` to the meta schema
|
||||
- `packages/trpc/server/document-router/create-document.types.ts` — include in meta
|
||||
- `packages/trpc/server/document-router/update-document.types.ts` — include in meta
|
||||
- `packages/trpc/server/document-router/distribute-document.types.ts` — include in meta
|
||||
|
||||
### 6.3 Expose `expiresAt` in recipient responses
|
||||
|
||||
Ensure `expiresAt` and `expirationNotifiedAt` are returned when fetching recipients/documents so the UI can display expiration status.
|
||||
|
||||
### 6.4 Webhook / API schema updates
|
||||
|
||||
- Recipient schema includes `expiresAt` and `expirationNotifiedAt` fields (replacing the old `expired` field)
|
||||
- Update `packages/api/v1/schema.ts`, webhook payload types, zapier integration, and sample data generators
|
||||
|
||||
---
|
||||
|
||||
## 7. Edge Cases & Considerations
|
||||
|
||||
### 7.1 Already-sent documents
|
||||
|
||||
The migration should NOT retroactively expire existing recipients. `expiresAt` will be null for all existing recipients, meaning they never expire (backward-compatible).
|
||||
|
||||
### 7.2 Re-sending / redistributing
|
||||
|
||||
When `redistribute` is called on a PENDING document, `expiresAt` should be refreshed on all eligible recipients. Redistributing signals active intent, so the clock should restart.
|
||||
|
||||
**Implementation**: `resendDocument` refreshes `recipient.expiresAt` for all recipients that haven't signed/rejected yet.
|
||||
|
||||
### 7.3 Multi-recipient partial expiration
|
||||
|
||||
If some recipients have signed and others expire, the document stays PENDING. This is the key advantage over document-level expiration — the owner can resend to extend the expired recipients' deadlines without affecting those who've already signed.
|
||||
|
||||
### 7.4 Partial completion
|
||||
|
||||
Partial signatures are preserved. The document is not sealed/completed until all required recipients have signed (or the owner cancels).
|
||||
|
||||
### 7.5 Timezone handling
|
||||
|
||||
`expiresAt` is stored as UTC. Display in the sender's configured timezone.
|
||||
|
||||
### 7.6 Race condition: signing at expiration time
|
||||
|
||||
The signing guard checks `recipient.expiresAt` in application code before the signing operation. The notification job's guard (`expirationNotifiedAt IS NULL` + `signingStatus NOT IN (SIGNED, REJECTED)`) prevents double-notifications. If a recipient signs just before expiration, the sweep's `signingStatus` filter skips them.
|
||||
|
||||
### 7.7 Direct template flow
|
||||
|
||||
`create-document-from-direct-template.ts` creates envelopes directly as PENDING then calls `sendDocument`. Since `sendDocument` sets `recipient.expiresAt`, no special handling is needed in the direct template flow.
|
||||
|
||||
---
|
||||
|
||||
## 8. Migration Plan
|
||||
|
||||
1. Add Prisma schema changes (`expiresAt` + `expirationNotifiedAt` on Recipient, `Json?` fields on settings models, index)
|
||||
2. Generate and run migration
|
||||
3. Backfill: set `envelopeExpirationPeriod` to `{ "unit": "month", "amount": 1 }` on all existing `OrganisationGlobalSettings` rows
|
||||
4. No backfill on `Recipient.expiresAt` — existing recipients keep null (never expire)
|
||||
5. Deploy backend changes (jobs, guards, email template)
|
||||
6. Deploy frontend changes (settings UI, document editor, signing page, embeds)
|
||||
|
||||
---
|
||||
|
||||
## 9. Files to Create or Modify
|
||||
|
||||
### New Files
|
||||
|
||||
- `packages/lib/constants/envelope-expiration.ts` — `ZEnvelopeExpirationPeriod` schema, types, `DEFAULT_ENVELOPE_EXPIRATION_PERIOD`, `getEnvelopeExpirationDuration()`, `resolveExpiresAt()` helper
|
||||
- `packages/lib/jobs/definitions/internal/expire-recipients-sweep.ts` — cron sweep job definition
|
||||
- `packages/lib/jobs/definitions/internal/expire-recipients-sweep.handler.ts` — cron sweep handler
|
||||
- `packages/lib/jobs/definitions/internal/notify-recipient-expired.ts` — individual notification job definition
|
||||
- `packages/lib/jobs/definitions/internal/notify-recipient-expired.handler.ts` — notification handler (includes inline email sending)
|
||||
- `packages/email/templates/recipient-expired.tsx` — email template wrapper
|
||||
- `packages/email/template-components/template-recipient-expired.tsx` — email template body
|
||||
- `apps/remix/app/components/embed/embed-recipient-expired.tsx` — embed expired component
|
||||
|
||||
### Modified Files
|
||||
|
||||
**Job system (cron infrastructure):**
|
||||
|
||||
- `packages/lib/jobs/client/_internal/job.ts` — add optional `cron` field to `trigger` type
|
||||
- `packages/lib/jobs/client/local.ts` — add cron poller + deterministic `BackgroundJob.id` dedupe
|
||||
- `packages/lib/jobs/client/inngest.ts` — wire up `{ cron: ... }` in `createFunction`
|
||||
- `packages/lib/jobs/client/_internal/*` — add cron helper utilities (slot calc + run ID)
|
||||
- `packages/lib/jobs/client.ts` — register new jobs
|
||||
|
||||
**Schema & data layer:**
|
||||
|
||||
- `packages/prisma/schema.prisma` — model changes + index
|
||||
- `packages/lib/utils/document.ts` — `extractDerivedDocumentMeta` (add `envelopeExpirationPeriod`)
|
||||
- `packages/lib/server-only/document/send-document.ts` — resolve settings + compute and set `recipient.expiresAt`
|
||||
- `packages/lib/server-only/template/create-document-from-direct-template.ts` — no changes (sendDocument handles it)
|
||||
- `packages/lib/server-only/document/resend-document.ts` — refresh `recipient.expiresAt` on redistribute
|
||||
- `packages/lib/server-only/document/complete-document-with-token.ts` — recipient expiration guard
|
||||
- `packages/lib/server-only/field/sign-field-with-token.ts` — recipient expiration guard
|
||||
- `packages/lib/server-only/field/remove-signed-field-with-token.ts` — recipient expiration guard
|
||||
- `packages/lib/server-only/document/reject-document-with-token.ts` — recipient expiration guard
|
||||
|
||||
**Error handling:**
|
||||
|
||||
- `packages/lib/errors/app-error.ts` — add `RECIPIENT_EXPIRED` error code
|
||||
|
||||
**Audit logs:**
|
||||
|
||||
- `packages/lib/types/document-audit-logs.ts` — add `DOCUMENT_RECIPIENT_EXPIRED` type with `recipientEmail`/`recipientName` data fields
|
||||
- `packages/lib/utils/document-audit-logs.ts` — add human-readable rendering for `DOCUMENT_RECIPIENT_EXPIRED`
|
||||
|
||||
**Signing page:**
|
||||
|
||||
- `packages/lib/server-only/envelope/get-envelope-for-recipient-signing.ts` — derive `isExpired` from `recipient.expiresAt`
|
||||
- `apps/remix/app/routes/_recipient+/sign.$token+/_index.tsx` — keep redirect to expired page using `isExpired`
|
||||
|
||||
**Embeds:**
|
||||
|
||||
- `apps/remix/app/routes/embed+/_v0+/sign.$token.tsx` — check recipient expiration in V1/V2 loaders
|
||||
- `apps/remix/app/routes/embed+/_v0+/_layout.tsx` — handle `embed-recipient-expired` in error boundary
|
||||
|
||||
**Webhook / API:**
|
||||
|
||||
- `packages/lib/types/recipient.ts` — add `expiresAt`/`expirationNotifiedAt` to recipient type
|
||||
- `packages/lib/types/webhook-payload.ts` — add `expiresAt`/`expirationNotifiedAt` to webhook recipient
|
||||
- `packages/lib/server-only/webhooks/trigger/generate-sample-data.ts` — update sample data
|
||||
- `packages/lib/server-only/webhooks/zapier/list-documents.ts` — update zapier recipient shape
|
||||
- `packages/api/v1/schema.ts` — add `expiresAt` to API recipient schema
|
||||
|
||||
**TRPC / settings:**
|
||||
|
||||
- `packages/trpc/server/organisation-router/update-organisation-settings.types.ts`
|
||||
- `packages/trpc/server/team-router/update-team-settings.types.ts`
|
||||
- `packages/lib/types/document-meta.ts`
|
||||
|
||||
**UI:**
|
||||
|
||||
- `apps/remix/app/components/forms/document-preferences-form.tsx` — add expiration period picker
|
||||
- `packages/ui/primitives/document-flow/add-settings.tsx` — add expiration field
|
||||
- `packages/ui/primitives/document-flow/add-settings.types.ts` — add to schema
|
||||
@@ -0,0 +1,151 @@
|
||||
---
|
||||
date: 2026-03-04
|
||||
title: Swap Subscription Between Orgs
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Add the ability for admins to move a subscription (and its associated Stripe customerId) from one organisation to another, when viewing a user in the admin panel. The target org must be owned by the same user and must be on the free plan (no existing active subscription).
|
||||
|
||||
## Context & Data Model
|
||||
|
||||
- `Organisation` has a 1:1 optional `Subscription` and a `customerId` (Stripe customer ID, `@unique`)
|
||||
- `Organisation` has a 1:1 `OrganisationClaim` that tracks entitlements (team count, member count, feature flags)
|
||||
- `Subscription` also stores a redundant `customerId` and has `organisationId` (`@unique`)
|
||||
- When a subscription is removed from an org, its `OrganisationClaim` should be reset to the FREE claim
|
||||
- Relationship chain: `User --owns--> Organisation --has--> Subscription + OrganisationClaim`
|
||||
|
||||
## Constraints
|
||||
|
||||
- **paid → free only**: The target org must NOT have an active subscription (status ACTIVE or PAST_DUE). It must be on the free plan.
|
||||
- **same owner**: Both source and target orgs must be owned by the same user (the user being viewed).
|
||||
- The `customerId` must move with the subscription to the target org (cleared from source, set on target).
|
||||
- The Stripe subscription object itself is NOT modified — only the DB-level mapping changes. The Stripe customer stays the same; we just reassociate it to a different org.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### 1. Backend: TRPC Admin Route
|
||||
|
||||
**Files to create:**
|
||||
|
||||
- `packages/trpc/server/admin-router/swap-organisation-subscription.types.ts`
|
||||
- `packages/trpc/server/admin-router/swap-organisation-subscription.ts`
|
||||
|
||||
**Request schema (`ZSwapOrganisationSubscriptionRequestSchema`):**
|
||||
|
||||
```ts
|
||||
z.object({
|
||||
sourceOrganisationId: z.string(),
|
||||
targetOrganisationId: z.string(),
|
||||
});
|
||||
```
|
||||
|
||||
**Response schema:** `z.void()`
|
||||
|
||||
**Route logic (in a single `prisma.$transaction`):**
|
||||
|
||||
1. Fetch source org with `subscription` + `organisationClaim`
|
||||
2. Fetch target org with `subscription` + `organisationClaim`
|
||||
3. Validate:
|
||||
- Source org has an active subscription (status `ACTIVE` or `PAST_DUE`)
|
||||
- Target org does NOT have an active subscription (no subscription record, or status `INACTIVE`)
|
||||
- Both orgs have the same `ownerUserId`
|
||||
4. In a transaction:
|
||||
a. Clear `customerId` on source org (set to `null`)
|
||||
b. Set `customerId` on target org to the source's `customerId`
|
||||
c. Move the `Subscription` record: update `organisationId` to target org ID
|
||||
d. Copy the source org's `OrganisationClaim` entitlements to the target org's `OrganisationClaim` (`originalSubscriptionClaimId`, `teamCount`, `memberCount`, `envelopeItemCount`, `flags`)
|
||||
e. Reset the source org's `OrganisationClaim` to the FREE claim (using `createOrganisationClaimUpsertData(internalClaims[INTERNAL_CLAIM_ID.FREE])` pattern from `on-subscription-deleted.ts`)
|
||||
|
||||
**Note on ordering:** Because `Organisation.customerId` is `@unique`, we must clear the source first, then set the target — or do both in a transaction that handles the constraint. Prisma transactions handle this correctly as they apply all writes atomically.
|
||||
|
||||
**Register the route:**
|
||||
|
||||
- Import in `packages/trpc/server/admin-router/router.ts`
|
||||
- Add under `organisation` as `swapSubscription`
|
||||
- Call path: `trpc.admin.organisation.swapSubscription`
|
||||
|
||||
### 2. Frontend: Dialog Component
|
||||
|
||||
**File to create:**
|
||||
|
||||
- `apps/remix/app/components/dialogs/admin-swap-subscription-dialog.tsx`
|
||||
|
||||
**Props:**
|
||||
|
||||
```ts
|
||||
type AdminSwapSubscriptionDialogProps = {
|
||||
trigger?: React.ReactNode;
|
||||
sourceOrganisationId: string;
|
||||
sourceOrganisationName: string;
|
||||
userId: number;
|
||||
} & Omit<DialogPrimitive.DialogProps, 'children'>;
|
||||
```
|
||||
|
||||
**Dialog behavior:**
|
||||
|
||||
1. Opens when the trigger is clicked (from the organisations table actions dropdown)
|
||||
2. Fetches the user's owned orgs via `trpc.admin.organisation.find.useQuery({ ownerUserId: userId })`
|
||||
3. Filters to only show orgs that are on the free plan (no active subscription) and excludes the source org
|
||||
4. Displays a select dropdown to pick the target org
|
||||
5. Shows a warning alert: "This will move the subscription from {source} to {target}. The source organisation will be reset to the free plan."
|
||||
6. On submit, calls `trpc.admin.organisation.swapSubscription.useMutation()`
|
||||
7. On success, shows a toast, invalidates relevant queries, and closes the dialog
|
||||
|
||||
**UI layout (following existing dialog patterns like `admin-organisation-create-dialog.tsx`):**
|
||||
|
||||
- `DialogHeader` with title "Move Subscription" and description
|
||||
- A select dropdown listing eligible target orgs (name + url)
|
||||
- An `Alert` explaining what will happen
|
||||
- `DialogFooter` with Cancel + "Move Subscription" buttons (submit button uses `loading` prop)
|
||||
|
||||
### 3. Frontend: Wire into the Organisations Table
|
||||
|
||||
**File to modify:**
|
||||
|
||||
- `apps/remix/app/components/tables/admin-organisations-table.tsx`
|
||||
|
||||
**Changes:**
|
||||
|
||||
- Import the `AdminSwapSubscriptionDialog`
|
||||
- Add a new prop `ownerUserId?: number` to `AdminOrganisationsTableOptions` (needed so the dialog can query other owned orgs)
|
||||
- Add a new dropdown menu item in the actions column: "Move Subscription" with `ArrowRightLeftIcon` from lucide
|
||||
- Only render this item when the org row has an active subscription (`subscription?.status === 'ACTIVE' || subscription?.status === 'PAST_DUE'`)
|
||||
- The menu item renders inside `AdminSwapSubscriptionDialog` with `trigger` prop as the menu item
|
||||
|
||||
### 4. Frontend: Pass userId from User Detail Page
|
||||
|
||||
**File to modify:**
|
||||
|
||||
- `apps/remix/app/routes/_authenticated+/admin+/users.$id.tsx`
|
||||
|
||||
**Changes:**
|
||||
|
||||
- Pass `ownerUserId={user.id}` to `<AdminOrganisationsTable>` so it can forward this to the swap dialog
|
||||
|
||||
## File Change Summary
|
||||
|
||||
| File | Action | Description |
|
||||
| --------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------- |
|
||||
| `packages/trpc/server/admin-router/swap-organisation-subscription.types.ts` | **Create** | Request/response Zod schemas + TS types |
|
||||
| `packages/trpc/server/admin-router/swap-organisation-subscription.ts` | **Create** | Admin mutation with prisma transaction |
|
||||
| `packages/trpc/server/admin-router/router.ts` | **Modify** | Register route at `organisation.swapSubscription` |
|
||||
| `apps/remix/app/components/dialogs/admin-swap-subscription-dialog.tsx` | **Create** | Dialog for selecting target org |
|
||||
| `apps/remix/app/components/tables/admin-organisations-table.tsx` | **Modify** | Add "Move Subscription" action + accept `ownerUserId` prop |
|
||||
| `apps/remix/app/routes/_authenticated+/admin+/users.$id.tsx` | **Modify** | Pass `ownerUserId={user.id}` to table |
|
||||
|
||||
## Edge Cases & Considerations
|
||||
|
||||
1. **Stripe customer stays the same**: The Stripe subscription is tied to a Stripe customer. We move the `customerId` to the target org, so webhook lookups (`findFirst where customerId`) will correctly resolve to the target org going forward.
|
||||
|
||||
2. **`@unique` constraint on `Organisation.customerId`**: Must clear source before setting target within the transaction. Prisma interactive transactions handle this correctly.
|
||||
|
||||
3. **`@unique` constraint on `Subscription.organisationId`**: Since the target org should not have a subscription record, updating the existing subscription's `organisationId` to the target should work. If the target has an INACTIVE subscription record, we need to delete it first.
|
||||
|
||||
4. **Target org has INACTIVE subscription**: The target org might have a stale INACTIVE subscription from a previous cancellation. In this case, delete the target's old subscription record before moving the source's subscription over.
|
||||
|
||||
5. **Seat-based plans**: If the subscription is seat-based, the Stripe quantity may not match the target org's member count. Consider calling `syncMemberCountWithStripeSeatPlan` after the swap as a post-transaction step.
|
||||
|
||||
6. **OrganisationClaim transfer**: Copy `originalSubscriptionClaimId`, `teamCount`, `memberCount`, `envelopeItemCount`, and `flags` from source claim to target claim. Reset source claim to FREE.
|
||||
|
||||
7. **No Stripe API calls needed**: This is purely a DB-level reassociation. The Stripe subscription, customer, and payment method all remain unchanged.
|
||||
@@ -0,0 +1,168 @@
|
||||
---
|
||||
date: 2026-02-11
|
||||
title: Cert Page Width Mismatch
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
Certificate and audit log pages are generated with hardcoded A4 dimensions (`PDF_SIZE_A4_72PPI`: 595×842) regardless of the actual document page sizes. When the source document uses a different page size (e.g., Letter, Legal, or custom dimensions), the certificate/audit log pages end up with a different width than the document pages. This causes problems with courts that expect uniform page dimensions throughout a PDF.
|
||||
|
||||
**Both width and height must match** the last page of the document so the entire PDF prints uniformly.
|
||||
|
||||
**Root cause**: In `seal-document.handler.ts` (lines 186-187), the certificate payload always uses:
|
||||
|
||||
```ts
|
||||
pageWidth: PDF_SIZE_A4_72PPI.width, // 595
|
||||
pageHeight: PDF_SIZE_A4_72PPI.height, // 842
|
||||
```
|
||||
|
||||
These hardcoded values flow into `generateCertificatePdf`, `generateAuditLogPdf`, `renderCertificate`, and `renderAuditLogs` — all of which use `pageWidth`/`pageHeight` to set Konva stage dimensions and layout content.
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Role |
|
||||
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
|
||||
| `packages/lib/jobs/definitions/internal/seal-document.handler.ts` | Orchestrates sealing; passes page dimensions to cert/audit generators |
|
||||
| `packages/lib/constants/pdf.ts` | Defines `PDF_SIZE_A4_72PPI` (595×842) |
|
||||
| `packages/lib/server-only/pdf/generate-certificate-pdf.ts` | Generates certificate PDF; accepts `pageWidth`/`pageHeight` |
|
||||
| `packages/lib/server-only/pdf/generate-audit-log-pdf.ts` | Generates audit log PDF; accepts `pageWidth`/`pageHeight` |
|
||||
| `packages/lib/server-only/pdf/render-certificate.ts` | Renders certificate pages via Konva; uses `pageWidth`/`pageHeight` for stage + layout |
|
||||
| `packages/lib/server-only/pdf/render-audit-logs.ts` | Renders audit log pages via Konva; uses `pageWidth`/`pageHeight` for stage + layout |
|
||||
| `packages/lib/server-only/pdf/get-page-size.ts` | Existing utility — extend with `@libpdf/core` version |
|
||||
| `packages/trpc/server/document-router/download-document-certificate.ts` | Standalone certificate download (also hardcodes A4) |
|
||||
| `packages/trpc/server/document-router/download-document-audit-logs.ts` | Standalone audit log download (also hardcodes A4) |
|
||||
|
||||
## Architecture
|
||||
|
||||
### Current Flow
|
||||
|
||||
1. **One cert PDF + one audit log PDF** generated per envelope with hardcoded A4 dims
|
||||
2. Both appended to **every** envelope item (document) via `decorateAndSignPdf` → `pdfDoc.copyPagesFrom()`
|
||||
3. The audit log is envelope-level (all recipients, all events across all docs) — one per envelope, not per document
|
||||
|
||||
### Multi-Document Envelopes
|
||||
|
||||
- V1 envelopes: single document only
|
||||
- V2 envelopes: support multiple documents (envelope items)
|
||||
- Each envelope item gets both cert + audit log pages appended to it
|
||||
- If documents have different page sizes → need size-matched cert/audit for each
|
||||
|
||||
### Reading Page Dimensions (`@libpdf/core` only)
|
||||
|
||||
Use `@libpdf/core`'s `PDF` class — NOT `@cantoo/pdf-lib`:
|
||||
|
||||
```ts
|
||||
const pdfDoc = await PDF.load(pdfData);
|
||||
const lastPage = pdfDoc.getPage(pdfDoc.getPageCount() - 1);
|
||||
const { width, height } = lastPage; // e.g. 612, 792 for Letter
|
||||
```
|
||||
|
||||
Already used this way in `seal-document.handler.ts` lines 403-410 for V2 field insertion.
|
||||
"Last page" = last page of the original document, before cert/audit pages are appended.
|
||||
|
||||
### Content Layout Adaptation
|
||||
|
||||
Both renderers already handle variable dimensions gracefully:
|
||||
|
||||
- **Width**: `render-certificate.ts:713` / `render-audit-logs.ts:588` — `Math.min(pageWidth - minimumMargin * 2, contentMaxWidth)` with `contentMaxWidth = 768`. Wider pages get more margin, narrower pages tighter margins.
|
||||
- **Height**: Both renderers paginate content into pages using `groupRowsIntoPages()` which respects `pageHeight` via `maxTableHeight = pageHeight - pageTopMargin - pageBottomMargin`. Shorter pages just mean more pages; taller pages fit more rows per page.
|
||||
|
||||
### Playwright PDF Path — Out of Scope
|
||||
|
||||
The `NEXT_PRIVATE_USE_PLAYWRIGHT_PDF` toggle enables a deprecated Playwright-based PDF generation path (`get-certificate-pdf.ts`, `get-audit-logs-pdf.ts`) that also hardcodes `format: 'A4'` in `page.pdf()`. This path is **not being updated** as part of this fix:
|
||||
|
||||
- Both files are marked `@deprecated`
|
||||
- The Konva-based path is the default and recommended path
|
||||
- The Playwright path is behind a feature flag and will be removed
|
||||
|
||||
No changes needed. Add a code comment noting the A4 limitation if the Playwright path is ever re-enabled.
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. Extend `get-page-size.ts` with `@libpdf/core` utility
|
||||
|
||||
Add a `getLastPageDimensions` function to the existing `packages/lib/server-only/pdf/get-page-size.ts` file. This consolidates page-size logic in one place (the file already has the legacy `@cantoo/pdf-lib` version).
|
||||
|
||||
```ts
|
||||
export const getLastPageDimensions = (pdfDoc: PDF): { width: number; height: number } => {
|
||||
const lastPage = pdfDoc.getPage(pdfDoc.getPageCount() - 1);
|
||||
const width = Math.round(lastPage.width);
|
||||
const height = Math.round(lastPage.height);
|
||||
|
||||
if (width < MIN_CERT_PAGE_WIDTH || height < MIN_CERT_PAGE_HEIGHT) {
|
||||
return { width: PDF_SIZE_A4_72PPI.width, height: PDF_SIZE_A4_72PPI.height };
|
||||
}
|
||||
|
||||
return { width, height };
|
||||
};
|
||||
```
|
||||
|
||||
**Dimension rounding**: `Math.round()` both width and height. PDF points at 72ppi are typically whole numbers; rounding avoids spurious float-precision mismatches (e.g., 612.0 vs 612.00001) that would cause unnecessary duplicate cert/audit PDF generation.
|
||||
|
||||
**Minimum page dimensions**: Enforce a minimum threshold (e.g., 300pt for both width and height). If either dimension falls below the minimum, fall back to A4 (595×842). The certificate and audit log renderers have headers, table rows, margins, and QR codes that require a minimum viable area.
|
||||
|
||||
### 2. Read last page dimensions from each envelope item's PDF
|
||||
|
||||
In `seal-document.handler.ts`, before generating cert/audit PDFs:
|
||||
|
||||
- For each `envelopeItem`, load the PDF and read the **last page's width and height** using `getLastPageDimensions`
|
||||
- Use `PDF.load()` then pass the loaded doc to the utility
|
||||
|
||||
**Resealing consideration**: When `isResealing` is true, envelope items are remapped to use `initialData` (lines 152-158) before this point. Page-size extraction must operate on the same data source that `decorateAndSignPdf` will use. Since the `envelopeItems` array is already remapped by the time we read dimensions, reading from `envelopeItem.documentData` will naturally give the correct (initial) data. No special handling needed beyond ensuring the dimension read happens **after** the resealing remap.
|
||||
|
||||
### 3. Generate cert/audit PDFs per unique page size
|
||||
|
||||
Current flow generates one cert + one audit log doc per envelope. Change to:
|
||||
|
||||
1. Collect `{ width, height }` of the last page for each envelope item
|
||||
2. Deduplicate by `"${width}x${height}"` key (using the already-rounded integers)
|
||||
3. For each unique size, generate cert PDF and audit log PDF with those dimensions
|
||||
4. Store in a `Map<string, { certificateDoc, auditLogDoc }>` keyed by `"${width}x${height}"`
|
||||
|
||||
For the common single-document case, this is one generation — same perf as today.
|
||||
|
||||
### 4. Thread the correct docs into `decorateAndSignPdf`
|
||||
|
||||
In the envelope item loop, look up the item's last-page dimensions in the map and pass the matching cert/audit docs. Signature of `decorateAndSignPdf` doesn't change — it still receives a single `certificateDoc` and `auditLogDoc`, just the right ones per item.
|
||||
|
||||
### 5. Update standalone download routes
|
||||
|
||||
`download-document-certificate.ts` and `download-document-audit-logs.ts` also hardcode A4:
|
||||
|
||||
- Both routes have `documentId` which maps to a specific envelope item
|
||||
- Fetch **that specific document's** PDF data, load it, read last page width + height via `getLastPageDimensions`
|
||||
- Pass `{ pageWidth, pageHeight }` to the generator
|
||||
- This ensures the standalone download matches the dimensions the user would see in the sealed PDF for that document
|
||||
|
||||
### 6. Edge cases
|
||||
|
||||
| Scenario | Behavior |
|
||||
| --------------------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| Mixed page sizes within one PDF | Use last page's dimensions (per spec) |
|
||||
| Page dimensions below minimum threshold | Fall back to A4 (595×842) |
|
||||
| Landscape pages | width/height just swap roles; renderers adapt via `Math.min()` capping. No special handling |
|
||||
| Fallback if page dims unreadable | Default to A4 (595×842) |
|
||||
| Resealing | Dimensions read after `initialData` remap — correct source automatically |
|
||||
| Playwright PDF path enabled | Remains A4 — out of scope, deprecated |
|
||||
| Single-doc envelope (most common) | One generation, same perf as today |
|
||||
| Multi-doc envelope, same page sizes | Dedup key matches → one generation |
|
||||
| Multi-doc envelope, different sizes | One generation per unique size |
|
||||
|
||||
### 7. Tests
|
||||
|
||||
- Add assertion-based E2E test (no visual regression / reference images needed)
|
||||
- Seal a Letter-size (612×792) PDF through the full flow
|
||||
- Load the sealed output and assert all pages (document + cert + audit) have matching width/height
|
||||
- Can be added to `envelope-alignment.spec.ts` or as a new focused test
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Extend `get-page-size.ts`** — add `getLastPageDimensions(pdfDoc: PDF): { width: number; height: number }` using `@libpdf/core`, with `Math.round()` and minimum dimension enforcement
|
||||
2. **In `seal-document.handler.ts`**:
|
||||
a. After the resealing remap (line ~159), load each envelope item's PDF via `PDF.load()` and collect last page `{ width, height }` using `getLastPageDimensions`
|
||||
b. Deduplicate by `"${width}x${height}"` key
|
||||
c. Generate cert/audit PDFs per unique size (parallel via `Promise.all`)
|
||||
d. In envelope item loop, look up matching cert/audit doc by size key
|
||||
3. **Fix `download-document-certificate.ts`** — load the specific document's PDF, read last page dims via `getLastPageDimensions`, pass to generator
|
||||
4. **Fix `download-document-audit-logs.ts`** — same as above, using the specific `documentId`'s PDF
|
||||
5. **Add E2E test** — assertion-based test with a Letter-size document verifying all page dimensions match after sealing
|
||||
@@ -0,0 +1,138 @@
|
||||
---
|
||||
date: 2026-05-06
|
||||
title: Platform Signing Page Branding
|
||||
---
|
||||
|
||||
## What
|
||||
|
||||
Platform-plan organisations (and their teams) can customise the **non-embed
|
||||
signing pages** (`/sign/:token`, `/d/:token`, and the sibling
|
||||
complete/expired/rejected/waiting pages) with:
|
||||
|
||||
- Six brand colour tokens (background, foreground, primary, primary-foreground,
|
||||
border, ring) plus a border-radius length.
|
||||
- A free-text custom CSS block (up to 256 KB).
|
||||
|
||||
Settings live on `OrganisationGlobalSettings` and `TeamGlobalSettings`. Teams
|
||||
inherit from the org via the existing `brandingEnabled === null` mechanism.
|
||||
|
||||
## Why
|
||||
|
||||
- Embed customers already have white-label CSS; Platform customers want the
|
||||
same coverage on direct signing URLs that they iframe or link to.
|
||||
- Persisting on org/team (not per envelope) means it's set-and-forget.
|
||||
- Sanitising **on save** lets us inline the verbatim string at SSR — no
|
||||
per-render parsing cost, no `<style>.innerHTML` injection on the client.
|
||||
- Reusing the existing `embedSigningWhiteLabel` claim flag keeps "if you can
|
||||
white-label an embed, you can white-label this" as one decision.
|
||||
|
||||
## How
|
||||
|
||||
### Storage (`packages/prisma/schema.prisma`)
|
||||
|
||||
Two new fields on each settings model. No new tables.
|
||||
|
||||
| Field | Org type | Team type |
|
||||
| ---------------- | ------------------ | ------------------ |
|
||||
| `brandingColors` | `Json?` (nullable) | `Json?` (nullable) |
|
||||
| `brandingCss` | `String @default("")` | `String?` |
|
||||
|
||||
Colours are validated against `ZCssVarsSchema`. The team's `null` means
|
||||
"inherit"; an empty colour object is collapsed to `null` server-side so a
|
||||
team toggling `brandingEnabled = true` without filling in colours doesn't
|
||||
silently override the org's defaults with nothing.
|
||||
|
||||
### Sanitiser (`packages/lib/utils/sanitize-branding-css.ts`)
|
||||
|
||||
PostCSS + `postcss-selector-parser`. Runs on save only.
|
||||
|
||||
- Drops selectors containing `::before`/`::after`/`::backdrop`/`::marker` or
|
||||
the universal `*`.
|
||||
- Drops integrity-breaking properties (`display`, `position`, `transform`,
|
||||
layout-affecting dimensions, text-hiding properties).
|
||||
- Drops declaration values containing `url(`, `expression(`, `@import`,
|
||||
`javascript:`.
|
||||
- Strips `!important`.
|
||||
- Allows `@media` only; drops other at-rules.
|
||||
- **Does not** rewrite selectors. Scoping happens at render time via native
|
||||
CSS nesting under `.documenso-branded { ... }`.
|
||||
- Final-pass tripwire: if a literal `</style` somehow survives serialization,
|
||||
reject the entire output. PostCSS already escapes `<` to `\3c` whenever it
|
||||
would form `</...`; the explicit check is belt-and-braces in case a future
|
||||
serializer regresses.
|
||||
- Returns `{ css, warnings[] }`. Warnings are surfaced in the UI.
|
||||
|
||||
Border-radius is the only token interpolated raw into a `<style>` block; it
|
||||
is regex-validated (`CSS_LENGTH_REGEX`) at both the Zod schema and the
|
||||
runtime `toNativeCssVars` call. Belt-and-braces against schema drift.
|
||||
|
||||
### Render (`apps/remix/app/components/general/recipient-branding.tsx`)
|
||||
|
||||
Each recipient loader calls `loadRecipientBrandingByTeamId` and threads the
|
||||
payload through to `<RecipientBranding>`, which emits a single
|
||||
nonce-attributed `<style>`:
|
||||
|
||||
```
|
||||
.documenso-branded {
|
||||
--background: ...; ...
|
||||
<user css>
|
||||
}
|
||||
```
|
||||
|
||||
Native CSS nesting expands user rules under the wrapper. The body class is
|
||||
applied unconditionally to recipient routes in `root.tsx` via `useMatches()`
|
||||
so portaled Radix content (dialogs, popovers, tooltips, dropdowns) inherits
|
||||
the scope.
|
||||
|
||||
CSP for recipient routes already supports `<style nonce>`; no policy
|
||||
changes needed.
|
||||
|
||||
### Plan gate
|
||||
|
||||
`organisationClaim.flags.embedSigningWhiteLabel || !IS_BILLING_ENABLED()`.
|
||||
Self-hosted instances always allow. The outer paywall for logo/URL/details
|
||||
stays on `allowCustomBranding` (Team plan and up); only the new
|
||||
colour/CSS section is Platform-only.
|
||||
|
||||
### UI (`apps/remix/app/components/forms/branding-preferences-form.tsx`)
|
||||
|
||||
Extends the existing branding form. Six `<ColorPicker showHex>` (rewritten
|
||||
to use the native `<input type="color">` instead of `react-colorful`, which
|
||||
was removed) in a 2-col grid, plus a free-text radius input and an
|
||||
`<Accordion>` revealing a mono `<Textarea>`. Defaults come from
|
||||
`packages/lib/constants/theme.ts` (light-mode hex mirror of `theme.css`).
|
||||
|
||||
Warnings from the sanitiser are surfaced in an `<Alert variant="warning">`
|
||||
after save, and the `brandingCss` textarea is re-synced from the persisted
|
||||
value so the user sees exactly what was stored. Other fields are
|
||||
deliberately NOT reset on settings refetch — that would clobber in-flight
|
||||
edits.
|
||||
|
||||
### TRPC
|
||||
|
||||
`update-organisation-settings` and `update-team-settings` accept the new
|
||||
fields, run them through `sanitizeBrandingCss` + `normalizeBrandingColors`,
|
||||
and return any sanitiser warnings to the client. The team route treats
|
||||
`null` as "inherit"; an empty post-sanitisation string is collapsed to
|
||||
`null` (team) so an empty override doesn't mask the org's CSS.
|
||||
|
||||
## Known accepted limitations
|
||||
|
||||
- The sanitiser does not prevent hostile-but-syntactically-valid CSS
|
||||
(`color: transparent`, low-contrast values, etc.). The customer is
|
||||
branding **their own** signing pages — we focus on integrity (no
|
||||
overlay/hide/exfiltrate), not aesthetic policing.
|
||||
- User rules targeting `body`/`html`/`:root` no-op once nested under the
|
||||
wrapper class. Documented for users.
|
||||
- CSS nesting baseline is Chrome 120+ / Firefox 117+ / Safari 16.5+.
|
||||
Acceptable for the Platform-tier audience.
|
||||
- No automated `theme.css` ↔ `theme.ts` sync check; fat comment in
|
||||
`theme.ts` reminds devs to update both.
|
||||
- Per-section team inherit is coarse — `brandingEnabled = null` inherits
|
||||
everything from the org. Per-field inherit toggles are deferred.
|
||||
|
||||
## Out of scope
|
||||
|
||||
Live preview, embed-route sanitiser unification, email/PDF certificate
|
||||
branding, custom font upload, the full ~30 colour tokens in the picker UI,
|
||||
wiring `hidePoweredBy` through to the actual footer.
|
||||
@@ -0,0 +1,312 @@
|
||||
---
|
||||
date: 2026-02-26
|
||||
title: pnpm Migration
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Migrate from npm to pnpm to eliminate dependency resolution duplication issues that cause bundler problems. The current npm workspace setup results in nested `node_modules` copies that don't deduplicate reliably, requiring manual hoisting and `npm dedupe` cycles. pnpm's content-addressable store and strict symlink structure eliminates this class of problem entirely.
|
||||
|
||||
## Current State
|
||||
|
||||
- **Package manager:** npm@10.7.0 with `legacy-peer-deps=true` and `prefer-dedupe=true`
|
||||
- **Workspaces:** 18 total (3 apps, 15 packages) declared in root `package.json` `workspaces` field
|
||||
- **Lockfile:** `package-lock.json`
|
||||
- **Patches:** `patch-package` with one patch (`@ai-sdk+google-vertex+3.0.81`)
|
||||
- **Overrides:** `lodash`, `pdfjs-dist`, `typescript`, `zod` in root `package.json`
|
||||
- **Syncpack:** installed but unconfigured (no `.syncpackrc`)
|
||||
- **Heavy duplication:** `zod` in 7 workspaces, `ts-pattern` in 9, `luxon` in 8, `react` in 6, etc.
|
||||
- **Docker:** `turbo prune` → `npm ci` → `npm ci --only=production` multi-stage build
|
||||
- **Existing Dockerfiles:** `docker/Dockerfile` (primary, npm), `apps/remix/Dockerfile.pnpm` (already exists, needs review)
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Phase 1: Core Migration
|
||||
|
||||
#### Step 1: Enable pnpm via corepack
|
||||
|
||||
```bash
|
||||
corepack enable pnpm
|
||||
corepack use pnpm@latest
|
||||
```
|
||||
|
||||
This adds a `"packageManager"` field to root `package.json` (e.g. `"packageManager": "pnpm@10.x.x"`). Remove the existing `"engines"` npm constraint if present.
|
||||
|
||||
#### Step 2: Create `pnpm-workspace.yaml`
|
||||
|
||||
```yaml
|
||||
packages:
|
||||
- apps/*
|
||||
- packages/*
|
||||
```
|
||||
|
||||
Remove the `"workspaces"` field from root `package.json` — pnpm uses `pnpm-workspace.yaml` instead.
|
||||
|
||||
#### Step 3: Convert lockfile
|
||||
|
||||
```bash
|
||||
pnpm import
|
||||
```
|
||||
|
||||
This reads `package-lock.json` and generates `pnpm-lock.yaml`. After verifying, delete `package-lock.json`.
|
||||
|
||||
#### Step 4: Create `.npmrc` for pnpm
|
||||
|
||||
Replace the current `.npmrc` contents. The existing settings (`legacy-peer-deps=true`, `prefer-dedupe=true`) are npm-specific.
|
||||
|
||||
```ini
|
||||
# Hoist packages that expect to be resolvable from any workspace.
|
||||
# Start strict, add patterns here only as needed.
|
||||
shamefully-hoist=true
|
||||
```
|
||||
|
||||
> **Note:** `shamefully-hoist=true` is the pragmatic starting point. It mimics npm's flat `node_modules` layout. Once the migration is stable, this can be tightened to `hoist-pattern[]` entries for specific packages that need it, moving toward pnpm's strict isolation model.
|
||||
|
||||
#### Step 5: Clean install
|
||||
|
||||
```bash
|
||||
rm -rf node_modules apps/*/node_modules packages/*/node_modules
|
||||
pnpm install
|
||||
```
|
||||
|
||||
Verify the install completes without errors. Fix any peer dependency warnings — pnpm is stricter than npm with `legacy-peer-deps=true`.
|
||||
|
||||
#### Step 6: Convert `overrides` to `pnpm.overrides`
|
||||
|
||||
In root `package.json`, move the `overrides` block under `pnpm`:
|
||||
|
||||
```json
|
||||
{
|
||||
"pnpm": {
|
||||
"overrides": {
|
||||
"lodash": "4.17.23",
|
||||
"pdfjs-dist": "5.4.296",
|
||||
"typescript": "5.6.2",
|
||||
"zod": "^3.25.76"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Remove the top-level `overrides` field (that's npm-specific).
|
||||
|
||||
#### Step 7: Convert patch-package to pnpm patches
|
||||
|
||||
pnpm has native patching. Convert the existing `@ai-sdk+google-vertex+3.0.81` patch:
|
||||
|
||||
```bash
|
||||
# Remove patch-package dependency and postinstall script
|
||||
# Then use pnpm's native patching:
|
||||
pnpm patch @ai-sdk/google-vertex@3.0.81
|
||||
# Apply the same changes from patches/@ai-sdk+google-vertex+3.0.81.patch
|
||||
pnpm patch-commit <temp-dir>
|
||||
```
|
||||
|
||||
This adds a `pnpm.patchedDependencies` entry to root `package.json` and stores the patch in a `patches/` directory (pnpm's own format). Remove `patch-package` from dependencies and the `postinstall` script.
|
||||
|
||||
### Phase 2: Catalogs
|
||||
|
||||
#### Step 8: Identify catalog candidates
|
||||
|
||||
Packages duplicated across 3+ workspaces are prime candidates:
|
||||
|
||||
| Package | Workspaces | Catalog? |
|
||||
| ----------------------------------------------- | ---------- | ---------------------- |
|
||||
| `zod` | 7 | Yes |
|
||||
| `ts-pattern` | 9 | Yes |
|
||||
| `luxon` | 8 | Yes |
|
||||
| `react` / `react-dom` | 6 / 3 | Yes |
|
||||
| `typescript` | 6 | Yes |
|
||||
| `nanoid` | 4 | Yes |
|
||||
| `@lingui/core` / `macro` / `react` | 2-3 | Yes |
|
||||
| `@simplewebauthn/server` | 3 | Yes |
|
||||
| `@documenso/*` (internal) | varies | No (use `workspace:*`) |
|
||||
| `@aws-sdk/*` | 2 | Yes |
|
||||
| `hono` | 2 | Yes |
|
||||
| `posthog-node` / `posthog-js` | 2 | Yes |
|
||||
| `remeda` | 3 | Yes |
|
||||
| `@tanstack/react-query` | 2 | Yes |
|
||||
| `@trpc/*` | 2 | Yes |
|
||||
| `superjson` | 2 | Yes |
|
||||
| `kysely` | 2 | Yes |
|
||||
| `@types/react` / `@types/node` / `@types/luxon` | 3-4 | Yes |
|
||||
|
||||
#### Step 9: Define catalogs in `pnpm-workspace.yaml`
|
||||
|
||||
```yaml
|
||||
packages:
|
||||
- apps/*
|
||||
- packages/*
|
||||
|
||||
catalog:
|
||||
# Core
|
||||
react: ^18
|
||||
react-dom: ^18
|
||||
typescript: 5.6.2
|
||||
zod: ^3.25.76
|
||||
|
||||
# Shared utilities
|
||||
ts-pattern: <current-version>
|
||||
luxon: ^3.7.2
|
||||
nanoid: ^5.1.6
|
||||
remeda: <current-version>
|
||||
superjson: ^2.2.5
|
||||
|
||||
# AWS
|
||||
'@aws-sdk/client-s3': ^3.998.0
|
||||
'@aws-sdk/client-sesv2': ^3.998.0
|
||||
'@aws-sdk/cloudfront-signer': ^3.998.0
|
||||
'@aws-sdk/s3-request-presigner': ^3.998.0
|
||||
'@aws-sdk/signature-v4-crt': ^3.998.0
|
||||
|
||||
# Framework
|
||||
hono: 4.12.2
|
||||
'@tanstack/react-query': <current-version>
|
||||
'@trpc/client': 11.8.1
|
||||
'@trpc/react-query': 11.8.1
|
||||
'@trpc/server': 11.8.1
|
||||
|
||||
# i18n
|
||||
'@lingui/core': ^5.6.0
|
||||
'@lingui/macro': ^5.6.0
|
||||
'@lingui/react': ^5.6.0
|
||||
|
||||
# Auth
|
||||
'@simplewebauthn/server': <current-version>
|
||||
|
||||
# Observability
|
||||
posthog-node: 4.18.0
|
||||
posthog-js: <current-version>
|
||||
|
||||
# Database
|
||||
kysely: <current-version>
|
||||
'@prisma/client': ^6.19.0
|
||||
prisma: ^6.19.0
|
||||
|
||||
# Types
|
||||
'@types/react': <current-version>
|
||||
'@types/react-dom': <current-version>
|
||||
'@types/node': ^20
|
||||
'@types/luxon': <current-version>
|
||||
```
|
||||
|
||||
#### Step 10: Update workspace `package.json` files
|
||||
|
||||
Replace pinned versions with `catalog:` protocol for all cataloged packages:
|
||||
|
||||
```json
|
||||
{
|
||||
"dependencies": {
|
||||
"zod": "catalog:",
|
||||
"ts-pattern": "catalog:",
|
||||
"luxon": "catalog:"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is a mechanical find-and-replace across all workspace `package.json` files.
|
||||
|
||||
### Phase 3: Internal Workspace References
|
||||
|
||||
#### Step 11: Convert internal references to `workspace:*`
|
||||
|
||||
All `@documenso/*` internal package references currently use `"*"`. Convert to pnpm's `workspace:*` protocol:
|
||||
|
||||
```json
|
||||
{
|
||||
"dependencies": {
|
||||
"@documenso/lib": "workspace:*",
|
||||
"@documenso/prisma": "workspace:*"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This makes the workspace resolution explicit and prevents accidental resolution to a published version.
|
||||
|
||||
### Phase 4: Docker & CI
|
||||
|
||||
#### Step 12: Update primary Dockerfile (`docker/Dockerfile`)
|
||||
|
||||
The existing multi-stage build needs to change:
|
||||
|
||||
1. **base:** Add pnpm — `corepack enable pnpm` or install via `npm i -g pnpm`
|
||||
2. **builder:** `turbo prune` still works with pnpm. Output structure is the same.
|
||||
3. **installer:**
|
||||
- Replace `npm ci` with `pnpm install --frozen-lockfile`
|
||||
- Copy `pnpm-lock.yaml` and `pnpm-workspace.yaml` instead of `package-lock.json`
|
||||
- Remove `patch-package` from postinstall (pnpm patches are applied natively)
|
||||
4. **runner:**
|
||||
- Replace `npm ci --only=production` with `pnpm install --frozen-lockfile --prod`
|
||||
- Or use `pnpm deploy` for standalone output (copies only production deps to a flat directory)
|
||||
|
||||
Review `apps/remix/Dockerfile.pnpm` — it already exists and may have most of this solved. Reconcile with the primary `docker/Dockerfile`.
|
||||
|
||||
#### Step 13: Update CI workflows
|
||||
|
||||
Search for all `npm ci`, `npm install`, `npm run` in CI config files (`.github/workflows/`, etc.) and replace with `pnpm install --frozen-lockfile`, `pnpm run`, etc.
|
||||
|
||||
Ensure corepack is enabled in CI runners:
|
||||
|
||||
```yaml
|
||||
- run: corepack enable pnpm
|
||||
```
|
||||
|
||||
#### Step 14: Update turborepo config
|
||||
|
||||
Turbo works with pnpm out of the box. The `turbo.json` should not need changes. Verify `turbo prune` generates correct output with pnpm lockfile.
|
||||
|
||||
### Phase 5: Cleanup & Tighten
|
||||
|
||||
#### Step 15: Remove npm-specific tooling
|
||||
|
||||
- Remove `syncpack` (catalogs replace its purpose)
|
||||
- Remove `patch-package` (pnpm native patches replace it)
|
||||
- Remove `"workspaces"` from root `package.json` if not already done
|
||||
- Delete `package-lock.json`
|
||||
- Update `.gitignore` if needed (pnpm store is outside the repo by default)
|
||||
|
||||
#### Step 16: Tighten hoisting (optional, future)
|
||||
|
||||
Once stable, replace `shamefully-hoist=true` with targeted hoist patterns:
|
||||
|
||||
```ini
|
||||
shamefully-hoist=false
|
||||
hoist-pattern[]=*eslint*
|
||||
hoist-pattern[]=*prettier*
|
||||
# Add others as discovered
|
||||
```
|
||||
|
||||
This moves toward strict isolation where each package can only import what it declares. Catches phantom dependency issues. Do this incrementally — let the bundler tell you what breaks.
|
||||
|
||||
#### Step 17: Remove root-level dependency hoisting
|
||||
|
||||
With catalogs and strict resolution, dependencies currently hoisted to root `package.json` for deduplication purposes can be moved back to the workspaces that actually use them. The root `package.json` should only contain tooling deps (`turbo`, `prettier`, `eslint`, etc.) and `pnpm.overrides`.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
1. **Phantom dependencies surface:** pnpm's strict isolation will expose imports that work today only because npm hoisted them. `shamefully-hoist=true` defers this, but tightening later will reveal them.
|
||||
- **Mitigation:** Start with `shamefully-hoist=true`. Tighten incrementally after the migration is stable.
|
||||
|
||||
2. **Peer dependency strictness:** pnpm enforces peer deps by default. The current `.npmrc` has `legacy-peer-deps=true` which suppresses all peer dep errors.
|
||||
- **Mitigation:** Run `pnpm install` and address peer dep warnings. Most will be resolvable by adding missing peer deps to the relevant workspace.
|
||||
|
||||
3. **Docker build breakage:** The `turbo prune` + `npm ci` pipeline is battle-tested. Switching to pnpm changes the install semantics.
|
||||
- **Mitigation:** The existing `Dockerfile.pnpm` in `apps/remix/` provides a reference. Test the Docker build in CI before merging.
|
||||
|
||||
4. **CI cache invalidation:** Switching lockfiles invalidates all CI dependency caches.
|
||||
- **Mitigation:** Update cache keys to use `pnpm-lock.yaml` hash. First CI run will be slower, subsequent runs will cache normally.
|
||||
|
||||
5. **Turbo + pnpm compatibility:** Turbo has first-class pnpm support, but `turbo prune` output format may differ slightly.
|
||||
- **Mitigation:** Test `turbo prune --scope=@documenso/remix --docker` and verify output structure before updating Dockerfile.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [ ] `pnpm install` succeeds with no errors
|
||||
- [ ] `pnpm run build` succeeds (all workspaces)
|
||||
- [ ] `pnpm run lint` passes
|
||||
- [ ] `pnpm run dev` starts correctly
|
||||
- [ ] Docker build produces a working image
|
||||
- [ ] E2E tests pass (`pnpm run test:e2e`)
|
||||
- [ ] No duplicate package copies in `node_modules` for key deps (`zod`, `react`, `typescript`)
|
||||
- [ ] `pnpm audit` shows same or better results than current npm audit
|
||||
- [ ] CI pipeline passes end-to-end
|
||||
@@ -0,0 +1,551 @@
|
||||
---
|
||||
date: 2026-02-19
|
||||
title: Database Rate Limiting
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Replace the in-memory `hono-rate-limiter` with a database-backed rate limiting system using Prisma and PostgreSQL. The current in-memory approach is ineffective in multi-instance deployments since there are no sticky sessions. The new system uses **bucketed counters** (one row per key/action/time-bucket with atomic increment) to efficiently handle both high-throughput API rate limiting and granular auth/email rate limiting.
|
||||
|
||||
### Design Decisions
|
||||
|
||||
- **Bucketed counters** over row-per-request: high-throughput consumers would create thousands of rows per minute; bucketed counters reduce this to one row per key per time bucket
|
||||
- **Fixed time windows**: simpler than sliding windows, the 2x burst-at-boundary scenario is acceptable for rate limiting purposes
|
||||
- **Dual-key rate limiting**: per-identifier (`max`) and per-IP (`globalMax`) checked independently via separate rows with a `key` prefix (`id:` / `ip:`)
|
||||
- **Accept slight over-count**: the upsert is atomic (increment + return count in one operation) but concurrent requests near the limit may both see a count just under the threshold before either commits, allowing a slight overshoot
|
||||
- **Fail-open on errors**: if the rate limit DB query fails, allow the request through rather than blocking legitimate users
|
||||
- **Prisma upsert** with `{ increment: 1 }` for atomic counter updates, returns the updated row so count check is a single operation
|
||||
- **Application cron job** for cleanup of expired bucket rows
|
||||
|
||||
### Rate Limit Check Flow
|
||||
|
||||
```
|
||||
check({ ip, identifier }) ->
|
||||
1. Upsert IP row (ip:{ip} / action / bucket) with count + 1, RETURNING count
|
||||
-> if globalMax is set and count >= globalMax, return { isLimited: true }
|
||||
2. Upsert identifier row (id:{identifier} / action / bucket) with count + 1, RETURNING count
|
||||
-> if count >= max, return { isLimited: true }
|
||||
3. Neither limited -> return { isLimited: false }
|
||||
```
|
||||
|
||||
Each upsert atomically increments and returns the new count in a single operation. Both counters always increment on every check — there's no conditional logic to skip one based on the other. This keeps the implementation simple and avoids read-then-write race conditions. If only IP is provided (API rate limiting), only step 1 runs.
|
||||
|
||||
---
|
||||
|
||||
## 1. Database Schema
|
||||
|
||||
### 1.1 Prisma model
|
||||
|
||||
Add to `packages/prisma/schema.prisma` after the `Counter` model:
|
||||
|
||||
```prisma
|
||||
model RateLimit {
|
||||
key String
|
||||
action String
|
||||
bucket DateTime
|
||||
count Int @default(1)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@id([key, action, bucket])
|
||||
@@index([createdAt])
|
||||
}
|
||||
```
|
||||
|
||||
- **Composite primary key** `(key, action, bucket)` serves as both the unique constraint for upserts and the lookup index
|
||||
- **`key`** is prefixed: `ip:1.2.3.4` or `id:user@example.com`
|
||||
- **`action`** is the rate limit action name: `auth.forgot-password`, `api.v1`, etc.
|
||||
- **`bucket`** is the start of the time window, truncated to the window size (e.g., `2026-02-19T10:05:00Z` for a 5-minute bucket)
|
||||
- **`createdAt` index** is for the cleanup job to efficiently delete old rows
|
||||
- **`count`** starts at 1 (set by the create side of the upsert)
|
||||
|
||||
### 1.2 Migration
|
||||
|
||||
Generate with `npx prisma migrate dev --name add-rate-limits`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Rate Limit Library
|
||||
|
||||
### 2.1 Core module
|
||||
|
||||
Create `packages/lib/server-only/rate-limit/rate-limit.ts`:
|
||||
|
||||
```typescript
|
||||
type WindowUnit = 's' | 'm' | 'h' | 'd';
|
||||
type WindowStr = `${number}${WindowUnit}`;
|
||||
|
||||
type RateLimitConfig = {
|
||||
action: string;
|
||||
max: number;
|
||||
globalMax?: number;
|
||||
window: WindowStr;
|
||||
};
|
||||
|
||||
type CheckParams = {
|
||||
ip: string;
|
||||
identifier?: string;
|
||||
};
|
||||
|
||||
export const rateLimit = (config: RateLimitConfig) => {
|
||||
return {
|
||||
async check(params: CheckParams): Promise<{
|
||||
isLimited: boolean;
|
||||
remaining: number;
|
||||
limit: number;
|
||||
reset: Date;
|
||||
}> { ... }
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
### 2.2 Window parsing and bucket computation
|
||||
|
||||
```typescript
|
||||
const parseWindow = (window: WindowStr): number => {
|
||||
const value = parseInt(window.slice(0, -1), 10);
|
||||
const unit = window.slice(-1) as WindowUnit;
|
||||
const multipliers: Record<WindowUnit, number> = {
|
||||
s: 1000,
|
||||
m: 60 * 1000,
|
||||
h: 60 * 60 * 1000,
|
||||
d: 24 * 60 * 60 * 1000,
|
||||
};
|
||||
return value * multipliers[unit];
|
||||
};
|
||||
|
||||
const getBucket = (windowMs: number): Date => {
|
||||
const now = Date.now();
|
||||
return new Date(now - (now % windowMs));
|
||||
};
|
||||
```
|
||||
|
||||
### 2.3 Check implementation
|
||||
|
||||
The `check()` method:
|
||||
|
||||
1. Compute the current bucket from the window
|
||||
2. Compute `reset` as `bucket + windowMs` (the start of the next window)
|
||||
3. If `globalMax` is set, upsert the IP row and check count
|
||||
4. If `identifier` is provided, upsert the identifier row and check count
|
||||
5. Wrap in try/catch — **fail-open** on any database error (log the error, return `{ isLimited: false }`)
|
||||
|
||||
Each upsert uses Prisma's `upsert` with `{ increment: 1 }`:
|
||||
|
||||
```typescript
|
||||
const result = await prisma.rateLimit.upsert({
|
||||
where: {
|
||||
key_action_bucket: {
|
||||
key: `ip:${params.ip}`,
|
||||
action: config.action,
|
||||
bucket,
|
||||
},
|
||||
},
|
||||
create: {
|
||||
key: `ip:${params.ip}`,
|
||||
action: config.action,
|
||||
bucket,
|
||||
count: 1,
|
||||
},
|
||||
update: {
|
||||
count: { increment: 1 },
|
||||
},
|
||||
});
|
||||
|
||||
if (config.globalMax && result.count >= config.globalMax) {
|
||||
return { isLimited: true, remaining: 0, limit: config.globalMax };
|
||||
}
|
||||
```
|
||||
|
||||
### 2.4 Rate limit definitions
|
||||
|
||||
Create `packages/lib/server-only/rate-limit/rate-limits.ts` with all rate limit instances:
|
||||
|
||||
```typescript
|
||||
// ---- Auth (Tier 1 - Critical, sends emails) ----
|
||||
export const signupRateLimit = rateLimit({
|
||||
action: 'auth.signup',
|
||||
max: 5,
|
||||
globalMax: 10,
|
||||
window: '1h',
|
||||
});
|
||||
|
||||
export const forgotPasswordRateLimit = rateLimit({
|
||||
action: 'auth.forgot-password',
|
||||
max: 3,
|
||||
globalMax: 20,
|
||||
window: '1h',
|
||||
});
|
||||
|
||||
export const resendVerifyEmailRateLimit = rateLimit({
|
||||
action: 'auth.resend-verify-email',
|
||||
max: 3,
|
||||
globalMax: 20,
|
||||
window: '1h',
|
||||
});
|
||||
|
||||
export const request2FAEmailRateLimit = rateLimit({
|
||||
action: 'auth.request-2fa-email',
|
||||
max: 5,
|
||||
globalMax: 20,
|
||||
window: '15m',
|
||||
});
|
||||
|
||||
// ---- Auth (Tier 2 - Unauthenticated) ----
|
||||
export const loginRateLimit = rateLimit({
|
||||
action: 'auth.login',
|
||||
max: 10,
|
||||
globalMax: 50,
|
||||
window: '15m',
|
||||
});
|
||||
|
||||
export const resetPasswordRateLimit = rateLimit({
|
||||
action: 'auth.reset-password',
|
||||
max: 5,
|
||||
globalMax: 20,
|
||||
window: '1h',
|
||||
});
|
||||
|
||||
export const verifyEmailRateLimit = rateLimit({
|
||||
action: 'auth.verify-email',
|
||||
max: 5,
|
||||
globalMax: 20,
|
||||
window: '15m',
|
||||
});
|
||||
|
||||
export const passkeyRateLimit = rateLimit({
|
||||
action: 'auth.passkey',
|
||||
max: 10,
|
||||
globalMax: 50,
|
||||
window: '15m',
|
||||
});
|
||||
|
||||
export const oauthRateLimit = rateLimit({
|
||||
action: 'auth.oauth',
|
||||
max: 10,
|
||||
globalMax: 50,
|
||||
window: '15m',
|
||||
});
|
||||
|
||||
export const linkOrgAccountRateLimit = rateLimit({
|
||||
action: 'auth.link-org-account',
|
||||
max: 5,
|
||||
globalMax: 20,
|
||||
window: '1h',
|
||||
});
|
||||
|
||||
// ---- API (Tier 4 - Standard) ----
|
||||
export const apiV1RateLimit = rateLimit({
|
||||
action: 'api.v1',
|
||||
max: 100,
|
||||
window: '1m',
|
||||
});
|
||||
|
||||
export const apiV2RateLimit = rateLimit({
|
||||
action: 'api.v2',
|
||||
max: 100,
|
||||
window: '1m',
|
||||
});
|
||||
|
||||
export const apiTrpcRateLimit = rateLimit({
|
||||
action: 'api.trpc',
|
||||
max: 100,
|
||||
window: '1m',
|
||||
});
|
||||
|
||||
export const aiRateLimit = rateLimit({
|
||||
action: 'api.ai',
|
||||
max: 3,
|
||||
window: '1m',
|
||||
});
|
||||
|
||||
export const fileUploadRateLimit = rateLimit({
|
||||
action: 'api.file-upload',
|
||||
max: 20,
|
||||
window: '1m',
|
||||
});
|
||||
```
|
||||
|
||||
Exact limits are initial values — tune based on observed traffic patterns. These should be easy to adjust.
|
||||
|
||||
---
|
||||
|
||||
## 3. Integration Points
|
||||
|
||||
### 3.1 Hono middleware for API routes
|
||||
|
||||
Create a reusable Hono middleware factory in `packages/lib/server-only/rate-limit/rate-limit-middleware.ts` that wraps the `rateLimit` checker into Hono middleware:
|
||||
|
||||
```typescript
|
||||
import { type MiddlewareHandler } from 'hono';
|
||||
|
||||
import { getIpAddress } from '@documenso/lib/universal/get-ip-address';
|
||||
|
||||
export const createRateLimitMiddleware = (
|
||||
limiter: ReturnType<typeof rateLimit>,
|
||||
options?: { identifierFn?: (c: Context) => string | undefined },
|
||||
): MiddlewareHandler => {
|
||||
return async (c, next) => {
|
||||
let ip: string;
|
||||
try {
|
||||
ip = getIpAddress(c.req.raw);
|
||||
} catch {
|
||||
ip = 'unknown';
|
||||
}
|
||||
|
||||
const identifier = options?.identifierFn?.(c);
|
||||
|
||||
const result = await limiter.check({ ip, identifier });
|
||||
|
||||
c.header('X-RateLimit-Limit', String(result.limit));
|
||||
c.header('X-RateLimit-Remaining', String(result.remaining));
|
||||
c.header('X-RateLimit-Reset', String(Math.ceil(result.reset.getTime() / 1000)));
|
||||
|
||||
if (result.isLimited) {
|
||||
c.header('Retry-After', String(Math.ceil((result.reset.getTime() - Date.now()) / 1000)));
|
||||
return c.json({ error: 'Too many requests, please try again later.' }, 429);
|
||||
}
|
||||
|
||||
await next();
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
### 3.2 Replace existing Hono rate limiters
|
||||
|
||||
In `apps/remix/server/router.ts`:
|
||||
|
||||
- Remove `hono-rate-limiter` import and both `rateLimiter()` instances
|
||||
- Replace with `createRateLimitMiddleware()` calls using the defined rate limits
|
||||
- API routes use IP-only limiting (no identifier)
|
||||
- AI route uses IP-only limiting with the stricter 3/min limit
|
||||
|
||||
```typescript
|
||||
// Before
|
||||
import { rateLimiter } from 'hono-rate-limiter';
|
||||
const rateLimitMiddleware = rateLimiter({ ... });
|
||||
|
||||
// After
|
||||
import { createRateLimitMiddleware } from '@documenso/lib/server-only/rate-limit/rate-limit-middleware';
|
||||
import { apiV1RateLimit, apiV2RateLimit, aiRateLimit } from '@documenso/lib/server-only/rate-limit/rate-limits';
|
||||
|
||||
const apiV1RateLimitMiddleware = createRateLimitMiddleware(apiV1RateLimit);
|
||||
const apiV2RateLimitMiddleware = createRateLimitMiddleware(apiV2RateLimit);
|
||||
const aiRateLimitMiddleware = createRateLimitMiddleware(aiRateLimit);
|
||||
```
|
||||
|
||||
### 3.3 Response helpers for inline checks
|
||||
|
||||
For auth routes (Hono handlers) and tRPC routes where rate limiting is applied inline rather than via middleware, provide helpers that handle the response formatting and headers consistently.
|
||||
|
||||
**Hono helper** — returns a 429 `Response` with headers if limited, or `null` if allowed:
|
||||
|
||||
```typescript
|
||||
export const rateLimitResponse = (c: Context, result: RateLimitCheckResult): Response | null => {
|
||||
c.header('X-RateLimit-Limit', String(result.limit));
|
||||
c.header('X-RateLimit-Remaining', String(result.remaining));
|
||||
c.header('X-RateLimit-Reset', String(Math.ceil(result.reset.getTime() / 1000)));
|
||||
|
||||
if (result.isLimited) {
|
||||
c.header('Retry-After', String(Math.ceil((result.reset.getTime() - Date.now()) / 1000)));
|
||||
return c.json({ error: 'Too many requests, please try again later.' }, 429);
|
||||
}
|
||||
|
||||
return null;
|
||||
};
|
||||
```
|
||||
|
||||
Usage in auth routes:
|
||||
|
||||
```typescript
|
||||
const result = await loginRateLimit.check({
|
||||
ip: requestMetadata.ipAddress ?? 'unknown',
|
||||
identifier: input.email,
|
||||
});
|
||||
|
||||
const limited = rateLimitResponse(c, result);
|
||||
if (limited) return limited;
|
||||
```
|
||||
|
||||
**tRPC helper** — throws a `TRPCError` with rate limit headers if limited:
|
||||
|
||||
```typescript
|
||||
export const assertRateLimit = (result: RateLimitCheckResult): void => {
|
||||
if (result.isLimited) {
|
||||
throw new TRPCError({
|
||||
code: 'TOO_MANY_REQUESTS',
|
||||
});
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
Usage in tRPC routes:
|
||||
|
||||
```typescript
|
||||
const result = await request2FAEmailRateLimit.check({
|
||||
ip: ctx.requestMetadata.ipAddress ?? 'unknown',
|
||||
identifier: input.recipientId,
|
||||
});
|
||||
|
||||
assertRateLimit(result);
|
||||
```
|
||||
|
||||
Both helpers live in `packages/lib/server-only/rate-limit/rate-limit-middleware.ts` alongside the Hono middleware.
|
||||
|
||||
### 3.4 Auth endpoint rate limiting
|
||||
|
||||
In `packages/auth/server/routes/email-password.ts`, add rate limit checks at the start of each handler using the `rateLimitResponse` helper.
|
||||
|
||||
Apply to each endpoint per the tier list:
|
||||
|
||||
| Endpoint | Rate Limit |
|
||||
| --------------------------- | ----------------------------------------------------- |
|
||||
| `POST /signup` | `signupRateLimit` with `identifier: email` |
|
||||
| `POST /authorize` (login) | `loginRateLimit` with `identifier: email` |
|
||||
| `POST /forgot-password` | `forgotPasswordRateLimit` with `identifier: email` |
|
||||
| `POST /resend-verify-email` | `resendVerifyEmailRateLimit` with `identifier: email` |
|
||||
| `POST /verify-email` | `verifyEmailRateLimit` with `identifier: token` |
|
||||
| `POST /reset-password` | `resetPasswordRateLimit` with `identifier: token` |
|
||||
| `POST /passkey/authorize` | `passkeyRateLimit` (IP only, no identifier) |
|
||||
| `POST /oauth/authorize/*` | `oauthRateLimit` (IP only) |
|
||||
|
||||
### 3.4 tRPC unauthenticated route rate limiting
|
||||
|
||||
For unauthenticated tRPC routes that send emails, add rate limit checks at the start of the route handler:
|
||||
|
||||
| Route | Rate Limit | Identifier |
|
||||
| ---------------------------------------------------------- | ------------------------------------ | ---------------------- |
|
||||
| `document.accessAuth.request2FAEmail` | `request2FAEmailRateLimit` | `recipientId` or token |
|
||||
| `enterprise.organisation.authenticationPortal.linkAccount` | `linkOrgAccountRateLimit` | email |
|
||||
| `template.createDocumentFromDirectTemplate` | Dedicated direct template rate limit | IP only |
|
||||
|
||||
Access `requestMetadata` from the tRPC context (`ctx.requestMetadata.ipAddress`).
|
||||
|
||||
### 3.5 tRPC and file routes — general API rate limiting
|
||||
|
||||
Add rate limit middleware for currently unprotected routes:
|
||||
|
||||
- `/api/trpc/*` — apply `apiTrpcRateLimit` middleware
|
||||
- `/api/files/*` — apply `fileUploadRateLimit` middleware
|
||||
|
||||
---
|
||||
|
||||
## 4. Cleanup Job
|
||||
|
||||
### 4.1 Job definition
|
||||
|
||||
Create `packages/lib/jobs/definitions/internal/cleanup-rate-limits.ts`:
|
||||
|
||||
```typescript
|
||||
export const CLEANUP_RATE_LIMITS_JOB_DEFINITION = {
|
||||
id: 'internal.cleanup-rate-limits',
|
||||
name: 'Cleanup Rate Limits',
|
||||
version: '1.0.0',
|
||||
trigger: {
|
||||
name: 'internal.cleanup-rate-limits',
|
||||
schema: z.object({}),
|
||||
cron: '*/15 * * * *', // Every 15 minutes
|
||||
},
|
||||
handler: async ({ payload, io }) => {
|
||||
const handler = await import('./cleanup-rate-limits.handler');
|
||||
await handler.run({ payload, io });
|
||||
},
|
||||
} as const satisfies JobDefinition<...>;
|
||||
```
|
||||
|
||||
### 4.2 Job handler
|
||||
|
||||
Create `packages/lib/jobs/definitions/internal/cleanup-rate-limits.handler.ts`:
|
||||
|
||||
- Delete all `RateLimit` rows where `createdAt` is older than 24 hours (covers all possible windows with margin)
|
||||
- Use batched deletes to avoid long-running transactions
|
||||
- Batch in chunks of 10,000 rows
|
||||
|
||||
```typescript
|
||||
export const run = async () => {
|
||||
const cutoff = new Date(Date.now() - 24 * 60 * 60 * 1000);
|
||||
|
||||
let deleted = 0;
|
||||
do {
|
||||
// Prisma doesn't support DELETE with LIMIT, so use raw SQL for batching
|
||||
deleted = await prisma.$executeRaw`
|
||||
DELETE FROM "RateLimit"
|
||||
WHERE "createdAt" < ${cutoff}
|
||||
AND ctid IN (
|
||||
SELECT ctid FROM "RateLimit"
|
||||
WHERE "createdAt" < ${cutoff}
|
||||
LIMIT 10000
|
||||
)
|
||||
`;
|
||||
} while (deleted > 0);
|
||||
};
|
||||
```
|
||||
|
||||
### 4.3 Register in job client
|
||||
|
||||
Add `CLEANUP_RATE_LIMITS_JOB_DEFINITION` to the job registry in `packages/lib/jobs/client.ts`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Remove hono-rate-limiter Dependency
|
||||
|
||||
After the migration is complete:
|
||||
|
||||
- Remove `hono-rate-limiter` from `apps/remix/package.json`
|
||||
- Run `npm install` to clean up
|
||||
|
||||
---
|
||||
|
||||
## 6. Files to Create or Modify
|
||||
|
||||
### New Files
|
||||
|
||||
| File | Purpose |
|
||||
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
||||
| `packages/lib/server-only/rate-limit/rate-limit.ts` | Core rate limit factory (`rateLimit()`) with window parsing, bucket computation, Prisma upsert, fail-open |
|
||||
| `packages/lib/server-only/rate-limit/rate-limits.ts` | All rate limit instances (auth, API, AI, file upload) |
|
||||
| `packages/lib/server-only/rate-limit/rate-limit-middleware.ts` | Hono middleware factory, `rateLimitResponse` helper for Hono handlers, `assertRateLimit` helper for tRPC routes |
|
||||
| `packages/lib/jobs/definitions/internal/cleanup-rate-limits.ts` | Cleanup cron job definition |
|
||||
| `packages/lib/jobs/definitions/internal/cleanup-rate-limits.handler.ts` | Cleanup handler (batched deletes) |
|
||||
|
||||
### Modified Files
|
||||
|
||||
| File | Change |
|
||||
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| `packages/prisma/schema.prisma` | Add `RateLimit` model |
|
||||
| `apps/remix/server/router.ts` | Replace `hono-rate-limiter` with DB-backed middleware, add rate limits for `/api/trpc/*` and `/api/files/*` |
|
||||
| `apps/remix/package.json` | Remove `hono-rate-limiter` dependency |
|
||||
| `packages/auth/server/routes/email-password.ts` | Add rate limit checks to signup, login, forgot-password, resend-verify-email, verify-email, reset-password |
|
||||
| `packages/auth/server/routes/passkey.ts` | Add rate limit check to passkey authorize |
|
||||
| `packages/auth/server/routes/oauth.ts` | Add rate limit check to OAuth authorize endpoints |
|
||||
| `packages/trpc/server/document-router/access-auth-request-2fa-email.ts` | Add rate limit check (sends email, unauthenticated) |
|
||||
| `packages/trpc/server/enterprise-router/link-organisation-account.ts` | Add rate limit check (sends email, unauthenticated) |
|
||||
| `packages/lib/jobs/client.ts` | Register cleanup-rate-limits job definition |
|
||||
|
||||
---
|
||||
|
||||
## 7. Considerations
|
||||
|
||||
### 7.1 Fail-open
|
||||
|
||||
All rate limit checks must be wrapped in try/catch. On any DB error, log the error and allow the request through. Rate limiting should never block legitimate traffic due to infrastructure issues.
|
||||
|
||||
### 7.2 Performance
|
||||
|
||||
- Each API request adds 1 upsert query (~1ms)
|
||||
- Auth requests add 2 upsert queries (~2ms total)
|
||||
- The composite primary key ensures all lookups and upserts are index-only operations
|
||||
- No `COUNT(*)` queries — the count is stored directly in the row
|
||||
|
||||
### 7.3 Monitoring
|
||||
|
||||
Log rate limit hits at `warn` level with the action, key type (IP/identifier), and count. This provides visibility into traffic patterns and helps tune limits.
|
||||
|
||||
### 7.4 Testing
|
||||
|
||||
The rate limit module should be mockable in tests. Consider exporting the bucket computation and window parsing as standalone functions for unit testing. Integration tests can verify the upsert + count logic against a test database.
|
||||
|
||||
### 7.5 Future improvements
|
||||
|
||||
- **Redis backend**: if DB pressure from rate limiting becomes measurable, swap the Prisma upsert for Redis `INCR` + `EXPIRE` with no API changes
|
||||
- **System-wide circuit breaker**: add a `systemMax` config option that counts all requests for an action regardless of key
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
date: 2026-04-22
|
||||
title: Partial Signed Pdf Download
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Let team members fetch a PDF with all currently-inserted fields burned in while the envelope is still in `PENDING` status. Today the only available bytes for a pending envelope are the original (no fields) - the sealed PDF only materialises after the last recipient signs and the `seal-document` job runs.
|
||||
|
||||
Exposed in two places:
|
||||
|
||||
- v2 API: `GET /api/v2/envelope/item/{envelopeItemId}/download?version=pending` (API-token auth)
|
||||
- UI: a `Partial` button in the existing `EnvelopeDownloadDialog`, alongside `Original`. Replaces the `Signed` slot when the envelope is `PENDING`. Backed by the existing session-authed file route `GET /api/files/envelope/{envelopeId}/envelopeItem/{id}/download/pending`.
|
||||
|
||||
## Scope
|
||||
|
||||
- v2 API only (no v1).
|
||||
- `internalVersion === 2` envelopes only. Legacy v1 returns 400 `ENVELOPE_LEGACY`.
|
||||
- Team-side / owner only. No recipient-token download path - recipients have the in-app overlay viewer for verification, and a downloadable half-signed PDF is a leak vector for partially-executed contracts. Enforced both at the server (the recipient-token file route does not accept `pending`) and at the UI (the dialog hides the Partial button when a recipient token is set).
|
||||
- No PKI signature, no certificate page, no audit log appendix - the response is explicitly not a final executed document.
|
||||
- No watermark or banner text. The filename suffix (`_pending.pdf`), the `Cache-Control: no-store, private` header, and the absence of a PKI signature are sufficient to signal draft status.
|
||||
|
||||
## Behaviour
|
||||
|
||||
API response matrix (both `/api/v2/envelope/item/{id}/download?version=pending` and the UI-facing `/api/files/envelope/{envelopeId}/envelopeItem/{id}/download/pending`):
|
||||
|
||||
| Envelope status | Response |
|
||||
|---|---|
|
||||
| `PENDING` (v2) | 200, PDF with currently-inserted fields burned in |
|
||||
| `PENDING` (v1) | 400 `ENVELOPE_LEGACY` |
|
||||
| `DRAFT` | 400 `ENVELOPE_DRAFT` |
|
||||
| `COMPLETED` | 400 `ENVELOPE_COMPLETED` |
|
||||
| `REJECTED` | 400 `ENVELOPE_REJECTED` |
|
||||
|
||||
All v1-vs-v2 / status-mismatch errors are 4xx so callers can cleanly separate them from real server failures (5xx). Specifically v1 PENDING returns 400 not 501: 5xx is reserved for actual server problems, while "this envelope can't satisfy this request shape" is a client-addressable condition.
|
||||
|
||||
Filename: `{title}_pending.pdf`.
|
||||
|
||||
ETag is content-addressed over `sha256(envelope.status + sorted((field.id, field.customText, field.signature?.id, field.signature?.created) for inserted===true fields))`. Returns 304 on `If-None-Match` match.
|
||||
|
||||
No persistent caching. Generated on-demand per request when ETag misses.
|
||||
|
||||
Error response shape (envelope item v2 download route and the team-side file route): preserves the existing `{ error: <message> }` field for backwards compatibility and adds `code: <APP_ERROR_CODE>` as a new field for callers that want to branch on it. The document download route (`/document/{documentId}/download`) is untouched.
|
||||
|
||||
## UI
|
||||
|
||||
`apps/remix/app/components/dialogs/envelope-download-dialog.tsx`:
|
||||
|
||||
- The dialog shows `Original` plus one of:
|
||||
- `Signed` when status is `COMPLETED` (existing behaviour)
|
||||
- `Partial` when status is `PENDING`, there is no recipient token, and the envelope is not legacy (`!isLegacy`)
|
||||
- nothing otherwise
|
||||
- New optional prop `isLegacy?: boolean`. Only consulted to gate the `Partial` button, so callers whose status can never be `PENDING` (DRAFT/COMPLETED/REJECTED hardcoded, or `isComplete: true` matchers) and callers that always set a recipient token can omit it. Three call sites pass it (`isLegacy={envelope.internalVersion === 1}`): `documents-table-action-dropdown.tsx`, `envelope-editor.tsx`, `document-page-view-dropdown.tsx`. The other eight callers were left alone.
|
||||
|
||||
Trade-off: a future team-side dialog usage where status could be PENDING but the dev forgets `isLegacy` will silently not render the Partial button. The status gate prevents an actively broken click; missing button is discoverable in testing. Required-prop alternative was rejected because eight of eleven call sites would carry a meaningless value.
|
||||
|
||||
## Files
|
||||
|
||||
Server:
|
||||
|
||||
- `apps/remix/server/api/download/download.types.ts` - added `'pending'` to the `version` enum; split the validator into `param` (envelopeItemId) + `query` (version). The original wiring as a path-param validator was a pre-existing bug: requests like `?version=original` were silently returning the signed PDF since `version` actually arrives as a query string. Fixed as a side effect.
|
||||
- `packages/trpc/server/envelope-router/download-envelope-item.types.ts` - mirrored the enum change in the OpenAPI schema.
|
||||
- `apps/remix/server/api/download/download.ts` - the envelope item v2 route now fetches envelope recipients alongside the envelope, branches on `version` when calling the helper, and emits AppError responses as `{ error, code }` consistently across all status codes.
|
||||
- `apps/remix/server/api/files/files.types.ts` - added `'pending'` to the team-side download schema only. The recipient-token download schema is untouched, so `/api/files/token/.../download/pending` is rejected by the schema validator.
|
||||
- `apps/remix/server/api/files/files.ts` - the team-side download handler fetches envelope recipients and dispatches the `pending` branch through the same `handleEnvelopeItemFileRequest` helper. Wrapped in a try/catch that returns `{ error, code }` for AppErrors.
|
||||
- `apps/remix/server/api/files/files.helpers.ts` - `handleEnvelopeItemFileRequest` is now a single entry point taking a discriminated-union options type. The static-file flow (`signed`/`original`) and the on-demand pending flow are private helpers in the same module.
|
||||
- `packages/lib/server-only/pdf/generate-partial-signed-pdf.ts` (new) - small orchestrator that loads the original PDF, groups inserted fields by page, calls the existing `insertFieldInPDFV2` overlay helper for each page, flattens, and saves.
|
||||
- `packages/lib/errors/app-error.ts` - added `ENVELOPE_DRAFT`, `ENVELOPE_COMPLETED`, `ENVELOPE_REJECTED`, `ENVELOPE_LEGACY` codes, all mapped to 400. The legacy-envelope case deliberately returns 4xx rather than 501 to keep "this resource can't satisfy this operation" distinct from real 5xx server failures in caller logs/metrics.
|
||||
|
||||
Client:
|
||||
|
||||
- `packages/lib/utils/envelope-download.ts` - `EnvelopeItemPdfUrlOptions` download variant now allows `'pending'` as a version. The recipient-token URL builder will produce a URL the server rejects, but the dialog gates on no-token at the call site.
|
||||
- `packages/lib/client-only/download-pdf.ts` - `DocumentVersion` extended; filename suffix logic moved into a small switch (`_signed.pdf`, `_pending.pdf`, `.pdf`).
|
||||
- `apps/remix/app/components/dialogs/envelope-download-dialog.tsx` - secondary download derivation with the new `Partial` branch, optional `isLegacy` prop.
|
||||
- `apps/remix/app/components/tables/documents-table-action-dropdown.tsx`, `apps/remix/app/components/general/envelope-editor/envelope-editor.tsx`, `apps/remix/app/components/general/document/document-page-view-dropdown.tsx` - pass `isLegacy={envelope.internalVersion === 1}` (or `row.internalVersion === 1`) to the dialog.
|
||||
|
||||
## Verification
|
||||
|
||||
1. E2E (`packages/app-tests/e2e/api/v2/partial-signed-pdf-download.spec.ts`):
|
||||
- Pending envelope, recipient 1 signs, API token download with `?version=pending` returns 200 + PDF; subsequent call with `If-None-Match: <etag>` returns 304; after recipient 2 completes the envelope flips to `COMPLETED` and the same call returns 400 `ENVELOPE_COMPLETED`; `?version=signed` then succeeds.
|
||||
- Draft envelope returns 400 `ENVELOPE_DRAFT`.
|
||||
- `internalVersion === 1` pending envelope returns 400 `ENVELOPE_LEGACY`.
|
||||
|
||||
2. `npx tsc --noEmit -p apps/remix/tsconfig.json` and `npm run lint`.
|
||||
|
||||
3. Manual: open the Documents table or envelope editor on a PENDING envelope (v2), open the download dialog, confirm `Partial` appears alongside `Original` and produces a `_pending.pdf` with current fields burned in. Same dialog on a COMPLETED envelope shows `Signed`. Same dialog on a v1 PENDING envelope shows neither (status gate would show Partial, but the `isLegacy` flag filters it out).
|
||||
|
||||
## Out of Scope / Follow-ups
|
||||
|
||||
- Recipient-token download path (API and UI) - decided against. Revisit if there is concrete demand and a story for limiting the leak vector.
|
||||
- v1 API parity / v1 partial rendering - not building. Implementing partial for v1 would require porting `legacy_insertFieldInPDF` / `insertFieldInPDFV1` into a partial-only flow, which is code with no long-term home as v1 is being phased out.
|
||||
- Document download route (`/document/{documentId}/download`) - untouched. Same error shape and validator wiring as before. Consider normalising to the same `{ error, code }` shape in a follow-up if any caller wants to branch on `code` from that route.
|
||||
- Persistent caching layer / job-queue generation - revisit if p95 latency on large PDFs becomes an issue.
|
||||
- Specific toast for `ENVELOPE_LEGACY` in the dialog - currently the catch-all "Something went wrong" handles it. Worth a polish if v1 PENDING envelopes are common in your data and we see complaints. (Note: with the `isLegacy` gate at the UI, the error is unreachable from the dialog itself; the API can still surface it for direct callers.)
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
date: 2026-02-24
|
||||
title: Custom Email Domain Sync And Recovery
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Custom email domains configured via AWS SES can get stuck in a `PENDING` state or fail validation silently. Currently, there is **no automated verification** -- users must manually click "Sync" in the UI to check domain status. If a domain fails to validate, the only option is to delete it and recreate it, which generates new DKIM keys and requires the user to update their DNS records.
|
||||
|
||||
### Current Pain Points
|
||||
|
||||
1. **No background sync** -- Domain verification status is never checked automatically; users must manually click "Sync"
|
||||
2. **Stuck domains** -- Domains can remain in `PENDING` state indefinitely with no alerting or auto-recovery
|
||||
3. **Failed recovery requires DNS changes** -- Deleting and recreating a domain generates new keys, forcing the user to update DNS records
|
||||
4. **No visibility into failure duration** -- There's no tracking of how long a domain has been pending
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
### 1. Hourly Background Sync Job
|
||||
|
||||
Create a new cron job (`internal.sync-email-domains`) that runs every hour to automatically verify all `PENDING` email domains.
|
||||
|
||||
**Job Definition:** `packages/lib/jobs/definitions/internal/sync-email-domains.ts`
|
||||
**Job Handler:** `packages/lib/jobs/definitions/internal/sync-email-domains.handler.ts`
|
||||
|
||||
**Pattern:** Follow the existing `cleanup-rate-limits` cron job pattern:
|
||||
|
||||
- `cron: '0 * * * *'` (every hour, on the hour)
|
||||
- Empty `z.object({})` schema (no payload needed)
|
||||
- Register in `packages/lib/jobs/client.ts`
|
||||
|
||||
**Handler Logic:**
|
||||
|
||||
1. Query all `EmailDomain` records with `status: 'PENDING'`
|
||||
2. For each domain, call `verifyEmailDomain(emailDomainId)` which:
|
||||
- Calls AWS SES `GetEmailIdentityCommand` to check current verification status
|
||||
- Updates DB status to `ACTIVE` if verified, keeps `PENDING` otherwise
|
||||
3. Log results via `io.logger` (how many checked, how many transitioned to ACTIVE)
|
||||
4. Process domains in batches to avoid overwhelming SES API rate limits
|
||||
5. Add error handling per-domain so one failure doesn't stop the entire sweep
|
||||
|
||||
### 2. Schema Changes -- Track Pending Duration
|
||||
|
||||
Add a `lastVerifiedAt` column to the `EmailDomain` model to track when verification was last attempted, enabling "stale domain" detection.
|
||||
|
||||
**File:** `packages/prisma/schema.prisma`
|
||||
|
||||
```prisma
|
||||
model EmailDomain {
|
||||
// ... existing fields ...
|
||||
lastVerifiedAt DateTime? // Last time verification was checked against SES
|
||||
}
|
||||
```
|
||||
|
||||
**Migration:** Create a new Prisma migration for this column addition.
|
||||
|
||||
**Updates needed:**
|
||||
|
||||
- `verify-email-domain.ts` -- Update `lastVerifiedAt` when verification is checked
|
||||
- The sync job handler -- Use `lastVerifiedAt` to avoid re-checking domains that were just verified
|
||||
|
||||
### 3. Domain Re-registration (Recovery) -- Delete & Recreate in SES Without Changing Keys
|
||||
|
||||
Add a new "Re-register" action that deletes the SES identity and recreates it using the **same** DKIM key pair stored in the database, so the user's DNS records remain valid.
|
||||
|
||||
#### 3a. New Service Function
|
||||
|
||||
**File:** `packages/ee/server-only/lib/reregister-email-domain.ts`
|
||||
|
||||
```typescript
|
||||
export const reregisterEmailDomain = async (options: { emailDomainId: string }) => {
|
||||
// 1. Fetch the EmailDomain record (including encrypted privateKey)
|
||||
// 2. Decrypt the private key using DOCUMENSO_ENCRYPTION_KEY
|
||||
// 3. Call DeleteEmailIdentityCommand on SES (ignore NotFoundException)
|
||||
// 4. Call CreateEmailIdentityCommand with BYODKIM using the SAME selector + private key
|
||||
// 5. Update EmailDomain status back to PENDING, update lastVerifiedAt
|
||||
// 6. Return the updated domain
|
||||
};
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- Uses the existing encrypted `privateKey` from the DB -- no new key generation
|
||||
- Uses the existing `selector` -- DNS records stay the same
|
||||
- Deletes first, then recreates -- handles cases where SES state is corrupted
|
||||
- Resets status to `PENDING` since verification will need to re-occur
|
||||
- Uses `verifyDomainWithDKIM()` from `create-email-domain.ts` (may need to extract/export this helper)
|
||||
|
||||
#### 3b. Admin TRPC Routes (Find, Get, Re-register)
|
||||
|
||||
All email domain admin routes use `adminProcedure` -- requires system-level `Role.ADMIN`.
|
||||
|
||||
**Find (list) route:**
|
||||
**File:** `packages/trpc/server/admin-router/find-email-domains.ts`
|
||||
**Types:** `packages/trpc/server/admin-router/find-email-domains.types.ts`
|
||||
|
||||
- Query route: `admin.emailDomain.find`
|
||||
- Input: `{ query?: string, page?: number, perPage?: number, status?: EmailDomainStatus }`
|
||||
- Extends `ZFindSearchParamsSchema` with optional `status` filter
|
||||
- Returns standard `ZFindResultResponse` with email domain data including: id, domain, status, selector, createdAt, lastVerifiedAt, organisation name, email count
|
||||
- Prisma query filters by domain name (LIKE search on `query`), optional status, joins organisation for name, counts emails
|
||||
|
||||
**Get (detail) route:**
|
||||
**File:** `packages/trpc/server/admin-router/get-email-domain.ts`
|
||||
**Types:** `packages/trpc/server/admin-router/get-email-domain.types.ts`
|
||||
|
||||
- Query route: `admin.emailDomain.get`
|
||||
- Input: `{ emailDomainId: string }`
|
||||
- Returns full email domain detail: all fields (except privateKey), organisation info, list of associated emails, DNS records (generated from publicKey + selector)
|
||||
- Omits `privateKey` from response
|
||||
|
||||
**Re-register (mutation) route:**
|
||||
**File:** `packages/trpc/server/admin-router/reregister-email-domain.ts`
|
||||
**Types:** `packages/trpc/server/admin-router/reregister-email-domain.types.ts`
|
||||
|
||||
- Mutation route: `admin.emailDomain.reregister`
|
||||
- Input: `{ emailDomainId: string }`
|
||||
- Calls `reregisterEmailDomain()`
|
||||
- Rationale: Re-registration is a recovery/operational action that deletes and recreates an SES identity. This is a privileged operation that should only be performed by platform operators, not self-service by org admins.
|
||||
|
||||
#### 3c. Register in Admin Router
|
||||
|
||||
**File:** `packages/trpc/server/admin-router/router.ts`
|
||||
|
||||
Add a new `emailDomain` namespace to the admin router:
|
||||
|
||||
```typescript
|
||||
emailDomain: {
|
||||
find: findEmailDomainsRoute,
|
||||
get: getEmailDomainRoute,
|
||||
reregister: reregisterEmailDomainRoute,
|
||||
},
|
||||
```
|
||||
|
||||
#### 3d. Admin Panel UI -- Email Domains Section
|
||||
|
||||
**List page:** `apps/remix/app/routes/_authenticated+/admin+/email-domains._index.tsx`
|
||||
|
||||
- New admin panel page at `/admin/email-domains`
|
||||
- Follow the existing admin documents list pattern (client-side TRPC data fetching)
|
||||
- Search input (debounced) filtering by domain name
|
||||
- Status filter dropdown (All / Pending / Active)
|
||||
- DataTable with columns: Domain, Organisation, Status (badge), Email Count, Created, Last Verified, Actions
|
||||
- Actions dropdown per row: View details, Re-register
|
||||
- Pagination via `DataTablePagination`
|
||||
|
||||
**Detail page:** `apps/remix/app/routes/_authenticated+/admin+/email-domains.$id.tsx`
|
||||
|
||||
- Shows full domain details: domain, selector, status, organisation, created date, last verified date
|
||||
- Shows DNS records (DKIM + SPF) with copy buttons (reuse `organisation-email-domain-records-dialog` pattern)
|
||||
- Table of associated organisation emails
|
||||
- "Re-register" button with confirmation dialog explaining the action (SES identity will be deleted and recreated with the same keys)
|
||||
- "Verify Now" button to manually trigger a verification check
|
||||
- Shows how long the domain has been pending (using `lastVerifiedAt` or `createdAt`)
|
||||
|
||||
**Navigation:** Add menu item to admin sidebar in `_layout.tsx`:
|
||||
|
||||
```tsx
|
||||
<Button
|
||||
variant="ghost"
|
||||
className={cn(
|
||||
'justify-start md:w-full',
|
||||
pathname?.startsWith('/admin/email-domains') && 'bg-secondary',
|
||||
)}
|
||||
asChild
|
||||
>
|
||||
<Link to="/admin/email-domains">
|
||||
<MailIcon className="mr-2 h-5 w-5" />
|
||||
<Trans>Email Domains</Trans>
|
||||
</Link>
|
||||
</Button>
|
||||
```
|
||||
|
||||
**Table component:** `apps/remix/app/components/tables/admin-email-domains-table.tsx` (optional -- can be inline in the route file like the documents page)
|
||||
|
||||
#### 3e. Automatic Re-registration in Sync Job (Optional Enhancement)
|
||||
|
||||
In the hourly sync job, after checking verification status, if a domain has been `PENDING` for more than 48 hours:
|
||||
|
||||
- Automatically call `reregisterEmailDomain()` to attempt recovery
|
||||
- Log the auto-recovery attempt
|
||||
- This provides a self-healing mechanism without user intervention
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Background Sync Job (Core)
|
||||
|
||||
1. Create `sync-email-domains.ts` job definition with hourly cron
|
||||
2. Create `sync-email-domains.handler.ts` with batch verification logic
|
||||
3. Register job in `packages/lib/jobs/client.ts`
|
||||
4. Add error handling and logging
|
||||
|
||||
### Phase 2: Schema Enhancement
|
||||
|
||||
5. Add `lastVerifiedAt` column to `EmailDomain` model
|
||||
6. Create Prisma migration
|
||||
7. Update `verifyEmailDomain()` to set `lastVerifiedAt` on each check
|
||||
8. Update sync job to use `lastVerifiedAt` for intelligent scheduling
|
||||
|
||||
### Phase 3: Admin Email Domains Panel
|
||||
|
||||
9. Create `find-email-domains` admin TRPC route + types (list/search with pagination and status filter)
|
||||
10. Create `get-email-domain` admin TRPC route + types (detail view with org info, emails, DNS records)
|
||||
11. Register find + get routes in admin router under `emailDomain` namespace
|
||||
12. Create admin list page (`admin+/email-domains._index.tsx`) with search, status filter, DataTable
|
||||
13. Create admin detail page (`admin+/email-domains.$id.tsx`) with domain info, emails table, DNS records
|
||||
14. Add "Email Domains" menu item to admin sidebar (`_layout.tsx`)
|
||||
|
||||
### Phase 4: Re-registration Feature
|
||||
|
||||
15. Extract `verifyDomainWithDKIM()` as a shared helper (if not already exported)
|
||||
16. Create `reregisterEmailDomain()` service function
|
||||
17. Create `reregister-email-domain` admin TRPC mutation route + types
|
||||
18. Register reregister route in admin router under `emailDomain.reregister`
|
||||
19. Add "Re-register" button + confirmation dialog on admin detail page
|
||||
|
||||
### Phase 5: Auto-Recovery (Optional)
|
||||
|
||||
20. Add 48-hour stale detection logic to sync job
|
||||
21. Auto-trigger re-registration for stale domains
|
||||
22. Add logging/notifications for auto-recovery events
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
### New Files
|
||||
|
||||
- `packages/lib/jobs/definitions/internal/sync-email-domains.ts`
|
||||
- `packages/lib/jobs/definitions/internal/sync-email-domains.handler.ts`
|
||||
- `packages/ee/server-only/lib/reregister-email-domain.ts`
|
||||
- `packages/trpc/server/admin-router/find-email-domains.ts`
|
||||
- `packages/trpc/server/admin-router/find-email-domains.types.ts`
|
||||
- `packages/trpc/server/admin-router/get-email-domain.ts`
|
||||
- `packages/trpc/server/admin-router/get-email-domain.types.ts`
|
||||
- `packages/trpc/server/admin-router/reregister-email-domain.ts`
|
||||
- `packages/trpc/server/admin-router/reregister-email-domain.types.ts`
|
||||
- `apps/remix/app/routes/_authenticated+/admin+/email-domains._index.tsx`
|
||||
- `apps/remix/app/routes/_authenticated+/admin+/email-domains.$id.tsx`
|
||||
|
||||
### Modified Files
|
||||
|
||||
- `packages/prisma/schema.prisma` -- Add `lastVerifiedAt` field
|
||||
- `packages/lib/jobs/client.ts` -- Register new sync job
|
||||
- `packages/ee/server-only/lib/verify-email-domain.ts` -- Update `lastVerifiedAt`
|
||||
- `packages/ee/server-only/lib/create-email-domain.ts` -- Export `verifyDomainWithDKIM` helper
|
||||
- `packages/trpc/server/admin-router/router.ts` -- Add `emailDomain.{find, get, reregister}` routes
|
||||
- `apps/remix/app/routes/_authenticated+/admin+/_layout.tsx` -- Add "Email Domains" nav item to sidebar
|
||||
- New Prisma migration file
|
||||
|
||||
## Technical Considerations
|
||||
|
||||
1. **SES API Rate Limits** -- AWS SES has rate limits on `GetEmailIdentityCommand`. The sync job should process domains in batches with small delays between calls (e.g., 5-10 per batch with 1s delay).
|
||||
|
||||
2. **Concurrency** -- The local job provider has deterministic deduplication via SHA-256 IDs, so multiple app instances won't run the same cron tick twice.
|
||||
|
||||
3. **Error Isolation** -- Each domain verification in the sync job should be wrapped in try/catch so one failing domain doesn't prevent others from being checked.
|
||||
|
||||
4. **Re-registration Safety** -- The re-register function should be idempotent. Deleting a non-existent SES identity should be handled gracefully (already done in `deleteEmailDomain`).
|
||||
|
||||
5. **Private Key Security** -- The private key is encrypted at rest and should only be decrypted transiently during re-registration. It should never be logged or exposed in API responses.
|
||||
|
||||
6. **Feature Gating** -- The sync job should only process domains belonging to organisations with active `emailDomains` claim flags. This prevents processing domains for orgs that have downgraded.
|
||||
|
||||
7. **Observability** -- Add structured logging to the sync job so operations teams can monitor domain verification health across all tenants.
|
||||
@@ -0,0 +1,210 @@
|
||||
---
|
||||
date: 2026-02-10
|
||||
title: Add Folder Support To V1 Api
|
||||
status: ready
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The `GET /api/v1/documents` endpoint does not return documents inside folders. The underlying `findDocuments()` function defaults to `folderId: null` when no `folderId` is provided, meaning only root-level documents are returned. The V1 API never passes `folderId`, so folder documents are invisible to API consumers.
|
||||
|
||||
Additionally, neither the list endpoint nor the single-document endpoint exposes `folderId` in the response, so consumers cannot know which folder a document belongs to.
|
||||
|
||||
## Root Cause
|
||||
|
||||
In `packages/lib/server-only/document/find-documents.ts` (line 222-226):
|
||||
|
||||
```ts
|
||||
if (folderId !== undefined) {
|
||||
whereClause.folderId = folderId;
|
||||
} else {
|
||||
whereClause.folderId = null; // Only root documents returned
|
||||
}
|
||||
```
|
||||
|
||||
The V1 `getDocuments` handler in `packages/api/v1/implementation.ts` (line 61-70) only passes `page` and `perPage` to `findDocuments` — it never extracts or forwards a `folderId` from the query string.
|
||||
|
||||
## Decisions
|
||||
|
||||
These decisions were made during the spec interview:
|
||||
|
||||
1. **Fix V1 directly** — The V1 API is deprecated but still actively used. This is a quick, low-risk fix. No need to defer to a newer API.
|
||||
2. **Breaking change accepted** — Returning ALL documents by default (instead of root-only) is intentional. The current root-only behavior is a bug, not a feature.
|
||||
3. **No root-only query option needed** — Not all documents are in folders, so consumers can filter client-side using the `folderId` field in the response if needed.
|
||||
4. **No folder existence validation** — `?folderId=nonexistent` returns empty array, not 404. Consistent with V1 list endpoint patterns.
|
||||
5. **Add `folderId` to both endpoints** — Both `GET /api/v1/documents` (list) and `GET /api/v1/documents/:id` (single) will include `folderId` in the response.
|
||||
6. **Top-level `skipFolderFilter` is sufficient** — The inner helper filters (`findDocumentsFilter`, `findTeamDocumentsFilter`) receive `folderId: undefined` when skip is active. Prisma ignores `undefined` values in WHERE clauses, so these inner filters will not constrain by folder. No propagation needed.
|
||||
7. **Scope is minimal** — Only `folderId` support. No other filters (status, period, query, senderIds) added in this change.
|
||||
|
||||
## Scope
|
||||
|
||||
Three files need changes. No new files.
|
||||
|
||||
| File | Change |
|
||||
| ----------------------------------------------------- | ------------------------------------------------------------------------ |
|
||||
| `packages/api/v1/schema.ts` | Add `folderId` to query schema + both response schemas |
|
||||
| `packages/api/v1/implementation.ts` | Pass `folderId` through in `getDocuments`, add to `getDocument` response |
|
||||
| `packages/lib/server-only/document/find-documents.ts` | Add `skipFolderFilter` option |
|
||||
|
||||
## Changes
|
||||
|
||||
### 1. `packages/api/v1/schema.ts` — Add `folderId` to query + response schemas
|
||||
|
||||
**Query schema** (`ZGetDocumentsQuerySchema`, line 35-38):
|
||||
|
||||
```ts
|
||||
export const ZGetDocumentsQuerySchema = z.object({
|
||||
page: z.coerce.number().min(1).optional().default(1),
|
||||
perPage: z.coerce.number().min(1).optional().default(10),
|
||||
folderId: z
|
||||
.string()
|
||||
.describe(
|
||||
'Filter documents by folder ID. When omitted, returns all documents regardless of folder.',
|
||||
)
|
||||
.optional(),
|
||||
});
|
||||
```
|
||||
|
||||
**List response schema** (`ZSuccessfulDocumentResponseSchema`, line 46-56):
|
||||
|
||||
Add `folderId: z.string().nullish()` so consumers can see which folder each document belongs to.
|
||||
|
||||
**Single document response schema** (`ZSuccessfulGetDocumentResponseSchema`, line 58-79):
|
||||
|
||||
Add `folderId: z.string().nullish()` to the extended schema as well.
|
||||
|
||||
### 2. `packages/api/v1/implementation.ts` — Pass `folderId` through + add to responses
|
||||
|
||||
**`getDocuments` handler** (line 61-70):
|
||||
|
||||
```ts
|
||||
getDocuments: authenticatedMiddleware(async (args, user, team) => {
|
||||
const page = Number(args.query.page) || 1;
|
||||
const perPage = Number(args.query.perPage) || 10;
|
||||
|
||||
const { data: documents, totalPages } = await findDocuments({
|
||||
page,
|
||||
perPage,
|
||||
userId: user.id,
|
||||
teamId: team.id,
|
||||
folderId: args.query.folderId,
|
||||
skipFolderFilter: args.query.folderId === undefined,
|
||||
});
|
||||
|
||||
return {
|
||||
status: 200,
|
||||
body: {
|
||||
documents: documents.map((document) => ({
|
||||
id: mapSecondaryIdToDocumentId(document.secondaryId),
|
||||
externalId: document.externalId,
|
||||
userId: document.userId,
|
||||
teamId: document.teamId,
|
||||
folderId: document.folderId,
|
||||
title: document.title,
|
||||
status: document.status,
|
||||
createdAt: document.createdAt,
|
||||
updatedAt: document.updatedAt,
|
||||
completedAt: document.completedAt,
|
||||
})),
|
||||
totalPages,
|
||||
},
|
||||
};
|
||||
}),
|
||||
```
|
||||
|
||||
**`getDocument` handler** (line 91-197):
|
||||
|
||||
Add `folderId: envelope.folderId` to the response body mapping (alongside `id`, `externalId`, etc.).
|
||||
|
||||
### 3. `packages/lib/server-only/document/find-documents.ts` — Handle "return all" semantics
|
||||
|
||||
Add `skipFolderFilter` to the options type and modify the WHERE clause logic:
|
||||
|
||||
```ts
|
||||
export type FindDocumentsOptions = {
|
||||
// ... existing fields ...
|
||||
folderId?: string;
|
||||
skipFolderFilter?: boolean;
|
||||
};
|
||||
```
|
||||
|
||||
Modify the folderId logic (line 222-226):
|
||||
|
||||
```ts
|
||||
if (!skipFolderFilter) {
|
||||
if (folderId !== undefined) {
|
||||
whereClause.folderId = folderId;
|
||||
} else {
|
||||
whereClause.folderId = null;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When `skipFolderFilter` is true:
|
||||
|
||||
- The top-level `whereClause.folderId` is never set — no folder constraint at the top level.
|
||||
- The inner helpers (`findDocumentsFilter`, `findTeamDocumentsFilter`) receive `folderId: undefined`, which Prisma ignores in WHERE objects — no folder constraint at the inner level either.
|
||||
- Result: all documents returned regardless of folder.
|
||||
|
||||
When `skipFolderFilter` is false (default, used by UI/tRPC callers):
|
||||
|
||||
- Existing behavior is completely unchanged. `folderId: undefined` still defaults to root-only.
|
||||
|
||||
## Why `skipFolderFilter` (Option B)
|
||||
|
||||
Two approaches were considered:
|
||||
|
||||
**Option A: Change `folderId: undefined` semantics to mean "all documents"**
|
||||
|
||||
- Risky: would affect all callers of `findDocuments` (UI, tRPC) unless every caller is audited.
|
||||
- The UI intentionally shows root-only when no folder is selected.
|
||||
|
||||
**Option B (chosen): Add `skipFolderFilter` boolean**
|
||||
|
||||
- Additive — no existing callers pass this flag, so they're unaffected.
|
||||
- Explicit — the intent is clear in the code.
|
||||
- Safe — zero risk to UI/tRPC behavior.
|
||||
|
||||
## Behavior Matrix
|
||||
|
||||
| Request | Current Behavior | New Behavior |
|
||||
| -------------------------------------------- | ------------------- | ------------------------- |
|
||||
| `GET /api/v1/documents` | Root docs only | ALL docs (root + folders) |
|
||||
| `GET /api/v1/documents?folderId=abc` | Not supported | Docs in folder `abc` only |
|
||||
| `GET /api/v1/documents?folderId=nonexistent` | Not supported | Empty array, 200 OK |
|
||||
| `GET /api/v1/documents/:id` response | No `folderId` field | Includes `folderId` |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- `folderId` is a `String?` on the `Envelope` model in Prisma, not a number.
|
||||
- The `findDocuments` function already accepts `folderId` in its options type — it just needs the `skipFolderFilter` escape hatch.
|
||||
- No need to propagate `skipFolderFilter` into `findDocumentsFilter` or `findTeamDocumentsFilter`. When `folderId` is `undefined`, those helpers embed `folderId: undefined` in their Prisma WHERE objects. Prisma strips `undefined` keys, so no folder constraint is applied. This is well-documented Prisma behavior.
|
||||
- The `createDocument` endpoint already supports `folderId` in the request body (line 139-144 of schema.ts), confirming the pattern.
|
||||
- The `getDocument` handler fetches from `prisma.envelope.findFirstOrThrow` which already includes `folderId` on the envelope — just needs to be added to the response mapping.
|
||||
|
||||
## Testing
|
||||
|
||||
Manual and automated test cases:
|
||||
|
||||
1. `GET /api/v1/documents` returns docs from root AND subfolders.
|
||||
2. `GET /api/v1/documents?folderId=<valid-id>` returns only docs in that folder.
|
||||
3. `GET /api/v1/documents?folderId=<nonexistent-id>` returns empty array with 200 status.
|
||||
4. List response includes `folderId` field on each document (null for root docs, string for folder docs).
|
||||
5. `GET /api/v1/documents/:id` response includes `folderId` field.
|
||||
6. Existing UI/tRPC callers of `findDocuments` are unaffected (they don't pass `skipFolderFilter`).
|
||||
7. Pagination: verify `totalPages` correctly reflects the larger result set when all docs are returned.
|
||||
|
||||
## Breaking Change Notice
|
||||
|
||||
This is a **breaking change** for existing V1 API consumers:
|
||||
|
||||
- **Before**: `GET /api/v1/documents` returned only root-level documents (those not in any folder).
|
||||
- **After**: `GET /api/v1/documents` returns all documents regardless of folder placement.
|
||||
|
||||
Impact:
|
||||
|
||||
- Consumers paginating through results will see more documents in the total count.
|
||||
- Consumers building UIs will now display folder documents they previously didn't see.
|
||||
- The new `folderId` field is additive and won't break existing response parsing.
|
||||
|
||||
This is considered a **bug fix**, not a feature removal. The previous behavior silently hid documents from API consumers.
|
||||
Reference in New Issue
Block a user