fix(platform): repair age-rating migration drift, resolve object storage stream leaks, and add Quadlet template (#497)

* fix(server): repair age-rating migration drift

The age-ratings change (97c6f2c8) appended the AgeRatingOrganization
enum and GameAgeRating table to 20260224145112, a migration that had
already been applied to existing databases. Prisma never re-runs applied
migrations, so those databases were left without the enum/table; the
later 20260726041153_add_user_groups migration then failed with
"type AgeRatingOrganization does not exist" and game pages 500'd.

Restore 20260224145112 to its original applied content, make
20260726041153 self-healing by creating the enum when it is missing, and
add an idempotent 20260915000000_add_game_age_ratings migration that
creates the type, table, index and foreign key. All statements are
guarded so they are safe on databases already repaired out-of-band.

* fix(server): eliminate file descriptor double-close and stream leaks in object storage

* feat(server): add generic Podman Quadlet deployment template

* fix(server): address platform-stability review feedback

Migrations are now append-only: 20260224145112 and 20260726041153 are left untouched and 20260915000000_add_game_age_ratings is replaced by 20260726041152_repair_age_rating_schema, named to sort before 20260726041153 so drifted databases create the enum before add_user_groups runs. Databases whose add_user_groups row already failed still need migrate resolve --rolled-back, which is documented in the migration.

FsObjectBackend#write now truncates before writing, so overwriting an object with shorter content cannot leave stale trailing bytes. FsObjectBackend#fetchHash computes and stores the hash on a database miss (FsHashStore#get returns undefined, not null).

Quadlet: drop the README and the chunk-cache volume/env keys (the chunk cache is not implemented in this repository), fix rootless boot via default.target, add a PostgreSQL healthcheck, use the drop-postgres DNS name, and fix the uninstall volume glob.

* fix(server): grant save owners read access and enforce save size limits

Save snapshots were created with no permissions, so the desktop client
could never pull an archive back, and pruning relied on deleteWithPermission
(which requires a delete grant the object never has), silently leaking every
pruned snapshot. Grant `<userId>:read` and reclaim as the system.

The advertised saveSlotSizeLimit was also never enforced; measure the stream
and reject oversized payloads with a 413 before the full archive is written.

---------

Co-authored-by: John Smith <you@example.com>
This commit is contained in:
BillyOutlast
2026-09-15 10:15:31 +10:00
committed by GitHub
co-authored by John Smith
parent 344b89c5b9
commit 6c3671714d
15 changed files with 413 additions and 84 deletions
+2
View File
@@ -31,6 +31,8 @@ logs
deploy-template/*
!deploy-template/compose.yml
!deploy-template/quadlet/
!deploy-template/quadlet/**
# generated prisma client
/prisma/client
@@ -0,0 +1,2 @@
[Volume]
VolumeName=drop-data
@@ -0,0 +1,2 @@
[Volume]
VolumeName=drop-db
@@ -0,0 +1,6 @@
[Network]
NetworkName=drop-network
Driver=bridge
Subnet=172.20.0.0/16
Subnet=fd00:172:20::/64
IPv6=true
@@ -0,0 +1,27 @@
[Unit]
Description=Drop PostgreSQL Database
Wants=network-online.target
After=network-online.target
[Container]
ContainerName=drop-postgres
Image=docker.io/postgres:14-alpine
AutoUpdate=registry
Network=drop-network.network
IP=172.20.0.21
Volume=drop-db.volume:/var/lib/postgresql/data
EnvironmentFile=%E/containers/systemd/drop.env
Environment=POSTGRES_DB=drop
Environment=HOSTNAME=drop-postgres
SecurityLabelDisable=true
HealthCmd=pg_isready -d drop -U drop
HealthInterval=30s
HealthTimeout=60s
HealthRetries=5
HealthStartPeriod=10s
[Service]
Restart=always
[Install]
WantedBy=multi-user.target default.target
@@ -0,0 +1,27 @@
[Unit]
Description=Drop Game Distribution Platform
Wants=network-online.target
After=network-online.target drop-postgres.service
Requires=drop-postgres.service
[Container]
ContainerName=drop
Image=ghcr.io/drop-oss/drop:nightly
AutoUpdate=registry
Network=drop-network.network
IP=172.20.0.20
PublishPort=3000:3000
Volume=drop-data.volume:/data
# Game storage library mount (uncomment and point to your library path)
# Volume=/mnt/storage/games:/library:ro
EnvironmentFile=%E/containers/systemd/drop.env
Environment=HOSTNAME=drop
SecurityLabelDisable=true
[Service]
# PostgreSQL may still be initializing when this unit starts; Restart=always
# recovers until the database accepts connections.
Restart=always
[Install]
WantedBy=multi-user.target default.target
+9
View File
@@ -0,0 +1,9 @@
# Drop Server & Database Configuration
POSTGRES_USER=drop
POSTGRES_PASSWORD=drop
DATABASE_URL=postgres://drop:drop@drop-postgres:5432/drop
EXTERNAL_URL=http://localhost:3000/
# Optional: IGDB Metadata Provider
# IGDB_CLIENT_ID=
# IGDB_CLIENT_SECRET=
+60
View File
@@ -0,0 +1,60 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
for arg in "$@"; do
case "$arg" in
--help|-h)
echo "Usage: $0"
echo " Installs the base Drop Quadlet stack (server + PostgreSQL)."
exit 0
;;
*)
echo "Unknown option: $arg (try --help)" >&2
exit 1
;;
esac
done
# Detect rootful vs rootless execution
if [[ $EUID -eq 0 ]]; then
QUADLET_DIR="/etc/containers/systemd"
SYSTEMCTL="systemctl"
echo "==> Deploying in system-wide mode (${QUADLET_DIR})"
else
QUADLET_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/containers/systemd"
SYSTEMCTL="systemctl --user"
echo "==> Deploying in rootless mode (${QUADLET_DIR})"
fi
mkdir -p "${QUADLET_DIR}"
echo "==> 1. Copying base Drop Quadlet units..."
cp -v "${SCRIPT_DIR}/base/"*.network "${QUADLET_DIR}/"
cp -v "${SCRIPT_DIR}/base/"*.volume "${QUADLET_DIR}/"
cp -v "${SCRIPT_DIR}/base/"*.container "${QUADLET_DIR}/"
echo "==> 2. Initializing environment configuration files..."
if [[ ! -f "${QUADLET_DIR}/drop.env" ]]; then
echo " Creating ${QUADLET_DIR}/drop.env from example template..."
cp "${SCRIPT_DIR}/env/drop.env.example" "${QUADLET_DIR}/drop.env"
fi
echo "==> 3. Reloading systemd daemon to generate service units..."
${SYSTEMCTL} daemon-reload
echo "==> 4. Starting services..."
${SYSTEMCTL} restart drop-network-network.service
${SYSTEMCTL} restart drop-postgres.service
${SYSTEMCTL} restart drop.service
echo ""
echo "==============================================================="
echo " Drop Quadlet deployment completed successfully!"
echo " Drop Web App: http://localhost:3000"
if [[ $EUID -ne 0 ]]; then
echo " Rootless note: run 'loginctl enable-linger ${USER}' to start Drop"
echo " on boot without an active login session."
fi
echo "==============================================================="
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env bash
set -euo pipefail
REMOVE_VOLUMES=false
for arg in "$@"; do
case "$arg" in
--purge-data)
REMOVE_VOLUMES=true
shift
;;
--help|-h)
echo "Usage: $0 [--purge-data]"
echo " --purge-data Remove all persistent Podman named volumes and database storage"
exit 0
;;
*)
echo "Unknown option: $arg (try --help)" >&2
exit 1
;;
esac
done
if [[ $EUID -eq 0 ]]; then
QUADLET_DIR="/etc/containers/systemd"
SYSTEMCTL="systemctl"
else
QUADLET_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/containers/systemd"
SYSTEMCTL="systemctl --user"
fi
echo "==> 1. Stopping Drop Quadlet services..."
${SYSTEMCTL} stop drop.service drop-postgres.service drop-network-network.service 2>/dev/null || true
${SYSTEMCTL} reset-failed drop*.service 2>/dev/null || true
echo "==> 2. Removing Quadlet unit files from ${QUADLET_DIR}..."
rm -f "${QUADLET_DIR}/drop-network.network" \
"${QUADLET_DIR}/drop-postgres.container" \
"${QUADLET_DIR}/drop.container" \
"${QUADLET_DIR}"/drop-*.volume
echo "==> 3. Reloading systemd daemon..."
${SYSTEMCTL} daemon-reload
if [[ "$REMOVE_VOLUMES" = true ]]; then
echo "==> 4. Purging Podman named volumes..."
podman volume rm -f systemd-drop-db systemd-drop-data 2>/dev/null || true
podman volume rm -f drop-db drop-data 2>/dev/null || true
rm -f "${QUADLET_DIR}/drop.env"
fi
echo ""
echo "==============================================================="
echo " Drop Quadlet deployment uninstalled."
if [[ "$REMOVE_VOLUMES" = false ]]; then
echo " Persistent volumes and configuration files were preserved in ${QUADLET_DIR}."
echo " To delete all data volumes, re-run with: $0 --purge-data"
else
echo " All persistent volumes and configuration files have been purged."
fi
echo "==============================================================="
@@ -0,0 +1,46 @@
-- Repair migration for the age-rating schema drift.
--
-- The "AgeRatingOrganization" enum and "GameAgeRating" table were originally
-- added by editing the already-applied
-- "20260224145112_add_non_null_default_to_carousel_object_ids" migration.
-- Databases that applied that migration before the age-rating change never
-- created those objects, which makes "20260726041153_add_user_groups" fail
-- with `type "AgeRatingOrganization" does not exist`.
--
-- This migration is intentionally named to sort *before*
-- "20260726041153_add_user_groups" so that prisma migrate deploy creates the
-- enum ahead of that migration, while remaining a no-op on databases that
-- already have these objects.
--
-- Note: databases that already recorded "20260726041153_add_user_groups" as
-- failed must first mark it rolled back:
-- pnpm prisma migrate resolve --rolled-back 20260726041153_add_user_groups
-- CreateEnum (guarded; databases that ran the edited carousel migration
-- already have this type)
DO $$ BEGIN
CREATE TYPE "AgeRatingOrganization" AS ENUM ('ESRB', 'PEGI', 'CERO', 'USK', 'GRAC', 'ClassInd', 'ACB');
EXCEPTION
WHEN duplicate_object THEN null;
END $$;
-- CreateTable
CREATE TABLE IF NOT EXISTS "GameAgeRating" (
"id" TEXT NOT NULL,
"organization" "AgeRatingOrganization" NOT NULL,
"rating" TEXT NOT NULL,
"ratingCoverUrl" TEXT,
"gameId" TEXT NOT NULL,
CONSTRAINT "GameAgeRating_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE UNIQUE INDEX IF NOT EXISTS "GameAgeRating_gameId_organization_key" ON "GameAgeRating"("gameId", "organization");
-- AddForeignKey
DO $$ BEGIN
ALTER TABLE "GameAgeRating" ADD CONSTRAINT "GameAgeRating_gameId_fkey" FOREIGN KEY ("gameId") REFERENCES "Game"("id") ON DELETE CASCADE ON UPDATE CASCADE;
EXCEPTION
WHEN duplicate_object THEN null;
END $$;
+16 -11
View File
@@ -10,13 +10,22 @@ export default defineEventHandler(async (h3) => {
const userId = await aclManager.getUserIdACL(h3, ["object:read"]);
const id = sanitize(unsafeId);
const object = await objectHandler.fetchWithPermissions(id, userId);
if (!object)
const permission = await objectHandler.checkPermission(id, userId);
if (!permission)
throw createError({ statusCode: 404, statusMessage: "Object not found" });
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/ETag
const etagRequestValue = h3.headers.get("If-None-Match");
const etagActualValue = await objectHandler.fetchHash(id);
setHeader(h3, "ETag", etagActualValue ?? "");
setHeader(h3, "Content-Type", permission.mime);
setHeader(
h3,
"Cache-Control",
"private, max-age=31536000, s-maxage=31536000, immutable",
);
if (
etagRequestValue &&
etagActualValue &&
@@ -27,13 +36,9 @@ export default defineEventHandler(async (h3) => {
return null;
}
// TODO: fix undefined etagValue
setHeader(h3, "ETag", etagActualValue ?? "");
setHeader(h3, "Content-Type", object.mime);
setHeader(
h3,
"Cache-Control",
"private, max-age=31536000, s-maxage=31536000, immutable",
);
return object.data;
const source = await objectHandler.fetch(id);
if (!source)
throw createError({ statusCode: 404, statusMessage: "Object not found" });
return source;
});
@@ -11,13 +11,17 @@ export default defineEventHandler(async (h3) => {
const userId = await aclManager.getUserIdACL(h3, ["object:read"]);
const id = sanitize(unsafeId);
const object = await objectHandler.fetchWithPermissions(id, userId);
if (!object)
const permission = await objectHandler.checkPermission(id, userId);
if (!permission)
throw createError({ statusCode: 404, statusMessage: "Object not found" });
setHeader(h3, "Content-Type", permission.mime);
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/ETag
const etagRequestValue = h3.headers.get("If-None-Match");
const etagActualValue = await objectHandler.fetchHash(id);
setHeader(h3, "ETag", etagActualValue ?? "");
if (etagRequestValue !== null && etagActualValue === etagRequestValue) {
// would compare if etag is valid, but objects should never change
setResponseStatus(h3, 304);
+71 -58
View File
@@ -1,10 +1,10 @@
import type { ObjectMetadata, ObjectReference, Source } from "./objectHandler";
import { ObjectBackend, objectMetadata } from "./objectHandler";
import fs from "fs";
import path from "path";
import { Readable } from "stream";
import { createHash } from "crypto";
import fs from "node:fs";
import path from "node:path";
import Stream, { Readable } from "node:stream";
import { createHash } from "node:crypto";
import prisma from "../db/database";
import cacheHandler from "../cache";
import { systemConfig } from "../config/sys-conf";
@@ -32,37 +32,71 @@ export class FsObjectBackend extends ObjectBackend {
async fetch(id: ObjectReference) {
const objectPath = path.join(this.baseObjectPath, id);
if (!fs.existsSync(objectPath)) return undefined;
return fs.createReadStream(objectPath);
let handle: fs.promises.FileHandle;
try {
handle = await fs.promises.open(objectPath, "r");
} catch {
return undefined;
}
try {
const stat = await handle.stat();
if (!stat.isFile()) {
await handle.close();
return undefined;
}
} catch {
await handle.close();
return undefined;
}
// createReadStream on the handle keeps reads tied to the opened inode and closes the handle on completion.
return handle.createReadStream({ autoClose: true });
}
async write(id: ObjectReference, source: Source): Promise<boolean> {
const objectPath = path.join(this.baseObjectPath, id);
if (!fs.existsSync(objectPath)) return false;
let handle: fs.promises.FileHandle;
try {
handle = await fs.promises.open(objectPath, "r+");
} catch {
return false;
}
// remove item from cache
await this.hashStore.delete(id);
if (source instanceof Readable) {
const outputStream = fs.createWriteStream(objectPath);
source.pipe(outputStream, { end: true });
await new Promise((r, _j) => source.on("end", r));
return true;
}
try {
if (source instanceof Readable) {
// Truncate first so overwriting a longer object doesn't leave stale bytes
await handle.truncate(0);
const outputStream = handle.createWriteStream({ autoClose: true });
await Stream.promises.pipeline(source, outputStream);
return true;
}
if (source instanceof Buffer) {
fs.writeFileSync(objectPath, source);
return true;
}
if (source instanceof Buffer) {
await handle.truncate(0);
await handle.writeFile(source);
return true;
}
return false;
return false;
} finally {
await handle.close().catch(() => {});
}
}
async startWriteStream(id: ObjectReference) {
const objectPath = path.join(this.baseObjectPath, id);
if (!fs.existsSync(objectPath)) return undefined;
// remove item from cache
await this.hashStore.delete(id);
return fs.createWriteStream(objectPath);
try {
const handle = await fs.promises.open(objectPath, "r+");
// remove item from cache
await this.hashStore.delete(id);
return handle.createWriteStream({ autoClose: true });
} catch {
return undefined;
}
}
async create(
id: string,
source: Source,
@@ -80,10 +114,11 @@ export class FsObjectBackend extends ObjectBackend {
fs.writeFileSync(objectPath, "");
// Call write
this.write(id, source);
await this.write(id, source);
return id;
}
async createWithWriteStream(id: string, metadata: ObjectMetadata) {
const objectPath = path.join(this.baseObjectPath, id);
const metadataPath = path.join(this.baseMetadataPath, `${id}.json`);
@@ -100,6 +135,7 @@ export class FsObjectBackend extends ObjectBackend {
if (!stream) throw new Error("Could not create write stream");
return stream;
}
async delete(id: ObjectReference): Promise<boolean> {
const objectPath = path.join(this.baseObjectPath, id);
if (!fs.existsSync(objectPath)) return true;
@@ -112,6 +148,7 @@ export class FsObjectBackend extends ObjectBackend {
await this.hashStore.delete(id);
return true;
}
async fetchMetadata(
id: ObjectReference,
): Promise<ObjectMetadata | undefined> {
@@ -132,6 +169,7 @@ export class FsObjectBackend extends ObjectBackend {
await this.metadataCache.set(id, metadata);
return metadata;
}
async writeMetadata(
id: ObjectReference,
metadata: ObjectMetadata,
@@ -142,38 +180,26 @@ export class FsObjectBackend extends ObjectBackend {
await this.metadataCache.set(id, metadata);
return true;
}
async fetchHash(id: ObjectReference): Promise<string | undefined> {
const cacheResult = await this.hashStore.get(id);
if (cacheResult !== null) return cacheResult;
// FsHashStore#get returns undefined on a database miss, not null
if (cacheResult) return cacheResult;
const obj = await this.fetch(id);
if (obj === undefined) return;
// hash object
// Upstream Drop uses md5 for ETag hashing
const hash = createHash("md5");
hash.setEncoding("hex");
// local variable to point to object
const store = this.hashStore;
let hashResult = "";
const objEnd = new Promise<void>((r) => {
obj.on("end", async function () {
hash.end();
hashResult = hash.read();
r();
});
});
// read obj into hash
obj.pipe(hash);
await objEnd;
// if hash isn't a string somehow, mark as unknown hash
if (typeof hashResult !== "string") {
try {
await Stream.promises.pipeline(obj, hash);
const hashResult = hash.digest("hex");
await this.hashStore.save(id, hashResult);
return hashResult;
} catch {
return undefined;
}
await store.save(id, hashResult);
return typeof hashResult;
}
async listAll(): Promise<string[]> {
@@ -212,11 +238,6 @@ export class FsObjectBackend extends ObjectBackend {
class FsHashStore {
private cache = cacheHandler.createCache<string>("ObjectHashStore");
/**
* Gets hash of object
* @param id
* @returns
*/
async get(id: ObjectReference) {
const cacheRes = await this.cache.get(id);
if (cacheRes !== null) {
@@ -236,10 +257,6 @@ class FsHashStore {
return objectHash.hash;
}
/**
* Saves hash of object
* @param id
*/
async save(id: ObjectReference, hash: string) {
await prisma.objectHash.upsert({
where: {
@@ -256,10 +273,6 @@ class FsHashStore {
await this.cache.set(id, hash);
}
/**
* Hash is no longer valid for whatever reason
* @param id
*/
async delete(id: ObjectReference) {
await this.cache.remove(id);
await prisma.objectHash.deleteMany({
@@ -155,6 +155,33 @@ export class ObjectHandler {
);
}
/**
* Checks if user has perms to access the object without opening the file stream
* @param id object id
* @param userId user to check, or act as anon user
* @returns metadata summary if permitted, undefined otherwise
*/
async checkPermission(
id: string,
userId?: string,
): Promise<{ mime: string } | undefined> {
const metadata = await this.backend.fetchMetadata(id);
if (!metadata) return undefined;
if (!this.hasAnyPermissions(metadata.permissions, userId)) return undefined;
return { mime: metadata.mime };
}
/**
* Fetches raw object source from backend
* @param id object id
* @returns source
*/
async fetch(id: string) {
return await this.backend.fetch(id);
}
/**
* Fetches object, but also checks if user has perms to access it
* @param id object id
+51 -13
View File
@@ -5,14 +5,29 @@ import objectHandler from "../objects";
import { randomUUID, createHash } from "node:crypto";
import type { IncomingMessage } from "node:http";
/** Raised by the measuring stream when a save exceeds `saveSlotSizeLimit`. */
class SaveSizeLimitError extends Error {
constructor() {
super("save exceeds saveSlotSizeLimit");
this.name = "SaveSizeLimitError";
}
}
class SaveManager {
async deleteObjectFromSave(
gameId: string,
userId: string,
index: number,
objectId: string,
) {
await objectHandler.deleteWithPermission(objectId, userId);
): Promise<boolean> {
void gameId;
void userId;
void index;
// Save objects are system-tracked (`${userId}:read`) and the id is taken
// from the caller's own slot history, so reclaim them as the system.
// `deleteWithPermission` would require a `delete` grant the object is
// deliberately never given, silently leaking every pruned snapshot.
return await objectHandler.deleteAsSystem(objectId);
}
async pushSave(
@@ -38,7 +53,9 @@ class SaveManager {
const newSaveStream = await objectHandler.createWithStream(
newSaveObjectId,
{ saveSlot: JSON.stringify({ userId, gameId, index }) },
[],
// System-tracked object: grant the owner read access (same convention as
// screenshots) so the desktop client can pull the archive back down.
[`${userId}:read`],
);
if (!newSaveStream)
throw createError({
@@ -46,19 +63,40 @@ class SaveManager {
statusMessage: "Failed to create writing stream to storage backend.",
});
let hash: string | undefined;
const hashPromise = Stream.promises.pipeline(
stream,
createHash("sha256").setEncoding("hex"),
async function (source) {
// @ts-expect-error Not sure how to get this to be typed
hash = (await source.toArray())[0];
const sizeLimitMb = await applicationSettings.get("saveSlotSizeLimit");
const sizeLimitBytes = sizeLimitMb * 1024 * 1024;
const hashStream = createHash("sha256");
let totalBytes = 0;
// Measure while streaming so an oversized payload is rejected before the
// whole archive reaches disk, rather than after an unbounded write.
const measuringStream = new Stream.Transform({
transform(chunk: Buffer, _encoding, callback) {
totalBytes += chunk.length;
if (totalBytes > sizeLimitBytes) {
callback(new SaveSizeLimitError());
return;
}
hashStream.update(chunk);
callback(null, chunk);
},
);
});
const uploadStream = Stream.promises.pipeline(stream, newSaveStream);
try {
await Stream.promises.pipeline(stream, measuringStream, newSaveStream);
} catch (error) {
await objectHandler.deleteAsSystem(newSaveObjectId);
if (error instanceof SaveSizeLimitError) {
throw createError({
statusCode: 413,
statusMessage: "Save exceeds saveSlotSizeLimit",
});
}
throw error;
}
await Promise.all([hashPromise, uploadStream]);
const hash = hashStream.digest("hex");
if (!hash) {
await objectHandler.deleteAsSystem(newSaveObjectId);