mirror of
https://github.com/Drop-OSS/drop.git
synced 2026-10-02 17:54:09 +10:00
fix(platform): repair age-rating migration drift, resolve object storage stream leaks, and add Quadlet template (#497)
* fix(server): repair age-rating migration drift
The age-ratings change (97c6f2c8) appended the AgeRatingOrganization
enum and GameAgeRating table to 20260224145112, a migration that had
already been applied to existing databases. Prisma never re-runs applied
migrations, so those databases were left without the enum/table; the
later 20260726041153_add_user_groups migration then failed with
"type AgeRatingOrganization does not exist" and game pages 500'd.
Restore 20260224145112 to its original applied content, make
20260726041153 self-healing by creating the enum when it is missing, and
add an idempotent 20260915000000_add_game_age_ratings migration that
creates the type, table, index and foreign key. All statements are
guarded so they are safe on databases already repaired out-of-band.
* fix(server): eliminate file descriptor double-close and stream leaks in object storage
* feat(server): add generic Podman Quadlet deployment template
* fix(server): address platform-stability review feedback
Migrations are now append-only: 20260224145112 and 20260726041153 are left untouched and 20260915000000_add_game_age_ratings is replaced by 20260726041152_repair_age_rating_schema, named to sort before 20260726041153 so drifted databases create the enum before add_user_groups runs. Databases whose add_user_groups row already failed still need migrate resolve --rolled-back, which is documented in the migration.
FsObjectBackend#write now truncates before writing, so overwriting an object with shorter content cannot leave stale trailing bytes. FsObjectBackend#fetchHash computes and stores the hash on a database miss (FsHashStore#get returns undefined, not null).
Quadlet: drop the README and the chunk-cache volume/env keys (the chunk cache is not implemented in this repository), fix rootless boot via default.target, add a PostgreSQL healthcheck, use the drop-postgres DNS name, and fix the uninstall volume glob.
* fix(server): grant save owners read access and enforce save size limits
Save snapshots were created with no permissions, so the desktop client
could never pull an archive back, and pruning relied on deleteWithPermission
(which requires a delete grant the object never has), silently leaking every
pruned snapshot. Grant `<userId>:read` and reclaim as the system.
The advertised saveSlotSizeLimit was also never enforced; measure the stream
and reject oversized payloads with a 413 before the full archive is written.
---------
Co-authored-by: John Smith <you@example.com>
This commit is contained in:
co-authored by
John Smith
parent
344b89c5b9
commit
6c3671714d
@@ -31,6 +31,8 @@ logs
|
||||
deploy-template/*
|
||||
|
||||
!deploy-template/compose.yml
|
||||
!deploy-template/quadlet/
|
||||
!deploy-template/quadlet/**
|
||||
|
||||
# generated prisma client
|
||||
/prisma/client
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
[Volume]
|
||||
VolumeName=drop-data
|
||||
@@ -0,0 +1,2 @@
|
||||
[Volume]
|
||||
VolumeName=drop-db
|
||||
@@ -0,0 +1,6 @@
|
||||
[Network]
|
||||
NetworkName=drop-network
|
||||
Driver=bridge
|
||||
Subnet=172.20.0.0/16
|
||||
Subnet=fd00:172:20::/64
|
||||
IPv6=true
|
||||
@@ -0,0 +1,27 @@
|
||||
[Unit]
|
||||
Description=Drop PostgreSQL Database
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=drop-postgres
|
||||
Image=docker.io/postgres:14-alpine
|
||||
AutoUpdate=registry
|
||||
Network=drop-network.network
|
||||
IP=172.20.0.21
|
||||
Volume=drop-db.volume:/var/lib/postgresql/data
|
||||
EnvironmentFile=%E/containers/systemd/drop.env
|
||||
Environment=POSTGRES_DB=drop
|
||||
Environment=HOSTNAME=drop-postgres
|
||||
SecurityLabelDisable=true
|
||||
HealthCmd=pg_isready -d drop -U drop
|
||||
HealthInterval=30s
|
||||
HealthTimeout=60s
|
||||
HealthRetries=5
|
||||
HealthStartPeriod=10s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target default.target
|
||||
@@ -0,0 +1,27 @@
|
||||
[Unit]
|
||||
Description=Drop Game Distribution Platform
|
||||
Wants=network-online.target
|
||||
After=network-online.target drop-postgres.service
|
||||
Requires=drop-postgres.service
|
||||
|
||||
[Container]
|
||||
ContainerName=drop
|
||||
Image=ghcr.io/drop-oss/drop:nightly
|
||||
AutoUpdate=registry
|
||||
Network=drop-network.network
|
||||
IP=172.20.0.20
|
||||
PublishPort=3000:3000
|
||||
Volume=drop-data.volume:/data
|
||||
# Game storage library mount (uncomment and point to your library path)
|
||||
# Volume=/mnt/storage/games:/library:ro
|
||||
EnvironmentFile=%E/containers/systemd/drop.env
|
||||
Environment=HOSTNAME=drop
|
||||
SecurityLabelDisable=true
|
||||
|
||||
[Service]
|
||||
# PostgreSQL may still be initializing when this unit starts; Restart=always
|
||||
# recovers until the database accepts connections.
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target default.target
|
||||
@@ -0,0 +1,9 @@
|
||||
# Drop Server & Database Configuration
|
||||
POSTGRES_USER=drop
|
||||
POSTGRES_PASSWORD=drop
|
||||
DATABASE_URL=postgres://drop:drop@drop-postgres:5432/drop
|
||||
EXTERNAL_URL=http://localhost:3000/
|
||||
|
||||
# Optional: IGDB Metadata Provider
|
||||
# IGDB_CLIENT_ID=
|
||||
# IGDB_CLIENT_SECRET=
|
||||
Executable
+60
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--help|-h)
|
||||
echo "Usage: $0"
|
||||
echo " Installs the base Drop Quadlet stack (server + PostgreSQL)."
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $arg (try --help)" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Detect rootful vs rootless execution
|
||||
if [[ $EUID -eq 0 ]]; then
|
||||
QUADLET_DIR="/etc/containers/systemd"
|
||||
SYSTEMCTL="systemctl"
|
||||
echo "==> Deploying in system-wide mode (${QUADLET_DIR})"
|
||||
else
|
||||
QUADLET_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/containers/systemd"
|
||||
SYSTEMCTL="systemctl --user"
|
||||
echo "==> Deploying in rootless mode (${QUADLET_DIR})"
|
||||
fi
|
||||
|
||||
mkdir -p "${QUADLET_DIR}"
|
||||
|
||||
echo "==> 1. Copying base Drop Quadlet units..."
|
||||
cp -v "${SCRIPT_DIR}/base/"*.network "${QUADLET_DIR}/"
|
||||
cp -v "${SCRIPT_DIR}/base/"*.volume "${QUADLET_DIR}/"
|
||||
cp -v "${SCRIPT_DIR}/base/"*.container "${QUADLET_DIR}/"
|
||||
|
||||
echo "==> 2. Initializing environment configuration files..."
|
||||
if [[ ! -f "${QUADLET_DIR}/drop.env" ]]; then
|
||||
echo " Creating ${QUADLET_DIR}/drop.env from example template..."
|
||||
cp "${SCRIPT_DIR}/env/drop.env.example" "${QUADLET_DIR}/drop.env"
|
||||
fi
|
||||
|
||||
echo "==> 3. Reloading systemd daemon to generate service units..."
|
||||
${SYSTEMCTL} daemon-reload
|
||||
|
||||
echo "==> 4. Starting services..."
|
||||
${SYSTEMCTL} restart drop-network-network.service
|
||||
${SYSTEMCTL} restart drop-postgres.service
|
||||
${SYSTEMCTL} restart drop.service
|
||||
|
||||
echo ""
|
||||
echo "==============================================================="
|
||||
echo " Drop Quadlet deployment completed successfully!"
|
||||
echo " Drop Web App: http://localhost:3000"
|
||||
if [[ $EUID -ne 0 ]]; then
|
||||
echo " Rootless note: run 'loginctl enable-linger ${USER}' to start Drop"
|
||||
echo " on boot without an active login session."
|
||||
fi
|
||||
echo "==============================================================="
|
||||
Executable
+61
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
REMOVE_VOLUMES=false
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--purge-data)
|
||||
REMOVE_VOLUMES=true
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
echo "Usage: $0 [--purge-data]"
|
||||
echo " --purge-data Remove all persistent Podman named volumes and database storage"
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $arg (try --help)" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ $EUID -eq 0 ]]; then
|
||||
QUADLET_DIR="/etc/containers/systemd"
|
||||
SYSTEMCTL="systemctl"
|
||||
else
|
||||
QUADLET_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/containers/systemd"
|
||||
SYSTEMCTL="systemctl --user"
|
||||
fi
|
||||
|
||||
echo "==> 1. Stopping Drop Quadlet services..."
|
||||
${SYSTEMCTL} stop drop.service drop-postgres.service drop-network-network.service 2>/dev/null || true
|
||||
${SYSTEMCTL} reset-failed drop*.service 2>/dev/null || true
|
||||
|
||||
echo "==> 2. Removing Quadlet unit files from ${QUADLET_DIR}..."
|
||||
rm -f "${QUADLET_DIR}/drop-network.network" \
|
||||
"${QUADLET_DIR}/drop-postgres.container" \
|
||||
"${QUADLET_DIR}/drop.container" \
|
||||
"${QUADLET_DIR}"/drop-*.volume
|
||||
|
||||
echo "==> 3. Reloading systemd daemon..."
|
||||
${SYSTEMCTL} daemon-reload
|
||||
|
||||
if [[ "$REMOVE_VOLUMES" = true ]]; then
|
||||
echo "==> 4. Purging Podman named volumes..."
|
||||
podman volume rm -f systemd-drop-db systemd-drop-data 2>/dev/null || true
|
||||
podman volume rm -f drop-db drop-data 2>/dev/null || true
|
||||
rm -f "${QUADLET_DIR}/drop.env"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "==============================================================="
|
||||
echo " Drop Quadlet deployment uninstalled."
|
||||
if [[ "$REMOVE_VOLUMES" = false ]]; then
|
||||
echo " Persistent volumes and configuration files were preserved in ${QUADLET_DIR}."
|
||||
echo " To delete all data volumes, re-run with: $0 --purge-data"
|
||||
else
|
||||
echo " All persistent volumes and configuration files have been purged."
|
||||
fi
|
||||
echo "==============================================================="
|
||||
@@ -0,0 +1,46 @@
|
||||
-- Repair migration for the age-rating schema drift.
|
||||
--
|
||||
-- The "AgeRatingOrganization" enum and "GameAgeRating" table were originally
|
||||
-- added by editing the already-applied
|
||||
-- "20260224145112_add_non_null_default_to_carousel_object_ids" migration.
|
||||
-- Databases that applied that migration before the age-rating change never
|
||||
-- created those objects, which makes "20260726041153_add_user_groups" fail
|
||||
-- with `type "AgeRatingOrganization" does not exist`.
|
||||
--
|
||||
-- This migration is intentionally named to sort *before*
|
||||
-- "20260726041153_add_user_groups" so that prisma migrate deploy creates the
|
||||
-- enum ahead of that migration, while remaining a no-op on databases that
|
||||
-- already have these objects.
|
||||
--
|
||||
-- Note: databases that already recorded "20260726041153_add_user_groups" as
|
||||
-- failed must first mark it rolled back:
|
||||
-- pnpm prisma migrate resolve --rolled-back 20260726041153_add_user_groups
|
||||
|
||||
-- CreateEnum (guarded; databases that ran the edited carousel migration
|
||||
-- already have this type)
|
||||
DO $$ BEGIN
|
||||
CREATE TYPE "AgeRatingOrganization" AS ENUM ('ESRB', 'PEGI', 'CERO', 'USK', 'GRAC', 'ClassInd', 'ACB');
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE IF NOT EXISTS "GameAgeRating" (
|
||||
"id" TEXT NOT NULL,
|
||||
"organization" "AgeRatingOrganization" NOT NULL,
|
||||
"rating" TEXT NOT NULL,
|
||||
"ratingCoverUrl" TEXT,
|
||||
"gameId" TEXT NOT NULL,
|
||||
|
||||
CONSTRAINT "GameAgeRating_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "GameAgeRating_gameId_organization_key" ON "GameAgeRating"("gameId", "organization");
|
||||
|
||||
-- AddForeignKey
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "GameAgeRating" ADD CONSTRAINT "GameAgeRating_gameId_fkey" FOREIGN KEY ("gameId") REFERENCES "Game"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
@@ -10,13 +10,22 @@ export default defineEventHandler(async (h3) => {
|
||||
const userId = await aclManager.getUserIdACL(h3, ["object:read"]);
|
||||
|
||||
const id = sanitize(unsafeId);
|
||||
const object = await objectHandler.fetchWithPermissions(id, userId);
|
||||
if (!object)
|
||||
const permission = await objectHandler.checkPermission(id, userId);
|
||||
if (!permission)
|
||||
throw createError({ statusCode: 404, statusMessage: "Object not found" });
|
||||
|
||||
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/ETag
|
||||
const etagRequestValue = h3.headers.get("If-None-Match");
|
||||
const etagActualValue = await objectHandler.fetchHash(id);
|
||||
|
||||
setHeader(h3, "ETag", etagActualValue ?? "");
|
||||
setHeader(h3, "Content-Type", permission.mime);
|
||||
setHeader(
|
||||
h3,
|
||||
"Cache-Control",
|
||||
"private, max-age=31536000, s-maxage=31536000, immutable",
|
||||
);
|
||||
|
||||
if (
|
||||
etagRequestValue &&
|
||||
etagActualValue &&
|
||||
@@ -27,13 +36,9 @@ export default defineEventHandler(async (h3) => {
|
||||
return null;
|
||||
}
|
||||
|
||||
// TODO: fix undefined etagValue
|
||||
setHeader(h3, "ETag", etagActualValue ?? "");
|
||||
setHeader(h3, "Content-Type", object.mime);
|
||||
setHeader(
|
||||
h3,
|
||||
"Cache-Control",
|
||||
"private, max-age=31536000, s-maxage=31536000, immutable",
|
||||
);
|
||||
return object.data;
|
||||
const source = await objectHandler.fetch(id);
|
||||
if (!source)
|
||||
throw createError({ statusCode: 404, statusMessage: "Object not found" });
|
||||
|
||||
return source;
|
||||
});
|
||||
|
||||
@@ -11,13 +11,17 @@ export default defineEventHandler(async (h3) => {
|
||||
const userId = await aclManager.getUserIdACL(h3, ["object:read"]);
|
||||
|
||||
const id = sanitize(unsafeId);
|
||||
const object = await objectHandler.fetchWithPermissions(id, userId);
|
||||
if (!object)
|
||||
const permission = await objectHandler.checkPermission(id, userId);
|
||||
if (!permission)
|
||||
throw createError({ statusCode: 404, statusMessage: "Object not found" });
|
||||
|
||||
setHeader(h3, "Content-Type", permission.mime);
|
||||
|
||||
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/ETag
|
||||
const etagRequestValue = h3.headers.get("If-None-Match");
|
||||
const etagActualValue = await objectHandler.fetchHash(id);
|
||||
setHeader(h3, "ETag", etagActualValue ?? "");
|
||||
|
||||
if (etagRequestValue !== null && etagActualValue === etagRequestValue) {
|
||||
// would compare if etag is valid, but objects should never change
|
||||
setResponseStatus(h3, 304);
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import type { ObjectMetadata, ObjectReference, Source } from "./objectHandler";
|
||||
import { ObjectBackend, objectMetadata } from "./objectHandler";
|
||||
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { Readable } from "stream";
|
||||
import { createHash } from "crypto";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import Stream, { Readable } from "node:stream";
|
||||
import { createHash } from "node:crypto";
|
||||
import prisma from "../db/database";
|
||||
import cacheHandler from "../cache";
|
||||
import { systemConfig } from "../config/sys-conf";
|
||||
@@ -32,37 +32,71 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
|
||||
async fetch(id: ObjectReference) {
|
||||
const objectPath = path.join(this.baseObjectPath, id);
|
||||
if (!fs.existsSync(objectPath)) return undefined;
|
||||
return fs.createReadStream(objectPath);
|
||||
let handle: fs.promises.FileHandle;
|
||||
try {
|
||||
handle = await fs.promises.open(objectPath, "r");
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
try {
|
||||
const stat = await handle.stat();
|
||||
if (!stat.isFile()) {
|
||||
await handle.close();
|
||||
return undefined;
|
||||
}
|
||||
} catch {
|
||||
await handle.close();
|
||||
return undefined;
|
||||
}
|
||||
// createReadStream on the handle keeps reads tied to the opened inode and closes the handle on completion.
|
||||
return handle.createReadStream({ autoClose: true });
|
||||
}
|
||||
|
||||
async write(id: ObjectReference, source: Source): Promise<boolean> {
|
||||
const objectPath = path.join(this.baseObjectPath, id);
|
||||
if (!fs.existsSync(objectPath)) return false;
|
||||
let handle: fs.promises.FileHandle;
|
||||
try {
|
||||
handle = await fs.promises.open(objectPath, "r+");
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
|
||||
// remove item from cache
|
||||
await this.hashStore.delete(id);
|
||||
|
||||
if (source instanceof Readable) {
|
||||
const outputStream = fs.createWriteStream(objectPath);
|
||||
source.pipe(outputStream, { end: true });
|
||||
await new Promise((r, _j) => source.on("end", r));
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
if (source instanceof Readable) {
|
||||
// Truncate first so overwriting a longer object doesn't leave stale bytes
|
||||
await handle.truncate(0);
|
||||
const outputStream = handle.createWriteStream({ autoClose: true });
|
||||
await Stream.promises.pipeline(source, outputStream);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (source instanceof Buffer) {
|
||||
fs.writeFileSync(objectPath, source);
|
||||
return true;
|
||||
}
|
||||
if (source instanceof Buffer) {
|
||||
await handle.truncate(0);
|
||||
await handle.writeFile(source);
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
return false;
|
||||
} finally {
|
||||
await handle.close().catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
async startWriteStream(id: ObjectReference) {
|
||||
const objectPath = path.join(this.baseObjectPath, id);
|
||||
if (!fs.existsSync(objectPath)) return undefined;
|
||||
// remove item from cache
|
||||
await this.hashStore.delete(id);
|
||||
return fs.createWriteStream(objectPath);
|
||||
try {
|
||||
const handle = await fs.promises.open(objectPath, "r+");
|
||||
// remove item from cache
|
||||
await this.hashStore.delete(id);
|
||||
return handle.createWriteStream({ autoClose: true });
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
async create(
|
||||
id: string,
|
||||
source: Source,
|
||||
@@ -80,10 +114,11 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
fs.writeFileSync(objectPath, "");
|
||||
|
||||
// Call write
|
||||
this.write(id, source);
|
||||
await this.write(id, source);
|
||||
|
||||
return id;
|
||||
}
|
||||
|
||||
async createWithWriteStream(id: string, metadata: ObjectMetadata) {
|
||||
const objectPath = path.join(this.baseObjectPath, id);
|
||||
const metadataPath = path.join(this.baseMetadataPath, `${id}.json`);
|
||||
@@ -100,6 +135,7 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
if (!stream) throw new Error("Could not create write stream");
|
||||
return stream;
|
||||
}
|
||||
|
||||
async delete(id: ObjectReference): Promise<boolean> {
|
||||
const objectPath = path.join(this.baseObjectPath, id);
|
||||
if (!fs.existsSync(objectPath)) return true;
|
||||
@@ -112,6 +148,7 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
await this.hashStore.delete(id);
|
||||
return true;
|
||||
}
|
||||
|
||||
async fetchMetadata(
|
||||
id: ObjectReference,
|
||||
): Promise<ObjectMetadata | undefined> {
|
||||
@@ -132,6 +169,7 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
await this.metadataCache.set(id, metadata);
|
||||
return metadata;
|
||||
}
|
||||
|
||||
async writeMetadata(
|
||||
id: ObjectReference,
|
||||
metadata: ObjectMetadata,
|
||||
@@ -142,38 +180,26 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
await this.metadataCache.set(id, metadata);
|
||||
return true;
|
||||
}
|
||||
|
||||
async fetchHash(id: ObjectReference): Promise<string | undefined> {
|
||||
const cacheResult = await this.hashStore.get(id);
|
||||
if (cacheResult !== null) return cacheResult;
|
||||
// FsHashStore#get returns undefined on a database miss, not null
|
||||
if (cacheResult) return cacheResult;
|
||||
|
||||
const obj = await this.fetch(id);
|
||||
if (obj === undefined) return;
|
||||
|
||||
// hash object
|
||||
// Upstream Drop uses md5 for ETag hashing
|
||||
const hash = createHash("md5");
|
||||
hash.setEncoding("hex");
|
||||
|
||||
// local variable to point to object
|
||||
const store = this.hashStore;
|
||||
let hashResult = "";
|
||||
|
||||
const objEnd = new Promise<void>((r) => {
|
||||
obj.on("end", async function () {
|
||||
hash.end();
|
||||
hashResult = hash.read();
|
||||
r();
|
||||
});
|
||||
});
|
||||
// read obj into hash
|
||||
obj.pipe(hash);
|
||||
await objEnd;
|
||||
|
||||
// if hash isn't a string somehow, mark as unknown hash
|
||||
if (typeof hashResult !== "string") {
|
||||
try {
|
||||
await Stream.promises.pipeline(obj, hash);
|
||||
const hashResult = hash.digest("hex");
|
||||
await this.hashStore.save(id, hashResult);
|
||||
return hashResult;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
await store.save(id, hashResult);
|
||||
return typeof hashResult;
|
||||
}
|
||||
|
||||
async listAll(): Promise<string[]> {
|
||||
@@ -212,11 +238,6 @@ export class FsObjectBackend extends ObjectBackend {
|
||||
class FsHashStore {
|
||||
private cache = cacheHandler.createCache<string>("ObjectHashStore");
|
||||
|
||||
/**
|
||||
* Gets hash of object
|
||||
* @param id
|
||||
* @returns
|
||||
*/
|
||||
async get(id: ObjectReference) {
|
||||
const cacheRes = await this.cache.get(id);
|
||||
if (cacheRes !== null) {
|
||||
@@ -236,10 +257,6 @@ class FsHashStore {
|
||||
return objectHash.hash;
|
||||
}
|
||||
|
||||
/**
|
||||
* Saves hash of object
|
||||
* @param id
|
||||
*/
|
||||
async save(id: ObjectReference, hash: string) {
|
||||
await prisma.objectHash.upsert({
|
||||
where: {
|
||||
@@ -256,10 +273,6 @@ class FsHashStore {
|
||||
await this.cache.set(id, hash);
|
||||
}
|
||||
|
||||
/**
|
||||
* Hash is no longer valid for whatever reason
|
||||
* @param id
|
||||
*/
|
||||
async delete(id: ObjectReference) {
|
||||
await this.cache.remove(id);
|
||||
await prisma.objectHash.deleteMany({
|
||||
|
||||
@@ -155,6 +155,33 @@ export class ObjectHandler {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if user has perms to access the object without opening the file stream
|
||||
* @param id object id
|
||||
* @param userId user to check, or act as anon user
|
||||
* @returns metadata summary if permitted, undefined otherwise
|
||||
*/
|
||||
async checkPermission(
|
||||
id: string,
|
||||
userId?: string,
|
||||
): Promise<{ mime: string } | undefined> {
|
||||
const metadata = await this.backend.fetchMetadata(id);
|
||||
if (!metadata) return undefined;
|
||||
|
||||
if (!this.hasAnyPermissions(metadata.permissions, userId)) return undefined;
|
||||
|
||||
return { mime: metadata.mime };
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches raw object source from backend
|
||||
* @param id object id
|
||||
* @returns source
|
||||
*/
|
||||
async fetch(id: string) {
|
||||
return await this.backend.fetch(id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches object, but also checks if user has perms to access it
|
||||
* @param id object id
|
||||
|
||||
@@ -5,14 +5,29 @@ import objectHandler from "../objects";
|
||||
import { randomUUID, createHash } from "node:crypto";
|
||||
import type { IncomingMessage } from "node:http";
|
||||
|
||||
/** Raised by the measuring stream when a save exceeds `saveSlotSizeLimit`. */
|
||||
class SaveSizeLimitError extends Error {
|
||||
constructor() {
|
||||
super("save exceeds saveSlotSizeLimit");
|
||||
this.name = "SaveSizeLimitError";
|
||||
}
|
||||
}
|
||||
|
||||
class SaveManager {
|
||||
async deleteObjectFromSave(
|
||||
gameId: string,
|
||||
userId: string,
|
||||
index: number,
|
||||
objectId: string,
|
||||
) {
|
||||
await objectHandler.deleteWithPermission(objectId, userId);
|
||||
): Promise<boolean> {
|
||||
void gameId;
|
||||
void userId;
|
||||
void index;
|
||||
// Save objects are system-tracked (`${userId}:read`) and the id is taken
|
||||
// from the caller's own slot history, so reclaim them as the system.
|
||||
// `deleteWithPermission` would require a `delete` grant the object is
|
||||
// deliberately never given, silently leaking every pruned snapshot.
|
||||
return await objectHandler.deleteAsSystem(objectId);
|
||||
}
|
||||
|
||||
async pushSave(
|
||||
@@ -38,7 +53,9 @@ class SaveManager {
|
||||
const newSaveStream = await objectHandler.createWithStream(
|
||||
newSaveObjectId,
|
||||
{ saveSlot: JSON.stringify({ userId, gameId, index }) },
|
||||
[],
|
||||
// System-tracked object: grant the owner read access (same convention as
|
||||
// screenshots) so the desktop client can pull the archive back down.
|
||||
[`${userId}:read`],
|
||||
);
|
||||
if (!newSaveStream)
|
||||
throw createError({
|
||||
@@ -46,19 +63,40 @@ class SaveManager {
|
||||
statusMessage: "Failed to create writing stream to storage backend.",
|
||||
});
|
||||
|
||||
let hash: string | undefined;
|
||||
const hashPromise = Stream.promises.pipeline(
|
||||
stream,
|
||||
createHash("sha256").setEncoding("hex"),
|
||||
async function (source) {
|
||||
// @ts-expect-error Not sure how to get this to be typed
|
||||
hash = (await source.toArray())[0];
|
||||
const sizeLimitMb = await applicationSettings.get("saveSlotSizeLimit");
|
||||
const sizeLimitBytes = sizeLimitMb * 1024 * 1024;
|
||||
|
||||
const hashStream = createHash("sha256");
|
||||
let totalBytes = 0;
|
||||
|
||||
// Measure while streaming so an oversized payload is rejected before the
|
||||
// whole archive reaches disk, rather than after an unbounded write.
|
||||
const measuringStream = new Stream.Transform({
|
||||
transform(chunk: Buffer, _encoding, callback) {
|
||||
totalBytes += chunk.length;
|
||||
if (totalBytes > sizeLimitBytes) {
|
||||
callback(new SaveSizeLimitError());
|
||||
return;
|
||||
}
|
||||
hashStream.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
const uploadStream = Stream.promises.pipeline(stream, newSaveStream);
|
||||
try {
|
||||
await Stream.promises.pipeline(stream, measuringStream, newSaveStream);
|
||||
} catch (error) {
|
||||
await objectHandler.deleteAsSystem(newSaveObjectId);
|
||||
if (error instanceof SaveSizeLimitError) {
|
||||
throw createError({
|
||||
statusCode: 413,
|
||||
statusMessage: "Save exceeds saveSlotSizeLimit",
|
||||
});
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
await Promise.all([hashPromise, uploadStream]);
|
||||
const hash = hashStream.digest("hex");
|
||||
|
||||
if (!hash) {
|
||||
await objectHandler.deleteAsSystem(newSaveObjectId);
|
||||
|
||||
Reference in New Issue
Block a user