BlueGit

timfa filemanager

Reduce subtitle buffer stalls

Keep bitmap subtitle caches from thrashing after only a few videos by removing the small default item caps and relying on byte limits instead. This reduces the cold reparse delays that could make later videos start with slightly late subtitle images.

Also smooth HLS prune cleanup by deleting old segment files in bounded batches and expose debug timing counters for subtitle parsing, frame rendering, and prune work so it is easier to see whether stalls come from cache misses or cleanup pressure.

timfa Tim Falken · pushed by timfa

Authored 2026-05-25 · 5 months ago

Hash
f21c3881b10d83f941b33f6454921a912943745d
Parent
37482082
5 files changed +214 −20
.github/technical-reference.md +6 −5
@@ -11,13 +11,13 @@
11 11
12 12 ## server/image-subtitles.js
13 13 - parseRenderOptions(options) - Normalises subtitle-frame request options and validates filePath/streamIndex/time/canvas dimensions; stream index `0` is valid and must not be treated as missing; canvas is scaled down within a 1280x720 ceiling, then multiplied by `IMAGE_SUBTITLE_RENDER_SCALE` (default `0.75`) while preserving requested aspect ratio.
14 - renderImageSubtitleFrame(options) - Extracts bitmap subtitle streams to cached `.sup`, parses PGS display sets, and returns {frame: PNG buffer, left, top, width, height, canvasWidth, canvasHeight} with metadata for proper client-side upscaling; the server preserves positioning info instead of throwing it away, allowing the client to calculate percentage-based sizing for downscaled frames.
14 - renderImageSubtitleFrame(options) - Extracts bitmap subtitle streams to cached `.sup`, parses PGS display sets, and returns {frame: PNG buffer, left, top, width, height, canvasWidth, canvasHeight} with metadata for proper client-side upscaling; the server preserves positioning info instead of throwing it away, allowing the client to calculate percentage-based sizing for downscaled frames; request/build/render timing counters are tracked for debug diagnostics.
15 15 - clearCachesForFile(filePath) - Removes all event and frame cache entries for a given file path using path-prefixed cache keys, also deletes the corresponding `.sup` file from disk for each evicted event entry; called when video stream sessions end to prevent memory and disk accumulation.
16 16 - pruneOldImageSubtitleCaches(filePath, slowestViewerTimeSeconds) - Drops subtitle frame caches for events that have passed their retention threshold of end time + event duration + 5 seconds (e.g., event ending at 30s with 10s duration is dropped after slowest viewer reaches 45s); used during segment pruning to keep memory usage bounded on long videos, especially on resource-limited hardware like Raspberry Pi.
17 17 - purgeSubtitleCacheDir() - Removes the entire subtitle `.sup` file cache directory from disk; called once at server startup to clean up any files left over from a previous run.
18 18 - touchCachedEntry(cache, key, ttlMs) - Refreshes the TTL of an existing cache entry without changing its value; used to keep actively-watched subtitle tracks alive past their initial 5-minute TTL and prevent expensive full re-parses during long playback sessions.
19 - getImageSubtitleDebugSnapshot() - Returns event/frame cache totals and per-video cache usage ({event entries/bytes/count, frame entries/bytes}) so admin diagnostics can list cached subtitle data even when no viewer is connected.
20 - IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS / IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES / IMAGE_SUBTITLE_FRAME_CACHE_MAX_BYTES / IMAGE_SUBTITLE_RENDER_SCALE / IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS - Environment knobs for subtitle cache retention, item caps, byte caps, render resolution, and prune grace window.
19 - getImageSubtitleDebugSnapshot() - Returns event/frame cache totals, timing counters (cache hits/misses, event build timing, frame render/request timing), and per-video cache usage ({event entries/bytes/count, frame entries/bytes}) so admin diagnostics can distinguish cache-thrash or cold-parse stalls from steady-state rendering.
20 - IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS / IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES / IMAGE_SUBTITLE_FRAME_CACHE_MAX_BYTES / IMAGE_SUBTITLE_RENDER_SCALE / IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS - Environment knobs for subtitle cache retention, optional item caps (`0`/`unbounded` disables the cap so byte limits govern), byte caps, render resolution, and prune grace window.
21 21
22 22 ## server/thumbnail-cache.js
23 23 - syncThumbnailCacheOnce() - Mirrors configured roots and userPath into public/thumbs and generates 99x99-max WebP thumbnails for image files, video files (ffprobe/ffmpeg frame at ~10%), and 3D model files (STL/OBJ, isometric SVG projection via Sharp); removes stale thumbnails and updates manifest hashes.
@@ -132,17 +132,18 @@
132 132 - sendVideoStreamAsset(sessionId, assetPath, req, res) - Resolves viewer leases to a shared stream session, serves HLS assets, tracks lease activity, and prunes segments using the slowest active viewer position.
133 133 - releaseVideoStreamSession(sessionId) - Releases one viewer lease; only destroys the underlying transcode session when no active leases remain; cleanup only clears image subtitle frame/event caches once no other active stream session for the same source file remains.
134 134 - parseSegmentIndexFromAssetPath(assetPath) - Extracts the numeric segment index from an HLS segment filename; matches both numeric variant identifiers (stream_0_segment_000000.ts) and named variants (stream_video_segment_000000.ts, stream_Audio-1-jpn_segment_000000.ts) — ffmpeg substitutes the var_stream_map name: value for %v when named streams are used.
135 - runSegmentPrune(session) - Removes HLS segment files that fall behind the slowest active viewer and calls pruneImageSubtitleCachesForSession() using the slowest active viewer across all live sessions for the same source file, so file-wide subtitle buffers are not pruned ahead of a newly started second session; segment filename regex also uses named-variant pattern to match production filenames correctly.
135 - runSegmentPrune(session) - Removes HLS segment files that fall behind the slowest active viewer and calls pruneImageSubtitleCachesForSession() using the slowest active viewer across all live sessions for the same source file, so file-wide subtitle buffers are not pruned ahead of a newly started second session; segment deletion now runs in bounded batches and records last prune duration/deleted-segment counts for diagnostics; segment filename regex also uses named-variant pattern to match production filenames correctly.
136 136 - hasInitialHlsAssets(outputDir) - Returns true once master.m3u8 or at least one variant playlist/segment exists in an HLS output directory; accepts named variant filenames (stream_video.m3u8) in addition to numeric ones.
137 137 - purgeStreamRootDir() - Best-effort recursive delete of the temporary HLS stream root; used at startup and by idle sweeps to remove orphaned stream session directories from previous runs.
138 138 - cleanupIdleStreamArtifacts() - Periodic idle-time cleaner that purges stream-root artifacts when there are no active sessions or leases, preventing stale segment directories from accumulating after failures.
139 139 - getVideoStreamDebugSnapshot() - Returns active stream sessions with viewer counts and output-dir stats (segment counts/bytes), plus orphaned output directories left under the stream root for diagnostics.
140 - getVideoStreamDebugSnapshot() - Includes segment retention and prune telemetry per session (segmentRetentionCount, slowestViewerSegmentIndex, segmentPrunedUntil) to explain growth in segment counts.
140 - getVideoStreamDebugSnapshot() - Includes segment retention and prune telemetry per session (segmentRetentionCount, slowestViewerSegmentIndex, segmentPrunedUntil, lastSegmentPrune, streamPruneDeleteBatchSize) to explain growth or cleanup stalls in segment counts.
141 141 - STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
142 142 - STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
143 143 - STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
144 144 - STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
145 145 - MAX_STREAM_SEGMENT_RETENTION_COUNT - Upper safety cap for retention (360) to avoid runaway in-session segment growth from oversized overrides.
146 - VIDEO_STREAM_PRUNE_DELETE_BATCH_SIZE - Optional environment knob that limits how many old HLS segment files are unlinked in parallel during one prune batch; smaller values reduce I/O spikes at the cost of slower cleanup.
146 147
147 148 ## server/db.js
148 149 - initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
@@ -11,13 +11,13 @@
11
11
12## server/image-subtitles.js
12## server/image-subtitles.js
13- parseRenderOptions(options) - Normalises subtitle-frame request options and validates filePath/streamIndex/time/canvas dimensions; stream index `0` is valid and must not be treated as missing; canvas is scaled down within a 1280x720 ceiling, then multiplied by `IMAGE_SUBTITLE_RENDER_SCALE` (default `0.75`) while preserving requested aspect ratio.
13- parseRenderOptions(options) - Normalises subtitle-frame request options and validates filePath/streamIndex/time/canvas dimensions; stream index `0` is valid and must not be treated as missing; canvas is scaled down within a 1280x720 ceiling, then multiplied by `IMAGE_SUBTITLE_RENDER_SCALE` (default `0.75`) while preserving requested aspect ratio.
14- renderImageSubtitleFrame(options) - Extracts bitmap subtitle streams to cached `.sup`, parses PGS display sets, and returns {frame: PNG buffer, left, top, width, height, canvasWidth, canvasHeight} with metadata for proper client-side upscaling; the server preserves positioning info instead of throwing it away, allowing the client to calculate percentage-based sizing for downscaled frames.
14- renderImageSubtitleFrame(options) - Extracts bitmap subtitle streams to cached `.sup`, parses PGS display sets, and returns {frame: PNG buffer, left, top, width, height, canvasWidth, canvasHeight} with metadata for proper client-side upscaling; the server preserves positioning info instead of throwing it away, allowing the client to calculate percentage-based sizing for downscaled frames; request/build/render timing counters are tracked for debug diagnostics.
15- clearCachesForFile(filePath) - Removes all event and frame cache entries for a given file path using path-prefixed cache keys, also deletes the corresponding `.sup` file from disk for each evicted event entry; called when video stream sessions end to prevent memory and disk accumulation.
15- clearCachesForFile(filePath) - Removes all event and frame cache entries for a given file path using path-prefixed cache keys, also deletes the corresponding `.sup` file from disk for each evicted event entry; called when video stream sessions end to prevent memory and disk accumulation.
16- pruneOldImageSubtitleCaches(filePath, slowestViewerTimeSeconds) - Drops subtitle frame caches for events that have passed their retention threshold of end time + event duration + 5 seconds (e.g., event ending at 30s with 10s duration is dropped after slowest viewer reaches 45s); used during segment pruning to keep memory usage bounded on long videos, especially on resource-limited hardware like Raspberry Pi.
16- pruneOldImageSubtitleCaches(filePath, slowestViewerTimeSeconds) - Drops subtitle frame caches for events that have passed their retention threshold of end time + event duration + 5 seconds (e.g., event ending at 30s with 10s duration is dropped after slowest viewer reaches 45s); used during segment pruning to keep memory usage bounded on long videos, especially on resource-limited hardware like Raspberry Pi.
17- purgeSubtitleCacheDir() - Removes the entire subtitle `.sup` file cache directory from disk; called once at server startup to clean up any files left over from a previous run.
17- purgeSubtitleCacheDir() - Removes the entire subtitle `.sup` file cache directory from disk; called once at server startup to clean up any files left over from a previous run.
18- touchCachedEntry(cache, key, ttlMs) - Refreshes the TTL of an existing cache entry without changing its value; used to keep actively-watched subtitle tracks alive past their initial 5-minute TTL and prevent expensive full re-parses during long playback sessions.
18- touchCachedEntry(cache, key, ttlMs) - Refreshes the TTL of an existing cache entry without changing its value; used to keep actively-watched subtitle tracks alive past their initial 5-minute TTL and prevent expensive full re-parses during long playback sessions.
19- getImageSubtitleDebugSnapshot() - Returns event/frame cache totals and per-video cache usage ({event entries/bytes/count, frame entries/bytes}) so admin diagnostics can list cached subtitle data even when no viewer is connected.
19- getImageSubtitleDebugSnapshot() - Returns event/frame cache totals, timing counters (cache hits/misses, event build timing, frame render/request timing), and per-video cache usage ({event entries/bytes/count, frame entries/bytes}) so admin diagnostics can distinguish cache-thrash or cold-parse stalls from steady-state rendering.
20- IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS / IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES / IMAGE_SUBTITLE_FRAME_CACHE_MAX_BYTES / IMAGE_SUBTITLE_RENDER_SCALE / IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS - Environment knobs for subtitle cache retention, item caps, byte caps, render resolution, and prune grace window.
20- IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS / IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS / IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES / IMAGE_SUBTITLE_FRAME_CACHE_MAX_BYTES / IMAGE_SUBTITLE_RENDER_SCALE / IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS - Environment knobs for subtitle cache retention, optional item caps (`0`/`unbounded` disables the cap so byte limits govern), byte caps, render resolution, and prune grace window.
21
21
22## server/thumbnail-cache.js
22## server/thumbnail-cache.js
23- syncThumbnailCacheOnce() - Mirrors configured roots and userPath into public/thumbs and generates 99x99-max WebP thumbnails for image files, video files (ffprobe/ffmpeg frame at ~10%), and 3D model files (STL/OBJ, isometric SVG projection via Sharp); removes stale thumbnails and updates manifest hashes.
23- syncThumbnailCacheOnce() - Mirrors configured roots and userPath into public/thumbs and generates 99x99-max WebP thumbnails for image files, video files (ffprobe/ffmpeg frame at ~10%), and 3D model files (STL/OBJ, isometric SVG projection via Sharp); removes stale thumbnails and updates manifest hashes.
@@ -132,17 +132,18 @@
132- sendVideoStreamAsset(sessionId, assetPath, req, res) - Resolves viewer leases to a shared stream session, serves HLS assets, tracks lease activity, and prunes segments using the slowest active viewer position.
132- sendVideoStreamAsset(sessionId, assetPath, req, res) - Resolves viewer leases to a shared stream session, serves HLS assets, tracks lease activity, and prunes segments using the slowest active viewer position.
133- releaseVideoStreamSession(sessionId) - Releases one viewer lease; only destroys the underlying transcode session when no active leases remain; cleanup only clears image subtitle frame/event caches once no other active stream session for the same source file remains.
133- releaseVideoStreamSession(sessionId) - Releases one viewer lease; only destroys the underlying transcode session when no active leases remain; cleanup only clears image subtitle frame/event caches once no other active stream session for the same source file remains.
134- parseSegmentIndexFromAssetPath(assetPath) - Extracts the numeric segment index from an HLS segment filename; matches both numeric variant identifiers (stream_0_segment_000000.ts) and named variants (stream_video_segment_000000.ts, stream_Audio-1-jpn_segment_000000.ts) — ffmpeg substitutes the var_stream_map name: value for %v when named streams are used.
134- parseSegmentIndexFromAssetPath(assetPath) - Extracts the numeric segment index from an HLS segment filename; matches both numeric variant identifiers (stream_0_segment_000000.ts) and named variants (stream_video_segment_000000.ts, stream_Audio-1-jpn_segment_000000.ts) — ffmpeg substitutes the var_stream_map name: value for %v when named streams are used.
135- runSegmentPrune(session) - Removes HLS segment files that fall behind the slowest active viewer and calls pruneImageSubtitleCachesForSession() using the slowest active viewer across all live sessions for the same source file, so file-wide subtitle buffers are not pruned ahead of a newly started second session; segment filename regex also uses named-variant pattern to match production filenames correctly.
135- runSegmentPrune(session) - Removes HLS segment files that fall behind the slowest active viewer and calls pruneImageSubtitleCachesForSession() using the slowest active viewer across all live sessions for the same source file, so file-wide subtitle buffers are not pruned ahead of a newly started second session; segment deletion now runs in bounded batches and records last prune duration/deleted-segment counts for diagnostics; segment filename regex also uses named-variant pattern to match production filenames correctly.
136- hasInitialHlsAssets(outputDir) - Returns true once master.m3u8 or at least one variant playlist/segment exists in an HLS output directory; accepts named variant filenames (stream_video.m3u8) in addition to numeric ones.
136- hasInitialHlsAssets(outputDir) - Returns true once master.m3u8 or at least one variant playlist/segment exists in an HLS output directory; accepts named variant filenames (stream_video.m3u8) in addition to numeric ones.
137- purgeStreamRootDir() - Best-effort recursive delete of the temporary HLS stream root; used at startup and by idle sweeps to remove orphaned stream session directories from previous runs.
137- purgeStreamRootDir() - Best-effort recursive delete of the temporary HLS stream root; used at startup and by idle sweeps to remove orphaned stream session directories from previous runs.
138- cleanupIdleStreamArtifacts() - Periodic idle-time cleaner that purges stream-root artifacts when there are no active sessions or leases, preventing stale segment directories from accumulating after failures.
138- cleanupIdleStreamArtifacts() - Periodic idle-time cleaner that purges stream-root artifacts when there are no active sessions or leases, preventing stale segment directories from accumulating after failures.
139- getVideoStreamDebugSnapshot() - Returns active stream sessions with viewer counts and output-dir stats (segment counts/bytes), plus orphaned output directories left under the stream root for diagnostics.
139- getVideoStreamDebugSnapshot() - Returns active stream sessions with viewer counts and output-dir stats (segment counts/bytes), plus orphaned output directories left under the stream root for diagnostics.
140- getVideoStreamDebugSnapshot() - Includes segment retention and prune telemetry per session (segmentRetentionCount, slowestViewerSegmentIndex, segmentPrunedUntil) to explain growth in segment counts.
140- getVideoStreamDebugSnapshot() - Includes segment retention and prune telemetry per session (segmentRetentionCount, slowestViewerSegmentIndex, segmentPrunedUntil, lastSegmentPrune, streamPruneDeleteBatchSize) to explain growth or cleanup stalls in segment counts.
141- STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
141- STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
142- STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
142- STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
143- STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
143- STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
144- STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
144- STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
145- MAX_STREAM_SEGMENT_RETENTION_COUNT - Upper safety cap for retention (360) to avoid runaway in-session segment growth from oversized overrides.
145- MAX_STREAM_SEGMENT_RETENTION_COUNT - Upper safety cap for retention (360) to avoid runaway in-session segment growth from oversized overrides.
146- VIDEO_STREAM_PRUNE_DELETE_BATCH_SIZE - Optional environment knob that limits how many old HLS segment files are unlinked in parallel during one prune batch; smaller values reduce I/O spikes at the cost of slower cleanup.
146
147
147## server/db.js
148## server/db.js
148- initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
149- initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
server/image-subtitles.js +122 −6
@@ -11,8 +11,8 @@ const sharp = require('sharp');
11 11 const SUBTITLE_CACHE_DIR = path.join(os.tmpdir(), 'filemanager-video-subtitles');
12 12 const EVENT_CACHE_TTL_MS = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS, 5 * 60 * 1000);
13 13 const FRAME_CACHE_TTL_MS = toPositiveInt(process.env.IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS, 2 * 60 * 1000);
14 const MAX_EVENT_CACHE_ITEMS = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS, 3);
15 const MAX_FRAME_CACHE_ITEMS = toPositiveInt(process.env.IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS, 50);
14 const MAX_EVENT_CACHE_ITEMS = toCacheItemLimit(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS, Infinity);
15 const MAX_FRAME_CACHE_ITEMS = toCacheItemLimit(process.env.IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS, Infinity);
16 16 // Removed byte size cap for event cache to prevent thrashing
17 17 const MAX_EVENT_CACHE_BYTES = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES, 50 * 1024 * 1024);
18 18 const MIN_EVENT_WINDOW_SECONDS = 30; // Always keep at least this much for seeking
@@ -23,9 +23,60 @@ const IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS = toPositiveInt(process.env.IMAGE_SUBTI
23 23
24 24 const eventCache = createCacheState(MAX_EVENT_CACHE_ITEMS, MAX_EVENT_CACHE_BYTES);
25 25 const frameCache = createCacheState(MAX_FRAME_CACHE_ITEMS, MAX_FRAME_CACHE_BYTES);
26 const subtitleDebugStats = createSubtitleDebugStats();
26 27
27 28 let parseDisplaySets = null;
28 29
30 function createDurationStats ()
31 {
32 return {
33 count: 0,
34 totalMs: 0,
35 maxMs: 0,
36 lastMs: 0,
37 };
38 }
39
40 function createSubtitleDebugStats ()
41 {
42 return {
43 eventCacheHits: 0,
44 eventCacheMisses: 0,
45 eventCachePendingHits: 0,
46 eventCacheBuilds: 0,
47 eventCacheBuildErrors: 0,
48 frameCacheHits: 0,
49 frameCacheMisses: 0,
50 frameRenders: 0,
51 eventBuildDurationMs: createDurationStats(),
52 frameRenderDurationMs: createDurationStats(),
53 frameRequestDurationMs: createDurationStats(),
54 };
55 }
56
57 function recordDuration (stats, startedAt)
58 {
59 if (!stats || !Number.isFinite(startedAt)) return 0;
60
61 const elapsedMs = Math.max(0, Date.now() - startedAt);
62 stats.count += 1;
63 stats.totalMs += elapsedMs;
64 stats.lastMs = elapsedMs;
65 if (elapsedMs > stats.maxMs) stats.maxMs = elapsedMs;
66 return elapsedMs;
67 }
68
69 function averageDurationMs (stats)
70 {
71 if (!stats || !stats.count) return 0;
72 return Number((stats.totalMs / stats.count).toFixed(1));
73 }
74
75 function debugLimitValue (value)
76 {
77 return Number.isFinite(value) ? value : null;
78 }
79
29 80 function ensureImageDataGlobal ()
30 81 {
31 82 if (typeof globalThis.ImageData === 'function') return;
@@ -68,6 +119,20 @@ function toPositiveInt (value, fallback)
68 119 return fallback;
69 120 }
70 121
122 function toCacheItemLimit (value, fallback)
123 {
124 const raw = String(value || '').trim().toLowerCase();
125 if (!raw) return fallback;
126 if (raw === '0' || raw === 'none' || raw === 'infinity' || raw === 'inf' || raw === 'unbounded' || raw === 'unlimited')
127 {
128 return Infinity;
129 }
130
131 const number = Number.parseInt(raw, 10);
132 if (Number.isInteger(number) && number > 0) return number;
133 return fallback;
134 }
135
71 136 function toBoundedNumber (value, fallback, min, max)
72 137 {
73 138 const numeric = Number.parseFloat(String(value || ''));
@@ -385,20 +450,27 @@ async function getEventCacheEntry (filePath, streamIndex)
385 450 const cached = getCachedValue(eventCache, identityKey);
386 451 if (cached)
387 452 {
453 if (typeof cached.then === 'function') subtitleDebugStats.eventCachePendingHits += 1;
454 else subtitleDebugStats.eventCacheHits += 1;
388 455 touchCachedEntry(eventCache, identityKey, EVENT_CACHE_TTL_MS);
389 456 return cached;
390 457 }
391 458
459 subtitleDebugStats.eventCacheMisses += 1;
460 subtitleDebugStats.eventCacheBuilds += 1;
461 const buildStartedAt = Date.now();
392 462 const pending = buildEventCacheEntry(normalisedPath, streamIndex, identityKey);
393 463 setCachedValue(eventCache, identityKey, pending, EVENT_CACHE_TTL_MS, estimateEventCacheBytes);
394 464
395 465 try
396 466 {
397 467 const resolved = await pending;
468 recordDuration(subtitleDebugStats.eventBuildDurationMs, buildStartedAt);
398 469 setCachedValue(eventCache, identityKey, resolved, EVENT_CACHE_TTL_MS, estimateEventCacheBytes);
399 470 return resolved;
400 471 } catch (error)
401 472 {
473 subtitleDebugStats.eventCacheBuildErrors += 1;
402 474 deleteCachedValue(eventCache, identityKey);
403 475 throw error;
404 476 }
@@ -511,32 +583,50 @@ function parseRenderOptions (options)
511 583
512 584 async function renderImageSubtitleFrame (options)
513 585 {
586 const requestStartedAt = Date.now();
514 587 const parsed = parseRenderOptions(options);
515 588 const { filePath, streamIndex, timeSeconds, canvasWidth, canvasHeight } = parsed;
516 589
517 590 if (!parsed.isValid)
518 591 {
592 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
519 593 return null;
520 594 }
521 595
522 596 const entry = await getEventCacheEntry(filePath, streamIndex);
523 if (!entry || !Array.isArray(entry.events) || entry.events.length === 0) return null;
597 if (!entry || !Array.isArray(entry.events) || entry.events.length === 0)
598 {
599 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
600 return null;
601 }
524 602
525 603 const eventIndex = findActiveEventIndex(entry.events, timeSeconds);
526 if (eventIndex < 0) return null;
604 if (eventIndex < 0)
605 {
606 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
607 return null;
608 }
527 609
528 610 const frameKey = `${entry.cacheKey}|${eventIndex}|${canvasWidth}x${canvasHeight}`;
529 611 const cachedFrame = getCachedValue(frameCache, frameKey);
530 612 if (cachedFrame)
531 613 {
614 subtitleDebugStats.frameCacheHits += 1;
615 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
532 616 return cachedFrame;
533 617 }
534 618
619 subtitleDebugStats.frameCacheMisses += 1;
620 subtitleDebugStats.frameRenders += 1;
621 const renderStartedAt = Date.now();
535 622 const frameData = await renderEventFrame(entry.events[eventIndex], canvasWidth, canvasHeight);
536 623 if (!frameData)
537 624 {
625 recordDuration(subtitleDebugStats.frameRenderDurationMs, renderStartedAt);
626 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
538 627 return null;
539 628 }
629 recordDuration(subtitleDebugStats.frameRenderDurationMs, renderStartedAt);
540 630
541 631 const event = entry.events[eventIndex];
542 632 const cacheValue = {
@@ -557,6 +647,7 @@ async function renderImageSubtitleFrame (options)
557 647 FRAME_CACHE_TTL_MS,
558 648 estimateFrameCacheBytes,
559 649 );
650 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
560 651 return cacheValue;
561 652 }
562 653
@@ -750,15 +841,40 @@ function getImageSubtitleDebugSnapshot ()
750 841 eventCache: {
751 842 entries: eventCache.map.size,
752 843 bytes: eventCache.totalBytes,
753 maxItems: eventCache.maxItems,
844 maxItems: debugLimitValue(eventCache.maxItems),
754 845 maxBytes: eventCache.maxBytes,
755 846 },
756 847 frameCache: {
757 848 entries: frameCache.map.size,
758 849 bytes: frameCache.totalBytes,
759 maxItems: frameCache.maxItems,
850 maxItems: debugLimitValue(frameCache.maxItems),
760 851 maxBytes: frameCache.maxBytes,
761 852 },
853 metrics: {
854 eventCacheHits: subtitleDebugStats.eventCacheHits,
855 eventCacheMisses: subtitleDebugStats.eventCacheMisses,
856 eventCachePendingHits: subtitleDebugStats.eventCachePendingHits,
857 eventCacheBuilds: subtitleDebugStats.eventCacheBuilds,
858 eventCacheBuildErrors: subtitleDebugStats.eventCacheBuildErrors,
859 frameCacheHits: subtitleDebugStats.frameCacheHits,
860 frameCacheMisses: subtitleDebugStats.frameCacheMisses,
861 frameRenders: subtitleDebugStats.frameRenders,
862 eventBuildDurationMs: {
863 last: subtitleDebugStats.eventBuildDurationMs.lastMs,
864 max: subtitleDebugStats.eventBuildDurationMs.maxMs,
865 average: averageDurationMs(subtitleDebugStats.eventBuildDurationMs),
866 },
867 frameRenderDurationMs: {
868 last: subtitleDebugStats.frameRenderDurationMs.lastMs,
869 max: subtitleDebugStats.frameRenderDurationMs.maxMs,
870 average: averageDurationMs(subtitleDebugStats.frameRenderDurationMs),
871 },
872 frameRequestDurationMs: {
873 last: subtitleDebugStats.frameRequestDurationMs.lastMs,
874 max: subtitleDebugStats.frameRequestDurationMs.maxMs,
875 average: averageDurationMs(subtitleDebugStats.frameRequestDurationMs),
876 },
877 },
762 878 byFile,
763 879 };
764 880 }
@@ -11,8 +11,8 @@ const sharp = require('sharp');
11const SUBTITLE_CACHE_DIR = path.join(os.tmpdir(), 'filemanager-video-subtitles');
11const SUBTITLE_CACHE_DIR = path.join(os.tmpdir(), 'filemanager-video-subtitles');
12const EVENT_CACHE_TTL_MS = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS, 5 * 60 * 1000);
12const EVENT_CACHE_TTL_MS = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_TTL_MS, 5 * 60 * 1000);
13const FRAME_CACHE_TTL_MS = toPositiveInt(process.env.IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS, 2 * 60 * 1000);
13const FRAME_CACHE_TTL_MS = toPositiveInt(process.env.IMAGE_SUBTITLE_FRAME_CACHE_TTL_MS, 2 * 60 * 1000);
14const MAX_EVENT_CACHE_ITEMS = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS, 3);
14const MAX_EVENT_CACHE_ITEMS = toCacheItemLimit(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_ITEMS, Infinity);
15const MAX_FRAME_CACHE_ITEMS = toPositiveInt(process.env.IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS, 50);
15const MAX_FRAME_CACHE_ITEMS = toCacheItemLimit(process.env.IMAGE_SUBTITLE_FRAME_CACHE_MAX_ITEMS, Infinity);
16// Removed byte size cap for event cache to prevent thrashing
16// Removed byte size cap for event cache to prevent thrashing
17const MAX_EVENT_CACHE_BYTES = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES, 50 * 1024 * 1024);
17const MAX_EVENT_CACHE_BYTES = toPositiveInt(process.env.IMAGE_SUBTITLE_EVENT_CACHE_MAX_BYTES, 50 * 1024 * 1024);
18const MIN_EVENT_WINDOW_SECONDS = 30; // Always keep at least this much for seeking
18const MIN_EVENT_WINDOW_SECONDS = 30; // Always keep at least this much for seeking
@@ -23,9 +23,60 @@ const IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS = toPositiveInt(process.env.IMAGE_SUBTI
23
23
24const eventCache = createCacheState(MAX_EVENT_CACHE_ITEMS, MAX_EVENT_CACHE_BYTES);
24const eventCache = createCacheState(MAX_EVENT_CACHE_ITEMS, MAX_EVENT_CACHE_BYTES);
25const frameCache = createCacheState(MAX_FRAME_CACHE_ITEMS, MAX_FRAME_CACHE_BYTES);
25const frameCache = createCacheState(MAX_FRAME_CACHE_ITEMS, MAX_FRAME_CACHE_BYTES);
26const subtitleDebugStats = createSubtitleDebugStats();
26
27
27let parseDisplaySets = null;
28let parseDisplaySets = null;
28
29
30function createDurationStats ()
31{
32 return {
33 count: 0,
34 totalMs: 0,
35 maxMs: 0,
36 lastMs: 0,
37 };
38}
39
40function createSubtitleDebugStats ()
41{
42 return {
43 eventCacheHits: 0,
44 eventCacheMisses: 0,
45 eventCachePendingHits: 0,
46 eventCacheBuilds: 0,
47 eventCacheBuildErrors: 0,
48 frameCacheHits: 0,
49 frameCacheMisses: 0,
50 frameRenders: 0,
51 eventBuildDurationMs: createDurationStats(),
52 frameRenderDurationMs: createDurationStats(),
53 frameRequestDurationMs: createDurationStats(),
54 };
55}
56
57function recordDuration (stats, startedAt)
58{
59 if (!stats || !Number.isFinite(startedAt)) return 0;
60
61 const elapsedMs = Math.max(0, Date.now() - startedAt);
62 stats.count += 1;
63 stats.totalMs += elapsedMs;
64 stats.lastMs = elapsedMs;
65 if (elapsedMs > stats.maxMs) stats.maxMs = elapsedMs;
66 return elapsedMs;
67}
68
69function averageDurationMs (stats)
70{
71 if (!stats || !stats.count) return 0;
72 return Number((stats.totalMs / stats.count).toFixed(1));
73}
74
75function debugLimitValue (value)
76{
77 return Number.isFinite(value) ? value : null;
78}
79
29function ensureImageDataGlobal ()
80function ensureImageDataGlobal ()
30{
81{
31 if (typeof globalThis.ImageData === 'function') return;
82 if (typeof globalThis.ImageData === 'function') return;
@@ -68,6 +119,20 @@ function toPositiveInt (value, fallback)
68 return fallback;
119 return fallback;
69}
120}
70
121
122function toCacheItemLimit (value, fallback)
123{
124 const raw = String(value || '').trim().toLowerCase();
125 if (!raw) return fallback;
126 if (raw === '0' || raw === 'none' || raw === 'infinity' || raw === 'inf' || raw === 'unbounded' || raw === 'unlimited')
127 {
128 return Infinity;
129 }
130
131 const number = Number.parseInt(raw, 10);
132 if (Number.isInteger(number) && number > 0) return number;
133 return fallback;
134}
135
71function toBoundedNumber (value, fallback, min, max)
136function toBoundedNumber (value, fallback, min, max)
72{
137{
73 const numeric = Number.parseFloat(String(value || ''));
138 const numeric = Number.parseFloat(String(value || ''));
@@ -385,20 +450,27 @@ async function getEventCacheEntry (filePath, streamIndex)
385 const cached = getCachedValue(eventCache, identityKey);
450 const cached = getCachedValue(eventCache, identityKey);
386 if (cached)
451 if (cached)
387 {
452 {
453 if (typeof cached.then === 'function') subtitleDebugStats.eventCachePendingHits += 1;
454 else subtitleDebugStats.eventCacheHits += 1;
388 touchCachedEntry(eventCache, identityKey, EVENT_CACHE_TTL_MS);
455 touchCachedEntry(eventCache, identityKey, EVENT_CACHE_TTL_MS);
389 return cached;
456 return cached;
390 }
457 }
391
458
459 subtitleDebugStats.eventCacheMisses += 1;
460 subtitleDebugStats.eventCacheBuilds += 1;
461 const buildStartedAt = Date.now();
392 const pending = buildEventCacheEntry(normalisedPath, streamIndex, identityKey);
462 const pending = buildEventCacheEntry(normalisedPath, streamIndex, identityKey);
393 setCachedValue(eventCache, identityKey, pending, EVENT_CACHE_TTL_MS, estimateEventCacheBytes);
463 setCachedValue(eventCache, identityKey, pending, EVENT_CACHE_TTL_MS, estimateEventCacheBytes);
394
464
395 try
465 try
396 {
466 {
397 const resolved = await pending;
467 const resolved = await pending;
468 recordDuration(subtitleDebugStats.eventBuildDurationMs, buildStartedAt);
398 setCachedValue(eventCache, identityKey, resolved, EVENT_CACHE_TTL_MS, estimateEventCacheBytes);
469 setCachedValue(eventCache, identityKey, resolved, EVENT_CACHE_TTL_MS, estimateEventCacheBytes);
399 return resolved;
470 return resolved;
400 } catch (error)
471 } catch (error)
401 {
472 {
473 subtitleDebugStats.eventCacheBuildErrors += 1;
402 deleteCachedValue(eventCache, identityKey);
474 deleteCachedValue(eventCache, identityKey);
403 throw error;
475 throw error;
404 }
476 }
@@ -511,32 +583,50 @@ function parseRenderOptions (options)
511
583
512async function renderImageSubtitleFrame (options)
584async function renderImageSubtitleFrame (options)
513{
585{
586 const requestStartedAt = Date.now();
514 const parsed = parseRenderOptions(options);
587 const parsed = parseRenderOptions(options);
515 const { filePath, streamIndex, timeSeconds, canvasWidth, canvasHeight } = parsed;
588 const { filePath, streamIndex, timeSeconds, canvasWidth, canvasHeight } = parsed;
516
589
517 if (!parsed.isValid)
590 if (!parsed.isValid)
518 {
591 {
592 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
519 return null;
593 return null;
520 }
594 }
521
595
522 const entry = await getEventCacheEntry(filePath, streamIndex);
596 const entry = await getEventCacheEntry(filePath, streamIndex);
523 if (!entry || !Array.isArray(entry.events) || entry.events.length === 0) return null;
597 if (!entry || !Array.isArray(entry.events) || entry.events.length === 0)
598 {
599 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
600 return null;
601 }
524
602
525 const eventIndex = findActiveEventIndex(entry.events, timeSeconds);
603 const eventIndex = findActiveEventIndex(entry.events, timeSeconds);
526 if (eventIndex < 0) return null;
604 if (eventIndex < 0)
605 {
606 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
607 return null;
608 }
527
609
528 const frameKey = `${entry.cacheKey}|${eventIndex}|${canvasWidth}x${canvasHeight}`;
610 const frameKey = `${entry.cacheKey}|${eventIndex}|${canvasWidth}x${canvasHeight}`;
529 const cachedFrame = getCachedValue(frameCache, frameKey);
611 const cachedFrame = getCachedValue(frameCache, frameKey);
530 if (cachedFrame)
612 if (cachedFrame)
531 {
613 {
614 subtitleDebugStats.frameCacheHits += 1;
615 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
532 return cachedFrame;
616 return cachedFrame;
533 }
617 }
534
618
619 subtitleDebugStats.frameCacheMisses += 1;
620 subtitleDebugStats.frameRenders += 1;
621 const renderStartedAt = Date.now();
535 const frameData = await renderEventFrame(entry.events[eventIndex], canvasWidth, canvasHeight);
622 const frameData = await renderEventFrame(entry.events[eventIndex], canvasWidth, canvasHeight);
536 if (!frameData)
623 if (!frameData)
537 {
624 {
625 recordDuration(subtitleDebugStats.frameRenderDurationMs, renderStartedAt);
626 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
538 return null;
627 return null;
539 }
628 }
629 recordDuration(subtitleDebugStats.frameRenderDurationMs, renderStartedAt);
540
630
541 const event = entry.events[eventIndex];
631 const event = entry.events[eventIndex];
542 const cacheValue = {
632 const cacheValue = {
@@ -557,6 +647,7 @@ async function renderImageSubtitleFrame (options)
557 FRAME_CACHE_TTL_MS,
647 FRAME_CACHE_TTL_MS,
558 estimateFrameCacheBytes,
648 estimateFrameCacheBytes,
559 );
649 );
650 recordDuration(subtitleDebugStats.frameRequestDurationMs, requestStartedAt);
560 return cacheValue;
651 return cacheValue;
561}
652}
562
653
@@ -750,15 +841,40 @@ function getImageSubtitleDebugSnapshot ()
750 eventCache: {
841 eventCache: {
751 entries: eventCache.map.size,
842 entries: eventCache.map.size,
752 bytes: eventCache.totalBytes,
843 bytes: eventCache.totalBytes,
753 maxItems: eventCache.maxItems,
844 maxItems: debugLimitValue(eventCache.maxItems),
754 maxBytes: eventCache.maxBytes,
845 maxBytes: eventCache.maxBytes,
755 },
846 },
756 frameCache: {
847 frameCache: {
757 entries: frameCache.map.size,
848 entries: frameCache.map.size,
758 bytes: frameCache.totalBytes,
849 bytes: frameCache.totalBytes,
759 maxItems: frameCache.maxItems,
850 maxItems: debugLimitValue(frameCache.maxItems),
760 maxBytes: frameCache.maxBytes,
851 maxBytes: frameCache.maxBytes,
761 },
852 },
853 metrics: {
854 eventCacheHits: subtitleDebugStats.eventCacheHits,
855 eventCacheMisses: subtitleDebugStats.eventCacheMisses,
856 eventCachePendingHits: subtitleDebugStats.eventCachePendingHits,
857 eventCacheBuilds: subtitleDebugStats.eventCacheBuilds,
858 eventCacheBuildErrors: subtitleDebugStats.eventCacheBuildErrors,
859 frameCacheHits: subtitleDebugStats.frameCacheHits,
860 frameCacheMisses: subtitleDebugStats.frameCacheMisses,
861 frameRenders: subtitleDebugStats.frameRenders,
862 eventBuildDurationMs: {
863 last: subtitleDebugStats.eventBuildDurationMs.lastMs,
864 max: subtitleDebugStats.eventBuildDurationMs.maxMs,
865 average: averageDurationMs(subtitleDebugStats.eventBuildDurationMs),
866 },
867 frameRenderDurationMs: {
868 last: subtitleDebugStats.frameRenderDurationMs.lastMs,
869 max: subtitleDebugStats.frameRenderDurationMs.maxMs,
870 average: averageDurationMs(subtitleDebugStats.frameRenderDurationMs),
871 },
872 frameRequestDurationMs: {
873 last: subtitleDebugStats.frameRequestDurationMs.lastMs,
874 max: subtitleDebugStats.frameRequestDurationMs.maxMs,
875 average: averageDurationMs(subtitleDebugStats.frameRequestDurationMs),
876 },
877 },
762 byFile,
878 byFile,
763 };
879 };
764}
880}
server/video-stream.js +51 −8
@@ -13,6 +13,7 @@ const STREAM_READY_TIMEOUT_MS = 20 * 1000;
13 13 const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
14 14 const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
15 15 const ESTIMATED_SEGMENT_DURATION_SECONDS = 6;
16 const STREAM_PRUNE_DELETE_BATCH_SIZE = parsePruneDeleteBatchSize(process.env.VIDEO_STREAM_PRUNE_DELETE_BATCH_SIZE);
16 17 const MAX_STREAM_SEGMENT_RETENTION_COUNT = 360;
17 18 const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
18 19 const IMAGE_BASED_SUBTITLE_CODECS = new Set([
@@ -68,6 +69,16 @@ function parseSegmentRetentionCount (value)
68 69 return defaultSegmentRetentionCount();
69 70 }
70 71
72 function parsePruneDeleteBatchSize (value)
73 {
74 const parsed = Number.parseInt(String(value || ''), 10);
75 if (Number.isInteger(parsed) && parsed >= 1)
76 {
77 return clampNumber(parsed, 1, 128);
78 }
79 return 16;
80 }
81
71 82 function readRaspberryPiModel ()
72 83 {
73 84 try
@@ -400,6 +411,7 @@ function runSegmentPrune (session)
400 411 if (session.segmentPrunePendingCutoff <= session.segmentPrunedUntil) return;
401 412
402 413 const cutoff = session.segmentPrunePendingCutoff;
414 const pruneStartedAt = Date.now();
403 415 session.segmentPruneRunning = true;
404 416
405 417 fs.promises.readdir(session.outputDir)
@@ -414,18 +426,20 @@ function runSegmentPrune (session)
414 426 if (!Number.isInteger(segmentIndex)) continue;
415 427 if (segmentIndex > cutoff) continue;
416 428
417 removals.push(
418 fs.promises.unlink(path.join(session.outputDir, entryName)).catch(() =>
419 {
420 // Ignore races while ffmpeg rotates temp files and request handlers read segments.
421 })
422 );
429 removals.push(path.join(session.outputDir, entryName));
423 430 }
424 return Promise.all(removals);
431 return deleteFilesInBatches(removals, STREAM_PRUNE_DELETE_BATCH_SIZE);
425 432 })
426 .then(() =>
433 .then((deletedCount) =>
427 434 {
435 const completedAt = Date.now();
428 436 session.segmentPrunedUntil = Math.max(session.segmentPrunedUntil, cutoff);
437 session.lastSegmentPrune = {
438 cutoff,
439 deletedSegments: deletedCount,
440 durationMs: Math.max(0, completedAt - pruneStartedAt),
441 completedAt,
442 };
429 443 pruneImageSubtitleCachesForSession(session);
430 444 if (session.segmentPrunedUntil >= 0 && reusableSessionIdBySource.get(session.sourcePath) === session.id)
431 445 {
@@ -446,6 +460,31 @@ function runSegmentPrune (session)
446 460 });
447 461 }
448 462
463 async function deleteFilesInBatches (pathsToDelete, batchSize)
464 {
465 const paths = Array.isArray(pathsToDelete) ? pathsToDelete : [];
466 if (!paths.length) return 0;
467
468 let deletedCount = 0;
469 for (let index = 0; index < paths.length; index += batchSize)
470 {
471 const batch = paths.slice(index, index + batchSize);
472 await Promise.all(batch.map((targetPath) =>
473 fs.promises.unlink(targetPath)
474 .then(() =>
475 {
476 deletedCount += 1;
477 })
478 .catch(() =>
479 {
480 // Ignore races while ffmpeg rotates temp files and request handlers read segments.
481 })
482 ));
483 }
484
485 return deletedCount;
486 }
487
449 488 function slowestLeaseSegmentIndex (session)
450 489 {
451 490 if (!session.leaseIds.size) return null;
@@ -993,6 +1032,7 @@ async function createVideoStreamSession (sourcePath)
993 1032 segmentPruneRunning: false,
994 1033 segmentPrunePendingCutoff: -1,
995 1034 segmentPrunedUntil: -1,
1035 lastSegmentPrune: null,
996 1036 leaseIds: new Set(),
997 1037 };
998 1038
@@ -1209,9 +1249,11 @@ function getVideoStreamDebugSnapshot ()
1209 1249 activeRequests: Number(session.activeRequests || 0),
1210 1250 expiresAt: Number(session.expiresAt || 0),
1211 1251 segmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1252 segmentPruneDeleteBatchSize: STREAM_PRUNE_DELETE_BATCH_SIZE,
1212 1253 segmentPrunePendingCutoff: Number(session.segmentPrunePendingCutoff || 0),
1213 1254 segmentPrunedUntil: Number(session.segmentPrunedUntil || 0),
1214 1255 slowestViewerSegmentIndex: slowestLeaseSegmentIndex(session) ?? -1,
1256 lastSegmentPrune: session.lastSegmentPrune || null,
1215 1257 output: collectOutputDirStats(session.outputDir),
1216 1258 });
1217 1259 }
@@ -1240,6 +1282,7 @@ function getVideoStreamDebugSnapshot ()
1240 1282 streamRootDir: STREAM_ROOT_DIR,
1241 1283 streamSegmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1242 1284 streamSegmentRetentionMax: MAX_STREAM_SEGMENT_RETENTION_COUNT,
1285 streamPruneDeleteBatchSize: STREAM_PRUNE_DELETE_BATCH_SIZE,
1243 1286 activeSessionCount: sessions.size,
1244 1287 activeLeaseCount: leases.size,
1245 1288 activeStreams,
@@ -13,6 +13,7 @@ const STREAM_READY_TIMEOUT_MS = 20 * 1000;
13const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
13const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
14const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
14const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
15const ESTIMATED_SEGMENT_DURATION_SECONDS = 6;
15const ESTIMATED_SEGMENT_DURATION_SECONDS = 6;
16const STREAM_PRUNE_DELETE_BATCH_SIZE = parsePruneDeleteBatchSize(process.env.VIDEO_STREAM_PRUNE_DELETE_BATCH_SIZE);
16const MAX_STREAM_SEGMENT_RETENTION_COUNT = 360;
17const MAX_STREAM_SEGMENT_RETENTION_COUNT = 360;
17const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
18const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
18const IMAGE_BASED_SUBTITLE_CODECS = new Set([
19const IMAGE_BASED_SUBTITLE_CODECS = new Set([
@@ -68,6 +69,16 @@ function parseSegmentRetentionCount (value)
68 return defaultSegmentRetentionCount();
69 return defaultSegmentRetentionCount();
69}
70}
70
71
72function parsePruneDeleteBatchSize (value)
73{
74 const parsed = Number.parseInt(String(value || ''), 10);
75 if (Number.isInteger(parsed) && parsed >= 1)
76 {
77 return clampNumber(parsed, 1, 128);
78 }
79 return 16;
80}
81
71function readRaspberryPiModel ()
82function readRaspberryPiModel ()
72{
83{
73 try
84 try
@@ -400,6 +411,7 @@ function runSegmentPrune (session)
400 if (session.segmentPrunePendingCutoff <= session.segmentPrunedUntil) return;
411 if (session.segmentPrunePendingCutoff <= session.segmentPrunedUntil) return;
401
412
402 const cutoff = session.segmentPrunePendingCutoff;
413 const cutoff = session.segmentPrunePendingCutoff;
414 const pruneStartedAt = Date.now();
403 session.segmentPruneRunning = true;
415 session.segmentPruneRunning = true;
404
416
405 fs.promises.readdir(session.outputDir)
417 fs.promises.readdir(session.outputDir)
@@ -414,18 +426,20 @@ function runSegmentPrune (session)
414 if (!Number.isInteger(segmentIndex)) continue;
426 if (!Number.isInteger(segmentIndex)) continue;
415 if (segmentIndex > cutoff) continue;
427 if (segmentIndex > cutoff) continue;
416
428
417 removals.push(
429 removals.push(path.join(session.outputDir, entryName));
418 fs.promises.unlink(path.join(session.outputDir, entryName)).catch(() =>
419 {
420 // Ignore races while ffmpeg rotates temp files and request handlers read segments.
421 })
422 );
423 }
430 }
424 return Promise.all(removals);
431 return deleteFilesInBatches(removals, STREAM_PRUNE_DELETE_BATCH_SIZE);
425 })
432 })
426 .then(() =>
433 .then((deletedCount) =>
427 {
434 {
435 const completedAt = Date.now();
428 session.segmentPrunedUntil = Math.max(session.segmentPrunedUntil, cutoff);
436 session.segmentPrunedUntil = Math.max(session.segmentPrunedUntil, cutoff);
437 session.lastSegmentPrune = {
438 cutoff,
439 deletedSegments: deletedCount,
440 durationMs: Math.max(0, completedAt - pruneStartedAt),
441 completedAt,
442 };
429 pruneImageSubtitleCachesForSession(session);
443 pruneImageSubtitleCachesForSession(session);
430 if (session.segmentPrunedUntil >= 0 && reusableSessionIdBySource.get(session.sourcePath) === session.id)
444 if (session.segmentPrunedUntil >= 0 && reusableSessionIdBySource.get(session.sourcePath) === session.id)
431 {
445 {
@@ -446,6 +460,31 @@ function runSegmentPrune (session)
446 });
460 });
447}
461}
448
462
463async function deleteFilesInBatches (pathsToDelete, batchSize)
464{
465 const paths = Array.isArray(pathsToDelete) ? pathsToDelete : [];
466 if (!paths.length) return 0;
467
468 let deletedCount = 0;
469 for (let index = 0; index < paths.length; index += batchSize)
470 {
471 const batch = paths.slice(index, index + batchSize);
472 await Promise.all(batch.map((targetPath) =>
473 fs.promises.unlink(targetPath)
474 .then(() =>
475 {
476 deletedCount += 1;
477 })
478 .catch(() =>
479 {
480 // Ignore races while ffmpeg rotates temp files and request handlers read segments.
481 })
482 ));
483 }
484
485 return deletedCount;
486}
487
449function slowestLeaseSegmentIndex (session)
488function slowestLeaseSegmentIndex (session)
450{
489{
451 if (!session.leaseIds.size) return null;
490 if (!session.leaseIds.size) return null;
@@ -993,6 +1032,7 @@ async function createVideoStreamSession (sourcePath)
993 segmentPruneRunning: false,
1032 segmentPruneRunning: false,
994 segmentPrunePendingCutoff: -1,
1033 segmentPrunePendingCutoff: -1,
995 segmentPrunedUntil: -1,
1034 segmentPrunedUntil: -1,
1035 lastSegmentPrune: null,
996 leaseIds: new Set(),
1036 leaseIds: new Set(),
997 };
1037 };
998
1038
@@ -1209,9 +1249,11 @@ function getVideoStreamDebugSnapshot ()
1209 activeRequests: Number(session.activeRequests || 0),
1249 activeRequests: Number(session.activeRequests || 0),
1210 expiresAt: Number(session.expiresAt || 0),
1250 expiresAt: Number(session.expiresAt || 0),
1211 segmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1251 segmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1252 segmentPruneDeleteBatchSize: STREAM_PRUNE_DELETE_BATCH_SIZE,
1212 segmentPrunePendingCutoff: Number(session.segmentPrunePendingCutoff || 0),
1253 segmentPrunePendingCutoff: Number(session.segmentPrunePendingCutoff || 0),
1213 segmentPrunedUntil: Number(session.segmentPrunedUntil || 0),
1254 segmentPrunedUntil: Number(session.segmentPrunedUntil || 0),
1214 slowestViewerSegmentIndex: slowestLeaseSegmentIndex(session) ?? -1,
1255 slowestViewerSegmentIndex: slowestLeaseSegmentIndex(session) ?? -1,
1256 lastSegmentPrune: session.lastSegmentPrune || null,
1215 output: collectOutputDirStats(session.outputDir),
1257 output: collectOutputDirStats(session.outputDir),
1216 });
1258 });
1217 }
1259 }
@@ -1240,6 +1282,7 @@ function getVideoStreamDebugSnapshot ()
1240 streamRootDir: STREAM_ROOT_DIR,
1282 streamRootDir: STREAM_ROOT_DIR,
1241 streamSegmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1283 streamSegmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1242 streamSegmentRetentionMax: MAX_STREAM_SEGMENT_RETENTION_COUNT,
1284 streamSegmentRetentionMax: MAX_STREAM_SEGMENT_RETENTION_COUNT,
1285 streamPruneDeleteBatchSize: STREAM_PRUNE_DELETE_BATCH_SIZE,
1243 activeSessionCount: sessions.size,
1286 activeSessionCount: sessions.size,
1244 activeLeaseCount: leases.size,
1287 activeLeaseCount: leases.size,
1245 activeStreams,
1288 activeStreams,
tests/image-subtitles.test.js +22 −1
@@ -3,7 +3,7 @@
3 3 const fs = require('fs');
4 4 const os = require('os');
5 5 const path = require('path');
6 const { parseRenderOptions, pruneOldImageSubtitleCaches, purgeSubtitleCacheDir } = require('../server/image-subtitles');
6 const { parseRenderOptions, getImageSubtitleDebugSnapshot, purgeSubtitleCacheDir } = require('../server/image-subtitles');
7 7
8 8 describe('image subtitle option parsing', () =>
9 9 {
@@ -90,6 +90,27 @@ describe('image subtitle option parsing', () =>
90 90
91 91 describe('subtitle cache directory management', () =>
92 92 {
93 test('debug snapshot reports unbounded item caps and timing metrics', () =>
94 {
95 const snapshot = getImageSubtitleDebugSnapshot();
96
97 expect(snapshot.eventCache.maxItems).toBeNull();
98 expect(snapshot.frameCache.maxItems).toBeNull();
99 expect(snapshot.metrics).toEqual(expect.objectContaining({
100 eventCacheHits: expect.any(Number),
101 eventCacheMisses: expect.any(Number),
102 eventCacheBuilds: expect.any(Number),
103 frameCacheHits: expect.any(Number),
104 frameCacheMisses: expect.any(Number),
105 frameRenders: expect.any(Number),
106 }));
107 expect(snapshot.metrics.eventBuildDurationMs).toEqual(expect.objectContaining({
108 last: expect.any(Number),
109 max: expect.any(Number),
110 average: expect.any(Number),
111 }));
112 });
113
93 114 test('purgeSubtitleCacheDir removes the subtitle cache directory and its contents', () =>
94 115 {
95 116 const testDir = path.join(os.tmpdir(), 'filemanager-video-subtitles');
@@ -3,7 +3,7 @@
3const fs = require('fs');
3const fs = require('fs');
4const os = require('os');
4const os = require('os');
5const path = require('path');
5const path = require('path');
6const { parseRenderOptions, pruneOldImageSubtitleCaches, purgeSubtitleCacheDir } = require('../server/image-subtitles');
6const { parseRenderOptions, getImageSubtitleDebugSnapshot, purgeSubtitleCacheDir } = require('../server/image-subtitles');
7
7
8describe('image subtitle option parsing', () =>
8describe('image subtitle option parsing', () =>
9{
9{
@@ -90,6 +90,27 @@ describe('image subtitle option parsing', () =>
90
90
91describe('subtitle cache directory management', () =>
91describe('subtitle cache directory management', () =>
92{
92{
93 test('debug snapshot reports unbounded item caps and timing metrics', () =>
94 {
95 const snapshot = getImageSubtitleDebugSnapshot();
96
97 expect(snapshot.eventCache.maxItems).toBeNull();
98 expect(snapshot.frameCache.maxItems).toBeNull();
99 expect(snapshot.metrics).toEqual(expect.objectContaining({
100 eventCacheHits: expect.any(Number),
101 eventCacheMisses: expect.any(Number),
102 eventCacheBuilds: expect.any(Number),
103 frameCacheHits: expect.any(Number),
104 frameCacheMisses: expect.any(Number),
105 frameRenders: expect.any(Number),
106 }));
107 expect(snapshot.metrics.eventBuildDurationMs).toEqual(expect.objectContaining({
108 last: expect.any(Number),
109 max: expect.any(Number),
110 average: expect.any(Number),
111 }));
112 });
113
93 test('purgeSubtitleCacheDir removes the subtitle cache directory and its contents', () =>
114 test('purgeSubtitleCacheDir removes the subtitle cache directory and its contents', () =>
94 {
115 {
95 const testDir = path.join(os.tmpdir(), 'filemanager-video-subtitles');
116 const testDir = path.join(os.tmpdir(), 'filemanager-video-subtitles');
tests/video-stream.test.js +13 −0
@@ -517,6 +517,19 @@ describe('video stream bootstrap', () =>
517 517 expect(fs.existsSync(path.join(sessionDir, 'stream_Audio-1_segment_000041.ts'))).toBe(true);
518 518 expect(pruneOldImageSubtitleCaches).toHaveBeenCalled();
519 519
520 const debugSnapshot = streamModule.getVideoStreamDebugSnapshot();
521 const activeStream = debugSnapshot.activeStreams.find((stream) => stream.sourcePath === sourceFile);
522 expect(debugSnapshot.streamPruneDeleteBatchSize).toBe(16);
523 expect(activeStream).toEqual(expect.objectContaining({
524 lastSegmentPrune: expect.objectContaining({
525 cutoff: 40,
526 deletedSegments: expect.any(Number),
527 durationMs: expect.any(Number),
528 completedAt: expect.any(Number),
529 }),
530 }));
531 expect(activeStream.lastSegmentPrune.deletedSegments).toBeGreaterThan(0);
532
520 533 streamModule.releaseVideoStreamSession(session.sessionId);
521 534 });
522 535
@@ -517,6 +517,19 @@ describe('video stream bootstrap', () =>
517 expect(fs.existsSync(path.join(sessionDir, 'stream_Audio-1_segment_000041.ts'))).toBe(true);
517 expect(fs.existsSync(path.join(sessionDir, 'stream_Audio-1_segment_000041.ts'))).toBe(true);
518 expect(pruneOldImageSubtitleCaches).toHaveBeenCalled();
518 expect(pruneOldImageSubtitleCaches).toHaveBeenCalled();
519
519
520 const debugSnapshot = streamModule.getVideoStreamDebugSnapshot();
521 const activeStream = debugSnapshot.activeStreams.find((stream) => stream.sourcePath === sourceFile);
522 expect(debugSnapshot.streamPruneDeleteBatchSize).toBe(16);
523 expect(activeStream).toEqual(expect.objectContaining({
524 lastSegmentPrune: expect.objectContaining({
525 cutoff: 40,
526 deletedSegments: expect.any(Number),
527 durationMs: expect.any(Number),
528 completedAt: expect.any(Number),
529 }),
530 }));
531 expect(activeStream.lastSegmentPrune.deletedSegments).toBeGreaterThan(0);
532
520 streamModule.releaseVideoStreamSession(session.sessionId);
533 streamModule.releaseVideoStreamSession(session.sessionId);
521 });
534 });
522
535