BlueGit

timfa filemanager

feat: add RAM and segment-prune telemetry to debug dashboard

Adds host/process RAM metrics and stream retention/prune telemetry to the admin debug dashboard so lag can be correlated with memory and segment buildup in real time. Includes minor formatting-only cleanups in touched UI/subtitle files without behavioural change.

timfa Tim Falken · pushed by timfa

Authored 2026-05-20 · 6 months ago

Hash
5e2d9b32896189bf9a07431a1b338b04eb70b157
Parent
84ad254e
7 files changed +73 −16
.github/technical-reference.md +4 −0
@@ -67,6 +67,7 @@
67 67
68 68 ## server/routes/debug.js
69 69 - GET /api/admin/debug/stream-stats - Computes debug dashboard data by combining auth_sessions activity, root disk usage scans, stream snapshot data, and subtitle cache snapshot data into a per-video view for leak/performance monitoring.
70 - GET /api/admin/debug/stream-stats - Also reports host/process memory ({total, free, used, usedPercent, processRssBytes, processHeapUsedBytes}) so streaming lag can be correlated with RAM pressure.
70 71
71 72 ## server/routes/files.js
72 73 - GET /api/files/roots - Returns readable roots with display label (alias when configured), originalName, and path.
@@ -132,10 +133,12 @@
132 133 - purgeStreamRootDir() - Best-effort recursive delete of the temporary HLS stream root; used at startup and by idle sweeps to remove orphaned stream session directories from previous runs.
133 134 - cleanupIdleStreamArtifacts() - Periodic idle-time cleaner that purges stream-root artifacts when there are no active sessions or leases, preventing stale segment directories from accumulating after failures.
134 135 - getVideoStreamDebugSnapshot() - Returns active stream sessions with viewer counts and output-dir stats (segment counts/bytes), plus orphaned output directories left under the stream root for diagnostics.
136 - getVideoStreamDebugSnapshot() - Includes segment retention and prune telemetry per session (segmentRetentionCount, slowestViewerSegmentIndex, segmentPrunedUntil) to explain growth in segment counts.
135 137 - STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
136 138 - STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
137 139 - STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
138 140 - STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
141 - MAX_STREAM_SEGMENT_RETENTION_COUNT - Upper safety cap for retention (360) to avoid runaway in-session segment growth from oversized overrides.
139 142
140 143 ## server/db.js
141 144 - initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
@@ -159,6 +162,7 @@
159 162 - Admin panel - Users section includes a conditional orphaned-user-folders subsection that appears only when non-empty orphan folders exist.
160 163 - Debug dashboard modal - Shows active signed-in users, total/per-root storage usage, active stream footprint, cached subtitle usage by video, and orphaned stream output directories.
161 164 - Debug dashboard modal - Each diagnostics sub-panel includes a copy action for quick report sharing: users/storage copy JSON, stream/cache and orphaned output tables copy CSV.
165 - Debug dashboard modal - Storage cards include RAM usage and process RSS; stream/cache table includes retention and prune progress columns for troubleshooting segment buildup.
162 166 - Admin modal - Includes User Folders controls (user path + per-user size limit) and a dedicated save action for that config.
163 167 - Import map - Maps `three` and `three/addons/` to local vendored modules for client-side STL/OBJ previews.
164 168
@@ -67,6 +67,7 @@
67
67
68## server/routes/debug.js
68## server/routes/debug.js
69- GET /api/admin/debug/stream-stats - Computes debug dashboard data by combining auth_sessions activity, root disk usage scans, stream snapshot data, and subtitle cache snapshot data into a per-video view for leak/performance monitoring.
69- GET /api/admin/debug/stream-stats - Computes debug dashboard data by combining auth_sessions activity, root disk usage scans, stream snapshot data, and subtitle cache snapshot data into a per-video view for leak/performance monitoring.
70- GET /api/admin/debug/stream-stats - Also reports host/process memory ({total, free, used, usedPercent, processRssBytes, processHeapUsedBytes}) so streaming lag can be correlated with RAM pressure.
70
71
71## server/routes/files.js
72## server/routes/files.js
72- GET /api/files/roots - Returns readable roots with display label (alias when configured), originalName, and path.
73- GET /api/files/roots - Returns readable roots with display label (alias when configured), originalName, and path.
@@ -132,10 +133,12 @@
132- purgeStreamRootDir() - Best-effort recursive delete of the temporary HLS stream root; used at startup and by idle sweeps to remove orphaned stream session directories from previous runs.
133- purgeStreamRootDir() - Best-effort recursive delete of the temporary HLS stream root; used at startup and by idle sweeps to remove orphaned stream session directories from previous runs.
133- cleanupIdleStreamArtifacts() - Periodic idle-time cleaner that purges stream-root artifacts when there are no active sessions or leases, preventing stale segment directories from accumulating after failures.
134- cleanupIdleStreamArtifacts() - Periodic idle-time cleaner that purges stream-root artifacts when there are no active sessions or leases, preventing stale segment directories from accumulating after failures.
134- getVideoStreamDebugSnapshot() - Returns active stream sessions with viewer counts and output-dir stats (segment counts/bytes), plus orphaned output directories left under the stream root for diagnostics.
135- getVideoStreamDebugSnapshot() - Returns active stream sessions with viewer counts and output-dir stats (segment counts/bytes), plus orphaned output directories left under the stream root for diagnostics.
136- getVideoStreamDebugSnapshot() - Includes segment retention and prune telemetry per session (segmentRetentionCount, slowestViewerSegmentIndex, segmentPrunedUntil) to explain growth in segment counts.
135- STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
137- STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
136- STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
138- STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
137- STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
139- STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
138- STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
140- STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
141- MAX_STREAM_SEGMENT_RETENTION_COUNT - Upper safety cap for retention (360) to avoid runaway in-session segment growth from oversized overrides.
139
142
140## server/db.js
143## server/db.js
141- initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
144- initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
@@ -159,6 +162,7 @@
159- Admin panel - Users section includes a conditional orphaned-user-folders subsection that appears only when non-empty orphan folders exist.
162- Admin panel - Users section includes a conditional orphaned-user-folders subsection that appears only when non-empty orphan folders exist.
160- Debug dashboard modal - Shows active signed-in users, total/per-root storage usage, active stream footprint, cached subtitle usage by video, and orphaned stream output directories.
163- Debug dashboard modal - Shows active signed-in users, total/per-root storage usage, active stream footprint, cached subtitle usage by video, and orphaned stream output directories.
161- Debug dashboard modal - Each diagnostics sub-panel includes a copy action for quick report sharing: users/storage copy JSON, stream/cache and orphaned output tables copy CSV.
164- Debug dashboard modal - Each diagnostics sub-panel includes a copy action for quick report sharing: users/storage copy JSON, stream/cache and orphaned output tables copy CSV.
165- Debug dashboard modal - Storage cards include RAM usage and process RSS; stream/cache table includes retention and prune progress columns for troubleshooting segment buildup.
162- Admin modal - Includes User Folders controls (user path + per-user size limit) and a dedicated save action for that config.
166- Admin modal - Includes User Folders controls (user path + per-user size limit) and a dedicated save action for that config.
163- Import map - Maps `three` and `three/addons/` to local vendored modules for client-side STL/OBJ previews.
167- Import map - Maps `three` and `three/addons/` to local vendored modules for client-side STL/OBJ previews.
164
168
public/app.js +16 −1
@@ -2994,7 +2994,7 @@ function addContextMenuSubmenu (menu, label, options)
2994 2994 {
2995 2995 const row = document.createElement('div');
2996 2996 row.className = 'custom-context-menu-row';
2997 const trigger = addContextMenuButton(row, label, () => {}, { hasSubmenu: true });
2997 const trigger = addContextMenuButton(row, label, () => { }, { hasSubmenu: true });
2998 2998 trigger.addEventListener('click', (e) => e.preventDefault());
2999 2999
3000 3000 const panel = document.createElement('div');
@@ -5846,15 +5846,20 @@ function renderDebugDashboardData (data)
5846 5846 if (summary)
5847 5847 {
5848 5848 const storage = data.storage || {};
5849 const memory = data.memory || {};
5849 5850 const roots = Array.isArray(storage.roots) ? storage.roots.length : 0;
5850 5851 const activeStreams = Number(data.videoStream && data.videoStream.activeSessionCount) || 0;
5851 5852 const activeLeases = Number(data.videoStream && data.videoStream.activeLeaseCount) || 0;
5853 const segmentRetention = Number(data.videoStream && data.videoStream.streamSegmentRetentionCount) || 0;
5852 5854 summary.innerHTML = [
5853 5855 ['Configured roots', String(roots)],
5854 5856 ['Total used', formatBytes(storage.totalBytes || 0)],
5855 5857 ['Total files', String(storage.totalFiles || 0)],
5858 ['RAM used', `${formatBytes(memory.usedBytes || 0)} / ${formatBytes(memory.totalBytes || 0)} (${Number(memory.usedPercent || 0).toFixed(1)}%)`],
5859 ['Process RSS', formatBytes(memory.processRssBytes || 0)],
5856 5860 ['Active streams', String(activeStreams)],
5857 5861 ['Active viewers', String(activeLeases)],
5862 ['Segment retention', String(segmentRetention)],
5858 5863 ].map(([k, v]) => `<div class="debug-summary-item"><span class="k">${escapeHtml(k)}</span><span class="v">${escapeHtml(v)}</span></div>`).join('');
5859 5864 }
5860 5865
@@ -5882,6 +5887,9 @@ function renderDebugDashboardData (data)
5882 5887 { label: 'Event PNG bytes', render: (row) => escapeHtml(formatBytes(row.subtitleCache ? row.subtitleCache.eventImageBytes : 0)) },
5883 5888 { label: 'Frame cache', render: (row) => escapeHtml(String(row.subtitleCache ? row.subtitleCache.frameEntries : 0)) },
5884 5889 { label: 'Frame bytes', render: (row) => escapeHtml(formatBytes(row.subtitleCache ? row.subtitleCache.frameBytes : 0)) },
5890 { label: 'Retention', render: (row) => escapeHtml(String(row.stream ? row.stream.segmentRetentionCount : 0)) },
5891 { label: 'Slowest seg', render: (row) => escapeHtml(String(row.stream ? row.stream.slowestViewerSegmentIndex : 0)) },
5892 { label: 'Pruned until', render: (row) => escapeHtml(String(row.stream ? row.stream.segmentPrunedUntil : 0)) },
5885 5893 { label: 'Session ID', render: (row) => escapeHtml((row.activeSessionIds || []).join(', ') || '—') },
5886 5894 ],
5887 5895 Array.isArray(data.videoFiles) ? data.videoFiles : [],
@@ -5945,6 +5953,7 @@ function copyDebugStorageJson ()
5945 5953 const payload = {
5946 5954 generatedAt: _debugLatestData && _debugLatestData.generatedAt,
5947 5955 storage: _debugLatestData && _debugLatestData.storage,
5956 memory: _debugLatestData && _debugLatestData.memory,
5948 5957 };
5949 5958 void copyDebugText(JSON.stringify(payload, null, 2), 'Copied storage JSON.');
5950 5959 }
@@ -5965,6 +5974,9 @@ function copyDebugVideoFilesCsv ()
5965 5974 eventPngBytes: Number(row.subtitleCache ? row.subtitleCache.eventImageBytes : 0),
5966 5975 frameEntries: Number(row.subtitleCache ? row.subtitleCache.frameEntries : 0),
5967 5976 frameBytes: Number(row.subtitleCache ? row.subtitleCache.frameBytes : 0),
5977 segmentRetentionCount: Number(row.stream ? row.stream.segmentRetentionCount : 0),
5978 slowestViewerSegmentIndex: Number(row.stream ? row.stream.slowestViewerSegmentIndex : 0),
5979 segmentPrunedUntil: Number(row.stream ? row.stream.segmentPrunedUntil : 0),
5968 5980 sessionIds: Array.isArray(row.activeSessionIds) ? row.activeSessionIds.join(';') : '',
5969 5981 }));
5970 5982
@@ -5978,6 +5990,9 @@ function copyDebugVideoFilesCsv ()
5978 5990 'eventPngBytes',
5979 5991 'frameEntries',
5980 5992 'frameBytes',
5993 'segmentRetentionCount',
5994 'slowestViewerSegmentIndex',
5995 'segmentPrunedUntil',
5981 5996 'sessionIds',
5982 5997 ]);
5983 5998 void copyDebugText(csv, 'Copied video/cache CSV.');
@@ -2994,7 +2994,7 @@ function addContextMenuSubmenu (menu, label, options)
2994{
2994{
2995 const row = document.createElement('div');
2995 const row = document.createElement('div');
2996 row.className = 'custom-context-menu-row';
2996 row.className = 'custom-context-menu-row';
2997 const trigger = addContextMenuButton(row, label, () => {}, { hasSubmenu: true });
2997 const trigger = addContextMenuButton(row, label, () => { }, { hasSubmenu: true });
2998 trigger.addEventListener('click', (e) => e.preventDefault());
2998 trigger.addEventListener('click', (e) => e.preventDefault());
2999
2999
3000 const panel = document.createElement('div');
3000 const panel = document.createElement('div');
@@ -5846,15 +5846,20 @@ function renderDebugDashboardData (data)
5846 if (summary)
5846 if (summary)
5847 {
5847 {
5848 const storage = data.storage || {};
5848 const storage = data.storage || {};
5849 const memory = data.memory || {};
5849 const roots = Array.isArray(storage.roots) ? storage.roots.length : 0;
5850 const roots = Array.isArray(storage.roots) ? storage.roots.length : 0;
5850 const activeStreams = Number(data.videoStream && data.videoStream.activeSessionCount) || 0;
5851 const activeStreams = Number(data.videoStream && data.videoStream.activeSessionCount) || 0;
5851 const activeLeases = Number(data.videoStream && data.videoStream.activeLeaseCount) || 0;
5852 const activeLeases = Number(data.videoStream && data.videoStream.activeLeaseCount) || 0;
5853 const segmentRetention = Number(data.videoStream && data.videoStream.streamSegmentRetentionCount) || 0;
5852 summary.innerHTML = [
5854 summary.innerHTML = [
5853 ['Configured roots', String(roots)],
5855 ['Configured roots', String(roots)],
5854 ['Total used', formatBytes(storage.totalBytes || 0)],
5856 ['Total used', formatBytes(storage.totalBytes || 0)],
5855 ['Total files', String(storage.totalFiles || 0)],
5857 ['Total files', String(storage.totalFiles || 0)],
5858 ['RAM used', `${formatBytes(memory.usedBytes || 0)} / ${formatBytes(memory.totalBytes || 0)} (${Number(memory.usedPercent || 0).toFixed(1)}%)`],
5859 ['Process RSS', formatBytes(memory.processRssBytes || 0)],
5856 ['Active streams', String(activeStreams)],
5860 ['Active streams', String(activeStreams)],
5857 ['Active viewers', String(activeLeases)],
5861 ['Active viewers', String(activeLeases)],
5862 ['Segment retention', String(segmentRetention)],
5858 ].map(([k, v]) => `<div class="debug-summary-item"><span class="k">${escapeHtml(k)}</span><span class="v">${escapeHtml(v)}</span></div>`).join('');
5863 ].map(([k, v]) => `<div class="debug-summary-item"><span class="k">${escapeHtml(k)}</span><span class="v">${escapeHtml(v)}</span></div>`).join('');
5859 }
5864 }
5860
5865
@@ -5882,6 +5887,9 @@ function renderDebugDashboardData (data)
5882 { label: 'Event PNG bytes', render: (row) => escapeHtml(formatBytes(row.subtitleCache ? row.subtitleCache.eventImageBytes : 0)) },
5887 { label: 'Event PNG bytes', render: (row) => escapeHtml(formatBytes(row.subtitleCache ? row.subtitleCache.eventImageBytes : 0)) },
5883 { label: 'Frame cache', render: (row) => escapeHtml(String(row.subtitleCache ? row.subtitleCache.frameEntries : 0)) },
5888 { label: 'Frame cache', render: (row) => escapeHtml(String(row.subtitleCache ? row.subtitleCache.frameEntries : 0)) },
5884 { label: 'Frame bytes', render: (row) => escapeHtml(formatBytes(row.subtitleCache ? row.subtitleCache.frameBytes : 0)) },
5889 { label: 'Frame bytes', render: (row) => escapeHtml(formatBytes(row.subtitleCache ? row.subtitleCache.frameBytes : 0)) },
5890 { label: 'Retention', render: (row) => escapeHtml(String(row.stream ? row.stream.segmentRetentionCount : 0)) },
5891 { label: 'Slowest seg', render: (row) => escapeHtml(String(row.stream ? row.stream.slowestViewerSegmentIndex : 0)) },
5892 { label: 'Pruned until', render: (row) => escapeHtml(String(row.stream ? row.stream.segmentPrunedUntil : 0)) },
5885 { label: 'Session ID', render: (row) => escapeHtml((row.activeSessionIds || []).join(', ') || '—') },
5893 { label: 'Session ID', render: (row) => escapeHtml((row.activeSessionIds || []).join(', ') || '—') },
5886 ],
5894 ],
5887 Array.isArray(data.videoFiles) ? data.videoFiles : [],
5895 Array.isArray(data.videoFiles) ? data.videoFiles : [],
@@ -5945,6 +5953,7 @@ function copyDebugStorageJson ()
5945 const payload = {
5953 const payload = {
5946 generatedAt: _debugLatestData && _debugLatestData.generatedAt,
5954 generatedAt: _debugLatestData && _debugLatestData.generatedAt,
5947 storage: _debugLatestData && _debugLatestData.storage,
5955 storage: _debugLatestData && _debugLatestData.storage,
5956 memory: _debugLatestData && _debugLatestData.memory,
5948 };
5957 };
5949 void copyDebugText(JSON.stringify(payload, null, 2), 'Copied storage JSON.');
5958 void copyDebugText(JSON.stringify(payload, null, 2), 'Copied storage JSON.');
5950}
5959}
@@ -5965,6 +5974,9 @@ function copyDebugVideoFilesCsv ()
5965 eventPngBytes: Number(row.subtitleCache ? row.subtitleCache.eventImageBytes : 0),
5974 eventPngBytes: Number(row.subtitleCache ? row.subtitleCache.eventImageBytes : 0),
5966 frameEntries: Number(row.subtitleCache ? row.subtitleCache.frameEntries : 0),
5975 frameEntries: Number(row.subtitleCache ? row.subtitleCache.frameEntries : 0),
5967 frameBytes: Number(row.subtitleCache ? row.subtitleCache.frameBytes : 0),
5976 frameBytes: Number(row.subtitleCache ? row.subtitleCache.frameBytes : 0),
5977 segmentRetentionCount: Number(row.stream ? row.stream.segmentRetentionCount : 0),
5978 slowestViewerSegmentIndex: Number(row.stream ? row.stream.slowestViewerSegmentIndex : 0),
5979 segmentPrunedUntil: Number(row.stream ? row.stream.segmentPrunedUntil : 0),
5968 sessionIds: Array.isArray(row.activeSessionIds) ? row.activeSessionIds.join(';') : '',
5980 sessionIds: Array.isArray(row.activeSessionIds) ? row.activeSessionIds.join(';') : '',
5969 }));
5981 }));
5970
5982
@@ -5978,6 +5990,9 @@ function copyDebugVideoFilesCsv ()
5978 'eventPngBytes',
5990 'eventPngBytes',
5979 'frameEntries',
5991 'frameEntries',
5980 'frameBytes',
5992 'frameBytes',
5993 'segmentRetentionCount',
5994 'slowestViewerSegmentIndex',
5995 'segmentPrunedUntil',
5981 'sessionIds',
5996 'sessionIds',
5982 ]);
5997 ]);
5983 void copyDebugText(csv, 'Copied video/cache CSV.');
5998 void copyDebugText(csv, 'Copied video/cache CSV.');
public/index.html +2 −2
@@ -116,8 +116,8 @@
116 116 </svg>
117 117 Debug
118 118 </button>
119 <button id="btn-change-password" class="topbar-menu-item" type="button" role="menuitem" title="Change Password"
120 aria-label="Change Password">
119 <button id="btn-change-password" class="topbar-menu-item" type="button" role="menuitem"
120 title="Change Password" aria-label="Change Password">
121 121 <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
122 122 stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
123 123 <rect x="3" y="11" width="18" height="10" rx="2" ry="2" />
@@ -116,8 +116,8 @@
116 </svg>
116 </svg>
117 Debug
117 Debug
118 </button>
118 </button>
119 <button id="btn-change-password" class="topbar-menu-item" type="button" role="menuitem" title="Change Password"
119 <button id="btn-change-password" class="topbar-menu-item" type="button" role="menuitem"
120 aria-label="Change Password">
120 title="Change Password" aria-label="Change Password">
121 <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
121 <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
122 stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
122 stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
123 <rect x="3" y="11" width="18" height="10" rx="2" ry="2" />
123 <rect x="3" y="11" width="18" height="10" rx="2" ry="2" />
public/style.css +3 −3
@@ -777,8 +777,8 @@ button.danger.admin-update-highlight {
777 777 display: none;
778 778 }
779 779
780 .custom-context-menu-row:hover > .custom-context-submenu,
781 .custom-context-menu-row:focus-within > .custom-context-submenu {
780 .custom-context-menu-row:hover>.custom-context-submenu,
781 .custom-context-menu-row:focus-within>.custom-context-submenu {
782 782 display: block;
783 783 }
784 784
@@ -912,7 +912,7 @@ button.danger.admin-update-highlight {
912 912 padding: 0.6rem 0.8rem;
913 913 }
914 914
915 .update-commit-item + .update-commit-item {
915 .update-commit-item+.update-commit-item {
916 916 margin-top: 0.9rem;
917 917 padding-top: 0.9rem;
918 918 border-top: 1px solid var(--border);
@@ -777,8 +777,8 @@ button.danger.admin-update-highlight {
777 display: none;
777 display: none;
778}
778}
779
779
780.custom-context-menu-row:hover > .custom-context-submenu,
780.custom-context-menu-row:hover>.custom-context-submenu,
781.custom-context-menu-row:focus-within > .custom-context-submenu {
781.custom-context-menu-row:focus-within>.custom-context-submenu {
782 display: block;
782 display: block;
783}
783}
784
784
@@ -912,7 +912,7 @@ button.danger.admin-update-highlight {
912 padding: 0.6rem 0.8rem;
912 padding: 0.6rem 0.8rem;
913}
913}
914
914
915.update-commit-item + .update-commit-item {
915.update-commit-item+.update-commit-item {
916 margin-top: 0.9rem;
916 margin-top: 0.9rem;
917 padding-top: 0.9rem;
917 padding-top: 0.9rem;
918 border-top: 1px solid var(--border);
918 border-top: 1px solid var(--border);
server/image-subtitles.js +18 −9
@@ -588,7 +588,8 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
588 588
589 589 // Prune frameCache as before
590 590 const keysToDelete = [];
591 for (const key of frameCache.map.keys()) {
591 for (const key of frameCache.map.keys())
592 {
592 593 if (!key.startsWith(prefix)) continue;
593 594 const entry = frameCache.map.get(key);
594 595 if (!entry) continue;
@@ -597,11 +598,13 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
597 598 if (!Number.isFinite(startTimeSeconds) || !Number.isFinite(endTimeSeconds)) continue;
598 599 const duration = Math.max(0, endTimeSeconds - startTimeSeconds);
599 600 const dropThreshold = endTimeSeconds + duration + IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS;
600 if (slowestViewerTimeSeconds >= dropThreshold) {
601 if (slowestViewerTimeSeconds >= dropThreshold)
602 {
601 603 keysToDelete.push(key);
602 604 }
603 605 }
604 for (const key of keysToDelete) {
606 for (const key of keysToDelete)
607 {
605 608 deleteCachedValue(frameCache, key);
606 609 }
607 610
@@ -609,7 +612,8 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
609 612 let totalBytes = eventCache.totalBytes;
610 613 if (totalBytes <= MAX_EVENT_CACHE_BYTES) return;
611 614 // For each entry, prune events that end before (slowestViewerTimeSeconds - MIN_EVENT_WINDOW_SECONDS)
612 for (const [key, entry] of eventCache.map.entries()) {
615 for (const [key, entry] of eventCache.map.entries())
616 {
613 617 if (!key.startsWith(prefix)) continue;
614 618 const value = entry.value;
615 619 if (!value || !Array.isArray(value.events)) continue;
@@ -617,21 +621,26 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
617 621 // Find first event that ends after the retention threshold
618 622 const minKeepTime = Math.max(0, slowestViewerTimeSeconds - MIN_EVENT_WINDOW_SECONDS);
619 623 let pruneIdx = 0;
620 while (pruneIdx < events.length && events[pruneIdx].endTimeSeconds < minKeepTime) {
624 while (pruneIdx < events.length && events[pruneIdx].endTimeSeconds < minKeepTime)
625 {
621 626 pruneIdx++;
622 627 }
623 if (pruneIdx > 0) {
628 if (pruneIdx > 0)
629 {
624 630 // Remove pruned events and update memory usage
625 631 const removed = events.splice(0, pruneIdx);
626 for (const ev of removed) {
627 if (ev && Buffer.isBuffer(ev.imagePng)) {
632 for (const ev of removed)
633 {
634 if (ev && Buffer.isBuffer(ev.imagePng))
635 {
628 636 entry.sizeBytes -= ev.imagePng.length + 160;
629 637 eventCache.totalBytes -= ev.imagePng.length + 160;
630 638 }
631 639 }
632 640 }
633 641 // If all events are pruned, remove the entry
634 if (events.length === 0) {
642 if (events.length === 0)
643 {
635 644 deleteSupFileForKey(key);
636 645 deleteCachedValue(eventCache, key);
637 646 }
@@ -588,7 +588,8 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
588
588
589 // Prune frameCache as before
589 // Prune frameCache as before
590 const keysToDelete = [];
590 const keysToDelete = [];
591 for (const key of frameCache.map.keys()) {
591 for (const key of frameCache.map.keys())
592 {
592 if (!key.startsWith(prefix)) continue;
593 if (!key.startsWith(prefix)) continue;
593 const entry = frameCache.map.get(key);
594 const entry = frameCache.map.get(key);
594 if (!entry) continue;
595 if (!entry) continue;
@@ -597,11 +598,13 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
597 if (!Number.isFinite(startTimeSeconds) || !Number.isFinite(endTimeSeconds)) continue;
598 if (!Number.isFinite(startTimeSeconds) || !Number.isFinite(endTimeSeconds)) continue;
598 const duration = Math.max(0, endTimeSeconds - startTimeSeconds);
599 const duration = Math.max(0, endTimeSeconds - startTimeSeconds);
599 const dropThreshold = endTimeSeconds + duration + IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS;
600 const dropThreshold = endTimeSeconds + duration + IMAGE_SUBTITLE_PRUNE_GRACE_SECONDS;
600 if (slowestViewerTimeSeconds >= dropThreshold) {
601 if (slowestViewerTimeSeconds >= dropThreshold)
602 {
601 keysToDelete.push(key);
603 keysToDelete.push(key);
602 }
604 }
603 }
605 }
604 for (const key of keysToDelete) {
606 for (const key of keysToDelete)
607 {
605 deleteCachedValue(frameCache, key);
608 deleteCachedValue(frameCache, key);
606 }
609 }
607
610
@@ -609,7 +612,8 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
609 let totalBytes = eventCache.totalBytes;
612 let totalBytes = eventCache.totalBytes;
610 if (totalBytes <= MAX_EVENT_CACHE_BYTES) return;
613 if (totalBytes <= MAX_EVENT_CACHE_BYTES) return;
611 // For each entry, prune events that end before (slowestViewerTimeSeconds - MIN_EVENT_WINDOW_SECONDS)
614 // For each entry, prune events that end before (slowestViewerTimeSeconds - MIN_EVENT_WINDOW_SECONDS)
612 for (const [key, entry] of eventCache.map.entries()) {
615 for (const [key, entry] of eventCache.map.entries())
616 {
613 if (!key.startsWith(prefix)) continue;
617 if (!key.startsWith(prefix)) continue;
614 const value = entry.value;
618 const value = entry.value;
615 if (!value || !Array.isArray(value.events)) continue;
619 if (!value || !Array.isArray(value.events)) continue;
@@ -617,21 +621,26 @@ function pruneOldImageSubtitleCaches (filePath, slowestViewerTimeSeconds)
617 // Find first event that ends after the retention threshold
621 // Find first event that ends after the retention threshold
618 const minKeepTime = Math.max(0, slowestViewerTimeSeconds - MIN_EVENT_WINDOW_SECONDS);
622 const minKeepTime = Math.max(0, slowestViewerTimeSeconds - MIN_EVENT_WINDOW_SECONDS);
619 let pruneIdx = 0;
623 let pruneIdx = 0;
620 while (pruneIdx < events.length && events[pruneIdx].endTimeSeconds < minKeepTime) {
624 while (pruneIdx < events.length && events[pruneIdx].endTimeSeconds < minKeepTime)
625 {
621 pruneIdx++;
626 pruneIdx++;
622 }
627 }
623 if (pruneIdx > 0) {
628 if (pruneIdx > 0)
629 {
624 // Remove pruned events and update memory usage
630 // Remove pruned events and update memory usage
625 const removed = events.splice(0, pruneIdx);
631 const removed = events.splice(0, pruneIdx);
626 for (const ev of removed) {
632 for (const ev of removed)
627 if (ev && Buffer.isBuffer(ev.imagePng)) {
633 {
634 if (ev && Buffer.isBuffer(ev.imagePng))
635 {
628 entry.sizeBytes -= ev.imagePng.length + 160;
636 entry.sizeBytes -= ev.imagePng.length + 160;
629 eventCache.totalBytes -= ev.imagePng.length + 160;
637 eventCache.totalBytes -= ev.imagePng.length + 160;
630 }
638 }
631 }
639 }
632 }
640 }
633 // If all events are pruned, remove the entry
641 // If all events are pruned, remove the entry
634 if (events.length === 0) {
642 if (events.length === 0)
643 {
635 deleteSupFileForKey(key);
644 deleteSupFileForKey(key);
636 deleteCachedValue(eventCache, key);
645 deleteCachedValue(eventCache, key);
637 }
646 }
server/routes/debug.js +19 −0
@@ -2,6 +2,7 @@
2 2
3 3 const express = require('express');
4 4 const fs = require('fs');
5 const os = require('os');
5 6 const path = require('path');
6 7
7 8 const router = express.Router();
@@ -181,6 +182,22 @@ function buildVideoDebugRows (videoSnapshot, subtitleSnapshot)
181 182 return [...perFile.values()].sort((a, b) => a.filePath.localeCompare(b.filePath));
182 183 }
183 184
185 function getMemorySnapshot ()
186 {
187 const totalBytes = Number(os.totalmem()) || 0;
188 const freeBytes = Number(os.freemem()) || 0;
189 const usedBytes = Math.max(0, totalBytes - freeBytes);
190
191 return {
192 totalBytes,
193 freeBytes,
194 usedBytes,
195 usedPercent: totalBytes > 0 ? Number(((usedBytes / totalBytes) * 100).toFixed(1)) : 0,
196 processRssBytes: Number(process.memoryUsage().rss || 0),
197 processHeapUsedBytes: Number(process.memoryUsage().heapUsed || 0),
198 };
199 }
200
184 201 router.get('/debug/stream-stats', (req, res) =>
185 202 {
186 203 const rootUsage = collectRootUsageStats();
@@ -188,11 +205,13 @@ router.get('/debug/stream-stats', (req, res) =>
188 205 const videoStream = getVideoStreamDebugSnapshot();
189 206 const subtitleCache = getImageSubtitleDebugSnapshot();
190 207 const videoFiles = buildVideoDebugRows(videoStream, subtitleCache);
208 const memory = getMemorySnapshot();
191 209
192 210 res.json({
193 211 generatedAt: Date.now(),
194 212 loggedInUsers,
195 213 storage: rootUsage,
214 memory,
196 215 videoStream,
197 216 subtitleCache,
198 217 videoFiles,
@@ -2,6 +2,7 @@
2
2
3const express = require('express');
3const express = require('express');
4const fs = require('fs');
4const fs = require('fs');
5const os = require('os');
5const path = require('path');
6const path = require('path');
6
7
7const router = express.Router();
8const router = express.Router();
@@ -181,6 +182,22 @@ function buildVideoDebugRows (videoSnapshot, subtitleSnapshot)
181 return [...perFile.values()].sort((a, b) => a.filePath.localeCompare(b.filePath));
182 return [...perFile.values()].sort((a, b) => a.filePath.localeCompare(b.filePath));
182}
183}
183
184
185function getMemorySnapshot ()
186{
187 const totalBytes = Number(os.totalmem()) || 0;
188 const freeBytes = Number(os.freemem()) || 0;
189 const usedBytes = Math.max(0, totalBytes - freeBytes);
190
191 return {
192 totalBytes,
193 freeBytes,
194 usedBytes,
195 usedPercent: totalBytes > 0 ? Number(((usedBytes / totalBytes) * 100).toFixed(1)) : 0,
196 processRssBytes: Number(process.memoryUsage().rss || 0),
197 processHeapUsedBytes: Number(process.memoryUsage().heapUsed || 0),
198 };
199}
200
184router.get('/debug/stream-stats', (req, res) =>
201router.get('/debug/stream-stats', (req, res) =>
185{
202{
186 const rootUsage = collectRootUsageStats();
203 const rootUsage = collectRootUsageStats();
@@ -188,11 +205,13 @@ router.get('/debug/stream-stats', (req, res) =>
188 const videoStream = getVideoStreamDebugSnapshot();
205 const videoStream = getVideoStreamDebugSnapshot();
189 const subtitleCache = getImageSubtitleDebugSnapshot();
206 const subtitleCache = getImageSubtitleDebugSnapshot();
190 const videoFiles = buildVideoDebugRows(videoStream, subtitleCache);
207 const videoFiles = buildVideoDebugRows(videoStream, subtitleCache);
208 const memory = getMemorySnapshot();
191
209
192 res.json({
210 res.json({
193 generatedAt: Date.now(),
211 generatedAt: Date.now(),
194 loggedInUsers,
212 loggedInUsers,
195 storage: rootUsage,
213 storage: rootUsage,
214 memory,
196 videoStream,
215 videoStream,
197 subtitleCache,
216 subtitleCache,
198 videoFiles,
217 videoFiles,
server/video-stream.js +11 −1
@@ -12,6 +12,7 @@ const STREAM_SESSION_TTL_MS = 30 * 60 * 1000;
12 12 const STREAM_READY_TIMEOUT_MS = 20 * 1000;
13 13 const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
14 14 const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
15 const MAX_STREAM_SEGMENT_RETENTION_COUNT = 360;
15 16 const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
16 17 const IMAGE_BASED_SUBTITLE_CODECS = new Set([
17 18 'hdmv_pgs_subtitle',
@@ -59,7 +60,10 @@ if (streamRootCleanupTimer && typeof streamRootCleanupTimer.unref === 'function'
59 60 function parseSegmentRetentionCount (value)
60 61 {
61 62 const parsed = Number.parseInt(String(value || ''), 10);
62 if (Number.isInteger(parsed) && parsed >= 30) return parsed;
63 if (Number.isInteger(parsed) && parsed >= 30)
64 {
65 return clampNumber(parsed, 30, MAX_STREAM_SEGMENT_RETENTION_COUNT);
66 }
63 67 return defaultSegmentRetentionCount();
64 68 }
65 69
@@ -1163,6 +1167,10 @@ function getVideoStreamDebugSnapshot ()
1163 1167 viewers: session.leaseIds ? session.leaseIds.size : 0,
1164 1168 activeRequests: Number(session.activeRequests || 0),
1165 1169 expiresAt: Number(session.expiresAt || 0),
1170 segmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1171 segmentPrunePendingCutoff: Number(session.segmentPrunePendingCutoff || 0),
1172 segmentPrunedUntil: Number(session.segmentPrunedUntil || 0),
1173 slowestViewerSegmentIndex: Number(slowestLeaseSegmentIndex(session) || 0),
1166 1174 output: collectOutputDirStats(session.outputDir),
1167 1175 });
1168 1176 }
@@ -1189,6 +1197,8 @@ function getVideoStreamDebugSnapshot ()
1189 1197
1190 1198 return {
1191 1199 streamRootDir: STREAM_ROOT_DIR,
1200 streamSegmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1201 streamSegmentRetentionMax: MAX_STREAM_SEGMENT_RETENTION_COUNT,
1192 1202 activeSessionCount: sessions.size,
1193 1203 activeLeaseCount: leases.size,
1194 1204 activeStreams,
@@ -12,6 +12,7 @@ const STREAM_SESSION_TTL_MS = 30 * 60 * 1000;
12const STREAM_READY_TIMEOUT_MS = 20 * 1000;
12const STREAM_READY_TIMEOUT_MS = 20 * 1000;
13const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
13const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
14const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
14const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
15const MAX_STREAM_SEGMENT_RETENTION_COUNT = 360;
15const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
16const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
16const IMAGE_BASED_SUBTITLE_CODECS = new Set([
17const IMAGE_BASED_SUBTITLE_CODECS = new Set([
17 'hdmv_pgs_subtitle',
18 'hdmv_pgs_subtitle',
@@ -59,7 +60,10 @@ if (streamRootCleanupTimer && typeof streamRootCleanupTimer.unref === 'function'
59function parseSegmentRetentionCount (value)
60function parseSegmentRetentionCount (value)
60{
61{
61 const parsed = Number.parseInt(String(value || ''), 10);
62 const parsed = Number.parseInt(String(value || ''), 10);
62 if (Number.isInteger(parsed) && parsed >= 30) return parsed;
63 if (Number.isInteger(parsed) && parsed >= 30)
64 {
65 return clampNumber(parsed, 30, MAX_STREAM_SEGMENT_RETENTION_COUNT);
66 }
63 return defaultSegmentRetentionCount();
67 return defaultSegmentRetentionCount();
64}
68}
65
69
@@ -1163,6 +1167,10 @@ function getVideoStreamDebugSnapshot ()
1163 viewers: session.leaseIds ? session.leaseIds.size : 0,
1167 viewers: session.leaseIds ? session.leaseIds.size : 0,
1164 activeRequests: Number(session.activeRequests || 0),
1168 activeRequests: Number(session.activeRequests || 0),
1165 expiresAt: Number(session.expiresAt || 0),
1169 expiresAt: Number(session.expiresAt || 0),
1170 segmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1171 segmentPrunePendingCutoff: Number(session.segmentPrunePendingCutoff || 0),
1172 segmentPrunedUntil: Number(session.segmentPrunedUntil || 0),
1173 slowestViewerSegmentIndex: Number(slowestLeaseSegmentIndex(session) || 0),
1166 output: collectOutputDirStats(session.outputDir),
1174 output: collectOutputDirStats(session.outputDir),
1167 });
1175 });
1168 }
1176 }
@@ -1189,6 +1197,8 @@ function getVideoStreamDebugSnapshot ()
1189
1197
1190 return {
1198 return {
1191 streamRootDir: STREAM_ROOT_DIR,
1199 streamRootDir: STREAM_ROOT_DIR,
1200 streamSegmentRetentionCount: STREAM_SEGMENT_RETENTION_COUNT,
1201 streamSegmentRetentionMax: MAX_STREAM_SEGMENT_RETENTION_COUNT,
1192 activeSessionCount: sessions.size,
1202 activeSessionCount: sessions.size,
1193 activeLeaseCount: leases.size,
1203 activeLeaseCount: leases.size,
1194 activeStreams,
1204 activeStreams,