BlueGit

timfa filemanager

Merge feat/pi5-retention-tuning

timfa Tim Falken · pushed by timfa

Authored 2026-05-14 · 6 months ago

Hash
57e6a59d385af278a2f4615929a5fa61bcdd5df7
3 files changed +347 −54
.github/technical-reference.md +6 −4
@@ -92,13 +92,13 @@
92 92 - ZIP_CACHE_DIR - Absolute path to the zip-cache/ directory (project root).
93 93
94 94 ## server/video-stream.js
95 - createVideoStreamSession(sourcePath) - Probes the input duration, attempts WebVTT conversion for all subtitle streams (skipping only failed conversions), starts ffmpeg HLS transcoding, and resolves only when HLS assets are present (not subtitle files alone); returns durationSeconds, subtitleTracks, and subtitleSummary {total, converted, skipped}.
96 - sendVideoStreamAsset(sessionId, assetPath, req, res) - Serves HLS playlists, segments, and extracted subtitle files, tracks request lifecycle, and prunes old .ts segments once playback has moved beyond the configured retention window.
97 - releaseVideoStreamSession(sessionId) - Explicitly stops an active stream session, terminates ffmpeg, and removes temporary assets.
95 - createVideoStreamSession(sourcePath) - Probes input metadata and reuses a compatible active transcode session per source file when possible; returns a per-viewer lease id as sessionId plus duration/subtitle data.
96 - sendVideoStreamAsset(sessionId, assetPath, req, res) - Resolves viewer leases to a shared stream session, serves HLS assets, tracks lease activity, and prunes segments using the slowest active viewer position.
97 - releaseVideoStreamSession(sessionId) - Releases one viewer lease; only destroys the underlying transcode session when no active leases remain.
98 98 - STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
99 99 - STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
100 100 - STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
101 - STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned (default 180, overridable via VIDEO_STREAM_SEGMENT_RETENTION_COUNT).
101 - STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
102 102
103 103 ## server/db.js
104 104 - initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
@@ -252,6 +252,8 @@
252 252 - createVideoStreamSession configures flat segment filenames that match playlist URLs - Verifies ffmpeg segment output paths match the segment URLs emitted in the generated HLS playlists.
253 253 - createVideoStreamSession does not become ready from subtitle files alone - Verifies stream readiness is tied to HLS output instead of extracted subtitle files.
254 254 - createVideoStreamSession keeps playback available when subtitle conversion fails - Verifies subtitle conversion failures are skipped without breaking stream playback.
255 - sendVideoStreamAsset prunes old segment files after playback moves far ahead - Verifies old segments are removed only after the retention window passes.
256 - createVideoStreamSession reuses one transcode for concurrent viewers of the same file - Verifies two viewers share one ffmpeg transcode while keeping independent lease release behaviour.
255 257
256 258 ## tests/share.page.load.test.js
257 259 - applies default caption settings for share-page video playback - Verifies requested caption defaults are applied and persisted on share-page video preview.
@@ -92,13 +92,13 @@
92- ZIP_CACHE_DIR - Absolute path to the zip-cache/ directory (project root).
92- ZIP_CACHE_DIR - Absolute path to the zip-cache/ directory (project root).
93
93
94## server/video-stream.js
94## server/video-stream.js
95- createVideoStreamSession(sourcePath) - Probes the input duration, attempts WebVTT conversion for all subtitle streams (skipping only failed conversions), starts ffmpeg HLS transcoding, and resolves only when HLS assets are present (not subtitle files alone); returns durationSeconds, subtitleTracks, and subtitleSummary {total, converted, skipped}.
95- createVideoStreamSession(sourcePath) - Probes input metadata and reuses a compatible active transcode session per source file when possible; returns a per-viewer lease id as sessionId plus duration/subtitle data.
96- sendVideoStreamAsset(sessionId, assetPath, req, res) - Serves HLS playlists, segments, and extracted subtitle files, tracks request lifecycle, and prunes old .ts segments once playback has moved beyond the configured retention window.
96- sendVideoStreamAsset(sessionId, assetPath, req, res) - Resolves viewer leases to a shared stream session, serves HLS assets, tracks lease activity, and prunes segments using the slowest active viewer position.
97- releaseVideoStreamSession(sessionId) - Explicitly stops an active stream session, terminates ffmpeg, and removes temporary assets.
97- releaseVideoStreamSession(sessionId) - Releases one viewer lease; only destroys the underlying transcode session when no active leases remain.
98- STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
98- STREAM_ROOT_DIR - Temporary root directory for active ffmpeg-backed stream sessions.
99- STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
99- STREAM_SESSION_TTL_MS - Idle lifetime for a stream session before cleanup.
100- STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
100- STREAM_IDLE_STOP_MS - Maximum client inactivity window before an active stream session is terminated (5 minutes).
101- STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned (default 180, overridable via VIDEO_STREAM_SEGMENT_RETENTION_COUNT).
101- STREAM_SEGMENT_RETENTION_COUNT - Number of most-recent HLS segment indices kept on disk while older segments are pruned; when not overridden with VIDEO_STREAM_SEGMENT_RETENTION_COUNT it is derived from hardware capacity (RAM + CPU), current memory/CPU pressure, and Raspberry Pi 5 profile bounds.
102
102
103## server/db.js
103## server/db.js
104- initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
104- initDb() - Initializes SQLite schema for users, file metadata, auth_sessions, and share_tokens (expires_at index).
@@ -252,6 +252,8 @@
252- createVideoStreamSession configures flat segment filenames that match playlist URLs - Verifies ffmpeg segment output paths match the segment URLs emitted in the generated HLS playlists.
252- createVideoStreamSession configures flat segment filenames that match playlist URLs - Verifies ffmpeg segment output paths match the segment URLs emitted in the generated HLS playlists.
253- createVideoStreamSession does not become ready from subtitle files alone - Verifies stream readiness is tied to HLS output instead of extracted subtitle files.
253- createVideoStreamSession does not become ready from subtitle files alone - Verifies stream readiness is tied to HLS output instead of extracted subtitle files.
254- createVideoStreamSession keeps playback available when subtitle conversion fails - Verifies subtitle conversion failures are skipped without breaking stream playback.
254- createVideoStreamSession keeps playback available when subtitle conversion fails - Verifies subtitle conversion failures are skipped without breaking stream playback.
255- sendVideoStreamAsset prunes old segment files after playback moves far ahead - Verifies old segments are removed only after the retention window passes.
256- createVideoStreamSession reuses one transcode for concurrent viewers of the same file - Verifies two viewers share one ffmpeg transcode while keeping independent lease release behaviour.
255
257
256## tests/share.page.load.test.js
258## tests/share.page.load.test.js
257- applies default caption settings for share-page video playback - Verifies requested caption defaults are applied and persisted on share-page video preview.
259- applies default caption settings for share-page video playback - Verifies requested caption defaults are applied and persisted on share-page video preview.
server/video-stream.js +258 −46
@@ -10,14 +10,88 @@ const STREAM_ROOT_DIR = path.join(os.tmpdir(), 'filemanager-video-streams');
10 10 const STREAM_SESSION_TTL_MS = 30 * 60 * 1000;
11 11 const STREAM_READY_TIMEOUT_MS = 20 * 1000;
12 12 const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
13 const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
13 14 const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
14 15 const sessions = new Map();
16 const leases = new Map();
17 const reusableSessionIdBySource = new Map();
15 18
16 19 function parseSegmentRetentionCount (value)
17 20 {
18 21 const parsed = Number.parseInt(String(value || ''), 10);
19 22 if (Number.isInteger(parsed) && parsed >= 30) return parsed;
20 return 180;
23 return defaultSegmentRetentionCount();
24 }
25
26 function readRaspberryPiModel ()
27 {
28 try
29 {
30 return fs.readFileSync('/proc/device-tree/model', 'utf8').replace(/\u0000/g, '').trim();
31 } catch
32 {
33 return '';
34 }
35 }
36
37 function clampNumber (value, min, max)
38 {
39 return Math.min(max, Math.max(min, value));
40 }
41
42 function currentCpuPressureFactor (cpuCount)
43 {
44 if (typeof os.loadavg !== 'function') return 1;
45 if (!Number.isFinite(cpuCount) || cpuCount <= 0) return 1;
46
47 const [load1] = os.loadavg();
48 if (!Number.isFinite(load1) || load1 < 0) return 1;
49
50 const saturation = load1 / cpuCount;
51 if (saturation >= 1.5) return 0.72;
52 if (saturation >= 1.2) return 0.8;
53 if (saturation >= 1.0) return 0.88;
54 if (saturation >= 0.8) return 0.94;
55 return 1;
56 }
57
58 function defaultSegmentRetentionCount ()
59 {
60 const totalMemBytes = Number(os.totalmem()) || 0;
61 const freeMemBytes = Number(os.freemem()) || 0;
62 const cpuCount = Math.max(1, ((os.cpus() || []).length || 1));
63 const arch = String(os.arch() || '').toLowerCase();
64 const piModel = readRaspberryPiModel().toLowerCase();
65 const isPi5 = piModel.includes('raspberry pi 5');
66 const isArm = arch === 'arm' || arch === 'arm64' || arch.startsWith('armv');
67 const cpuPressure = currentCpuPressureFactor(cpuCount);
68
69 const freeMemRatio = totalMemBytes > 0 ? (freeMemBytes / totalMemBytes) : 1;
70 const memoryPressure = freeMemRatio < 0.12 ? 0.72 : freeMemRatio < 0.2 ? 0.84 : freeMemRatio < 0.3 ? 0.92 : 1;
71
72 if (isPi5)
73 {
74 const memoryBudgetBytes = Math.floor(totalMemBytes * 0.03);
75 const byMemory = Math.floor(memoryBudgetBytes / ESTIMATED_SEGMENT_BYTES);
76 const byCpu = 120 + ((cpuCount - 2) * 20);
77 const combined = Math.min(byMemory, byCpu);
78 return clampNumber(Math.floor(combined * cpuPressure * memoryPressure), 100, 220);
79 }
80
81 if (isArm)
82 {
83 const memoryBudgetBytes = Math.floor(totalMemBytes * 0.025);
84 const byMemory = Math.floor(memoryBudgetBytes / ESTIMATED_SEGMENT_BYTES);
85 const byCpu = 90 + ((cpuCount - 2) * 14);
86 const combined = Math.min(byMemory, byCpu);
87 return clampNumber(Math.floor(combined * cpuPressure * memoryPressure), 80, 180);
88 }
89
90 const memoryBudgetBytes = Math.floor(totalMemBytes * 0.05);
91 const byMemory = Math.floor(memoryBudgetBytes / ESTIMATED_SEGMENT_BYTES);
92 const byCpu = 140 + ((cpuCount - 4) * 16);
93 const combined = Math.min(byMemory, byCpu);
94 return clampNumber(Math.floor(combined * cpuPressure * memoryPressure), 120, 320);
21 95 }
22 96
23 97 function hasInitialHlsAssets (outputDir)
@@ -64,6 +138,13 @@ function ffprobePath ()
64 138 function pruneExpiredSessions ()
65 139 {
66 140 const now = Date.now();
141
142 for (const [leaseId, lease] of leases)
143 {
144 if (lease.expiresAt > now) continue;
145 releaseVideoStreamSession(leaseId);
146 }
147
67 148 for (const [sessionId, session] of sessions)
68 149 {
69 150 if (session.expiresAt > now) continue;
@@ -76,6 +157,11 @@ function refreshSession (session)
76 157 session.expiresAt = Date.now() + STREAM_SESSION_TTL_MS;
77 158 }
78 159
160 function refreshLease (lease)
161 {
162 lease.expiresAt = Date.now() + STREAM_SESSION_TTL_MS;
163 }
164
79 165 function destroySession (sessionId)
80 166 {
81 167 const session = sessions.get(sessionId);
@@ -83,6 +169,20 @@ function destroySession (sessionId)
83 169
84 170 sessions.delete(sessionId);
85 171
172 if (reusableSessionIdBySource.get(session.sourcePath) === session.id)
173 {
174 reusableSessionIdBySource.delete(session.sourcePath);
175 }
176
177 if (session.leaseIds)
178 {
179 for (const leaseId of session.leaseIds)
180 {
181 leases.delete(leaseId);
182 }
183 session.leaseIds.clear();
184 }
185
86 186 if (session.idleTimer)
87 187 {
88 188 clearTimeout(session.idleTimer);
@@ -101,10 +201,46 @@ function destroySession (sessionId)
101 201 return true;
102 202 }
103 203
104 function touchSessionActivity (session)
204 function getSessionForLease (leaseId)
205 {
206 const lease = leases.get(leaseId);
207 if (!lease) return null;
208
209 const session = sessions.get(lease.sessionId);
210 if (!session)
211 {
212 leases.delete(leaseId);
213 return null;
214 }
215
216 return { lease, session };
217 }
218
219 function createLeaseForSession (session)
220 {
221 const leaseId = crypto.randomBytes(18).toString('hex');
222 const lease = {
223 id: leaseId,
224 sessionId: session.id,
225 expiresAt: Date.now() + STREAM_SESSION_TTL_MS,
226 lastActivityAt: Date.now(),
227 lastSegmentIndex: null,
228 };
229 leases.set(leaseId, lease);
230 session.leaseIds.add(leaseId);
231 return lease;
232 }
233
234 function touchSessionActivity (session, lease)
105 235 {
106 236 refreshSession(session);
107 237 session.lastActivityAt = Date.now();
238
239 if (lease)
240 {
241 refreshLease(lease);
242 lease.lastActivityAt = Date.now();
243 }
108 244 }
109 245
110 246 function clearIdleStopTimer (session)
@@ -117,13 +253,13 @@ function clearIdleStopTimer (session)
117 253 function scheduleIdleStop (session)
118 254 {
119 255 clearIdleStopTimer(session);
120 if (session.activeRequests > 0) return;
256 if (session.activeRequests > 0 || (session.leaseIds && session.leaseIds.size > 0)) return;
121 257
122 258 session.idleTimer = setTimeout(() =>
123 259 {
124 260 const activeSession = sessions.get(session.id);
125 261 if (!activeSession) return;
126 if (activeSession.activeRequests > 0) return;
262 if (activeSession.activeRequests > 0 || (activeSession.leaseIds && activeSession.leaseIds.size > 0)) return;
127 263 if (Date.now() - activeSession.lastActivityAt < STREAM_IDLE_STOP_MS)
128 264 {
129 265 scheduleIdleStop(activeSession);
@@ -133,18 +269,18 @@ function scheduleIdleStop (session)
133 269 }, STREAM_IDLE_STOP_MS);
134 270 }
135 271
136 function startAssetRequest (session)
272 function startAssetRequest (session, lease)
137 273 {
138 274 session.activeRequests += 1;
139 touchSessionActivity(session);
275 touchSessionActivity(session, lease);
140 276 clearIdleStopTimer(session);
141 277 }
142 278
143 function finishAssetRequest (session)
279 function finishAssetRequest (session, lease)
144 280 {
145 281 if (session.activeRequests > 0) session.activeRequests -= 1;
146 touchSessionActivity(session);
147 if (session.activeRequests === 0) scheduleIdleStop(session);
282 touchSessionActivity(session, lease);
283 if (session.activeRequests === 0 && session.leaseIds.size === 0) scheduleIdleStop(session);
148 284 }
149 285
150 286 function parseSegmentIndexFromAssetPath (assetPath)
@@ -189,6 +325,10 @@ function runSegmentPrune (session)
189 325 .then(() =>
190 326 {
191 327 session.segmentPrunedUntil = Math.max(session.segmentPrunedUntil, cutoff);
328 if (session.segmentPrunedUntil >= 0 && reusableSessionIdBySource.get(session.sourcePath) === session.id)
329 {
330 reusableSessionIdBySource.delete(session.sourcePath);
331 }
192 332 })
193 333 .catch(() =>
194 334 {
@@ -204,11 +344,31 @@ function runSegmentPrune (session)
204 344 });
205 345 }
206 346
207 function scheduleSegmentPrune (session, segmentIndex)
347 function slowestLeaseSegmentIndex (session)
348 {
349 if (!session.leaseIds.size) return null;
350
351 let minIndex = null;
352 for (const leaseId of session.leaseIds)
353 {
354 const lease = leases.get(leaseId);
355 if (!lease) continue;
356 if (!Number.isInteger(lease.lastSegmentIndex)) return null;
357 if (minIndex === null || lease.lastSegmentIndex < minIndex) minIndex = lease.lastSegmentIndex;
358 }
359
360 return minIndex;
361 }
362
363 function scheduleSegmentPrune (session, lease, segmentIndex)
208 364 {
209 365 if (!Number.isInteger(segmentIndex) || segmentIndex < 0) return;
210 366
211 const pruneCutoff = segmentIndex - STREAM_SEGMENT_RETENTION_COUNT;
367 lease.lastSegmentIndex = segmentIndex;
368 const slowestIndex = slowestLeaseSegmentIndex(session);
369 if (!Number.isInteger(slowestIndex)) return;
370
371 const pruneCutoff = slowestIndex - STREAM_SEGMENT_RETENTION_COUNT;
212 372 if (pruneCutoff <= session.segmentPrunedUntil) return;
213 373
214 374 session.segmentPrunePendingCutoff = Math.max(session.segmentPrunePendingCutoff, pruneCutoff);
@@ -602,39 +762,72 @@ async function createVideoStreamSession (sourcePath)
602 762 pruneExpiredSessions();
603 763 ensureStreamRootDir();
604 764
605 const sessionId = crypto.randomBytes(18).toString('hex');
606 const outputDir = path.join(STREAM_ROOT_DIR, sessionId);
607 fs.mkdirSync(outputDir, { recursive: true });
765 const sourceKey = String(sourcePath || '');
766 let session = null;
608 767
609 const session = {
610 id: sessionId,
611 sourcePath,
612 outputDir,
613 expiresAt: Date.now() + STREAM_SESSION_TTL_MS,
614 process: null,
615 error: null,
616 subtitleTracks: [],
617 subtitleSummary: { total: 0, converted: 0, skipped: 0 },
618 readyPromise: null,
619 activeRequests: 0,
620 lastActivityAt: Date.now(),
621 idleTimer: null,
622 segmentPruneRunning: false,
623 segmentPrunePendingCutoff: -1,
624 segmentPrunedUntil: -1,
625 };
768 const reusableSessionId = reusableSessionIdBySource.get(sourceKey);
769 if (reusableSessionId)
770 {
771 const reusable = sessions.get(reusableSessionId);
772 if (reusable && !reusable.error && reusable.segmentPrunedUntil < 0)
773 {
774 session = reusable;
775 } else
776 {
777 reusableSessionIdBySource.delete(sourceKey);
778 }
779 }
626 780
627 session.readyPromise = initialiseSession(session).catch((err) =>
781 if (!session)
628 782 {
629 session.error = err;
630 throw err;
631 });
783 const sessionId = crypto.randomBytes(18).toString('hex');
784 const outputDir = path.join(STREAM_ROOT_DIR, sessionId);
785 fs.mkdirSync(outputDir, { recursive: true });
786
787 session = {
788 id: sessionId,
789 sourcePath: sourceKey,
790 outputDir,
791 expiresAt: Date.now() + STREAM_SESSION_TTL_MS,
792 process: null,
793 error: null,
794 subtitleTracks: [],
795 subtitleSummary: { total: 0, converted: 0, skipped: 0 },
796 readyPromise: null,
797 activeRequests: 0,
798 lastActivityAt: Date.now(),
799 idleTimer: null,
800 segmentPruneRunning: false,
801 segmentPrunePendingCutoff: -1,
802 segmentPrunedUntil: -1,
803 leaseIds: new Set(),
804 };
632 805
633 sessions.set(sessionId, session);
634 await session.readyPromise;
806 session.readyPromise = initialiseSession(session).catch((err) =>
807 {
808 session.error = err;
809 throw err;
810 });
811
812 sessions.set(session.id, session);
813 reusableSessionIdBySource.set(sourceKey, session.id);
814 }
815
816 if (!session.leaseIds) session.leaseIds = new Set();
817 const lease = createLeaseForSession(session);
818 touchSessionActivity(session, lease);
819
820 try
821 {
822 await session.readyPromise;
823 } catch (err)
824 {
825 releaseVideoStreamSession(lease.id);
826 throw err;
827 }
635 828
636 829 return {
637 sessionId,
830 sessionId: lease.id,
638 831 durationSeconds: session.durationSeconds,
639 832 subtitleTracks: session.subtitleTracks.map((track) => ({ ...track })),
640 833 subtitleSummary: {
@@ -659,10 +852,12 @@ function contentTypeForAsset (assetPath)
659 852 async function sendVideoStreamAsset (sessionId, assetPath, req, res)
660 853 {
661 854 pruneExpiredSessions();
662 const session = sessions.get(sessionId);
663 if (!session) throw makeHttpError(404, 'Video stream session not found.');
855 const active = getSessionForLease(sessionId);
856 if (!active) throw makeHttpError(404, 'Video stream session not found.');
664 857
665 touchSessionActivity(session);
858 const { session, lease } = active;
859
860 touchSessionActivity(session, lease);
666 861 await session.readyPromise;
667 862
668 863 const relativeAssetPath = String(assetPath || 'master.m3u8').replace(/^[/\\]+/, '');
@@ -674,20 +869,20 @@ async function sendVideoStreamAsset (sessionId, assetPath, req, res)
674 869 }
675 870
676 871 const segmentIndex = parseSegmentIndexFromAssetPath(relativeAssetPath);
677 if (segmentIndex !== null) scheduleSegmentPrune(session, segmentIndex);
872 if (segmentIndex !== null) scheduleSegmentPrune(session, lease, segmentIndex);
678 873
679 874 await waitForFile(resolvedPath, session);
680 875 res.set('Cache-Control', 'private, no-store');
681 876 res.type(contentTypeForAsset(resolvedPath));
682 877
683 startAssetRequest(session);
878 startAssetRequest(session, lease);
684 879
685 880 let settled = false;
686 881 const finish = () =>
687 882 {
688 883 if (settled) return;
689 884 settled = true;
690 finishAssetRequest(session);
885 finishAssetRequest(session, lease);
691 886 };
692 887
693 888 req.on('aborted', finish);
@@ -719,14 +914,31 @@ async function sendVideoStreamAsset (sessionId, assetPath, req, res)
719 914
720 915 function releaseVideoStreamSession (sessionId)
721 916 {
722 pruneExpiredSessions();
723 return destroySession(String(sessionId || ''));
917 const leaseId = String(sessionId || '');
918 const active = getSessionForLease(leaseId);
919 if (!active)
920 {
921 leases.delete(leaseId);
922 return false;
923 }
924
925 const { session } = active;
926 leases.delete(leaseId);
927 session.leaseIds.delete(leaseId);
928
929 if (session.leaseIds.size === 0 && session.activeRequests === 0)
930 {
931 return destroySession(session.id);
932 }
933
934 return true;
724 935 }
725 936
726 937 module.exports = {
727 938 STREAM_ROOT_DIR,
728 939 STREAM_SESSION_TTL_MS,
729 940 STREAM_IDLE_STOP_MS,
941 STREAM_SEGMENT_RETENTION_COUNT,
730 942 createVideoStreamSession,
731 943 sendVideoStreamAsset,
732 944 releaseVideoStreamSession,
@@ -10,14 +10,88 @@ const STREAM_ROOT_DIR = path.join(os.tmpdir(), 'filemanager-video-streams');
10const STREAM_SESSION_TTL_MS = 30 * 60 * 1000;
10const STREAM_SESSION_TTL_MS = 30 * 60 * 1000;
11const STREAM_READY_TIMEOUT_MS = 20 * 1000;
11const STREAM_READY_TIMEOUT_MS = 20 * 1000;
12const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
12const STREAM_IDLE_STOP_MS = 5 * 60 * 1000;
13const ESTIMATED_SEGMENT_BYTES = 1024 * 1024;
13const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
14const STREAM_SEGMENT_RETENTION_COUNT = parseSegmentRetentionCount(process.env.VIDEO_STREAM_SEGMENT_RETENTION_COUNT);
14const sessions = new Map();
15const sessions = new Map();
16const leases = new Map();
17const reusableSessionIdBySource = new Map();
15
18
16function parseSegmentRetentionCount (value)
19function parseSegmentRetentionCount (value)
17{
20{
18 const parsed = Number.parseInt(String(value || ''), 10);
21 const parsed = Number.parseInt(String(value || ''), 10);
19 if (Number.isInteger(parsed) && parsed >= 30) return parsed;
22 if (Number.isInteger(parsed) && parsed >= 30) return parsed;
20 return 180;
23 return defaultSegmentRetentionCount();
24}
25
26function readRaspberryPiModel ()
27{
28 try
29 {
30 return fs.readFileSync('/proc/device-tree/model', 'utf8').replace(/\u0000/g, '').trim();
31 } catch
32 {
33 return '';
34 }
35}
36
37function clampNumber (value, min, max)
38{
39 return Math.min(max, Math.max(min, value));
40}
41
42function currentCpuPressureFactor (cpuCount)
43{
44 if (typeof os.loadavg !== 'function') return 1;
45 if (!Number.isFinite(cpuCount) || cpuCount <= 0) return 1;
46
47 const [load1] = os.loadavg();
48 if (!Number.isFinite(load1) || load1 < 0) return 1;
49
50 const saturation = load1 / cpuCount;
51 if (saturation >= 1.5) return 0.72;
52 if (saturation >= 1.2) return 0.8;
53 if (saturation >= 1.0) return 0.88;
54 if (saturation >= 0.8) return 0.94;
55 return 1;
56}
57
58function defaultSegmentRetentionCount ()
59{
60 const totalMemBytes = Number(os.totalmem()) || 0;
61 const freeMemBytes = Number(os.freemem()) || 0;
62 const cpuCount = Math.max(1, ((os.cpus() || []).length || 1));
63 const arch = String(os.arch() || '').toLowerCase();
64 const piModel = readRaspberryPiModel().toLowerCase();
65 const isPi5 = piModel.includes('raspberry pi 5');
66 const isArm = arch === 'arm' || arch === 'arm64' || arch.startsWith('armv');
67 const cpuPressure = currentCpuPressureFactor(cpuCount);
68
69 const freeMemRatio = totalMemBytes > 0 ? (freeMemBytes / totalMemBytes) : 1;
70 const memoryPressure = freeMemRatio < 0.12 ? 0.72 : freeMemRatio < 0.2 ? 0.84 : freeMemRatio < 0.3 ? 0.92 : 1;
71
72 if (isPi5)
73 {
74 const memoryBudgetBytes = Math.floor(totalMemBytes * 0.03);
75 const byMemory = Math.floor(memoryBudgetBytes / ESTIMATED_SEGMENT_BYTES);
76 const byCpu = 120 + ((cpuCount - 2) * 20);
77 const combined = Math.min(byMemory, byCpu);
78 return clampNumber(Math.floor(combined * cpuPressure * memoryPressure), 100, 220);
79 }
80
81 if (isArm)
82 {
83 const memoryBudgetBytes = Math.floor(totalMemBytes * 0.025);
84 const byMemory = Math.floor(memoryBudgetBytes / ESTIMATED_SEGMENT_BYTES);
85 const byCpu = 90 + ((cpuCount - 2) * 14);
86 const combined = Math.min(byMemory, byCpu);
87 return clampNumber(Math.floor(combined * cpuPressure * memoryPressure), 80, 180);
88 }
89
90 const memoryBudgetBytes = Math.floor(totalMemBytes * 0.05);
91 const byMemory = Math.floor(memoryBudgetBytes / ESTIMATED_SEGMENT_BYTES);
92 const byCpu = 140 + ((cpuCount - 4) * 16);
93 const combined = Math.min(byMemory, byCpu);
94 return clampNumber(Math.floor(combined * cpuPressure * memoryPressure), 120, 320);
21}
95}
22
96
23function hasInitialHlsAssets (outputDir)
97function hasInitialHlsAssets (outputDir)
@@ -64,6 +138,13 @@ function ffprobePath ()
64function pruneExpiredSessions ()
138function pruneExpiredSessions ()
65{
139{
66 const now = Date.now();
140 const now = Date.now();
141
142 for (const [leaseId, lease] of leases)
143 {
144 if (lease.expiresAt > now) continue;
145 releaseVideoStreamSession(leaseId);
146 }
147
67 for (const [sessionId, session] of sessions)
148 for (const [sessionId, session] of sessions)
68 {
149 {
69 if (session.expiresAt > now) continue;
150 if (session.expiresAt > now) continue;
@@ -76,6 +157,11 @@ function refreshSession (session)
76 session.expiresAt = Date.now() + STREAM_SESSION_TTL_MS;
157 session.expiresAt = Date.now() + STREAM_SESSION_TTL_MS;
77}
158}
78
159
160function refreshLease (lease)
161{
162 lease.expiresAt = Date.now() + STREAM_SESSION_TTL_MS;
163}
164
79function destroySession (sessionId)
165function destroySession (sessionId)
80{
166{
81 const session = sessions.get(sessionId);
167 const session = sessions.get(sessionId);
@@ -83,6 +169,20 @@ function destroySession (sessionId)
83
169
84 sessions.delete(sessionId);
170 sessions.delete(sessionId);
85
171
172 if (reusableSessionIdBySource.get(session.sourcePath) === session.id)
173 {
174 reusableSessionIdBySource.delete(session.sourcePath);
175 }
176
177 if (session.leaseIds)
178 {
179 for (const leaseId of session.leaseIds)
180 {
181 leases.delete(leaseId);
182 }
183 session.leaseIds.clear();
184 }
185
86 if (session.idleTimer)
186 if (session.idleTimer)
87 {
187 {
88 clearTimeout(session.idleTimer);
188 clearTimeout(session.idleTimer);
@@ -101,10 +201,46 @@ function destroySession (sessionId)
101 return true;
201 return true;
102}
202}
103
203
104function touchSessionActivity (session)
204function getSessionForLease (leaseId)
205{
206 const lease = leases.get(leaseId);
207 if (!lease) return null;
208
209 const session = sessions.get(lease.sessionId);
210 if (!session)
211 {
212 leases.delete(leaseId);
213 return null;
214 }
215
216 return { lease, session };
217}
218
219function createLeaseForSession (session)
220{
221 const leaseId = crypto.randomBytes(18).toString('hex');
222 const lease = {
223 id: leaseId,
224 sessionId: session.id,
225 expiresAt: Date.now() + STREAM_SESSION_TTL_MS,
226 lastActivityAt: Date.now(),
227 lastSegmentIndex: null,
228 };
229 leases.set(leaseId, lease);
230 session.leaseIds.add(leaseId);
231 return lease;
232}
233
234function touchSessionActivity (session, lease)
105{
235{
106 refreshSession(session);
236 refreshSession(session);
107 session.lastActivityAt = Date.now();
237 session.lastActivityAt = Date.now();
238
239 if (lease)
240 {
241 refreshLease(lease);
242 lease.lastActivityAt = Date.now();
243 }
108}
244}
109
245
110function clearIdleStopTimer (session)
246function clearIdleStopTimer (session)
@@ -117,13 +253,13 @@ function clearIdleStopTimer (session)
117function scheduleIdleStop (session)
253function scheduleIdleStop (session)
118{
254{
119 clearIdleStopTimer(session);
255 clearIdleStopTimer(session);
120 if (session.activeRequests > 0) return;
256 if (session.activeRequests > 0 || (session.leaseIds && session.leaseIds.size > 0)) return;
121
257
122 session.idleTimer = setTimeout(() =>
258 session.idleTimer = setTimeout(() =>
123 {
259 {
124 const activeSession = sessions.get(session.id);
260 const activeSession = sessions.get(session.id);
125 if (!activeSession) return;
261 if (!activeSession) return;
126 if (activeSession.activeRequests > 0) return;
262 if (activeSession.activeRequests > 0 || (activeSession.leaseIds && activeSession.leaseIds.size > 0)) return;
127 if (Date.now() - activeSession.lastActivityAt < STREAM_IDLE_STOP_MS)
263 if (Date.now() - activeSession.lastActivityAt < STREAM_IDLE_STOP_MS)
128 {
264 {
129 scheduleIdleStop(activeSession);
265 scheduleIdleStop(activeSession);
@@ -133,18 +269,18 @@ function scheduleIdleStop (session)
133 }, STREAM_IDLE_STOP_MS);
269 }, STREAM_IDLE_STOP_MS);
134}
270}
135
271
136function startAssetRequest (session)
272function startAssetRequest (session, lease)
137{
273{
138 session.activeRequests += 1;
274 session.activeRequests += 1;
139 touchSessionActivity(session);
275 touchSessionActivity(session, lease);
140 clearIdleStopTimer(session);
276 clearIdleStopTimer(session);
141}
277}
142
278
143function finishAssetRequest (session)
279function finishAssetRequest (session, lease)
144{
280{
145 if (session.activeRequests > 0) session.activeRequests -= 1;
281 if (session.activeRequests > 0) session.activeRequests -= 1;
146 touchSessionActivity(session);
282 touchSessionActivity(session, lease);
147 if (session.activeRequests === 0) scheduleIdleStop(session);
283 if (session.activeRequests === 0 && session.leaseIds.size === 0) scheduleIdleStop(session);
148}
284}
149
285
150function parseSegmentIndexFromAssetPath (assetPath)
286function parseSegmentIndexFromAssetPath (assetPath)
@@ -189,6 +325,10 @@ function runSegmentPrune (session)
189 .then(() =>
325 .then(() =>
190 {
326 {
191 session.segmentPrunedUntil = Math.max(session.segmentPrunedUntil, cutoff);
327 session.segmentPrunedUntil = Math.max(session.segmentPrunedUntil, cutoff);
328 if (session.segmentPrunedUntil >= 0 && reusableSessionIdBySource.get(session.sourcePath) === session.id)
329 {
330 reusableSessionIdBySource.delete(session.sourcePath);
331 }
192 })
332 })
193 .catch(() =>
333 .catch(() =>
194 {
334 {
@@ -204,11 +344,31 @@ function runSegmentPrune (session)
204 });
344 });
205}
345}
206
346
207function scheduleSegmentPrune (session, segmentIndex)
347function slowestLeaseSegmentIndex (session)
348{
349 if (!session.leaseIds.size) return null;
350
351 let minIndex = null;
352 for (const leaseId of session.leaseIds)
353 {
354 const lease = leases.get(leaseId);
355 if (!lease) continue;
356 if (!Number.isInteger(lease.lastSegmentIndex)) return null;
357 if (minIndex === null || lease.lastSegmentIndex < minIndex) minIndex = lease.lastSegmentIndex;
358 }
359
360 return minIndex;
361}
362
363function scheduleSegmentPrune (session, lease, segmentIndex)
208{
364{
209 if (!Number.isInteger(segmentIndex) || segmentIndex < 0) return;
365 if (!Number.isInteger(segmentIndex) || segmentIndex < 0) return;
210
366
211 const pruneCutoff = segmentIndex - STREAM_SEGMENT_RETENTION_COUNT;
367 lease.lastSegmentIndex = segmentIndex;
368 const slowestIndex = slowestLeaseSegmentIndex(session);
369 if (!Number.isInteger(slowestIndex)) return;
370
371 const pruneCutoff = slowestIndex - STREAM_SEGMENT_RETENTION_COUNT;
212 if (pruneCutoff <= session.segmentPrunedUntil) return;
372 if (pruneCutoff <= session.segmentPrunedUntil) return;
213
373
214 session.segmentPrunePendingCutoff = Math.max(session.segmentPrunePendingCutoff, pruneCutoff);
374 session.segmentPrunePendingCutoff = Math.max(session.segmentPrunePendingCutoff, pruneCutoff);
@@ -602,39 +762,72 @@ async function createVideoStreamSession (sourcePath)
602 pruneExpiredSessions();
762 pruneExpiredSessions();
603 ensureStreamRootDir();
763 ensureStreamRootDir();
604
764
605 const sessionId = crypto.randomBytes(18).toString('hex');
765 const sourceKey = String(sourcePath || '');
606 const outputDir = path.join(STREAM_ROOT_DIR, sessionId);
766 let session = null;
607 fs.mkdirSync(outputDir, { recursive: true });
608
767
609 const session = {
768 const reusableSessionId = reusableSessionIdBySource.get(sourceKey);
610 id: sessionId,
769 if (reusableSessionId)
611 sourcePath,
770 {
612 outputDir,
771 const reusable = sessions.get(reusableSessionId);
613 expiresAt: Date.now() + STREAM_SESSION_TTL_MS,
772 if (reusable && !reusable.error && reusable.segmentPrunedUntil < 0)
614 process: null,
773 {
615 error: null,
774 session = reusable;
616 subtitleTracks: [],
775 } else
617 subtitleSummary: { total: 0, converted: 0, skipped: 0 },
776 {
618 readyPromise: null,
777 reusableSessionIdBySource.delete(sourceKey);
619 activeRequests: 0,
778 }
620 lastActivityAt: Date.now(),
779 }
621 idleTimer: null,
622 segmentPruneRunning: false,
623 segmentPrunePendingCutoff: -1,
624 segmentPrunedUntil: -1,
625 };
626
780
627 session.readyPromise = initialiseSession(session).catch((err) =>
781 if (!session)
628 {
782 {
629 session.error = err;
783 const sessionId = crypto.randomBytes(18).toString('hex');
630 throw err;
784 const outputDir = path.join(STREAM_ROOT_DIR, sessionId);
631 });
785 fs.mkdirSync(outputDir, { recursive: true });
786
787 session = {
788 id: sessionId,
789 sourcePath: sourceKey,
790 outputDir,
791 expiresAt: Date.now() + STREAM_SESSION_TTL_MS,
792 process: null,
793 error: null,
794 subtitleTracks: [],
795 subtitleSummary: { total: 0, converted: 0, skipped: 0 },
796 readyPromise: null,
797 activeRequests: 0,
798 lastActivityAt: Date.now(),
799 idleTimer: null,
800 segmentPruneRunning: false,
801 segmentPrunePendingCutoff: -1,
802 segmentPrunedUntil: -1,
803 leaseIds: new Set(),
804 };
632
805
633 sessions.set(sessionId, session);
806 session.readyPromise = initialiseSession(session).catch((err) =>
634 await session.readyPromise;
807 {
808 session.error = err;
809 throw err;
810 });
811
812 sessions.set(session.id, session);
813 reusableSessionIdBySource.set(sourceKey, session.id);
814 }
815
816 if (!session.leaseIds) session.leaseIds = new Set();
817 const lease = createLeaseForSession(session);
818 touchSessionActivity(session, lease);
819
820 try
821 {
822 await session.readyPromise;
823 } catch (err)
824 {
825 releaseVideoStreamSession(lease.id);
826 throw err;
827 }
635
828
636 return {
829 return {
637 sessionId,
830 sessionId: lease.id,
638 durationSeconds: session.durationSeconds,
831 durationSeconds: session.durationSeconds,
639 subtitleTracks: session.subtitleTracks.map((track) => ({ ...track })),
832 subtitleTracks: session.subtitleTracks.map((track) => ({ ...track })),
640 subtitleSummary: {
833 subtitleSummary: {
@@ -659,10 +852,12 @@ function contentTypeForAsset (assetPath)
659async function sendVideoStreamAsset (sessionId, assetPath, req, res)
852async function sendVideoStreamAsset (sessionId, assetPath, req, res)
660{
853{
661 pruneExpiredSessions();
854 pruneExpiredSessions();
662 const session = sessions.get(sessionId);
855 const active = getSessionForLease(sessionId);
663 if (!session) throw makeHttpError(404, 'Video stream session not found.');
856 if (!active) throw makeHttpError(404, 'Video stream session not found.');
664
857
665 touchSessionActivity(session);
858 const { session, lease } = active;
859
860 touchSessionActivity(session, lease);
666 await session.readyPromise;
861 await session.readyPromise;
667
862
668 const relativeAssetPath = String(assetPath || 'master.m3u8').replace(/^[/\\]+/, '');
863 const relativeAssetPath = String(assetPath || 'master.m3u8').replace(/^[/\\]+/, '');
@@ -674,20 +869,20 @@ async function sendVideoStreamAsset (sessionId, assetPath, req, res)
674 }
869 }
675
870
676 const segmentIndex = parseSegmentIndexFromAssetPath(relativeAssetPath);
871 const segmentIndex = parseSegmentIndexFromAssetPath(relativeAssetPath);
677 if (segmentIndex !== null) scheduleSegmentPrune(session, segmentIndex);
872 if (segmentIndex !== null) scheduleSegmentPrune(session, lease, segmentIndex);
678
873
679 await waitForFile(resolvedPath, session);
874 await waitForFile(resolvedPath, session);
680 res.set('Cache-Control', 'private, no-store');
875 res.set('Cache-Control', 'private, no-store');
681 res.type(contentTypeForAsset(resolvedPath));
876 res.type(contentTypeForAsset(resolvedPath));
682
877
683 startAssetRequest(session);
878 startAssetRequest(session, lease);
684
879
685 let settled = false;
880 let settled = false;
686 const finish = () =>
881 const finish = () =>
687 {
882 {
688 if (settled) return;
883 if (settled) return;
689 settled = true;
884 settled = true;
690 finishAssetRequest(session);
885 finishAssetRequest(session, lease);
691 };
886 };
692
887
693 req.on('aborted', finish);
888 req.on('aborted', finish);
@@ -719,14 +914,31 @@ async function sendVideoStreamAsset (sessionId, assetPath, req, res)
719
914
720function releaseVideoStreamSession (sessionId)
915function releaseVideoStreamSession (sessionId)
721{
916{
722 pruneExpiredSessions();
917 const leaseId = String(sessionId || '');
723 return destroySession(String(sessionId || ''));
918 const active = getSessionForLease(leaseId);
919 if (!active)
920 {
921 leases.delete(leaseId);
922 return false;
923 }
924
925 const { session } = active;
926 leases.delete(leaseId);
927 session.leaseIds.delete(leaseId);
928
929 if (session.leaseIds.size === 0 && session.activeRequests === 0)
930 {
931 return destroySession(session.id);
932 }
933
934 return true;
724}
935}
725
936
726module.exports = {
937module.exports = {
727 STREAM_ROOT_DIR,
938 STREAM_ROOT_DIR,
728 STREAM_SESSION_TTL_MS,
939 STREAM_SESSION_TTL_MS,
729 STREAM_IDLE_STOP_MS,
940 STREAM_IDLE_STOP_MS,
941 STREAM_SEGMENT_RETENTION_COUNT,
730 createVideoStreamSession,
942 createVideoStreamSession,
731 sendVideoStreamAsset,
943 sendVideoStreamAsset,
732 releaseVideoStreamSession,
944 releaseVideoStreamSession,
tests/video-stream.test.js +83 −4
@@ -380,17 +380,32 @@ describe('video stream bootstrap', () =>
380 380 });
381 381
382 382 const streamModule = require('../server/video-stream');
383 const beforeDirs = new Set(
384 fs.existsSync(streamModule.STREAM_ROOT_DIR)
385 ? fs.readdirSync(streamModule.STREAM_ROOT_DIR)
386 : []
387 );
383 388 const session = await streamModule.createVideoStreamSession(sourceFile);
384 const sessionDir = path.join(streamModule.STREAM_ROOT_DIR, session.sessionId);
385
386 for (let segmentIndex = 0; segmentIndex <= 220; segmentIndex += 1)
389 const afterDirs = fs.readdirSync(streamModule.STREAM_ROOT_DIR);
390 const newDirName = afterDirs.find((dirName) => !beforeDirs.has(dirName));
391 expect(newDirName).toEqual(expect.any(String));
392 const sessionDir = path.join(streamModule.STREAM_ROOT_DIR, String(newDirName || ''));
393 expect(Number.isInteger(streamModule.STREAM_SEGMENT_RETENTION_COUNT)).toBe(true);
394 const targetSegmentIndex = streamModule.STREAM_SEGMENT_RETENTION_COUNT + 40;
395
396 for (let segmentIndex = 0; segmentIndex <= targetSegmentIndex; segmentIndex += 1)
387 397 {
388 398 const fileName = `stream_0_segment_${String(segmentIndex).padStart(6, '0')}.ts`;
389 399 fs.writeFileSync(path.join(sessionDir, fileName), `segment-${segmentIndex}`);
390 400 }
391 401
392 402 const { req, res } = makeReqRes();
393 await streamModule.sendVideoStreamAsset(session.sessionId, 'stream_0_segment_000220.ts', req, res);
403 await streamModule.sendVideoStreamAsset(
404 session.sessionId,
405 `stream_0_segment_${String(targetSegmentIndex).padStart(6, '0')}.ts`,
406 req,
407 res
408 );
394 409
395 410 await new Promise((resolve) => setTimeout(resolve, 80));
396 411
@@ -399,4 +414,68 @@ describe('video stream bootstrap', () =>
399 414
400 415 streamModule.releaseVideoStreamSession(session.sessionId);
401 416 });
417
418 test('createVideoStreamSession reuses one transcode for concurrent viewers of the same file', async () =>
419 {
420 mockSpawn.mockImplementation((command, args) =>
421 {
422 const child = makeChildProcess();
423 const exeName = path.basename(String(command || '')).toLowerCase();
424
425 if (exeName.includes('ffprobe'))
426 {
427 process.nextTick(() =>
428 {
429 child.stdout.emit('data', Buffer.from(JSON.stringify({
430 streams: [
431 { codec_type: 'video', codec_name: 'h264', tags: {}, disposition: {} },
432 { codec_type: 'audio', codec_name: 'aac', tags: {}, disposition: { default: 1 } },
433 ],
434 })));
435 child.emit('close', 0);
436 });
437 return child;
438 }
439
440 const playlistPath = String(args[args.length - 1] || '');
441 const outputDir = path.dirname(playlistPath);
442 process.nextTick(() =>
443 {
444 fs.mkdirSync(outputDir, { recursive: true });
445 fs.writeFileSync(path.join(outputDir, 'master.m3u8'), '#EXTM3U\n');
446 });
447 return child;
448 });
449
450 const streamModule = require('../server/video-stream');
451 const firstViewer = await streamModule.createVideoStreamSession(sourceFile);
452 const secondViewer = await streamModule.createVideoStreamSession(sourceFile);
453
454 expect(firstViewer.sessionId).not.toBe(secondViewer.sessionId);
455
456 const ffmpegSpawns = mockSpawn.mock.calls.filter(([command]) =>
457 path.basename(String(command || '')).toLowerCase().includes('ffmpeg')
458 );
459 expect(ffmpegSpawns).toHaveLength(1);
460
461 const firstReqRes = makeReqRes();
462 await streamModule.sendVideoStreamAsset(firstViewer.sessionId, 'master.m3u8', firstReqRes.req, firstReqRes.res);
463
464 const secondReqRes = makeReqRes();
465 await streamModule.sendVideoStreamAsset(secondViewer.sessionId, 'master.m3u8', secondReqRes.req, secondReqRes.res);
466
467 streamModule.releaseVideoStreamSession(firstViewer.sessionId);
468
469 const stillActiveReqRes = makeReqRes();
470 await expect(
471 streamModule.sendVideoStreamAsset(secondViewer.sessionId, 'master.m3u8', stillActiveReqRes.req, stillActiveReqRes.res)
472 ).resolves.toBeUndefined();
473
474 const releasedReqRes = makeReqRes();
475 await expect(
476 streamModule.sendVideoStreamAsset(firstViewer.sessionId, 'master.m3u8', releasedReqRes.req, releasedReqRes.res)
477 ).rejects.toMatchObject({ status: 404 });
478
479 streamModule.releaseVideoStreamSession(secondViewer.sessionId);
480 });
402 481 });
@@ -380,17 +380,32 @@ describe('video stream bootstrap', () =>
380 });
380 });
381
381
382 const streamModule = require('../server/video-stream');
382 const streamModule = require('../server/video-stream');
383 const beforeDirs = new Set(
384 fs.existsSync(streamModule.STREAM_ROOT_DIR)
385 ? fs.readdirSync(streamModule.STREAM_ROOT_DIR)
386 : []
387 );
383 const session = await streamModule.createVideoStreamSession(sourceFile);
388 const session = await streamModule.createVideoStreamSession(sourceFile);
384 const sessionDir = path.join(streamModule.STREAM_ROOT_DIR, session.sessionId);
389 const afterDirs = fs.readdirSync(streamModule.STREAM_ROOT_DIR);
385
390 const newDirName = afterDirs.find((dirName) => !beforeDirs.has(dirName));
386 for (let segmentIndex = 0; segmentIndex <= 220; segmentIndex += 1)
391 expect(newDirName).toEqual(expect.any(String));
392 const sessionDir = path.join(streamModule.STREAM_ROOT_DIR, String(newDirName || ''));
393 expect(Number.isInteger(streamModule.STREAM_SEGMENT_RETENTION_COUNT)).toBe(true);
394 const targetSegmentIndex = streamModule.STREAM_SEGMENT_RETENTION_COUNT + 40;
395
396 for (let segmentIndex = 0; segmentIndex <= targetSegmentIndex; segmentIndex += 1)
387 {
397 {
388 const fileName = `stream_0_segment_${String(segmentIndex).padStart(6, '0')}.ts`;
398 const fileName = `stream_0_segment_${String(segmentIndex).padStart(6, '0')}.ts`;
389 fs.writeFileSync(path.join(sessionDir, fileName), `segment-${segmentIndex}`);
399 fs.writeFileSync(path.join(sessionDir, fileName), `segment-${segmentIndex}`);
390 }
400 }
391
401
392 const { req, res } = makeReqRes();
402 const { req, res } = makeReqRes();
393 await streamModule.sendVideoStreamAsset(session.sessionId, 'stream_0_segment_000220.ts', req, res);
403 await streamModule.sendVideoStreamAsset(
404 session.sessionId,
405 `stream_0_segment_${String(targetSegmentIndex).padStart(6, '0')}.ts`,
406 req,
407 res
408 );
394
409
395 await new Promise((resolve) => setTimeout(resolve, 80));
410 await new Promise((resolve) => setTimeout(resolve, 80));
396
411
@@ -399,4 +414,68 @@ describe('video stream bootstrap', () =>
399
414
400 streamModule.releaseVideoStreamSession(session.sessionId);
415 streamModule.releaseVideoStreamSession(session.sessionId);
401 });
416 });
417
418 test('createVideoStreamSession reuses one transcode for concurrent viewers of the same file', async () =>
419 {
420 mockSpawn.mockImplementation((command, args) =>
421 {
422 const child = makeChildProcess();
423 const exeName = path.basename(String(command || '')).toLowerCase();
424
425 if (exeName.includes('ffprobe'))
426 {
427 process.nextTick(() =>
428 {
429 child.stdout.emit('data', Buffer.from(JSON.stringify({
430 streams: [
431 { codec_type: 'video', codec_name: 'h264', tags: {}, disposition: {} },
432 { codec_type: 'audio', codec_name: 'aac', tags: {}, disposition: { default: 1 } },
433 ],
434 })));
435 child.emit('close', 0);
436 });
437 return child;
438 }
439
440 const playlistPath = String(args[args.length - 1] || '');
441 const outputDir = path.dirname(playlistPath);
442 process.nextTick(() =>
443 {
444 fs.mkdirSync(outputDir, { recursive: true });
445 fs.writeFileSync(path.join(outputDir, 'master.m3u8'), '#EXTM3U\n');
446 });
447 return child;
448 });
449
450 const streamModule = require('../server/video-stream');
451 const firstViewer = await streamModule.createVideoStreamSession(sourceFile);
452 const secondViewer = await streamModule.createVideoStreamSession(sourceFile);
453
454 expect(firstViewer.sessionId).not.toBe(secondViewer.sessionId);
455
456 const ffmpegSpawns = mockSpawn.mock.calls.filter(([command]) =>
457 path.basename(String(command || '')).toLowerCase().includes('ffmpeg')
458 );
459 expect(ffmpegSpawns).toHaveLength(1);
460
461 const firstReqRes = makeReqRes();
462 await streamModule.sendVideoStreamAsset(firstViewer.sessionId, 'master.m3u8', firstReqRes.req, firstReqRes.res);
463
464 const secondReqRes = makeReqRes();
465 await streamModule.sendVideoStreamAsset(secondViewer.sessionId, 'master.m3u8', secondReqRes.req, secondReqRes.res);
466
467 streamModule.releaseVideoStreamSession(firstViewer.sessionId);
468
469 const stillActiveReqRes = makeReqRes();
470 await expect(
471 streamModule.sendVideoStreamAsset(secondViewer.sessionId, 'master.m3u8', stillActiveReqRes.req, stillActiveReqRes.res)
472 ).resolves.toBeUndefined();
473
474 const releasedReqRes = makeReqRes();
475 await expect(
476 streamModule.sendVideoStreamAsset(firstViewer.sessionId, 'master.m3u8', releasedReqRes.req, releasedReqRes.res)
477 ).rejects.toMatchObject({ status: 404 });
478
479 streamModule.releaseVideoStreamSession(secondViewer.sessionId);
480 });
402});
481});