BlueGit

timfa filemanager

Merge fix/subtitle-timeout-recovery-20260525

Improves subtitle reliability by recovering from hung frame fetches and validating timeout retry behavior with regression tests.

timfa Tim Falken · pushed by timfa

Authored 2026-05-25 · 5 months ago

Hash
0f3aa256202108104505c795bc4d8f1ba08304a6
5 files changed +309 −44
.github/technical-reference.md +1 −1
@@ -182,7 +182,7 @@
182 182 ## public/shared-video-player.js
183 183 - applyDefaultTextTrackPrefs() - Writes the default Video.js caption override settings (transparent background, uniform edge style) so all streamed players start from the same text-track baseline.
184 184 - bindTrackPrefs(player, prefs, savePrefs) - Restores subtitle/audio track selections by track name and persists later track changes; shared by the authenticated player and the share page so both resolve track matches the same way.
185 - bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, renders seek-bar triangle markers for last requested caption start time (`>`) and last received/ready caption start time (`<`), and disables the track after the first hard failure.
185 - bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), extends prefetch target to at least the latest buffered video end (not only fixed lookahead seconds), coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), applies per-request timeouts (`currentRequestTimeoutMs`, `prefetchRequestTimeoutMs`) so hung subtitle frame requests cannot stall updates indefinitely, paces prefetch polling (`prefetchRequestIntervalMs`) to reduce request bursts, logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, renders seek-bar bracket markers for last requested point (`]`) and last received/ready point (`[`), advances markers even when responses are 204/no-frame, and disables the track after the first hard failure.
186 186 - configureVideoJsBuffering(videojs, opts) - Applies the long forward-buffer settings used for file playback when VHS exposes those knobs.
187 187 - ensureVideoPlaybackUi(player, durationSeconds) - Forces file streams into VOD UI mode, normalises the reported duration, stops live-tracker state, and hides the live badge when present.
188 188
@@ -182,7 +182,7 @@
182## public/shared-video-player.js
182## public/shared-video-player.js
183- applyDefaultTextTrackPrefs() - Writes the default Video.js caption override settings (transparent background, uniform edge style) so all streamed players start from the same text-track baseline.
183- applyDefaultTextTrackPrefs() - Writes the default Video.js caption override settings (transparent background, uniform edge style) so all streamed players start from the same text-track baseline.
184- bindTrackPrefs(player, prefs, savePrefs) - Restores subtitle/audio track selections by track name and persists later track changes; shared by the authenticated player and the share page so both resolve track matches the same way.
184- bindTrackPrefs(player, prefs, savePrefs) - Restores subtitle/audio track selections by track name and persists later track changes; shared by the authenticated player and the share page so both resolve track matches the same way.
185- bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, renders seek-bar triangle markers for last requested caption start time (`>`) and last received/ready caption start time (`<`), and disables the track after the first hard failure.
185- bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), extends prefetch target to at least the latest buffered video end (not only fixed lookahead seconds), coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), applies per-request timeouts (`currentRequestTimeoutMs`, `prefetchRequestTimeoutMs`) so hung subtitle frame requests cannot stall updates indefinitely, paces prefetch polling (`prefetchRequestIntervalMs`) to reduce request bursts, logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, renders seek-bar bracket markers for last requested point (`]`) and last received/ready point (`[`), advances markers even when responses are 204/no-frame, and disables the track after the first hard failure.
186- configureVideoJsBuffering(videojs, opts) - Applies the long forward-buffer settings used for file playback when VHS exposes those knobs.
186- configureVideoJsBuffering(videojs, opts) - Applies the long forward-buffer settings used for file playback when VHS exposes those knobs.
187- ensureVideoPlaybackUi(player, durationSeconds) - Forces file streams into VOD UI mode, normalises the reported duration, stops live-tracker state, and hides the live badge when present.
187- ensureVideoPlaybackUi(player, durationSeconds) - Forces file streams into VOD UI mode, normalises the reported duration, stops live-tracker state, and hides the live badge when present.
188
188
public/app.js +2 −0
@@ -109,6 +109,7 @@ const FOLDER_GROUP_OPTIONS = [
109 109 const VIDEO_FORWARD_BUFFER_SECONDS = 60 * 60;
110 110 const VIDEO_FORWARD_BUFFER_OVERRIDE_ENABLED = false;
111 111 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS = 15;
112 const IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS = 40;
112 113 let videoJsLoaderPromise = null;
113 114 let threePreviewModulesPromise = null;
114 115
@@ -425,6 +426,7 @@ function bindImageSubtitleOverlay (player, videojs, prefs, session, item)
425 426 return shared.bindImageSubtitleOverlay(player, videojs, prefs, session, {
426 427 savePrefs: saveUserVideoPrefs,
427 428 lookaheadSeconds: IMAGE_SUBTITLE_LOOKAHEAD_SECONDS,
429 prefetchRequestIntervalMs: IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS,
428 430 sendSubtitleLogBatch,
429 431 buildFrameRequest: ({ currentTime, streamIndex, session: activeSession }) => ({
430 432 url: `/api/files/video-stream-subtitle-frame?filePath=${encodeURIComponent(item.path)}&streamIndex=${streamIndex}&timeSeconds=${encodeURIComponent(String(currentTime))}&videoWidth=${encodeURIComponent(String(activeSession.videoWidth || ''))}&videoHeight=${encodeURIComponent(String(activeSession.videoHeight || ''))}`,
@@ -109,6 +109,7 @@ const FOLDER_GROUP_OPTIONS = [
109const VIDEO_FORWARD_BUFFER_SECONDS = 60 * 60;
109const VIDEO_FORWARD_BUFFER_SECONDS = 60 * 60;
110const VIDEO_FORWARD_BUFFER_OVERRIDE_ENABLED = false;
110const VIDEO_FORWARD_BUFFER_OVERRIDE_ENABLED = false;
111const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS = 15;
111const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS = 15;
112const IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS = 40;
112let videoJsLoaderPromise = null;
113let videoJsLoaderPromise = null;
113let threePreviewModulesPromise = null;
114let threePreviewModulesPromise = null;
114
115
@@ -425,6 +426,7 @@ function bindImageSubtitleOverlay (player, videojs, prefs, session, item)
425 return shared.bindImageSubtitleOverlay(player, videojs, prefs, session, {
426 return shared.bindImageSubtitleOverlay(player, videojs, prefs, session, {
426 savePrefs: saveUserVideoPrefs,
427 savePrefs: saveUserVideoPrefs,
427 lookaheadSeconds: IMAGE_SUBTITLE_LOOKAHEAD_SECONDS,
428 lookaheadSeconds: IMAGE_SUBTITLE_LOOKAHEAD_SECONDS,
429 prefetchRequestIntervalMs: IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS,
428 sendSubtitleLogBatch,
430 sendSubtitleLogBatch,
429 buildFrameRequest: ({ currentTime, streamIndex, session: activeSession }) => ({
431 buildFrameRequest: ({ currentTime, streamIndex, session: activeSession }) => ({
430 url: `/api/files/video-stream-subtitle-frame?filePath=${encodeURIComponent(item.path)}&streamIndex=${streamIndex}&timeSeconds=${encodeURIComponent(String(currentTime))}&videoWidth=${encodeURIComponent(String(activeSession.videoWidth || ''))}&videoHeight=${encodeURIComponent(String(activeSession.videoHeight || ''))}`,
432 url: `/api/files/video-stream-subtitle-frame?filePath=${encodeURIComponent(item.path)}&streamIndex=${streamIndex}&timeSeconds=${encodeURIComponent(String(currentTime))}&videoWidth=${encodeURIComponent(String(activeSession.videoWidth || ''))}&videoHeight=${encodeURIComponent(String(activeSession.videoHeight || ''))}`,
public/share.html +2 −0
@@ -250,6 +250,7 @@
250 250 const token = location.pathname.split('/').pop();
251 251 const VIDEO_FORWARD_BUFFER_SECONDS = 60 * 60;
252 252 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS = 15;
253 const IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS = 40;
253 254 const SHARE_VIDEO_PREFS_KEY = 'filemanager.share-video-prefs.v1';
254 255 let expiresAt = null;
255 256 let timerInterval = null;
@@ -463,6 +464,7 @@
463 464 ? shared.bindImageSubtitleOverlay(shareVideoPlayer, window.videojs, prefs, data, {
464 465 savePrefs: saveShareVideoPrefs,
465 466 lookaheadSeconds: IMAGE_SUBTITLE_LOOKAHEAD_SECONDS,
467 prefetchRequestIntervalMs: IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS,
466 468 sendSubtitleLogBatch: (entries) =>
467 469 {
468 470 if (!Array.isArray(entries) || !entries.length) return Promise.resolve();
@@ -250,6 +250,7 @@
250 const token = location.pathname.split('/').pop();
250 const token = location.pathname.split('/').pop();
251 const VIDEO_FORWARD_BUFFER_SECONDS = 60 * 60;
251 const VIDEO_FORWARD_BUFFER_SECONDS = 60 * 60;
252 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS = 15;
252 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS = 15;
253 const IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS = 40;
253 const SHARE_VIDEO_PREFS_KEY = 'filemanager.share-video-prefs.v1';
254 const SHARE_VIDEO_PREFS_KEY = 'filemanager.share-video-prefs.v1';
254 let expiresAt = null;
255 let expiresAt = null;
255 let timerInterval = null;
256 let timerInterval = null;
@@ -463,6 +464,7 @@
463 ? shared.bindImageSubtitleOverlay(shareVideoPlayer, window.videojs, prefs, data, {
464 ? shared.bindImageSubtitleOverlay(shareVideoPlayer, window.videojs, prefs, data, {
464 savePrefs: saveShareVideoPrefs,
465 savePrefs: saveShareVideoPrefs,
465 lookaheadSeconds: IMAGE_SUBTITLE_LOOKAHEAD_SECONDS,
466 lookaheadSeconds: IMAGE_SUBTITLE_LOOKAHEAD_SECONDS,
467 prefetchRequestIntervalMs: IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS,
466 sendSubtitleLogBatch: (entries) =>
468 sendSubtitleLogBatch: (entries) =>
467 {
469 {
468 if (!Array.isArray(entries) || !entries.length) return Promise.resolve();
470 if (!Array.isArray(entries) || !entries.length) return Promise.resolve();
public/shared-video-player.js +148 −42
@@ -12,6 +12,9 @@
12 12 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_DEFAULT = 15;
13 13 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_MAX = 30;
14 14 const IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS = 0.5;
15 const IMAGE_SUBTITLE_CURRENT_REQUEST_TIMEOUT_MS_DEFAULT = 4000;
16 const IMAGE_SUBTITLE_PREFETCH_REQUEST_TIMEOUT_MS_DEFAULT = 4000;
17 const IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS_DEFAULT = 0;
15 18
16 19 function roundLogNumber (value, digits = 3)
17 20 {
@@ -261,6 +264,19 @@
261 264 const lookaheadSeconds = Number.isFinite(requestedLookaheadSeconds)
262 265 ? Math.max(0, Math.min(IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_MAX, requestedLookaheadSeconds))
263 266 : IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_DEFAULT;
267 const requestedCurrentRequestTimeoutMs = Number(opts.currentRequestTimeoutMs);
268 const currentRequestTimeoutMs = Number.isFinite(requestedCurrentRequestTimeoutMs)
269 ? Math.max(250, requestedCurrentRequestTimeoutMs)
270 : IMAGE_SUBTITLE_CURRENT_REQUEST_TIMEOUT_MS_DEFAULT;
271 const requestedPrefetchRequestTimeoutMs = Number(opts.prefetchRequestTimeoutMs);
272 const prefetchRequestTimeoutMs = Number.isFinite(requestedPrefetchRequestTimeoutMs)
273 ? Math.max(250, requestedPrefetchRequestTimeoutMs)
274 : IMAGE_SUBTITLE_PREFETCH_REQUEST_TIMEOUT_MS_DEFAULT;
275 const requestedPrefetchRequestIntervalMs = Number(opts.prefetchRequestIntervalMs);
276 const prefetchRequestIntervalMs = Number.isFinite(requestedPrefetchRequestIntervalMs)
277 ? Math.max(0, requestedPrefetchRequestIntervalMs)
278 : IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS_DEFAULT;
279 const logToConsole = opts.logToConsole !== false;
264 280 if (!buildFrameRequest) return null;
265 281
266 282 const trackByStreamIndex = new Map(imageTracks.map((track) => [track.streamIndex, track]));
@@ -338,12 +354,15 @@
338 354 playerTimeSeconds: roundLogNumber(player.currentTime()),
339 355 ...details,
340 356 };
341 try
342 {
343 console.log('[subtitle-log]', entry);
344 } catch
357 if (logToConsole)
345 358 {
346 // Ignore console availability issues.
359 try
360 {
361 console.log('[subtitle-log]', entry);
362 } catch
363 {
364 // Ignore console availability issues.
365 }
347 366 }
348 367
349 368 if (!sendSubtitleLogBatch) return;
@@ -374,6 +393,22 @@
374 393 return !!(error && typeof error === 'object' && error.name === 'AbortError');
375 394 };
376 395
396 const armRequestTimeout = (controller, timeoutMs, onTimeout) =>
397 {
398 if (!controller || !Number.isFinite(timeoutMs) || timeoutMs <= 0) return null;
399 return window.setTimeout(() =>
400 {
401 if (typeof onTimeout === 'function') onTimeout();
402 abortController(controller);
403 }, timeoutMs);
404 };
405
406 const clearRequestTimeout = (timeoutId) =>
407 {
408 if (timeoutId === null || timeoutId === undefined) return;
409 window.clearTimeout(timeoutId);
410 };
411
377 412 const cancelCurrentFrameRequest = () =>
378 413 {
379 414 currentFrameToken++;
@@ -490,27 +525,30 @@
490 525 seekMarkerContainerEl.style.pointerEvents = 'none';
491 526 seekMarkerContainerEl.style.zIndex = '5';
492 527
493 const buildTriangle = (direction) =>
528 const buildBracketMarker = (direction) =>
494 529 {
495 530 const markerEl = document.createElement('span');
496 531 markerEl.className = `vjs-image-subtitle-seek-marker vjs-image-subtitle-seek-marker-${direction}`;
497 532 markerEl.style.position = 'absolute';
498 markerEl.style.width = '12px';
533 markerEl.style.top = '50%';
534 markerEl.style.width = '10px';
499 535 markerEl.style.height = '10px';
500 markerEl.style.transform = 'translateX(-50%)';
501 markerEl.style.background = '#fff';
502 markerEl.style.border = '1px solid #000';
536 markerEl.style.transform = 'translate(-50%, -50%)';
537 markerEl.style.color = '#fff';
538 markerEl.style.fontFamily = 'monospace';
539 markerEl.style.fontSize = '12px';
540 markerEl.style.fontWeight = '700';
541 markerEl.style.lineHeight = '10px';
542 markerEl.style.textAlign = 'center';
543 markerEl.style.textShadow = '-1px 0 #000, 0 1px #000, 1px 0 #000, 0 -1px #000';
503 544 markerEl.style.display = 'none';
504 markerEl.style.boxSizing = 'border-box';
505 markerEl.style.clipPath = direction === 'left'
506 ? 'polygon(100% 0, 0 50%, 100% 100%)'
507 : 'polygon(0 0, 100% 50%, 0 100%)';
545 markerEl.textContent = direction === 'left' ? '[' : ']';
508 546 markerEl.setAttribute('aria-hidden', 'true');
509 547 return markerEl;
510 548 };
511 549
512 seekMarkerReceivedEl = buildTriangle('left');
513 seekMarkerRequestedEl = buildTriangle('right');
550 seekMarkerReceivedEl = buildBracketMarker('left');
551 seekMarkerRequestedEl = buildBracketMarker('right');
514 552 seekMarkerContainerEl.appendChild(seekMarkerReceivedEl);
515 553 seekMarkerContainerEl.appendChild(seekMarkerRequestedEl);
516 554
@@ -622,10 +660,35 @@
622 660 {
623 661 if (selectedStreamIndex === null) return;
624 662 if (trackDisabled) return;
625 if (lookaheadSeconds <= 0) return;
663
664 const bufferedVideoEndSeconds = (() =>
665 {
666 const fromPlayer = Number(player.bufferedEnd && player.bufferedEnd());
667 if (Number.isFinite(fromPlayer) && fromPlayer >= 0) return fromPlayer;
668
669 const tech = typeof player.tech === 'function' ? player.tech(true) : null;
670 const mediaEl = tech && typeof tech.el === 'function'
671 ? tech.el()
672 : (player.el && player.el() ? player.el().querySelector('video') : null);
673
674 const ranges = mediaEl && mediaEl.buffered;
675 if (!ranges || typeof ranges.length !== 'number' || ranges.length <= 0 || typeof ranges.end !== 'function')
676 {
677 return null;
678 }
679
680 try
681 {
682 const endTime = Number(ranges.end(ranges.length - 1));
683 return Number.isFinite(endTime) && endTime >= 0 ? endTime : null;
684 } catch
685 {
686 return null;
687 }
688 })();
626 689
627 690 const startTime = currentTime + IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS;
628 const endTime = currentTime + lookaheadSeconds;
691 const endTime = Math.max(currentTime + lookaheadSeconds, Number.isFinite(bufferedVideoEndSeconds) ? bufferedVideoEndSeconds : -1);
629 692 if (!Number.isFinite(startTime) || !Number.isFinite(endTime) || endTime <= startTime) return;
630 693
631 694 if (reset || prefetchedThroughTimeSeconds < 0 || prefetchedThroughTimeSeconds < currentTime - IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS)
@@ -670,6 +733,14 @@
670 733 requestedAtMs: prefetchRequestStartedAtMs,
671 734 requestedVideoTimeSeconds: roundLogNumber(nextTime),
672 735 });
736 setSeekMarkerTime('requested', nextTime);
737 updateSeekBarMarkers();
738
739 let prefetchTimedOut = false;
740 const timeoutId = armRequestTimeout(controller, prefetchRequestTimeoutMs, () =>
741 {
742 prefetchTimedOut = true;
743 });
673 744
674 745 try
675 746 {
@@ -679,6 +750,7 @@
679 750 credentials: request.credentials,
680 751 signal: controller ? controller.signal : undefined,
681 752 });
753 clearRequestTimeout(timeoutId);
682 754
683 755 if (workerToken !== prefetchWorkerToken || workerLifecycleToken !== lifecycleToken) return;
684 756
@@ -691,18 +763,20 @@
691 763 shouldShowAtVideoTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-Start-Time'),
692 764 subtitleEndTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-End-Time'),
693 765 responseHeadersReceivedAtMs: Date.now(),
766 requestedVideoTimeSeconds: roundLogNumber(nextTime),
694 767 };
695 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
696 {
697 setSeekMarkerTime('requested', responseMeta.shouldShowAtVideoTimeSeconds);
698 updateSeekBarMarkers();
699 }
700 768 logSubtitleEvent('subtitle_png_sent', {
701 769 ...responseMeta,
702 770 serverProcessingMs: diffMilliseconds(responseMeta.serverResponseSentAtMs, responseMeta.serverRequestReceivedAtMs),
703 771 sentToHeadersReceivedMs: diffMilliseconds(responseMeta.responseHeadersReceivedAtMs, responseMeta.serverResponseSentAtMs),
704 772 });
705 773
774 const receivedMarkerTime = Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds)
775 ? responseMeta.shouldShowAtVideoTimeSeconds
776 : nextTime;
777 setSeekMarkerTime('received', receivedMarkerTime);
778 updateSeekBarMarkers();
779
706 780 if (res.status === 204)
707 781 {
708 782 const frameStatus = String(res.headers.get('X-Subtitle-Frame-Status') || '').toLowerCase();
@@ -712,17 +786,16 @@
712 786 });
713 787 if (frameStatus === 'error') return;
714 788 prefetchedThroughTimeSeconds = nextTime;
789 if (prefetchRequestIntervalMs > 0)
790 {
791 await new Promise((resolve) => window.setTimeout(resolve, prefetchRequestIntervalMs));
792 }
715 793 continue;
716 794 }
717 795
718 796 if (!res.ok) return;
719 797 const payload = await res.arrayBuffer();
720 798 const receivedAtMs = Date.now();
721 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
722 {
723 setSeekMarkerTime('received', responseMeta.shouldShowAtVideoTimeSeconds);
724 updateSeekBarMarkers();
725 }
726 799 logSubtitleEvent('subtitle_png_received', {
727 800 ...responseMeta,
728 801 receivedAtMs,
@@ -731,8 +804,24 @@
731 804 blobBytes: Number(payload && payload.byteLength || 0),
732 805 });
733 806 prefetchedThroughTimeSeconds = nextTime;
807 if (prefetchRequestIntervalMs > 0)
808 {
809 await new Promise((resolve) => window.setTimeout(resolve, prefetchRequestIntervalMs));
810 }
734 811 } catch (error)
735 812 {
813 clearRequestTimeout(timeoutId);
814 if (prefetchTimedOut)
815 {
816 logSubtitleEvent('subtitle_png_request_failed', {
817 requestId: prefetchRequestId,
818 requestKind: 'prefetch',
819 requestedAtMs: prefetchRequestStartedAtMs,
820 errorName: 'AbortError',
821 errorMessage: `Prefetch subtitle request timed out after ${Math.round(prefetchRequestTimeoutMs)}ms`,
822 });
823 return;
824 }
736 825 if (isAbortError(error)) return;
737 826 logSubtitleEvent('subtitle_png_request_failed', {
738 827 requestId: prefetchRequestId,
@@ -809,9 +898,16 @@
809 898 requestedAtMs: requestStartedAtMs,
810 899 requestedVideoTimeSeconds: roundLogNumber(currentTime),
811 900 });
901 setSeekMarkerTime('requested', currentTime);
902 updateSeekBarMarkers();
812 903
813 904 currentFramePromise = (async () =>
814 905 {
906 let currentTimedOut = false;
907 const timeoutId = armRequestTimeout(controller, currentRequestTimeoutMs, () =>
908 {
909 currentTimedOut = true;
910 });
815 911 try
816 912 {
817 913 const res = await fetch(request.url, {
@@ -820,6 +916,7 @@
820 916 credentials: request.credentials,
821 917 signal: controller ? controller.signal : undefined,
822 918 });
919 clearRequestTimeout(timeoutId);
823 920
824 921 if (token !== currentFrameToken || refreshLifecycleToken !== lifecycleToken) return;
825 922
@@ -832,18 +929,20 @@
832 929 shouldShowAtVideoTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-Start-Time'),
833 930 subtitleEndTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-End-Time'),
834 931 responseHeadersReceivedAtMs: Date.now(),
932 requestedVideoTimeSeconds: roundLogNumber(currentTime),
835 933 };
836 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
837 {
838 setSeekMarkerTime('requested', responseMeta.shouldShowAtVideoTimeSeconds);
839 updateSeekBarMarkers();
840 }
841 934 logSubtitleEvent('subtitle_png_sent', {
842 935 ...responseMeta,
843 936 serverProcessingMs: diffMilliseconds(responseMeta.serverResponseSentAtMs, responseMeta.serverRequestReceivedAtMs),
844 937 sentToHeadersReceivedMs: diffMilliseconds(responseMeta.responseHeadersReceivedAtMs, responseMeta.serverResponseSentAtMs),
845 938 });
846 939
940 const receivedMarkerTime = Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds)
941 ? responseMeta.shouldShowAtVideoTimeSeconds
942 : currentTime;
943 setSeekMarkerTime('received', receivedMarkerTime);
944 updateSeekBarMarkers();
945
847 946 if (res.status === 204)
848 947 {
849 948 const frameStatus = String(res.headers.get('X-Subtitle-Frame-Status') || '').toLowerCase();
@@ -867,11 +966,6 @@
867 966 if (token !== currentFrameToken || refreshLifecycleToken !== lifecycleToken) return;
868 967
869 968 const receivedAtMs = Date.now();
870 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
871 {
872 setSeekMarkerTime('received', responseMeta.shouldShowAtVideoTimeSeconds);
873 updateSeekBarMarkers();
874 }
875 969 logSubtitleEvent('subtitle_png_received', {
876 970 ...responseMeta,
877 971 receivedAtMs,
@@ -890,10 +984,10 @@
890 984 activeObjectUrl = URL.createObjectURL(blob);
891 985 overlayImgEl.src = activeObjectUrl;
892 986
893 const frameWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Width') || ''), 10);
894 const frameHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Height') || ''), 10);
895 const canvasWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Width') || ''), 10);
896 const canvasHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Height') || ''), 10);
987 const frameWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Width') || ''), 10);
988 const frameHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Height') || ''), 10);
989 const canvasWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Width') || ''), 10);
990 const canvasHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Height') || ''), 10);
897 991
898 992 if (frameWidth > 0 && frameHeight > 0 && canvasWidth > 0 && canvasHeight > 0)
899 993 {
@@ -932,6 +1026,18 @@
932 1026 queueLookaheadPrefetch(currentTime, { reset: force });
933 1027 } catch (error)
934 1028 {
1029 clearRequestTimeout(timeoutId);
1030 if (currentTimedOut)
1031 {
1032 logSubtitleEvent('subtitle_png_request_failed', {
1033 requestId,
1034 requestKind: 'current',
1035 requestedAtMs: requestStartedAtMs,
1036 errorName: 'AbortError',
1037 errorMessage: `Current subtitle request timed out after ${Math.round(currentRequestTimeoutMs)}ms`,
1038 });
1039 return;
1040 }
935 1041 if (isAbortError(error)) return;
936 1042 logSubtitleEvent('subtitle_png_request_failed', {
937 1043 requestId,
@@ -12,6 +12,9 @@
12 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_DEFAULT = 15;
12 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_DEFAULT = 15;
13 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_MAX = 30;
13 const IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_MAX = 30;
14 const IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS = 0.5;
14 const IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS = 0.5;
15 const IMAGE_SUBTITLE_CURRENT_REQUEST_TIMEOUT_MS_DEFAULT = 4000;
16 const IMAGE_SUBTITLE_PREFETCH_REQUEST_TIMEOUT_MS_DEFAULT = 4000;
17 const IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS_DEFAULT = 0;
15
18
16 function roundLogNumber (value, digits = 3)
19 function roundLogNumber (value, digits = 3)
17 {
20 {
@@ -261,6 +264,19 @@
261 const lookaheadSeconds = Number.isFinite(requestedLookaheadSeconds)
264 const lookaheadSeconds = Number.isFinite(requestedLookaheadSeconds)
262 ? Math.max(0, Math.min(IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_MAX, requestedLookaheadSeconds))
265 ? Math.max(0, Math.min(IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_MAX, requestedLookaheadSeconds))
263 : IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_DEFAULT;
266 : IMAGE_SUBTITLE_LOOKAHEAD_SECONDS_DEFAULT;
267 const requestedCurrentRequestTimeoutMs = Number(opts.currentRequestTimeoutMs);
268 const currentRequestTimeoutMs = Number.isFinite(requestedCurrentRequestTimeoutMs)
269 ? Math.max(250, requestedCurrentRequestTimeoutMs)
270 : IMAGE_SUBTITLE_CURRENT_REQUEST_TIMEOUT_MS_DEFAULT;
271 const requestedPrefetchRequestTimeoutMs = Number(opts.prefetchRequestTimeoutMs);
272 const prefetchRequestTimeoutMs = Number.isFinite(requestedPrefetchRequestTimeoutMs)
273 ? Math.max(250, requestedPrefetchRequestTimeoutMs)
274 : IMAGE_SUBTITLE_PREFETCH_REQUEST_TIMEOUT_MS_DEFAULT;
275 const requestedPrefetchRequestIntervalMs = Number(opts.prefetchRequestIntervalMs);
276 const prefetchRequestIntervalMs = Number.isFinite(requestedPrefetchRequestIntervalMs)
277 ? Math.max(0, requestedPrefetchRequestIntervalMs)
278 : IMAGE_SUBTITLE_PREFETCH_REQUEST_INTERVAL_MS_DEFAULT;
279 const logToConsole = opts.logToConsole !== false;
264 if (!buildFrameRequest) return null;
280 if (!buildFrameRequest) return null;
265
281
266 const trackByStreamIndex = new Map(imageTracks.map((track) => [track.streamIndex, track]));
282 const trackByStreamIndex = new Map(imageTracks.map((track) => [track.streamIndex, track]));
@@ -338,12 +354,15 @@
338 playerTimeSeconds: roundLogNumber(player.currentTime()),
354 playerTimeSeconds: roundLogNumber(player.currentTime()),
339 ...details,
355 ...details,
340 };
356 };
341 try
357 if (logToConsole)
342 {
343 console.log('[subtitle-log]', entry);
344 } catch
345 {
358 {
346 // Ignore console availability issues.
359 try
360 {
361 console.log('[subtitle-log]', entry);
362 } catch
363 {
364 // Ignore console availability issues.
365 }
347 }
366 }
348
367
349 if (!sendSubtitleLogBatch) return;
368 if (!sendSubtitleLogBatch) return;
@@ -374,6 +393,22 @@
374 return !!(error && typeof error === 'object' && error.name === 'AbortError');
393 return !!(error && typeof error === 'object' && error.name === 'AbortError');
375 };
394 };
376
395
396 const armRequestTimeout = (controller, timeoutMs, onTimeout) =>
397 {
398 if (!controller || !Number.isFinite(timeoutMs) || timeoutMs <= 0) return null;
399 return window.setTimeout(() =>
400 {
401 if (typeof onTimeout === 'function') onTimeout();
402 abortController(controller);
403 }, timeoutMs);
404 };
405
406 const clearRequestTimeout = (timeoutId) =>
407 {
408 if (timeoutId === null || timeoutId === undefined) return;
409 window.clearTimeout(timeoutId);
410 };
411
377 const cancelCurrentFrameRequest = () =>
412 const cancelCurrentFrameRequest = () =>
378 {
413 {
379 currentFrameToken++;
414 currentFrameToken++;
@@ -490,27 +525,30 @@
490 seekMarkerContainerEl.style.pointerEvents = 'none';
525 seekMarkerContainerEl.style.pointerEvents = 'none';
491 seekMarkerContainerEl.style.zIndex = '5';
526 seekMarkerContainerEl.style.zIndex = '5';
492
527
493 const buildTriangle = (direction) =>
528 const buildBracketMarker = (direction) =>
494 {
529 {
495 const markerEl = document.createElement('span');
530 const markerEl = document.createElement('span');
496 markerEl.className = `vjs-image-subtitle-seek-marker vjs-image-subtitle-seek-marker-${direction}`;
531 markerEl.className = `vjs-image-subtitle-seek-marker vjs-image-subtitle-seek-marker-${direction}`;
497 markerEl.style.position = 'absolute';
532 markerEl.style.position = 'absolute';
498 markerEl.style.width = '12px';
533 markerEl.style.top = '50%';
534 markerEl.style.width = '10px';
499 markerEl.style.height = '10px';
535 markerEl.style.height = '10px';
500 markerEl.style.transform = 'translateX(-50%)';
536 markerEl.style.transform = 'translate(-50%, -50%)';
501 markerEl.style.background = '#fff';
537 markerEl.style.color = '#fff';
502 markerEl.style.border = '1px solid #000';
538 markerEl.style.fontFamily = 'monospace';
539 markerEl.style.fontSize = '12px';
540 markerEl.style.fontWeight = '700';
541 markerEl.style.lineHeight = '10px';
542 markerEl.style.textAlign = 'center';
543 markerEl.style.textShadow = '-1px 0 #000, 0 1px #000, 1px 0 #000, 0 -1px #000';
503 markerEl.style.display = 'none';
544 markerEl.style.display = 'none';
504 markerEl.style.boxSizing = 'border-box';
545 markerEl.textContent = direction === 'left' ? '[' : ']';
505 markerEl.style.clipPath = direction === 'left'
506 ? 'polygon(100% 0, 0 50%, 100% 100%)'
507 : 'polygon(0 0, 100% 50%, 0 100%)';
508 markerEl.setAttribute('aria-hidden', 'true');
546 markerEl.setAttribute('aria-hidden', 'true');
509 return markerEl;
547 return markerEl;
510 };
548 };
511
549
512 seekMarkerReceivedEl = buildTriangle('left');
550 seekMarkerReceivedEl = buildBracketMarker('left');
513 seekMarkerRequestedEl = buildTriangle('right');
551 seekMarkerRequestedEl = buildBracketMarker('right');
514 seekMarkerContainerEl.appendChild(seekMarkerReceivedEl);
552 seekMarkerContainerEl.appendChild(seekMarkerReceivedEl);
515 seekMarkerContainerEl.appendChild(seekMarkerRequestedEl);
553 seekMarkerContainerEl.appendChild(seekMarkerRequestedEl);
516
554
@@ -622,10 +660,35 @@
622 {
660 {
623 if (selectedStreamIndex === null) return;
661 if (selectedStreamIndex === null) return;
624 if (trackDisabled) return;
662 if (trackDisabled) return;
625 if (lookaheadSeconds <= 0) return;
663
664 const bufferedVideoEndSeconds = (() =>
665 {
666 const fromPlayer = Number(player.bufferedEnd && player.bufferedEnd());
667 if (Number.isFinite(fromPlayer) && fromPlayer >= 0) return fromPlayer;
668
669 const tech = typeof player.tech === 'function' ? player.tech(true) : null;
670 const mediaEl = tech && typeof tech.el === 'function'
671 ? tech.el()
672 : (player.el && player.el() ? player.el().querySelector('video') : null);
673
674 const ranges = mediaEl && mediaEl.buffered;
675 if (!ranges || typeof ranges.length !== 'number' || ranges.length <= 0 || typeof ranges.end !== 'function')
676 {
677 return null;
678 }
679
680 try
681 {
682 const endTime = Number(ranges.end(ranges.length - 1));
683 return Number.isFinite(endTime) && endTime >= 0 ? endTime : null;
684 } catch
685 {
686 return null;
687 }
688 })();
626
689
627 const startTime = currentTime + IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS;
690 const startTime = currentTime + IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS;
628 const endTime = currentTime + lookaheadSeconds;
691 const endTime = Math.max(currentTime + lookaheadSeconds, Number.isFinite(bufferedVideoEndSeconds) ? bufferedVideoEndSeconds : -1);
629 if (!Number.isFinite(startTime) || !Number.isFinite(endTime) || endTime <= startTime) return;
692 if (!Number.isFinite(startTime) || !Number.isFinite(endTime) || endTime <= startTime) return;
630
693
631 if (reset || prefetchedThroughTimeSeconds < 0 || prefetchedThroughTimeSeconds < currentTime - IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS)
694 if (reset || prefetchedThroughTimeSeconds < 0 || prefetchedThroughTimeSeconds < currentTime - IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS)
@@ -670,6 +733,14 @@
670 requestedAtMs: prefetchRequestStartedAtMs,
733 requestedAtMs: prefetchRequestStartedAtMs,
671 requestedVideoTimeSeconds: roundLogNumber(nextTime),
734 requestedVideoTimeSeconds: roundLogNumber(nextTime),
672 });
735 });
736 setSeekMarkerTime('requested', nextTime);
737 updateSeekBarMarkers();
738
739 let prefetchTimedOut = false;
740 const timeoutId = armRequestTimeout(controller, prefetchRequestTimeoutMs, () =>
741 {
742 prefetchTimedOut = true;
743 });
673
744
674 try
745 try
675 {
746 {
@@ -679,6 +750,7 @@
679 credentials: request.credentials,
750 credentials: request.credentials,
680 signal: controller ? controller.signal : undefined,
751 signal: controller ? controller.signal : undefined,
681 });
752 });
753 clearRequestTimeout(timeoutId);
682
754
683 if (workerToken !== prefetchWorkerToken || workerLifecycleToken !== lifecycleToken) return;
755 if (workerToken !== prefetchWorkerToken || workerLifecycleToken !== lifecycleToken) return;
684
756
@@ -691,18 +763,20 @@
691 shouldShowAtVideoTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-Start-Time'),
763 shouldShowAtVideoTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-Start-Time'),
692 subtitleEndTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-End-Time'),
764 subtitleEndTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-End-Time'),
693 responseHeadersReceivedAtMs: Date.now(),
765 responseHeadersReceivedAtMs: Date.now(),
766 requestedVideoTimeSeconds: roundLogNumber(nextTime),
694 };
767 };
695 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
696 {
697 setSeekMarkerTime('requested', responseMeta.shouldShowAtVideoTimeSeconds);
698 updateSeekBarMarkers();
699 }
700 logSubtitleEvent('subtitle_png_sent', {
768 logSubtitleEvent('subtitle_png_sent', {
701 ...responseMeta,
769 ...responseMeta,
702 serverProcessingMs: diffMilliseconds(responseMeta.serverResponseSentAtMs, responseMeta.serverRequestReceivedAtMs),
770 serverProcessingMs: diffMilliseconds(responseMeta.serverResponseSentAtMs, responseMeta.serverRequestReceivedAtMs),
703 sentToHeadersReceivedMs: diffMilliseconds(responseMeta.responseHeadersReceivedAtMs, responseMeta.serverResponseSentAtMs),
771 sentToHeadersReceivedMs: diffMilliseconds(responseMeta.responseHeadersReceivedAtMs, responseMeta.serverResponseSentAtMs),
704 });
772 });
705
773
774 const receivedMarkerTime = Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds)
775 ? responseMeta.shouldShowAtVideoTimeSeconds
776 : nextTime;
777 setSeekMarkerTime('received', receivedMarkerTime);
778 updateSeekBarMarkers();
779
706 if (res.status === 204)
780 if (res.status === 204)
707 {
781 {
708 const frameStatus = String(res.headers.get('X-Subtitle-Frame-Status') || '').toLowerCase();
782 const frameStatus = String(res.headers.get('X-Subtitle-Frame-Status') || '').toLowerCase();
@@ -712,17 +786,16 @@
712 });
786 });
713 if (frameStatus === 'error') return;
787 if (frameStatus === 'error') return;
714 prefetchedThroughTimeSeconds = nextTime;
788 prefetchedThroughTimeSeconds = nextTime;
789 if (prefetchRequestIntervalMs > 0)
790 {
791 await new Promise((resolve) => window.setTimeout(resolve, prefetchRequestIntervalMs));
792 }
715 continue;
793 continue;
716 }
794 }
717
795
718 if (!res.ok) return;
796 if (!res.ok) return;
719 const payload = await res.arrayBuffer();
797 const payload = await res.arrayBuffer();
720 const receivedAtMs = Date.now();
798 const receivedAtMs = Date.now();
721 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
722 {
723 setSeekMarkerTime('received', responseMeta.shouldShowAtVideoTimeSeconds);
724 updateSeekBarMarkers();
725 }
726 logSubtitleEvent('subtitle_png_received', {
799 logSubtitleEvent('subtitle_png_received', {
727 ...responseMeta,
800 ...responseMeta,
728 receivedAtMs,
801 receivedAtMs,
@@ -731,8 +804,24 @@
731 blobBytes: Number(payload && payload.byteLength || 0),
804 blobBytes: Number(payload && payload.byteLength || 0),
732 });
805 });
733 prefetchedThroughTimeSeconds = nextTime;
806 prefetchedThroughTimeSeconds = nextTime;
807 if (prefetchRequestIntervalMs > 0)
808 {
809 await new Promise((resolve) => window.setTimeout(resolve, prefetchRequestIntervalMs));
810 }
734 } catch (error)
811 } catch (error)
735 {
812 {
813 clearRequestTimeout(timeoutId);
814 if (prefetchTimedOut)
815 {
816 logSubtitleEvent('subtitle_png_request_failed', {
817 requestId: prefetchRequestId,
818 requestKind: 'prefetch',
819 requestedAtMs: prefetchRequestStartedAtMs,
820 errorName: 'AbortError',
821 errorMessage: `Prefetch subtitle request timed out after ${Math.round(prefetchRequestTimeoutMs)}ms`,
822 });
823 return;
824 }
736 if (isAbortError(error)) return;
825 if (isAbortError(error)) return;
737 logSubtitleEvent('subtitle_png_request_failed', {
826 logSubtitleEvent('subtitle_png_request_failed', {
738 requestId: prefetchRequestId,
827 requestId: prefetchRequestId,
@@ -809,9 +898,16 @@
809 requestedAtMs: requestStartedAtMs,
898 requestedAtMs: requestStartedAtMs,
810 requestedVideoTimeSeconds: roundLogNumber(currentTime),
899 requestedVideoTimeSeconds: roundLogNumber(currentTime),
811 });
900 });
901 setSeekMarkerTime('requested', currentTime);
902 updateSeekBarMarkers();
812
903
813 currentFramePromise = (async () =>
904 currentFramePromise = (async () =>
814 {
905 {
906 let currentTimedOut = false;
907 const timeoutId = armRequestTimeout(controller, currentRequestTimeoutMs, () =>
908 {
909 currentTimedOut = true;
910 });
815 try
911 try
816 {
912 {
817 const res = await fetch(request.url, {
913 const res = await fetch(request.url, {
@@ -820,6 +916,7 @@
820 credentials: request.credentials,
916 credentials: request.credentials,
821 signal: controller ? controller.signal : undefined,
917 signal: controller ? controller.signal : undefined,
822 });
918 });
919 clearRequestTimeout(timeoutId);
823
920
824 if (token !== currentFrameToken || refreshLifecycleToken !== lifecycleToken) return;
921 if (token !== currentFrameToken || refreshLifecycleToken !== lifecycleToken) return;
825
922
@@ -832,18 +929,20 @@
832 shouldShowAtVideoTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-Start-Time'),
929 shouldShowAtVideoTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-Start-Time'),
833 subtitleEndTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-End-Time'),
930 subtitleEndTimeSeconds: parseHeaderFloat(res, 'X-Subtitle-Frame-End-Time'),
834 responseHeadersReceivedAtMs: Date.now(),
931 responseHeadersReceivedAtMs: Date.now(),
932 requestedVideoTimeSeconds: roundLogNumber(currentTime),
835 };
933 };
836 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
837 {
838 setSeekMarkerTime('requested', responseMeta.shouldShowAtVideoTimeSeconds);
839 updateSeekBarMarkers();
840 }
841 logSubtitleEvent('subtitle_png_sent', {
934 logSubtitleEvent('subtitle_png_sent', {
842 ...responseMeta,
935 ...responseMeta,
843 serverProcessingMs: diffMilliseconds(responseMeta.serverResponseSentAtMs, responseMeta.serverRequestReceivedAtMs),
936 serverProcessingMs: diffMilliseconds(responseMeta.serverResponseSentAtMs, responseMeta.serverRequestReceivedAtMs),
844 sentToHeadersReceivedMs: diffMilliseconds(responseMeta.responseHeadersReceivedAtMs, responseMeta.serverResponseSentAtMs),
937 sentToHeadersReceivedMs: diffMilliseconds(responseMeta.responseHeadersReceivedAtMs, responseMeta.serverResponseSentAtMs),
845 });
938 });
846
939
940 const receivedMarkerTime = Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds)
941 ? responseMeta.shouldShowAtVideoTimeSeconds
942 : currentTime;
943 setSeekMarkerTime('received', receivedMarkerTime);
944 updateSeekBarMarkers();
945
847 if (res.status === 204)
946 if (res.status === 204)
848 {
947 {
849 const frameStatus = String(res.headers.get('X-Subtitle-Frame-Status') || '').toLowerCase();
948 const frameStatus = String(res.headers.get('X-Subtitle-Frame-Status') || '').toLowerCase();
@@ -867,11 +966,6 @@
867 if (token !== currentFrameToken || refreshLifecycleToken !== lifecycleToken) return;
966 if (token !== currentFrameToken || refreshLifecycleToken !== lifecycleToken) return;
868
967
869 const receivedAtMs = Date.now();
968 const receivedAtMs = Date.now();
870 if (Number.isFinite(responseMeta.shouldShowAtVideoTimeSeconds))
871 {
872 setSeekMarkerTime('received', responseMeta.shouldShowAtVideoTimeSeconds);
873 updateSeekBarMarkers();
874 }
875 logSubtitleEvent('subtitle_png_received', {
969 logSubtitleEvent('subtitle_png_received', {
876 ...responseMeta,
970 ...responseMeta,
877 receivedAtMs,
971 receivedAtMs,
@@ -890,10 +984,10 @@
890 activeObjectUrl = URL.createObjectURL(blob);
984 activeObjectUrl = URL.createObjectURL(blob);
891 overlayImgEl.src = activeObjectUrl;
985 overlayImgEl.src = activeObjectUrl;
892
986
893 const frameWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Width') || ''), 10);
987 const frameWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Width') || ''), 10);
894 const frameHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Height') || ''), 10);
988 const frameHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Height') || ''), 10);
895 const canvasWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Width') || ''), 10);
989 const canvasWidth = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Width') || ''), 10);
896 const canvasHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Height') || ''), 10);
990 const canvasHeight = Number.parseInt(String(res.headers.get('X-Subtitle-Frame-Canvas-Height') || ''), 10);
897
991
898 if (frameWidth > 0 && frameHeight > 0 && canvasWidth > 0 && canvasHeight > 0)
992 if (frameWidth > 0 && frameHeight > 0 && canvasWidth > 0 && canvasHeight > 0)
899 {
993 {
@@ -932,6 +1026,18 @@
932 queueLookaheadPrefetch(currentTime, { reset: force });
1026 queueLookaheadPrefetch(currentTime, { reset: force });
933 } catch (error)
1027 } catch (error)
934 {
1028 {
1029 clearRequestTimeout(timeoutId);
1030 if (currentTimedOut)
1031 {
1032 logSubtitleEvent('subtitle_png_request_failed', {
1033 requestId,
1034 requestKind: 'current',
1035 requestedAtMs: requestStartedAtMs,
1036 errorName: 'AbortError',
1037 errorMessage: `Current subtitle request timed out after ${Math.round(currentRequestTimeoutMs)}ms`,
1038 });
1039 return;
1040 }
935 if (isAbortError(error)) return;
1041 if (isAbortError(error)) return;
936 logSubtitleEvent('subtitle_png_request_failed', {
1042 logSubtitleEvent('subtitle_png_request_failed', {
937 requestId,
1043 requestId,
tests/shared-video-player.test.js +156 −1
@@ -67,6 +67,7 @@ function buildPlayerHarness (window)
67 67 playerEl.appendChild(progressControlEl);
68 68 let currentTime = 0;
69 69 let durationSeconds = 10;
70 let bufferedEndSeconds = null;
70 71
71 72 const player = {
72 73 ready (cb)
@@ -100,6 +101,14 @@ function buildPlayerHarness (window)
100 101 }
101 102 return durationSeconds;
102 103 },
104 bufferedEnd ()
105 {
106 return bufferedEndSeconds;
107 },
108 setBufferedEnd (next)
109 {
110 bufferedEndSeconds = Number.isFinite(next) ? Number(next) : null;
111 },
103 112 el ()
104 113 {
105 114 return playerEl;
@@ -183,6 +192,41 @@ describe('shared image subtitle prefetching', () =>
183 192 window.close();
184 193 });
185 194
195 test('prefetches at least through the buffered video end', async () =>
196 {
197 const window = createWindow();
198 const helper = loadSharedVideoPlayer(window);
199 const { player, videojs } = buildPlayerHarness(window);
200 const requestedTimes = [];
201
202 player.setBufferedEnd(3);
203
204 window.fetch = jest.fn(async (url) =>
205 {
206 const parsed = new URL(url, 'http://localhost/');
207 requestedTimes.push(Number(parsed.searchParams.get('timeSeconds')));
208 return new Response(null, {
209 status: 204,
210 headers: { 'X-Subtitle-Frame-Status': 'none' },
211 });
212 });
213
214 helper.bindImageSubtitleOverlay(player, videojs, {}, {
215 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
216 }, {
217 lookaheadSeconds: 0.5,
218 buildFrameRequest: ({ currentTime, streamIndex }) => ({
219 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
220 }),
221 });
222
223 await flushAsyncWork(window, 12);
224
225 expect(requestedTimes).toEqual(expect.arrayContaining([0, 0.5, 1, 1.5, 2, 2.5, 3]));
226
227 window.close();
228 });
229
186 230 test('does not restart current subtitle request while one is in flight', async () =>
187 231 {
188 232 const window = createWindow();
@@ -446,12 +490,123 @@ describe('shared image subtitle prefetching', () =>
446 490 expect(receivedMarker).toBeTruthy();
447 491 expect(requestedMarker.style.display).toBe('block');
448 492 expect(receivedMarker.style.display).toBe('block');
449 expect(requestedMarker.style.left).toBe('25.0000%');
493 expect(requestedMarker.style.left).toBe('0.0000%');
450 494 expect(receivedMarker.style.left).toBe('25.0000%');
495 expect(requestedMarker.textContent).toBe(']');
496 expect(receivedMarker.textContent).toBe('[');
497
498 controller.detach();
499 await new Promise((resolve) => window.setTimeout(resolve, 300));
500 await flushAsyncWork(window, 2);
501
502 window.close();
503 });
504
505 test('advances both seek markers when subtitle request returns 204 without frame timing headers', async () =>
506 {
507 const window = createWindow();
508 const helper = loadSharedVideoPlayer(window);
509 const { player, videojs } = buildPlayerHarness(window);
510
511 player.duration(10);
512 player.setCurrentTime(4);
513 window.fetch = jest.fn(async () =>
514 {
515 return new Response(null, {
516 status: 204,
517 headers: {
518 'X-Subtitle-Frame-Status': 'none',
519 'X-Subtitle-Request-Received-At-Ms': '1000',
520 'X-Subtitle-Response-Sent-At-Ms': '1010',
521 },
522 });
523 });
524
525 const controller = helper.bindImageSubtitleOverlay(player, videojs, {}, {
526 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
527 }, {
528 lookaheadSeconds: 0,
529 buildFrameRequest: ({ currentTime, streamIndex }) => ({
530 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
531 }),
532 });
533
534 await flushAsyncWork(window, 8);
535
536 const requestedMarker = player.el().querySelector('.vjs-image-subtitle-seek-marker-right');
537 const receivedMarker = player.el().querySelector('.vjs-image-subtitle-seek-marker-left');
538 expect(requestedMarker).toBeTruthy();
539 expect(receivedMarker).toBeTruthy();
540 expect(requestedMarker.style.left).toBe('40.0000%');
541 expect(receivedMarker.style.left).toBe('40.0000%');
542 expect(requestedMarker.textContent).toBe(']');
543 expect(receivedMarker.textContent).toBe('[');
451 544
452 545 controller.detach();
453 546 await flushAsyncWork(window, 2);
547 window.close();
548 });
549
550 test('recovers when a current subtitle request hangs and times out', async () =>
551 {
552 const window = createWindow();
553 const helper = loadSharedVideoPlayer(window);
554 const { player, videojs } = buildPlayerHarness(window);
555 const requestedTimes = [];
454 556
557 window.fetch = jest.fn((url, init = {}) =>
558 {
559 const parsed = new URL(url, 'http://localhost/');
560 const timeSeconds = Number(parsed.searchParams.get('timeSeconds'));
561 requestedTimes.push(timeSeconds);
562
563 if (requestedTimes.length === 1)
564 {
565 return new Promise((resolve, reject) =>
566 {
567 if (init.signal && typeof init.signal.addEventListener === 'function')
568 {
569 init.signal.addEventListener('abort', () =>
570 {
571 reject(new window.DOMException('Aborted', 'AbortError'));
572 }, { once: true });
573 }
574 });
575 }
576
577 return Promise.resolve(new Response(null, {
578 status: 204,
579 headers: { 'X-Subtitle-Frame-Status': 'none' },
580 }));
581 });
582
583 const controller = helper.bindImageSubtitleOverlay(player, videojs, {}, {
584 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
585 }, {
586 lookaheadSeconds: 0,
587 currentRequestTimeoutMs: 250,
588 logToConsole: false,
589 buildFrameRequest: ({ currentTime, streamIndex }) => ({
590 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
591 }),
592 });
593
594 await flushAsyncWork(window, 3);
595
596 player.setCurrentTime(1.5);
597 player.trigger('timeupdate');
598
599 await new Promise((resolve) => window.setTimeout(resolve, 320));
600 player.trigger('timeupdate');
601 await new Promise((resolve) => window.setTimeout(resolve, 220));
602 await flushAsyncWork(window, 20);
603
604 expect(window.fetch.mock.calls.length).toBeGreaterThanOrEqual(2);
605 expect(requestedTimes).toEqual(expect.arrayContaining([0, 1.5]));
606
607 controller.detach();
608 await new Promise((resolve) => window.setTimeout(resolve, 200));
609 await flushAsyncWork(window, 20);
455 610 window.close();
456 611 });
457 612 });
@@ -67,6 +67,7 @@ function buildPlayerHarness (window)
67 playerEl.appendChild(progressControlEl);
67 playerEl.appendChild(progressControlEl);
68 let currentTime = 0;
68 let currentTime = 0;
69 let durationSeconds = 10;
69 let durationSeconds = 10;
70 let bufferedEndSeconds = null;
70
71
71 const player = {
72 const player = {
72 ready (cb)
73 ready (cb)
@@ -100,6 +101,14 @@ function buildPlayerHarness (window)
100 }
101 }
101 return durationSeconds;
102 return durationSeconds;
102 },
103 },
104 bufferedEnd ()
105 {
106 return bufferedEndSeconds;
107 },
108 setBufferedEnd (next)
109 {
110 bufferedEndSeconds = Number.isFinite(next) ? Number(next) : null;
111 },
103 el ()
112 el ()
104 {
113 {
105 return playerEl;
114 return playerEl;
@@ -183,6 +192,41 @@ describe('shared image subtitle prefetching', () =>
183 window.close();
192 window.close();
184 });
193 });
185
194
195 test('prefetches at least through the buffered video end', async () =>
196 {
197 const window = createWindow();
198 const helper = loadSharedVideoPlayer(window);
199 const { player, videojs } = buildPlayerHarness(window);
200 const requestedTimes = [];
201
202 player.setBufferedEnd(3);
203
204 window.fetch = jest.fn(async (url) =>
205 {
206 const parsed = new URL(url, 'http://localhost/');
207 requestedTimes.push(Number(parsed.searchParams.get('timeSeconds')));
208 return new Response(null, {
209 status: 204,
210 headers: { 'X-Subtitle-Frame-Status': 'none' },
211 });
212 });
213
214 helper.bindImageSubtitleOverlay(player, videojs, {}, {
215 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
216 }, {
217 lookaheadSeconds: 0.5,
218 buildFrameRequest: ({ currentTime, streamIndex }) => ({
219 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
220 }),
221 });
222
223 await flushAsyncWork(window, 12);
224
225 expect(requestedTimes).toEqual(expect.arrayContaining([0, 0.5, 1, 1.5, 2, 2.5, 3]));
226
227 window.close();
228 });
229
186 test('does not restart current subtitle request while one is in flight', async () =>
230 test('does not restart current subtitle request while one is in flight', async () =>
187 {
231 {
188 const window = createWindow();
232 const window = createWindow();
@@ -446,12 +490,123 @@ describe('shared image subtitle prefetching', () =>
446 expect(receivedMarker).toBeTruthy();
490 expect(receivedMarker).toBeTruthy();
447 expect(requestedMarker.style.display).toBe('block');
491 expect(requestedMarker.style.display).toBe('block');
448 expect(receivedMarker.style.display).toBe('block');
492 expect(receivedMarker.style.display).toBe('block');
449 expect(requestedMarker.style.left).toBe('25.0000%');
493 expect(requestedMarker.style.left).toBe('0.0000%');
450 expect(receivedMarker.style.left).toBe('25.0000%');
494 expect(receivedMarker.style.left).toBe('25.0000%');
495 expect(requestedMarker.textContent).toBe(']');
496 expect(receivedMarker.textContent).toBe('[');
497
498 controller.detach();
499 await new Promise((resolve) => window.setTimeout(resolve, 300));
500 await flushAsyncWork(window, 2);
501
502 window.close();
503 });
504
505 test('advances both seek markers when subtitle request returns 204 without frame timing headers', async () =>
506 {
507 const window = createWindow();
508 const helper = loadSharedVideoPlayer(window);
509 const { player, videojs } = buildPlayerHarness(window);
510
511 player.duration(10);
512 player.setCurrentTime(4);
513 window.fetch = jest.fn(async () =>
514 {
515 return new Response(null, {
516 status: 204,
517 headers: {
518 'X-Subtitle-Frame-Status': 'none',
519 'X-Subtitle-Request-Received-At-Ms': '1000',
520 'X-Subtitle-Response-Sent-At-Ms': '1010',
521 },
522 });
523 });
524
525 const controller = helper.bindImageSubtitleOverlay(player, videojs, {}, {
526 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
527 }, {
528 lookaheadSeconds: 0,
529 buildFrameRequest: ({ currentTime, streamIndex }) => ({
530 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
531 }),
532 });
533
534 await flushAsyncWork(window, 8);
535
536 const requestedMarker = player.el().querySelector('.vjs-image-subtitle-seek-marker-right');
537 const receivedMarker = player.el().querySelector('.vjs-image-subtitle-seek-marker-left');
538 expect(requestedMarker).toBeTruthy();
539 expect(receivedMarker).toBeTruthy();
540 expect(requestedMarker.style.left).toBe('40.0000%');
541 expect(receivedMarker.style.left).toBe('40.0000%');
542 expect(requestedMarker.textContent).toBe(']');
543 expect(receivedMarker.textContent).toBe('[');
451
544
452 controller.detach();
545 controller.detach();
453 await flushAsyncWork(window, 2);
546 await flushAsyncWork(window, 2);
547 window.close();
548 });
549
550 test('recovers when a current subtitle request hangs and times out', async () =>
551 {
552 const window = createWindow();
553 const helper = loadSharedVideoPlayer(window);
554 const { player, videojs } = buildPlayerHarness(window);
555 const requestedTimes = [];
454
556
557 window.fetch = jest.fn((url, init = {}) =>
558 {
559 const parsed = new URL(url, 'http://localhost/');
560 const timeSeconds = Number(parsed.searchParams.get('timeSeconds'));
561 requestedTimes.push(timeSeconds);
562
563 if (requestedTimes.length === 1)
564 {
565 return new Promise((resolve, reject) =>
566 {
567 if (init.signal && typeof init.signal.addEventListener === 'function')
568 {
569 init.signal.addEventListener('abort', () =>
570 {
571 reject(new window.DOMException('Aborted', 'AbortError'));
572 }, { once: true });
573 }
574 });
575 }
576
577 return Promise.resolve(new Response(null, {
578 status: 204,
579 headers: { 'X-Subtitle-Frame-Status': 'none' },
580 }));
581 });
582
583 const controller = helper.bindImageSubtitleOverlay(player, videojs, {}, {
584 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
585 }, {
586 lookaheadSeconds: 0,
587 currentRequestTimeoutMs: 250,
588 logToConsole: false,
589 buildFrameRequest: ({ currentTime, streamIndex }) => ({
590 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
591 }),
592 });
593
594 await flushAsyncWork(window, 3);
595
596 player.setCurrentTime(1.5);
597 player.trigger('timeupdate');
598
599 await new Promise((resolve) => window.setTimeout(resolve, 320));
600 player.trigger('timeupdate');
601 await new Promise((resolve) => window.setTimeout(resolve, 220));
602 await flushAsyncWork(window, 20);
603
604 expect(window.fetch.mock.calls.length).toBeGreaterThanOrEqual(2);
605 expect(requestedTimes).toEqual(expect.arrayContaining([0, 1.5]));
606
607 controller.detach();
608 await new Promise((resolve) => window.setTimeout(resolve, 200));
609 await flushAsyncWork(window, 20);
455 window.close();
610 window.close();
456 });
611 });
457});
612});