BlueGit

timfa filemanager

Fix subtitle prefetch buffered frontier

Image subtitle prefetch now tracks the furthest buffered video point reliably.\n\nThis keeps subtitle lookahead aligned with buffered video instead of stalling near current playback time on new videos. It also pins the prefetch frontier to current playback to avoid replaying stale lag and adds regression tests for under-reported buffered end values and frontier catch-up behaviour.

timfa Tim Falken · pushed by timfa

Authored 2026-05-25 · 5 months ago

Hash
cd2795a6ffcbcba195c54706a9570feb3c845270
Parent
afd05e40
3 files changed +110 −13
.github/technical-reference.md +1 −1
@@ -182,7 +182,7 @@
182 182 ## public/shared-video-player.js
183 183 - applyDefaultTextTrackPrefs() - Writes the default Video.js caption override settings (transparent background, uniform edge style) so all streamed players start from the same text-track baseline.
184 184 - bindTrackPrefs(player, prefs, savePrefs) - Restores subtitle/audio track selections by track name and persists later track changes; shared by the authenticated player and the share page so both resolve track matches the same way.
185 - bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), extends prefetch target to at least the latest buffered video end (not only fixed lookahead seconds), coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), applies per-request timeouts (`currentRequestTimeoutMs`, `prefetchRequestTimeoutMs`) so hung subtitle frame requests cannot stall updates indefinitely, paces prefetch polling (`prefetchRequestIntervalMs`) to reduce request bursts, logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, caches received subtitle PNG frames client-side for the lifetime of the player session (including prefetched frames) so rewind playback can render from local cache even after server-side purge, ensures force-seek aborts still clear in-flight current-request state so subtitle refreshes continue instead of deadlocking, updates seek-bar bracket markers only after responses resolve (no in-flight jump-back flicker), keeps both marker positions forward-only (never regressing to earlier times), allows the two bracket glyphs to overlap naturally when positions match, advances markers even when responses are 204/no-frame, and disables the track after the first hard failure.
185 - bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), extends prefetch target to at least the latest buffered video end (not only fixed lookahead seconds), merges buffered-end signals from both Video.js/player APIs and underlying media buffered ranges so under-reported player values do not cap subtitle prefetch early, keeps the prefetch frontier pinned to at least current playback time so it chases buffered-ahead range instead of replaying stale lag after video/session changes, coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), applies per-request timeouts (`currentRequestTimeoutMs`, `prefetchRequestTimeoutMs`) so hung subtitle frame requests cannot stall updates indefinitely, paces prefetch polling (`prefetchRequestIntervalMs`) to reduce request bursts, logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, caches received subtitle PNG frames client-side for the lifetime of the player session (including prefetched frames) so rewind playback can render from local cache even after server-side purge, ensures force-seek aborts still clear in-flight current-request state so subtitle refreshes continue instead of deadlocking, updates seek-bar bracket markers only after responses resolve (no in-flight jump-back flicker), keeps both marker positions forward-only (never regressing to earlier times), allows the two bracket glyphs to overlap naturally when positions match, advances markers even when responses are 204/no-frame, and disables the track after the first hard failure.
186 186 - enableVideoStreamClientCache() - Registers a shared service worker (`/stream-cache-sw.js`) that caches HLS playlist/segment responses so already-downloaded stream assets remain playable on rewind after server-side segment pruning.
187 187 - clearVideoStreamClientCache(playlistUrl) - Best-effort cache cleanup helper that removes cached HLS entries for one stream session when preview playback is closed.
188 188
@@ -182,7 +182,7 @@
182## public/shared-video-player.js
182## public/shared-video-player.js
183- applyDefaultTextTrackPrefs() - Writes the default Video.js caption override settings (transparent background, uniform edge style) so all streamed players start from the same text-track baseline.
183- applyDefaultTextTrackPrefs() - Writes the default Video.js caption override settings (transparent background, uniform edge style) so all streamed players start from the same text-track baseline.
184- bindTrackPrefs(player, prefs, savePrefs) - Restores subtitle/audio track selections by track name and persists later track changes; shared by the authenticated player and the share page so both resolve track matches the same way.
184- bindTrackPrefs(player, prefs, savePrefs) - Restores subtitle/audio track selections by track name and persists later track changes; shared by the authenticated player and the share page so both resolve track matches the same way.
185- bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), extends prefetch target to at least the latest buffered video end (not only fixed lookahead seconds), coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), applies per-request timeouts (`currentRequestTimeoutMs`, `prefetchRequestTimeoutMs`) so hung subtitle frame requests cannot stall updates indefinitely, paces prefetch polling (`prefetchRequestIntervalMs`) to reduce request bursts, logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, caches received subtitle PNG frames client-side for the lifetime of the player session (including prefetched frames) so rewind playback can render from local cache even after server-side purge, ensures force-seek aborts still clear in-flight current-request state so subtitle refreshes continue instead of deadlocking, updates seek-bar bracket markers only after responses resolve (no in-flight jump-back flicker), keeps both marker positions forward-only (never regressing to earlier times), allows the two bracket glyphs to overlap naturally when positions match, advances markers even when responses are 204/no-frame, and disables the track after the first hard failure.
185- bindImageSubtitleOverlay(player, videojs, prefs, session, opts) - Adds the shared IM button/menu/overlay for bitmap subtitle tracks, restores saved image-subtitle selections, fetches rendered frames through a caller-supplied request builder, keeps a separate rolling lookahead-prefetch worker alive across timeupdates (including gaps where no subtitle is currently visible), extends prefetch target to at least the latest buffered video end (not only fixed lookahead seconds), merges buffered-end signals from both Video.js/player APIs and underlying media buffered ranges so under-reported player values do not cap subtitle prefetch early, keeps the prefetch frontier pinned to at least current playback time so it chases buffered-ahead range instead of replaying stale lag after video/session changes, coalesces overlapping current-frame refreshes into a single in-flight fetch (only force-refresh paths abort/restart), applies per-request timeouts (`currentRequestTimeoutMs`, `prefetchRequestTimeoutMs`) so hung subtitle frame requests cannot stall updates indefinitely, paces prefetch polling (`prefetchRequestIntervalMs`) to reduce request bursts, logs current and prefetch subtitle request/send/receive/not-available timing to the browser console and optional batch sink, caches received subtitle PNG frames client-side for the lifetime of the player session (including prefetched frames) so rewind playback can render from local cache even after server-side purge, ensures force-seek aborts still clear in-flight current-request state so subtitle refreshes continue instead of deadlocking, updates seek-bar bracket markers only after responses resolve (no in-flight jump-back flicker), keeps both marker positions forward-only (never regressing to earlier times), allows the two bracket glyphs to overlap naturally when positions match, advances markers even when responses are 204/no-frame, and disables the track after the first hard failure.
186- enableVideoStreamClientCache() - Registers a shared service worker (`/stream-cache-sw.js`) that caches HLS playlist/segment responses so already-downloaded stream assets remain playable on rewind after server-side segment pruning.
186- enableVideoStreamClientCache() - Registers a shared service worker (`/stream-cache-sw.js`) that caches HLS playlist/segment responses so already-downloaded stream assets remain playable on rewind after server-side segment pruning.
187- clearVideoStreamClientCache(playlistUrl) - Best-effort cache cleanup helper that removes cached HLS entries for one stream session when preview playback is closed.
187- clearVideoStreamClientCache(playlistUrl) - Best-effort cache cleanup helper that removes cached HLS entries for one stream session when preview playback is closed.
188
188
public/shared-video-player.js +20 −12
@@ -820,8 +820,10 @@
820 820
821 821 const bufferedVideoEndSeconds = (() =>
822 822 {
823 const candidates = [];
824
823 825 const fromPlayer = Number(player.bufferedEnd && player.bufferedEnd());
824 if (Number.isFinite(fromPlayer) && fromPlayer >= 0) return fromPlayer;
826 if (Number.isFinite(fromPlayer) && fromPlayer >= 0) candidates.push(fromPlayer);
825 827
826 828 const tech = typeof player.tech === 'function' ? player.tech(true) : null;
827 829 const mediaEl = tech && typeof tech.el === 'function'
@@ -829,29 +831,35 @@
829 831 : (player.el && player.el() ? player.el().querySelector('video') : null);
830 832
831 833 const ranges = mediaEl && mediaEl.buffered;
832 if (!ranges || typeof ranges.length !== 'number' || ranges.length <= 0 || typeof ranges.end !== 'function')
834 if (ranges && typeof ranges.length === 'number' && ranges.length > 0 && typeof ranges.end === 'function')
833 835 {
834 return null;
836 try
837 {
838 const endTime = Number(ranges.end(ranges.length - 1));
839 if (Number.isFinite(endTime) && endTime >= 0) candidates.push(endTime);
840 } catch
841 {
842 // Ignore buffered range read races.
843 }
835 844 }
836 845
837 try
838 {
839 const endTime = Number(ranges.end(ranges.length - 1));
840 return Number.isFinite(endTime) && endTime >= 0 ? endTime : null;
841 } catch
842 {
843 return null;
844 }
846 if (!candidates.length) return null;
847 return Math.max(...candidates);
845 848 })();
846 849
847 850 const startTime = currentTime + IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS;
848 851 const endTime = Math.max(currentTime + lookaheadSeconds, Number.isFinite(bufferedVideoEndSeconds) ? bufferedVideoEndSeconds : -1);
849 852 if (!Number.isFinite(startTime) || !Number.isFinite(endTime) || endTime <= startTime) return;
850 853
851 if (reset || prefetchedThroughTimeSeconds < 0 || prefetchedThroughTimeSeconds < currentTime - IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS)
854 if (reset || prefetchedThroughTimeSeconds < 0)
852 855 {
853 856 cancelPrefetchRequests({ resetProgress: true });
854 857 prefetchedThroughTimeSeconds = currentTime;
858 } else if (prefetchedThroughTimeSeconds < currentTime)
859 {
860 // Keep the prefetch frontier at or ahead of current playback so
861 // prefetch catches buffered-ahead range instead of replaying stale lag.
862 prefetchedThroughTimeSeconds = currentTime;
855 863 }
856 864
857 865 prefetchTargetTimeSeconds = Math.max(prefetchTargetTimeSeconds, endTime);
@@ -820,8 +820,10 @@
820
820
821 const bufferedVideoEndSeconds = (() =>
821 const bufferedVideoEndSeconds = (() =>
822 {
822 {
823 const candidates = [];
824
823 const fromPlayer = Number(player.bufferedEnd && player.bufferedEnd());
825 const fromPlayer = Number(player.bufferedEnd && player.bufferedEnd());
824 if (Number.isFinite(fromPlayer) && fromPlayer >= 0) return fromPlayer;
826 if (Number.isFinite(fromPlayer) && fromPlayer >= 0) candidates.push(fromPlayer);
825
827
826 const tech = typeof player.tech === 'function' ? player.tech(true) : null;
828 const tech = typeof player.tech === 'function' ? player.tech(true) : null;
827 const mediaEl = tech && typeof tech.el === 'function'
829 const mediaEl = tech && typeof tech.el === 'function'
@@ -829,29 +831,35 @@
829 : (player.el && player.el() ? player.el().querySelector('video') : null);
831 : (player.el && player.el() ? player.el().querySelector('video') : null);
830
832
831 const ranges = mediaEl && mediaEl.buffered;
833 const ranges = mediaEl && mediaEl.buffered;
832 if (!ranges || typeof ranges.length !== 'number' || ranges.length <= 0 || typeof ranges.end !== 'function')
834 if (ranges && typeof ranges.length === 'number' && ranges.length > 0 && typeof ranges.end === 'function')
833 {
835 {
834 return null;
836 try
837 {
838 const endTime = Number(ranges.end(ranges.length - 1));
839 if (Number.isFinite(endTime) && endTime >= 0) candidates.push(endTime);
840 } catch
841 {
842 // Ignore buffered range read races.
843 }
835 }
844 }
836
845
837 try
846 if (!candidates.length) return null;
838 {
847 return Math.max(...candidates);
839 const endTime = Number(ranges.end(ranges.length - 1));
840 return Number.isFinite(endTime) && endTime >= 0 ? endTime : null;
841 } catch
842 {
843 return null;
844 }
845 })();
848 })();
846
849
847 const startTime = currentTime + IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS;
850 const startTime = currentTime + IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS;
848 const endTime = Math.max(currentTime + lookaheadSeconds, Number.isFinite(bufferedVideoEndSeconds) ? bufferedVideoEndSeconds : -1);
851 const endTime = Math.max(currentTime + lookaheadSeconds, Number.isFinite(bufferedVideoEndSeconds) ? bufferedVideoEndSeconds : -1);
849 if (!Number.isFinite(startTime) || !Number.isFinite(endTime) || endTime <= startTime) return;
852 if (!Number.isFinite(startTime) || !Number.isFinite(endTime) || endTime <= startTime) return;
850
853
851 if (reset || prefetchedThroughTimeSeconds < 0 || prefetchedThroughTimeSeconds < currentTime - IMAGE_SUBTITLE_PREFETCH_STEP_SECONDS)
854 if (reset || prefetchedThroughTimeSeconds < 0)
852 {
855 {
853 cancelPrefetchRequests({ resetProgress: true });
856 cancelPrefetchRequests({ resetProgress: true });
854 prefetchedThroughTimeSeconds = currentTime;
857 prefetchedThroughTimeSeconds = currentTime;
858 } else if (prefetchedThroughTimeSeconds < currentTime)
859 {
860 // Keep the prefetch frontier at or ahead of current playback so
861 // prefetch catches buffered-ahead range instead of replaying stale lag.
862 prefetchedThroughTimeSeconds = currentTime;
855 }
863 }
856
864
857 prefetchTargetTimeSeconds = Math.max(prefetchTargetTimeSeconds, endTime);
865 prefetchTargetTimeSeconds = Math.max(prefetchTargetTimeSeconds, endTime);
tests/shared-video-player.test.js +89 −0
@@ -227,6 +227,95 @@ describe('shared image subtitle prefetching', () =>
227 227 window.close();
228 228 });
229 229
230 test('prefetch uses furthest buffered end when player.bufferedEnd under-reports', async () =>
231 {
232 const window = createWindow();
233 const helper = loadSharedVideoPlayer(window);
234 const { player, videojs } = buildPlayerHarness(window);
235 const requestedTimes = [];
236
237 player.setBufferedEnd(0);
238 player.tech = () => ({
239 el: () => ({
240 buffered: {
241 length: 1,
242 end: () => 6,
243 },
244 }),
245 });
246
247 window.fetch = jest.fn(async (url) =>
248 {
249 const parsed = new URL(url, 'http://localhost/');
250 requestedTimes.push(Number(parsed.searchParams.get('timeSeconds')));
251 return new Response(null, {
252 status: 204,
253 headers: { 'X-Subtitle-Frame-Status': 'none' },
254 });
255 });
256
257 helper.bindImageSubtitleOverlay(player, videojs, {}, {
258 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
259 }, {
260 lookaheadSeconds: 0,
261 buildFrameRequest: ({ currentTime, streamIndex }) => ({
262 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
263 }),
264 });
265
266 await flushAsyncWork(window, 16);
267
268 expect(requestedTimes).toEqual(expect.arrayContaining([0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6]));
269
270 window.close();
271 });
272
273 test('prefetch frontier jumps to current playback time instead of replaying stale lag', async () =>
274 {
275 const window = createWindow();
276 const helper = loadSharedVideoPlayer(window);
277 const { player, videojs } = buildPlayerHarness(window);
278 const requestedTimes = [];
279
280 player.setBufferedEnd(35);
281
282 window.fetch = jest.fn(async (url) =>
283 {
284 const parsed = new URL(url, 'http://localhost/');
285 const timeSeconds = Number(parsed.searchParams.get('timeSeconds'));
286 requestedTimes.push(timeSeconds);
287 return new Response(null, {
288 status: 204,
289 headers: { 'X-Subtitle-Frame-Status': 'none' },
290 });
291 });
292
293 helper.bindImageSubtitleOverlay(player, videojs, {}, {
294 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
295 }, {
296 lookaheadSeconds: 0,
297 buildFrameRequest: ({ currentTime, streamIndex }) => ({
298 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
299 }),
300 });
301
302 await flushAsyncWork(window, 8);
303 expect(requestedTimes).toEqual(expect.arrayContaining([0, 0.5]));
304
305 requestedTimes.length = 0;
306 player.setCurrentTime(20);
307 player.setBufferedEnd(40);
308 player.trigger('timeupdate');
309 await flushAsyncWork(window, 8);
310
311 expect(requestedTimes).toContain(20);
312 const prefetchAfterJump = requestedTimes.filter((value) => value > 20);
313 expect(prefetchAfterJump.length).toBeGreaterThan(0);
314 expect(Math.min(...prefetchAfterJump)).toBeGreaterThanOrEqual(20.5);
315
316 window.close();
317 });
318
230 319 test('does not restart current subtitle request while one is in flight', async () =>
231 320 {
232 321 const window = createWindow();
@@ -227,6 +227,95 @@ describe('shared image subtitle prefetching', () =>
227 window.close();
227 window.close();
228 });
228 });
229
229
230 test('prefetch uses furthest buffered end when player.bufferedEnd under-reports', async () =>
231 {
232 const window = createWindow();
233 const helper = loadSharedVideoPlayer(window);
234 const { player, videojs } = buildPlayerHarness(window);
235 const requestedTimes = [];
236
237 player.setBufferedEnd(0);
238 player.tech = () => ({
239 el: () => ({
240 buffered: {
241 length: 1,
242 end: () => 6,
243 },
244 }),
245 });
246
247 window.fetch = jest.fn(async (url) =>
248 {
249 const parsed = new URL(url, 'http://localhost/');
250 requestedTimes.push(Number(parsed.searchParams.get('timeSeconds')));
251 return new Response(null, {
252 status: 204,
253 headers: { 'X-Subtitle-Frame-Status': 'none' },
254 });
255 });
256
257 helper.bindImageSubtitleOverlay(player, videojs, {}, {
258 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
259 }, {
260 lookaheadSeconds: 0,
261 buildFrameRequest: ({ currentTime, streamIndex }) => ({
262 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
263 }),
264 });
265
266 await flushAsyncWork(window, 16);
267
268 expect(requestedTimes).toEqual(expect.arrayContaining([0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6]));
269
270 window.close();
271 });
272
273 test('prefetch frontier jumps to current playback time instead of replaying stale lag', async () =>
274 {
275 const window = createWindow();
276 const helper = loadSharedVideoPlayer(window);
277 const { player, videojs } = buildPlayerHarness(window);
278 const requestedTimes = [];
279
280 player.setBufferedEnd(35);
281
282 window.fetch = jest.fn(async (url) =>
283 {
284 const parsed = new URL(url, 'http://localhost/');
285 const timeSeconds = Number(parsed.searchParams.get('timeSeconds'));
286 requestedTimes.push(timeSeconds);
287 return new Response(null, {
288 status: 204,
289 headers: { 'X-Subtitle-Frame-Status': 'none' },
290 });
291 });
292
293 helper.bindImageSubtitleOverlay(player, videojs, {}, {
294 imageSubtitleTracks: [{ label: 'PGS English', streamIndex: 0, default: true }],
295 }, {
296 lookaheadSeconds: 0,
297 buildFrameRequest: ({ currentTime, streamIndex }) => ({
298 url: `/subtitle-frame?streamIndex=${streamIndex}&timeSeconds=${currentTime}`,
299 }),
300 });
301
302 await flushAsyncWork(window, 8);
303 expect(requestedTimes).toEqual(expect.arrayContaining([0, 0.5]));
304
305 requestedTimes.length = 0;
306 player.setCurrentTime(20);
307 player.setBufferedEnd(40);
308 player.trigger('timeupdate');
309 await flushAsyncWork(window, 8);
310
311 expect(requestedTimes).toContain(20);
312 const prefetchAfterJump = requestedTimes.filter((value) => value > 20);
313 expect(prefetchAfterJump.length).toBeGreaterThan(0);
314 expect(Math.min(...prefetchAfterJump)).toBeGreaterThanOrEqual(20.5);
315
316 window.close();
317 });
318
230 test('does not restart current subtitle request while one is in flight', async () =>
319 test('does not restart current subtitle request while one is in flight', async () =>
231 {
320 {
232 const window = createWindow();
321 const window = createWindow();