IP Library › Granted Patent US 12,301,920
Granted Patent B2
US 12,301,920 · App. 18/338,186 · Granted May 13, 2025

Audio transitions when streaming audiovisual media titles

Inventor: Mark Watson (San Francisco, CA)
Assignee: NETFLIX, INC.
H04N21/4307G06F16/7834G06V20/46G06V20/49H04N21/4394
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,301,920
App. No.
18/338,186
Granted
May 13, 2025
Kind
B2
Abstract

A playback application is configured to analyze audio frames associated with transitions between segments within a media title to identify one or more portions of extraneous audio. The playback application is configured to analyze the one or more portions of extraneous audio and then determine which of the one or more corresponding audio frames should be dropped. In doing so, the playback application can analyze a topology associated with the media title to determine whether any specific portions of extraneous audio are to be played outside of a logical ordering of audio samples set forth in the topology. These specific portions of extraneous audio are preferentially removed.

Claims (29)

1. A computer-implemented method for reducing playback of extraneous audio data, the method comprising:

identifying a first portion of extraneous audio data that does not align with either a first segment of video data or a second segment of video data that are played back sequentially when playing back a media title;

identifying a second portion of extraneous audio data that also does not align with either the first segment of video data or the second segment of video data;

determining, based on a topology of the media title that indicates that a third segment of video data that separates the first segment of video data and the second segment of video data is to be omitted during playback of the media title, that at least one of the first portion of extraneous audio data or the second portion of extraneous audio data conflicts with the topology; and

causing at least one of the first portion of extraneous audio data or the second portion of extraneous audio data to not be played back when playing back the first segment of video data and the second segment of video data.

2. The computer-implemented method of claim 1 , further comprising determining that one or more audio frames of the media title can be shifted relative to one or more video frames of the media title.

3. The computer-implemented method of claim 2 , wherein determining that one or more audio frames of the media title can be shifted comprises determining that shifting one or more audio frames of the media title would not cause a maximum allowable audio-visual synchronization error to be exceeded.

4. The computer-implemented method of claim 2 , wherein the first portion of extraneous audio data conflicts with the topology corresponding to the media title, but the second portion of extraneous audio data does not, and wherein causing at least one of the first portion of extraneous audio data or the second portion of extraneous audio data to not be played back comprises dropping one or more audio frames that include the first portion of extraneous audio data.

5. The computer-implemented method of claim 4 , further comprising shifting one or more audio frames that include the second portion of extraneous audio data to implement a transition from playing back audio data associated with the first segment of video data to playing back audio data associated with the second segment of video data.

6. The computer-implemented method of claim 2 , wherein both the first portion of extraneous audio data and the second portion of extraneous audio data conflict with the topology corresponding to the media title, and wherein the first portion of extraneous audio data is longer than the second portion of extraneous audio data, and wherein causing at least one of the first portion of extraneous audio data or the second portion of extraneous audio data to not be played back comprises dropping one or more audio frames that include the first portion of extraneous audio data.

7. The computer-implemented method of claim 1 , further comprising determining that one or more audio frames of the media title cannot be shifted relative to one or more video frames of the media title.

8. The computer-implemented method of claim 7 , wherein determining that one or more audio frames of the media title cannot be shifted comprises determining that shifting one or more audio frames of the media title would cause a maximum allowable audio-visual synchronization error to be exceeded.

9. The computer-implemented method of claim 7 , wherein the first portion of extraneous audio data does not conflict with the topology corresponding to the media title, and further comprising determining that a transition from playing back audio data associated with the first segment of video data to playing back audio data associated with the second segment of video data should be moved, and moving the transition past the first portion of extraneous audio data.

10. The computer-implemented method of claim 9 , wherein moving the transition past the first portion of extraneous audio data comprises causing an additional audio frame that is stored in physical memory immediately subsequent to an audio frame that includes the first portion of extraneous audio data to be played back.

11. The computer-implemented method of claim 10 , wherein the second portion of extraneous audio data does conflict with the topology corresponding to the media title, and wherein causing at least one of the first portion of extraneous audio data or the second portion of extraneous audio data to not be played back comprises dropping one or more audio frames that include the second portion of extraneous audio data.

12. The computer-implemented method of claim 7 , further comprising determining that a transition from playing back audio data associated with the first segment of video data to playing back audio data associated with the second segment of video data should not be moved.

13. The computer-implemented method of claim 12 , wherein the first portion of extraneous audio data does conflict with the topology corresponding to the media title, and wherein causing at least one of the first portion of extraneous audio data or the second portion of extraneous audio data to not be played back comprises replacing one or more audio frames that include the first portion of extraneous audio data with one or more audio frames that are adjacent to first portion of extraneous audio data.

14. The computer-implemented method of claim 13 , wherein the one or more audio frames that are adjacent to the first portion of extraneous audio data include audio samples that do not conflict with the topology corresponding to the media title.

15. The computer-implemented method of claim 1 , wherein the first segment of video data and the second segment of video data comprise sequential video segments in a logical ordering of video segments for playback.

16. The computer-implemented method of claim 15 , wherein the first segment of video data is not stored in physical memory immediately adjacent to the second segment of video data.

17. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform that step of:

identifying a first portion of extraneous audio data that does not align with either a first segment of video data or a second segment of video data that are played back sequentially when playing back a media title;

determining, based on a topology of the media title that indicates that a third segment of video data that separates the first segment of video data and the second segment of video data is to be omitted during playback of the media title, that the first portion of extraneous audio data conflicts with the topology; and

performing at least one operation that causes the first portion of extraneous audio data to not be played back when playing back the first segment of video data and the second segment of video data.

18. The one or more non-transitory computer-readable media of claim 17 , wherein the first segment of video data and the second segment of video data comprise sequential video segments in a logical ordering of video segments for playback.

19. The one or more non-transitory computer-readable media of claim 17 , wherein the first segment of video data is not stored in physical memory immediately adjacent to the second segment of video data.

20. The one or more non-transitory computer-readable media of claim 17 , further comprising determining that one or more audio frames of the media title can be shifted relative to one or more video frames of the media title.

21. The one or more non-transitory computer-readable media of claim 20 , wherein determining that one or more audio frames of the media title can be shifted comprises determining that shifting one or more audio frames of the media title would not cause a maximum allowable audio-visual synchronization error to be exceeded.

22. The one or more non-transitory computer-readable media of claim 21 , wherein performing the at least one operation comprises dropping one or more audio frames that include the first portion of extraneous audio data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2023
From: WATSON, MARK
To: NETFLIX, INC.
Reel/Frame 064051/0331 →
Continuity (3)
Continuation 17746823 · May 17, 2022
Continuation 16570880 · Sep 13, 2019
Related Publication 20230336809A1 · Oct 19, 2023
References Cited (39)
US 4430676A · Johnson · 1984 [cited by applicant]
US 6181383B1 · Fox et al. · 2001 [cited by applicant]
US 6630963B1 · Billmaier · 2003 [cited by applicant]
US 9082092B1 · Henry · 2015 [cited by applicant]
US 9400555B2 · Quigley et al. · 2016 [cited by applicant]
US 10250937B2 · Pantos et al. · 2019 [cited by applicant]
US 10638180B1 · Pringle et al. · 2020 [cited by applicant]
US 10681104B1 · Wu et al. · 2020 [cited by applicant]
US 11336947B2 · Watson · 2022 [cited by examiner]
US 11503264B2 · Watson · 2022 [cited by applicant]
US 11700415B2 · Watson · 2023 [cited by examiner]
US 20020018645A1 · Nakamatsu et al. · 2002 [cited by applicant]
US 20030033325A1 · Boogaard · 2003 [cited by applicant]
US 20040096190A1 · Uenaka et al. · 2004 [cited by applicant]
US 20040196989A1 · Friedman et al. · 2004 [cited by applicant]
US 20040226035A1 · Hauser, Jr. · 2004 [cited by applicant]
US 20080062998A1 · Kim et al. · 2008 [cited by applicant]
US 20080075417A1 · Mizuno · 2008 [cited by applicant]
US 20080095514A1 · Yahata · 2008 [cited by examiner]
US 20110138433A1 · Whiteing · 2011 [cited by applicant]
US 20110150099A1 · Owen · 2011 [cited by applicant]
US 20130129302A1 · Meadowcroft et al. · 2013 [cited by applicant]
US 20130159004A1 · Hoerich · 2013 [cited by applicant]
US 20140118468A1 · Purdy et al. · 2014 [cited by applicant]
US 20140230631A1 · Wieder · 2014 [cited by applicant]
US 20140354434A1 · Lalonde · 2014 [cited by applicant]
US 20150039302A1 · Jarvinen et al. · 2015 [cited by applicant]
US 20160012857A1 · Leppanen et al. · 2016 [cited by applicant]
US 20160322054A1 · Bloch et al. · 2016 [cited by applicant]
US 20170013329A1 · Um · 2017 [cited by applicant]
US 20170062006A1 · Plom et al. · 2017 [cited by applicant]
US 20180218749A1 · Fejgin et al. · 2018 [cited by applicant]
US 20180218756A1 · McIntosh et al. · 2018 [cited by applicant]
US 20190028525A1 · Kansara · 2019 [cited by applicant]
US 20190215542A1 · Gandhi et al. · 2019 [cited by applicant]
US 20200314496A1 · Wittke · 2020 [cited by applicant]
JP 2001008157A · 2001 [cited by applicant]
WO 2019055275A1 · 2019 [cited by applicant]
Krishnamoorthi et al., “Quality-Adaptive Prefetching for Interactive Branched Video using HTTP-based Adaptive Streaming”, Multimedia, ACM, XP058058715, DOI: 10.1145/2647868.2654951, Nov. 3-7, 2014, pp. 317-326. [cited by applicant]