IP Library Granted Patent US 12,271,420
Granted Patent B1
US 12,271,420 · App. 17/833,790 · Granted Apr 8, 2025

Video segments for a video related to a task

Inventors: Kerwell Liao (San Francisco, CA); Nikhil Sharma (Mountain View, CA); LaDawn Risenmay Jentzsch (Mountain View, CA); Jennifer Ellen Fernquist Seth (San Francisco, CA)
Assignee: GOOGLE LLC
G06F16/7837G06F3/0481G06F16/73G06F16/78G06F16/7844G06F16/7867G06F16/70G06F16/738G06F16/74G06F16/743G06F16/783G06F16/93G06F16/951G06F16/9535
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,271,420
App. No.
17/833,790
Granted
Apr 8, 2025
Kind
B1
Abstract

Methods and apparatus related to identifying a video for completing a task and determining a plurality of video segments of the identified video based on one or more attributes of the task. A task and a plurality of how-to videos related to the task may be identified. A how-to video may be selected and a plurality of video segments of the selected how-to video may be determined. One or more video segments may be associated with one or more task attributes that relate to performing the task. The selected video may be provided to a user and segmented, indexed, and/or annotated based on the associated video segments. In some implementations a given object utilized in performing the task may be identified and one or more video segments corresponding to the given object may be identified and/or provided to the user.

Claims (53)

1. A method implemented by one or more processors, the method comprising:

identifying, for a given video, one or more transitional indicators indicative of a transition of the given video;

determining a motion flow based on a gradient flow that is quantitatively grouped to identify the one or more transitional indicators;

segmenting the given video, based on a transcript and the motion flow of the given video, into a plurality of video segments;

determining that a given segment of the plurality of video segments is associated with a key phrase;

associating the given segment with the key phrase;

subsequent to the associating

receiving a query; and

in response to receiving the query:

identifying the given video as being responsive to the query,

selecting a given segment from the plurality of video segments of the identified given video, based on a confidence measure of the given segment satisfying a threshold, and

providing a search result for the query.

2. The method of claim 1 , wherein:

the key phrase is stored in a content database.

3. The method of claim 2 , wherein associating the given segment with the key phrase includes:

generating an entry for the given segment in the content database in association with the key phrase, wherein the entry for the given segment includes the selectable link to the given segment.

4. The method of claim 1 , wherein the query is a spoken query including one or more terms matching the key phrase.

5. The method of claim 1 , wherein the key phrase is detected in a transcript of the given segment, and wherein the transcript of the given video includes the transcript of the given segment.

6. The method of claim 1 , wherein the given segment is separated from the given video into a stand-alone video.

7. The method of claim 1 , wherein the given segment is enhanced with an improved image and/or sound quality.

8. The method of claim 1 , wherein the transcript of the given video includes a textual transcript acquired by transcribing an audio of the given video.

9. The method of claim 1 , wherein the one or more transitional indicators includes a transitional indicator indicative of a transition from the given segment to another video segment, of the plurality of video segments, of the given video.

10. The method of claim 9 , wherein the given video is segmented into the plurality of video segments additionally based on motion characteristics of the given video, wherein the motion characteristics of the given video include one or more motion vectors and the motion flow.

11. The method of claim 1 , wherein the one or more transitional indicators include one or more transitional terms and/or one or more shot transitions, and wherein the one or more shot transitions include hard cut, fade, or dissolve.

12. The method of claim 1 , wherein:

one or more critical values of the gradient flow correspond to the one or more transitional indicators respectively, and

a portion of the gradient flow between two critical values, of the one or more critical values, represent the given segment.

13. The method of claim 1 , wherein the given segment corresponds to a commercial and/or sponsored segment.

14. The method of claim 1 , wherein the given video is segmented into the plurality of video segments additionally based on a video text, wherein the video text includes one or more topic change markers identified in closed caption annotators.

15. The method of claim 1 , wherein the confidence measure of the given segment is determined at least based on a relevance score of the given segment to the query.

16. The method of claim 1 , wherein the confidence measure of the given segment is further determined based on a measure of popularity of the given segment.

17. The method of claim 1 , further comprising:

providing, in response to the query, a visual indication of the confidence measure of the given segment.

18. A method implemented by one or more processors, the method comprising:

identifying, for a given video, one or more transitional indicators indicative of a transition of the given video;

determining a motion flow based on a gradient flow that is quantitatively grouped to identify the one or more transitional indicators;

segmenting the given video, based on a transcript and the motion flow of the given video, into a plurality of video segments;

determining that a given segment of the plurality of video segments is associated with a key phrase detected from the transcript of the given video;

associating a link of the given segment with the key phrase detected from the transcript of the given video;

subsequent to the associating

receiving a query; and

in response to receiving the query:

identifying the given video as being responsive to the query,

selecting a given segment from the plurality of video segments of the identified given video, based on a confidence measure of the given segment, and

providing a search result for the query, wherein the search result identifies the given video and includes the link.

19. A method implemented by one or more processors, the method comprising:

receiving a query;

identifying a given video responsive to the query;

determining that one or more given segments of the given video are associated with a confidence measure of the given video that satisfies a threshold,

wherein the one or more given segments are segmented from the given video based on a transcript of the given video, and based on a motion flow that is determined based on a gradient flow that is quantitatively grouped to identify one or more transitional indicators identified for the given video,

wherein the one or more given segments are respectively associated with a key phrase detected from the transcript of the given video, and

wherein the confidence measure of the given video is determined at least based on matching the key phrases with which the one or more given segments are associated to one or more terms of the query; and

provide, in response to the query and based on the confidence measure satisfying the threshold, a link to a respective one of the one or more given segments of the given video.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2022
From: LIAO, KERWELL; SHARMA, NIKHIL; JENTZSCH, LADAWN RISENMAY
To: GOOGLE INC.
Reel/Frame 060133/0758 →
CHANGE OF NAME Recorded Jun 8, 2022
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 060310/0390 →
Continuity (3)
Continuation 16416882 · May 20, 2019
Continuation 15055000 · Feb 26, 2016
Continuation 13927533 · Jun 26, 2013
References Cited (13)
US 6032151A · Arnold et al. · 2000 [cited by applicant]
US 9304648B2 · Liao et al. · 2016 [cited by applicant]
US 10296640B1 · Liao et al. · 2019 [cited by applicant]
US 10394887B2 · Chowdhury · 2019 [cited by examiner]
US 20030011629A1 · Rouse · 2003 [cited by examiner]
US 20070071323A1 · Kontsevich et al. · 2007 [cited by applicant]
US 20070118873A1 · Houh · 2007 [cited by examiner]
US 20070244902A1 · Seide · 2007 [cited by examiner]
US 20080046406A1 · Seide · 2008 [cited by examiner]
US 20080086688A1 · Chandratillake · 2008 [cited by examiner]
US 20100046830A1 · Wang · 2010 [cited by examiner]
US 20110099195A1 · Patwardhan · 2011 [cited by examiner]
US 20120323891A1 · Jacobson · 2012 [cited by examiner]