IP Library › Granted Patent US 12,272,382
Granted Patent B2
US 12,272,382 · App. 17/921,148 · Granted Apr 8, 2025

Methods and systems for presenting media content with multiple media elements in an editing environment

Inventors: Ariel Braunstein (San Francisco, CA); Nicholas Matarese (San Francisco, CA)
Assignee: Google LLC
G11B27/031G06F3/0482G06F3/0485
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,272,382
App. No.
17/921,148
Filed
Oct 25, 2022
Granted
Apr 8, 2025
Kind
B2
Examiner
YANG, NIEN
Art Unit
2484
USPC
386/278
Abstract

In one example, a system comprises a hardware processor that is configured to: receive a request to modify a video content item containing a plurality of media elements; present a user interface that includes a video track representation of the video content item and a layered representation of the plurality of media elements, wherein each media element is represented by a media overlay element positioned proximal to the video track representation, wherein at least one media overlay element overlaps at least another media overlap element to provide a compact representation of the plurality of media elements; and in response to receiving a selected time position within the video track representation, update the layered representation within the user interface to present an expanded overlay list that includes media overlay elements corresponding to the subset of the plurality of media elements that occur at the selected time position.

Claims (43)

1. A method comprising:

receiving, by a computing device having a display, a request to modify a video content item containing a plurality of media elements;

presenting, by the computing device and using the display, a user interface that includes a video track representation of the video content item and a layered representation of the plurality of media elements occurring within the video content item, wherein each of the plurality of media elements is represented by a media overlay element of a plurality of media overlay elements that has one or more visual characteristics and is positioned proximal to the video track representation, wherein at least one of the plurality of media overlay elements overlaps at least another one of the plurality of media overlap elements in a z-order stacking to provide a compact representation of the plurality of media elements;

in response to receiving a selected time position within the video track representation, updating, by the computing device, the layered representation within the user interface to present an expanded overlay list that includes media overlay elements corresponding to a subset of the plurality of media elements that occur at the selected time position within the video content item; and

in response to receiving a selected media overlay element from the media overlay elements corresponding to the subset included in the expanded overlay list, updating, by the computing device, a media window in the user interface to present a video frame corresponding to the selected time position and the media element applied to the video frame corresponding to the selected media overlay element.

2. The method of claim 1 , wherein the plurality of media elements includes a plurality of visual layers and wherein the plurality of visual layers are presented in a first layered representation that is positioned above the video track representation in the user interface.

3. The method of claim 1 , wherein the plurality of media elements includes a plurality of audio layers and wherein the plurality of audio layers are presented in a second layered representation that is positioned below the video track representation in the user interface.

4. The method of claim 1 , wherein, in response to receiving the selected media overlay element from the expanded overlay list, positioning, by the computing device the selected media overlay element within the video track representation and presenting, by the computing device, unselected media overlay elements in the layered representation of the plurality of media elements occurring at the selected time position within the video content item.

5. The method of claim 1 , wherein the one or more visual characteristics includes at least one color applied to a media overlay element and at least one pattern applied to the media overlay element.

6. The method of claim 1 , wherein the expanded overlay list is presented in place of the layered representation of the plurality of media elements in response to receiving the selected time position within the video track representation and wherein the expanded overlay list is collapsed into the layered representation of the plurality of media elements in response to receiving the selected media overlay element from the expanded overlay list.

7. The method of claim 1 , further comprising providing, by the computing device, one or more editing tools associated with the media element applied to the video frame corresponding to the selected media overlay element.

8. The method of claim 1 , further comprising:

determining, by the computing device, that a first media element and a second media element share at least one timestamp on the video track representation; and

causing, by the computing device, a first media overlay element corresponding to the first media element to have a vertical offset on the video track representation from a second overlay element corresponding to the second media element.

9. The method of claim 1 , further comprising:

determining, by the computing device, that a height of the expanded overlay list being presented on the display of the computing device exceeds the height of the display of the computing device; and

in response to determining that the height of the expanded overlay list being presented on the display of the computing device exceeds the height of the display of the computing device, allowing, by the computing device, the expanded overlay list to be scrolled in a vertical direction to review each of the media overlay elements in the expanded overlay list.

10. The method of claim 1 , wherein each of the media overlay elements has a width that corresponds to a duration of the media element within the video content item.

11. A computing device system for editing media content comprising:

a display;

a hardware processor that is configured to:

receive a request to modify a video content item containing a plurality of media elements;

present, using the display, a user interface that includes a video track representation of the video content item and a layered representation of the plurality of media elements occurring within the video content item, wherein each of the plurality of media elements is represented by a media overlay element of a plurality of media overlay elements that has one or more visual characteristics and is positioned proximal to the video track representation, wherein at least one of the plurality of media overlay elements overlaps at least another one of the plurality of media overlap elements in a z-order stacking to provide a compact representation of the plurality of media elements;

in response to receiving a selected time position within the video track representation, update the layered representation within the user interface to present an expanded overlay list that includes media overlay elements corresponding to a subset of the plurality of media elements that occur at the selected time position within the video content item; and

in response to receiving a selected media overlay element from the media overlay elements corresponding to the subset included in the expanded overlay list, update a media window in the user interface to present a video frame corresponding to the selected time position and the media element applied to the video frame corresponding to the selected media overlay element.

12. The computing device of claim 11 , wherein the plurality of media elements includes a plurality of visual layers and wherein the plurality of visual layers are presented in a first layered representation that is positioned above the video track representation in the user interface.

13. The computing device of claim 11 , wherein the plurality of media elements includes a plurality of audio layers and wherein the plurality of audio layers are presented in a second layered representation that is positioned below the video track representation in the user interface.

14. The computing device of claim 11 , wherein, in response to receiving the selected media overlay element from the expanded overlay list, the hardware processor is further configured to position the selected media overlay element within the video track representation and present unselected media overlay elements in the layered representation of the plurality of media elements occurring at the selected time position within the video content item.

15. The computing device of claim 11 , wherein the one or more visual characteristics includes at least one color applied to a media overlay element and at least one pattern applied to the media overlay element.

16. The computing device of claim 11 , wherein the expanded overlay list is presented in place of the layered representation of the plurality of media elements in response to receiving the selected time position within the video track representation and wherein the expanded overlay list is collapsed into the layered representation of the plurality of media elements in response to receiving the selected media overlay element from the expanded overlay list.

17. The computing device of claim 11 , wherein the hardware processor is further configured to provide one or more editing tools associated with the media element applied to the video frame corresponding to the selected media overlay element.

18. The computing device of claim 11 , wherein the hardware processor is further configured to:

determine that a first media element and a second media element share at least one timestamp on the video track representation; and

cause a first media overlay element corresponding to the first media element to have a vertical offset on the video track representation from a second overlay element corresponding to the second media element.

19. The computing device of claim 11 , wherein the hardware processor is further configured to:

determine that a height of the expanded overlay list being presented on the display of the computing device exceeds the height of the display of the computing device; and

in response to determining that the height of the expanded overlay list being presented on the display of the computing device exceeds the height of the display of the computing device, allow the expanded overlay list to be scrolled in a vertical direction to review each of the media overlay elements in the expanded overlay list.

20. The computing device of claim 11 , wherein each of the media overlay elements has a width that corresponds to a duration of the media element within the video content item.

21. Non-transitory computer-readable storage media containing computer executable instructions that, when executed by a processor of a computing device having a display, cause the processor to:

receive a request to modify a video content item containing a plurality of media elements;

present, using the display, a user interface that includes a video track representation of the video content item and a layered representation of the plurality of media elements occurring within the video content item, wherein each of the plurality of media elements is represented by a media overlay element of a plurality of media overlay elements that has one or more visual characteristics and is positioned proximal to the video track representation, wherein at least one of the plurality of media overlay elements overlaps at least another one of the plurality of media overlap elements in a z-order stacking to provide a compact representation of the plurality of media elements;

in response to receiving a selected time position within the video track representation, update the layered representation within the user interface to present an expanded overlay list that includes media overlay elements corresponding to a subset of the plurality of media elements that occur at the selected time position within the video content item; and

in response to receiving a selected media overlay element from the media overlay elements corresponding to the subset included in the expanded overlay list, update a media window in the user interface to present a video frame corresponding to the selected time position and the media element applied to the video frame corresponding to the selected media overlay element.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2023
From: BRAUNSTEIN, ARIEL; MATARESE, NICHOLAS
To: GOOGLE LLC
Reel/Frame 065066/0480 →
Continuity (1)
Related Publication 20240233768A1 · Jul 11, 2024
References Cited (19)
US 10432559B2 · Baldwin et al. · 2019 [cited by applicant]
US 20060224940A1 · Lee · 2006 [cited by applicant]
US 20070239788A1 · Cunningham et al. · 2007 [cited by applicant]
US 20150019972A1 · Matsuda · 2015 [cited by examiner]
US 20170300189A1 · Pendergast · 2017 [cited by examiner]
US 20180364904A1 · Bernstein · 2018 [cited by examiner]
US 20200302029A1 · Holm · 2020 [cited by examiner]
JP 2018074337A · 2018 [cited by applicant]
KR 20100086136A · 2010 [cited by applicant]
KR 101862372B1 · 2018 [cited by applicant]
KR 101873897B1 · 2018 [cited by applicant]
KR 102274723B1 · 2020 [cited by applicant]
KR 20210059013A · 2021 [cited by applicant]
WO 2011128829A2 · 2011 [cited by applicant]
Office Action, and translation thereof, from counterpart Korean Application No. 10-2022-7039449 dated Sep. 26, 2023, 13 pp. [cited by applicant]
Notice of Intent to Grant, and translation thereof, from counterpart Korean Application No. 10-2022-7039449 dated Dec. 26, 2023, 5 pp. [cited by applicant]
Notice of Intent to Grant from counterpart Japanese Application No. 2022-568897 dated Jan. 22, 2024, 7 pp. [cited by applicant]
International Search Report and Written Opinion dated May 19, 2023 in International Patent Application No. PCT/US2021/058940. [cited by applicant]
Kost, J., “Using the New Video Features in Photoshop CS6 | Adobe”, YouTube, last updated Apr. 23, 2012, 1 page, available at: https://www.youtube.com/watch?v=praOG6jJxpc. [cited by applicant]