IP Library Granted Patent US 12,033,672
Granted Patent B1
US 12,033,672 · App. 18/531,151 · Granted Jul 9, 2024

Methods and systems for generating interactive composite media assets comprising common objects

Inventors: Michael Joseph Karlin (Bethesda, MD); Roy Feinson (Washington, DC)
Assignee: EmergeX, LLC
G11B27/34G06F16/435G06F16/487G06T3/60G06T5/50G06T5/70G06T5/77G06T7/66G06T7/74G11B27/036G06T2200/24G06T2207/10016G06T2207/20092G06T2207/20221G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,033,672
App. No.
18/531,151
Filed
Dec 6, 2023
Granted
Jul 9, 2024
Kind
B1
Art Unit
2484
USPC
386/282
Abstract

Methods and systems are disclosed herein to address the problems discussed above and in particular to provide coordination for uncoordinated content. Beyond simply archiving and/or organizing available content, the methods and systems generate composite media assets that provide enriched details and supplemental data that extends beyond any given source content. The methods and systems achieve this through a novel data filtration and synchronization process.

Claims (85)

1. A system for generating interactive composite media assets comprising common objects by coordinating uncoordinated content using best-fit models applied to perimeters about the common objects, the system comprising:

one or more processors; and

one or more non-transitory, computer-readable mediums comprising instructions that when executed by the one or more processors causes operations comprising:

receiving source content comprising uncoordinated media assets generating using a plurality of digital cameras;

receiving a first user input requesting a composite media asset based on a center-of-mass point for a shared geographic location at a shared time window;

in response to the first user input, performing a data filtration process by:

determining a shared object orientation based on the center-of-mass point for the shared geographic location, wherein the shared object orientation comprises a direction that a digital camera of the plurality of digital cameras faced when capturing a respective media asset from locations, as determined by applying a best-fit mechanism to the center-of-mass point, on a near-continuous perimeter about the center-of-mass point;

retrieving respective media asset data structures for two media assets, wherein the respective media asset data structures comprise a geographic location corresponding to the two media assets, a time corresponding to the two media assets, and an object included with the two media assets;

determining that the two media assets correspond to the geographic location and time;

in response to determining that the two media assets correspond to the geographic location and time, determining that the two media assets correspond to the shared object orientation based on (i) analyzing the object and (ii) determining that the two media assets correspond to one or more of the locations on the near-continuous perimeter about the center-of-mass point; and

in response to determining that the two media assets correspond to the shared object orientation, synchronizing the two media assets by generating the composite media asset.

2. A method for generating interactive composite media assets comprising common objects by coordinating uncoordinated content using best-fit models applied to perimeters about the common objects, the method comprising:

receiving a first user input requesting a composite media asset based on a center-of-mass point for a shared geographic location at a shared time window;

determining a shared object orientation based on the center-of-mass point for the shared geographic location, wherein the shared object orientation comprises a direction that a content capture device faced when capturing a respective media asset and locations on a near-continuous perimeter about the center-of-mass point;

retrieving a first media asset data structure for a first media asset, wherein the first media asset data structure comprises first location information, first time information, and first object information;

retrieving a second media asset data structure for a second media asset, wherein the second media asset data structure comprises second location information, second time information, and second object information;

determining that the first media asset and the second media asset correspond to the shared geographic location based on analyzing the first location information and the second location information;

in response to determining that the first media asset and the second media asset correspond to the shared geographic location, determining that the first media asset and the second media asset correspond to the shared time window based on analyzing the first time information and the second time information;

in response to determining that the first media asset and the second media asset correspond to the shared time window, determining that the first media asset and the second media asset correspond to the shared object orientation based on (i) analyzing the first object information and the second object information and (ii) determining that the first media asset and the second media asset correspond to one or more of the locations on the near-continuous perimeter about the center-of-mass point;

in response to determining that the first media asset and the second media asset correspond to the shared object orientation, generating the composite media asset, in a user interface, based on the first media asset and the second media asset, by merging the first media asset and the second media asset about the center-of-mass point;

receiving, via the user interface, a second user input requesting a movement within the shared geographic location; and

in response to the second user input, determining a new center-of-mass point for the composite media asset.

3. The method of claim 2 , wherein the locations on the near-continuous perimeter about the center-of-mass point are determined by filtering media assets to determine a smallest contained shape about the center-of-mass point.

4. The method of claim 2 , wherein the locations on the near-continuous perimeter about the center-of-mass point are determined by applying a best-fit mechanism to the center-of-mass point.

5. The method of claim 2 , wherein merging the first media asset and the second media asset about the center-of-mass point further comprises:

detecting an object detail in the first media asset; and

rotating the first media asset based on the object detail.

6. The method of claim 2 , wherein merging the first media asset and the second media asset about the center-of-mass point further comprises:

detecting a first object in the first media asset, wherein the first object has a first scale;

detecting a second object in the second media asset, wherein the second object has a second scale; and

using scale spaced merging to merge the first object and the second object.

7. The method of claim 2 , wherein merging the first media asset and the second media asset about the center-of-mass point further comprises:

determining a number of objects in the first media asset and the second media asset to be blurred; and

determining a level of blur based on the number of objects.

8. The method of claim 2 , further comprising:

generating for display, in the user interface, a mapping of available media assets for the composite media asset; and

generating for display, in the user interface, an indicator of the available media assets based on the first location information and the second location information.

9. The method of claim 2 , wherein determining that the first media asset and the second media asset correspond to the shared geographic location further comprises:

receiving a third user input indicating the shared geographic location; and

in response to receiving the third user input indicating the shared geographic location, filtering a plurality of available media assets based on a comparison of respective location information for the plurality of available media assets and the shared geographic location to generate a first subset of media assets.

10. The method of claim 9 , wherein determining that the first media asset and the second media asset correspond to the shared time window further comprises:

receiving a fourth user input indicating the shared time window; and

in response to receiving the fourth user input indicating the shared time window, filtering the first subset of media assets based on a comparison of respective time information for the first subset of media assets and the shared time window to generate a second subset of media assets.

11. The method of claim 2 , wherein determining that the first media asset and the second media asset correspond to the shared object orientation further comprises:

identifying a known object corresponding to the center-of-mass point;

retrieving a plurality of known object details for the known object at the shared object orientation; and

determining a known object detail of the plurality of known object details is in both the first media asset and the second media asset.

12. The method of claim 2 , wherein generating the composite media asset based on the first media asset and the second media asset by merging the first media asset and the second media asset further comprises:

identifying a shared object in both the first media asset and the second media asset; and

generating a representation of the shared object in the composite media asset using a first object detail from the first media asset and a second object detail from the second media asset,

wherein the second media asset does not comprise the first object detail and the first media asset does not comprise the second object detail.

13. The method of claim 2 , wherein:

the first location information indicates a first geographic location corresponding to the first media asset;

the first time information indicates a first time corresponding to the first media asset; and

the first object information indicates a first object included with the first media asset.

14. The method of claim 2 , wherein the first media asset comprises a plurality of frames, and wherein retrieving the first media asset data structure for the first media asset further comprises:

determining a first frame of the plurality of frames for generating the composite media asset;

determining a subset of the first media asset data structure that corresponds to the first frame; and

retrieving the first location information, the first time information, and the first object information from the subset of the first media asset data structure.

15. The method of claim 2 , wherein generating the composite media asset based on the first media asset and the second media asset further comprises:

identifying a first portion of the first media asset corresponding to an out-of-focus object; selecting a second portion of the second media asset corresponding to the out-of-focus object in the first media asset; and

replacing the first portion of the first media asset with the second portion.

16. The method of claim 2 , wherein receiving the first user input requesting the composite media asset based on the center-of-mass point for the shared geographic location at the shared time window comprises:

receiving a user selection of an object in the first media asset;

determining a geographic location in which the object is found;

assigning the geographic location as the shared geographic location; and

assigning a position of the object at the geographic location as the center-of-mass point.

17. A non-transitory, computer-readable medium comprising instructions that when executed by one or more processors cause operations comprising:

receiving a first user input requesting a composite media asset based on a center-of-mass point for a shared geographic location at a shared time window;

determining a shared object orientation based on the center-of-mass point for the shared geographic location, wherein the shared object orientation comprises a direction that a content capture device faced when capturing a respective media asset and locations, determined by filtering media assets to determine a smallest contained shape about the center-of-mass point, on a near-continuous perimeter about the center-of-mass point;

retrieving a first media asset data structure for a first media asset, wherein the first media asset data structure comprises first location information, first time information, and first object information;

retrieving a second media asset data structure for a second media asset, wherein the second media asset data structure comprises second location information, second time information, and second object information;

determining that the first media asset and the second media asset correspond based on comparing the first media asset data structure to the second media asset data structure;

in response to determining that the first media asset and the second media asset correspond, determining that the first media asset and the second media asset correspond to the shared object orientation based on (i) analyzing the first object information and the second object information and (ii) determining that the first media asset and the second media asset correspond to one or more of the locations on the near-continuous perimeter about the center-of-mass point; and

in response to determining that the first media asset and the second media asset correspond to the shared object orientation, generating the composite media asset based on the first media asset and the second media asset by merging the first media asset and the second media asset about the center-of-mass point, wherein the composite media asset comprises a portion of a 360 degree media asset from the locations on the near-continuous perimeter about the center-of-mass point.

18. The non-transitory, computer-readable medium of claim 17 , wherein determining that the first media asset and the second media asset correspond to the shared geographic location further comprises:

receiving a second user input indicating the shared geographic location; and

in response to receiving the second user input indicating the shared geographic location, filtering a plurality of available media assets based on a comparison of respective location information for the plurality of available media assets and the shared geographic location to generate a first subset of media assets.

19. The non-transitory, computer-readable medium of claim 18 , wherein determining that the first media asset and the second media asset correspond to the shared time window further comprises:

receiving a third user input indicating the shared time window; and

in response to receiving the third user input indicating the shared time window, filtering first subset of media assets based on a comparison of respective time information for the first subset of media assets and the shared time window to generate a second subset of media assets.

20. The non-transitory, computer-readable medium of claim 17 , wherein determining that the first media asset and the second media asset correspond to the shared object orientation further comprises:

identifying a known object corresponding to the center-of-mass point;

retrieving a plurality of known object details for the known object at the shared object orientation; and

determining a known object detail of the plurality of known object details is in both the first media asset and the second media asset.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 6, 2024
From: KARLIN, MICHAEL JOSEPH; FEINSON, ROY
To: EMERGEX, LLC
Reel/Frame 066672/0504 →
Continuity (2)
Continuation 17582979 · Jan 24, 2022
Provisional Application 63141171 · Jan 25, 2021
Cited By (3)
US 12,211,279 US 12,287,822 US 12,730,838