IP Library Granted Patent US 10,192,362
Granted Patent B2
US 10,192,362 · App. 15/336,713 · Granted Jan 29, 2019

Generating virtual reality and augmented reality content for a live event

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,192,362
App. No.
15/336,713
Granted
Jan 29, 2019
Kind
B2
Abstract

A content visualization system generates visual content for a visualization device based on visual content of a live event. The content visualization system collects visual content and source perspective data from visual content sources. The visualization device requests visual content from the content visualization system by providing device perspective data to the content visualization system. The content visualization system generates visual content for the visualization device based on the visual content from the visual content sources, the source perspective data, and the device perspective data. The content visualization system can determine visual content that is relevant to the device perspective by identifying source perspectives that overlap with the device perspective. The content visualization system generates visual content for the visualization device based on the identified visual content. The content visualization system can generate composited or non-composited visual content, based on the similarity of the source perspectives to the device perspective.

Claims (44)

1. A method comprising:

receiving visual content of a live event from a plurality of visual content sources;

receiving source perspective data from the plurality of visual content sources, the source perspective data describing, for each visual content source of the plurality of visual content sources, a source perspective of the live event;

receiving device perspective data from a visualization device, the device perspective data describing a device perspective of the visualization device, the device perspective being a perspective of the live event to be presented to the a user of the visualization device;

identifying one or more visual content sources of the plurality of visual content sources that are associated with a source perspective that overlaps with the device perspective, the one or more visual content sources being identified based on the source perspective data and the device perspective data;

computing one or more similarity scores for each of the identified one or more visual content sources based on the device perspective data and the source perspective data, each similarity score representing the similarity of a corresponding source perspective to the device perspective;

generating visualization content for the visualization device based on the one or more similarity scores and visual content from the identified one or more visual content sources;

wherein generating the visualization content for the visualization device comprises:

determining, based on the computed one or more similarity scores, whether to generate composite visualization content or non-composite visualization content;

generating, responsive to determining to generate the composite visualization content, the composite visualization content based on two or more of the identified one or more visual content sources; and

generating, responsive to determining to generate the non-composite visualization content, the non-composite visualization content based on exactly one of the identified one or more visual content sources; and

transmitting the generated visualization content to the visualization device.

2. The method of claim 1 , wherein the source perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of a visual content source, a frame rate of a visual content source, and a resolution of a visual content source.

3. The method of claim 1 , wherein the device perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of the visualization device, a frame rate of the visualization device, and a resolution of the visualization device.

4. The method of claim 1 , wherein the visual content comprises at least one of video data, image data, and audio data.

5. The method of claim 1 , wherein identifying the one or more visual content sources that are associated with the source perspective that overlaps with the device perspective further comprises generating a model of the device perspective and the source perspectives of the plurality of visual content sources.

6. The method of claim 1 , wherein the one or more similarity scores are computed based on amounts of overlap of the corresponding source perspectives with the device perspective.

7. The method of claim 1 , wherein the one or more similarity scores are computed based on differences in angle of direction of the corresponding source perspectives from the device perspective.

8. The method of claim 1 , wherein the one or more similarity scores are computed based on differences in position of the corresponding source perspectives from the device perspective.

9. The method of claim 1 , wherein determining whether to generate the composite visualization content or the non-composite visualization content comprises comparing the one or more similarity scores to a similarity score threshold.

10. The method of claim 1 , wherein generating the non-composite visualization content comprises:

identifying a visual content source with a highest similarity score of the one or more similarity scores; and

generating the non-composite visualization content based on visual content from the identified visual content source.

11. The method of claim 1 , wherein generating the composite visualization content comprises generating a linear combination of visual content from two or more of the identified one or more visual content sources.

12. A computer program product comprising a non-transitory computer-readable storage medium comprising instructions encoded thereon that, when executed by a processor, causes the processor to:

receive visual content of a live event from a plurality of visual content sources;

receive source perspective data from the plurality of visual content sources, the source perspective data describing, for each visual content source of the plurality of visual content sources, a source perspective of the live event;

receive device perspective data from a visualization device, the device perspective data describing a device perspective of the visualization device, the device perspective being a perspective of the live event to be presented to the a user of the visualization device;

identify one or more visual content sources of the plurality of visual content sources that are associated with a source perspective that overlaps with the device perspective, the one or more visual content sources being identified based on the source perspective data and the device perspective data;

compute one or more similarity scores for each of the identified one or more visual content sources based on the device perspective data and the source perspective data, each similarity score representing the similarity of a corresponding source perspective to the device perspective;

generate visualization content for the visualization device based on the one or more similarity scores and visual content from the identified one or more visual content sources;

wherein the instructions to generate the visualization content for the visualization device further comprises instructions that when executed causes the processor to:

determine, based on the computed one or more similarity scores, whether to generate composite visualization content or non-composite visualization content;

generate, responsive to determining to generate the composite visualization content, the composite visualization content based on two or more of the identified one or more visual content sources; and

generate, responsive to determining to generate the non-composite visualization content, the non-composite visualization content based on exactly one of the identified one or more visual content sources; and

transmit the generated visualization content to the visualization device.

13. The computer program product of claim 12 , wherein the source perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of a visual content source, a frame rate of a visual content source, and a resolution of a visual content source.

14. The computer program product of claim 12 , wherein the device perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of the visualization device, a frame rate of the visualization device, and a resolution of the visualization device.

15. The computer program product of claim 12 , wherein the one or more similarity scores are computed based on amounts of overlap of the corresponding source perspectives with the device perspective.

16. The computer program product of claim 12 , wherein the instructions to determine whether to generate the composite visualization content or the non-composite visualization content further comprises instructions that when executed causes the processor to compare the one or more similarity scores to a similarity score threshold.

17. The computer program product of claim 12 , wherein the instructions to generate the non-composite visualization content further comprises instructions that when executed causes the processor to:

identify a visual content source with a highest similarity score of the one or more similarity scores; and

generate the non-composite visualization content based on visual content from the identified visual content source.

18. The computer program product of claim 12 , wherein the instructions to generate the composite visualization content further comprises instructions that when executed causes the processor to generate a linear combination of visual content from two or more of the identified one or more visual content sources.

Assignments (4)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2016
From: CAMPBELL, SCOTT PATRICK; FONG, GARY
To: GOPRO, INC.
Reel/Frame 040203/0822 →