IP Library Granted Patent US 10,818,264
Granted Patent B2
US 10,818,264 · App. 16/232,500 · Granted Oct 27, 2020

Generating virtual reality and augmented reality content for a live event

Inventors: Scott Patrick Campbell (Belmont, CA); Gary Fong (Cupertino, CA)
Assignee: GoPro, Inc.
G09G5/006G06K9/00744G06K9/3233G06K9/6215H04N21/2187H04N21/21805H04N21/4402
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,818,264
App. No.
16/232,500
Granted
Oct 27, 2020
Kind
B2
Abstract

A content visualization system generates visual content for a visualization device based on visual content of an event. The content visualization system collects visual content and source perspective data from visual content sources. The visualization device requests visual content from the content visualization system by providing device perspective data to the content visualization system. The content visualization system generates visual content for the visualization device based on the visual content from the visual content sources, the source perspective data, and the device perspective data. The content visualization system can determine visual content that is relevant to the device perspective by identifying source perspectives that overlap with the device perspective. The content visualization system generates visual content for the visualization device based on the identified visual content. The content visualization system can generate composited or non-composited visual content, based on the similarity of the source perspectives to the device perspective.

Claims (36)

1. A method comprising:

receiving visual content of an event from a plurality of visual content sources;

receiving source perspective data for the plurality of visual content sources, the source perspective data describing source perspectives of the event for the plurality of visual content sources;

receiving device perspective data for a visualization device, the device perspective data describing a device perspective of the visualization device, the device perspective being a perspective of the event to be presented to a user of the visualization device;

identifying one or more visual content sources of the plurality of visual content sources based on the device perspective data and the source perspective data, the one or more visual content sources associated with one or more source perspectives that overlap with the device perspective;

computing one or more similarity scores for the one or more visual content sources based on the device perspective data and the source perspective data, a similarity score representing similarity of a corresponding source perspective to the device perspective;

generating, based on the one or more similarity scores, composite visualization content or non-composite visualization content, the composite visualization content generated based on visual content from two or more of the one or more visual content sources and the non-composite visualization content generated based on visual content from exactly one of the one or more visual content sources, wherein generating the composite visualization content comprises generating a linear combination of the visual content from the two or more of the one or more visual content sources; and

transmitting the composite visualization content or the non-composite visualization content to the visualization device.

2. The method of claim 1 , wherein the source perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of a visual content source, a frame rate of the visual content source, and a resolution of the visual content source.

3. The method of claim 1 , wherein the device perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of the visualization device, a frame rate of the visualization device, and a resolution of the visualization device.

4. The method of claim 1 , wherein the visual content comprises at least one of video data, image data, and audio data.

5. The method of claim 1 , wherein identifying the one or more visual content sources comprises generating a model of the device perspective and the source perspectives.

6. The method of claim 1 , wherein the one or more similarity scores are computed based on amounts of overlap of the corresponding source perspectives with the device perspective.

7. The method of claim 1 , wherein the one or more similarity scores are computed based on differences in angle of direction of the corresponding source perspectives from the device perspective.

8. The method of claim 1 , wherein the one or more similarity scores are computed based on differences in position of the corresponding source perspectives from the device perspective.

9. The method of claim 1 , wherein generating the composite visualization content or the non-composite visualization content comprises comparing the one or more similarity scores to a similarity score threshold and determining whether to generate the composite visualization content or the non-composite visualization content based on the comparison of the one or more similarity scores to the similarity score threshold.

10. The method of claim 1 , wherein generating the non-composite visualization content comprises:

identifying a visual content source with a highest similarity score of the one or more similarity scores; and

generating the non-composite visualization content based on visual content from the identified visual content source.

11. A computer program product comprising a non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, causes the processor to:

receive visual content of an event from a plurality of visual content sources;

receive source perspective data for the plurality of visual content sources, the source perspective data describing source perspectives of the event for the plurality of visual content sources;

receive device perspective data for a visualization device, the device perspective data describing a device perspective of the visualization device, the device perspective being a perspective of the event to be presented to a user of the visualization device;

identify one or more visual content sources of the plurality of visual content sources based on the device perspective data and the source perspective data, the one or more visual content sources associated with one or more source perspectives that overlap with the device perspective;

compute one or more similarity scores for the one or more visual content sources based on the device perspective data and the source perspective data, a similarity score representing similarity of a corresponding source perspective to the device perspective;

generate, based on the one or more similarity scores, composite visualization content or non-composite visualization content, the composite visualization content generated based on visual content from two or more of the one or more visual content sources and the non-composite visualization content generated based on visual content from exactly one of the one or more visual content sources, wherein the instructions to generate the composite visualization content further comprises instructions that when executed causes the processor to generate a linear combination of the visual content from the two or more of the one or more visual content sources; and

transmit the composite visualization content or the non-composite visualization content to the visualization device.

12. The computer program product of claim 11 , wherein the source perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of a visual content source, a frame rate of the visual content source, and a resolution of the visual content source.

13. The computer program product of claim 11 , wherein the device perspective data comprises at least one of: GPS data, accelerometer data, electronic compass data, a field of view of the visualization device, a frame rate of the visualization device, and a resolution of the visualization device.

14. The computer program product of claim 11 , wherein the one or more similarity scores are computed based on amounts of overlap of the corresponding source perspectives with the device perspective.

15. The computer program product of claim 11 , wherein the one or more similarity scores are computed based on differences in angle of direction of the corresponding source perspectives from the device perspective.

16. The computer program product of claim 11 , wherein the one or more similarity scores are computed based on differences in position of the corresponding source perspectives from the device perspective.

17. The computer program product of claim 11 , wherein the instructions to generate the composite visualization content or the non-composite visualization content further comprises instructions that when executed causes the processor to compare the one or more similarity scores to a similarity score threshold and determine whether to generate the composite visualization content or the non-composite visualization content based on the comparison of the one or more similarity scores to the similarity score threshold.

18. The computer program product of claim 11 , wherein the instructions to generate the non-composite visualization content further comprises instructions that when executed causes the processor to:

identify a visual content source with a highest similarity score of the one or more similarity scores; and

generate the non-composite visualization content based on visual content from the identified visual content source.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Mar 5, 2019
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 048508/0728 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 26, 2018
From: CAMPBELL, SCOTT PATRICK; FONG, GARY
To: GOPRO, INC.
Reel/Frame 047853/0155 →
Continuity (2)
Continuation 15336713 · Oct 27, 2016
Related Publication 20190130651A1 · May 2, 2019