IP Library Granted Patent US 11,215,817
Granted Patent B1
US 11,215,817 · App. 17/111,423 · Granted Jan 4, 2022

Systems and methods for generating spectator images of an artificial reality environment

Inventor: Volga Aksoy (Redwood City, CA)
Assignee: Facebook Technologies, LLC.
G02B27/0093G02B27/017G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,215,817
App. No.
17/111,423
Granted
Jan 4, 2022
Kind
B1
Abstract

A method includes a computing system receiving a first-person image that captures an artificial-reality environment from a first-person perspective of a user of a head-mounted device, wherein the first-person image is rendered based on a pose of the head-mounted device. Based on the pose of the head-mounted device, the computing system determines a spectator perspective wherein the spectator perspective and the first-person perspective are co-located in a three-dimensional space and the spectator perspective has a predetermined rotational orientation about a view axis. The computing system generates a spectator image that captures the first-person image from the spectator perspective, and causes the first-person image to be displayed by the head-mounted device and the spectator image to be displayed on a device separate from the head-mounted device.

Claims (37)

1. A method comprising, by a computing system:

receiving a first-person image that captures an artificial-reality environment from a first-person perspective of a user of a head-mounted device, wherein the first-person image is rendered based on a pose of the head-mounted device;

determining a spectator perspective based on the pose of the head-mounted device, wherein (1) the spectator perspective and the first-person perspective are co-located in a three-dimensional space and (2) the spectator perspective has a predetermined rotational orientation about a view axis;

generating a spectator image that captures the first-person image from the spectator perspective;

causing the first-person image to be displayed by the head-mounted device; and

causing the spectator image to be displayed on a device separate from the head-mounted device.

2. The method of claim 1 , wherein the predetermined rotational orientation of the spectator perspective is fixed, and the first-person perspective has a rotational orientation about the view axis of the user that changes depending on the pose of the head-mounted device.

3. The method of claim 1 , wherein the pose of the head-mounted device is used to determine, for the first-person perspective, first rotational orientations about the view axis, a pitch axis, and a yaw axis.

4. The method of claim 3 , wherein the spectator perspective comprises second rotational orientations about the pitch axis and the yaw axis that are respectively determined based on the first rotational orientations of the first-person perspective about the pitch axis and the yaw axis.

5. The method of claim 1 , wherein the predetermined rotational orientation about the view axis is zero.

6. The method of claim 1 , further comprising adjusting a field of view of the spectator perspective, wherein the adjusted field-of-view of the spectator perspective is less than a field-of-view of the first-person perspective.

7. The method of claim 1 , wherein determining the spectator perspective is performed by a mirror application associated with the computing system.

8. The method of claim 1 , wherein generating the spectator image is performed by a runtime engine associated with the computing system.

9. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive a first-person image that captures an artificial-reality environment from a first-person perspective of a user of a head-mounted device, wherein the first-person image is rendered based on a pose of the head-mounted device;

determine a spectator perspective based on the pose of the head-mounted device, wherein (1) the spectator perspective and the first-person perspective are co-located in a three-dimensional space and (2) the spectator perspective has a predetermined rotational orientation about a view axis;

generate a spectator image that captures the first-person image from the spectator perspective;

cause the first-person image to be displayed by the head-mounted device; and

cause the spectator image to be displayed on a device separate from the head-mounted device.

10. The media of claim 9 , wherein the predetermined rotational orientation of the spectator perspective is fixed, and the first-person perspective has a rotational orientation about the view axis of the user that changes depending on the pose of the head-mounted device.

11. The media of claim 9 , wherein the pose of the head-mounted device is used to determine, for the first-person perspective, first rotational orientations about the view axis, a pitch axis, and a yaw axis.

12. The media of claim 11 , wherein the spectator perspective comprises second rotational orientations about the pitch axis and the yaw axis that are respectively determined based on the first rotational orientations of the first-person perspective about the pitch axis and the yaw axis.

13. The media of claim 11 , wherein the predetermined rotational orientation about the view axis is zero.

14. The media of claim 9 , wherein the software is further operable when executed to adjust a field of view of the spectator perspective, wherein the adjusted field-of-view of the spectator perspective is less than a field-of-view of the first-person perspective.

15. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

receive a first-person image that captures an artificial-reality environment from a first-person perspective of a user of a head-mounted device, wherein the first-person image is rendered based on a pose of the head-mounted device;

determine a spectator perspective based on the pose of the head-mounted device, wherein (1) the spectator perspective and the first-person perspective are co-located in a three-dimensional space and (2) the spectator perspective has a predetermined rotational orientation about a view axis;

generate a spectator image that captures the first-person image from the spectator perspective;

cause the first-person image to be displayed by the head-mounted device; and

cause the spectator image to be displayed on a device separate from the head-mounted device.

16. The system of claim 15 , wherein the predetermined rotational orientation of the spectator perspective is fixed, and the first-person perspective has a rotational orientation about the view axis of the user that changes depending on the pose of the head-mounted device.

17. The system of claim 15 , wherein the pose of the head-mounted device is used to determine, for the first-person perspective, first rotational orientations about the view axis, a pitch axis, and a yaw axis.

18. The system of claim 17 , wherein the spectator perspective comprises second rotational orientations about the pitch axis and the yaw axis that are respectively determined based on the first rotational orientations of the first-person perspective about the pitch axis and the yaw axis.

19. The system of claim 17 , wherein the predetermined rotational orientation about the view axis is zero.

20. The system of claim 15 , wherein the processors are further operable when executing the instructions to adjust a field of view of the spectator perspective, wherein the adjusted field-of-view of the spectator perspective is less than a field-of-view of the first-person perspective.

Assignments (2)
CHANGE OF NAME Recorded Jul 6, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060591/0848 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2021
From: AKSOY, VOLGA
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 054991/0307 →