IP Library Granted Patent US 11,710,284
Granted Patent B2
US 11,710,284 · App. 17/551,020 · Granted Jul 25, 2023

Augmented reality collaboration system

Inventors: Avi Bar-Zeev (San Mateo, CA); Alexander Tyurin (San Mateo, CA); Gerald V. Wright, Jr. (San Mateo, CA)
Assignee: Campfire 3D, Inc.
G06T19/006G01S5/16G06F3/011G06T15/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,710,284
App. No.
17/551,020
Filed
Dec 14, 2021
Granted
Jul 25, 2023
Kind
B2
Examiner
GUO, XILIN
Art Unit
2616
USPC
345/419
Abstract

A system comprising: a user device, comprising: sensors configured to sense data related to a physical environment of the user device, displays; hardware processors; and a non-transitory machine-readable storage medium encoded with instructions executable by the hardware processors to: place a virtual object in a 3D scene displayed by the second user device, determine a pose of the user device with respect to the physical location in the physical environment of the user device, and generate an image of virtual content based on the pose of the user device with respect to the placed virtual object, wherein the image of the virtual content is projected by the one or more displays of the user device in a predetermined location relative to the physical location in the physical environment of the user device.

Claims (72)

1. A virtual content system configured to display virtual content, comprising:

a structure comprising a plurality of radiation emitters arranged in a predetermined pattern, and

a first user device, wherein the first user device comprises:

one or more sensors configured to sense outputs of the plurality of radiation emitters, and

one or more displays;

one or more hardware processors; and

a non-transitory machine-readable storage medium encoded with instructions executable by the one or more hardware processors to, for the first user device:

generate an image representing a physical environment of the first user device,

determine a pose of the first user device with respect to the structure based on the sensed outputs of the plurality of radiation emitters,

select a physical location in the physical environment of the first user device relative to the structure,

place the virtual content in a location in the image corresponding to the physical location in the physical environment of the first user device with a predetermined orientation relative to the structure,

generate an image of virtual content based on the pose of the first user device with respect to the structure,

add the image of the virtual content to the image representing the physical environment of the first user device,

generate a virtual proxy of a second user device that is viewing a second image of the virtual content in the second user device, with the position of the virtual proxy based on a pose of the second user device with respect to a second physical location in a physical environment of the second user device,

add the virtual proxy to the image representing the physical environment of the first user device, and

cause the one or more displays of the first user device to render the image representing the physical environment of the first user device with the image of the virtual content and the virtual proxy.

2. The system of claim 1 , wherein:

the second user device is remote from the virtual content system.

3. The system of claim 1 , wherein:

the plurality of radiation emitters is configured to emit infrared light;

the one or more second sensors on the first user device are configured to sense the output of the plurality of radiation emitters; and

the instructions are further executable by the one or more hardware processors to determine the pose of the first user device with respect to the structure based on the sensed output of the plurality of radiation emitters.

4. The system of claim 1 , wherein the instructions are further executable by the one or more hardware processors to perform, responsive to user input received by the first user device, at least one of:

selecting the virtual content, and

modifying the virtual content.

5. The system of claim 1 , wherein the first user device may be worn on the head of a user.

6. The system of claim 1 , wherein the first user device further comprises at least one optical element which facilitates viewing of the image integrated with a view of a physical environment of the first user device.

7. The system of claim 1 , wherein the structure is attached to a physical object.

8. One or more non-transitory machine-readable storage media encoded with instructions, wherein the instructions are executable by one or more hardware processors to, for a first user device in a virtual content system configured to display virtual content:

cause the first user device to sense outputs of a plurality of radiation emitters arranged in a structure in a predetermined pattern;

generate an image representing a physical environment of the first user device,

determine a pose of the first user device with respect to the structure based on the sensed outputs of the plurality of radiation emitters;

select a physical location in the physical environment of the first user device relative to the structure;

place the virtual content in a location in the image corresponding to the physical location in the physical environment of the first user device with a predetermined orientation relative to the structure;

generate an image of virtual content based on the pose of the first user device with respect to the structure;

add the image of the virtual content to the image representing the physical environment of the first user device;

generate a virtual proxy of a second user device that is viewing a second image of the virtual content in the second user device, with the position of the virtual proxy based on a pose of the second user device with respect to a second physical location in a physical environment of the second user device;

add the virtual proxy to the image representing the physical environment of the first user device; and

cause the one or more displays of the first user device to render the image representing the physical environment of the first user device with the image of the virtual content and the virtual proxy.

9. The media of claim 8 , wherein:

the second user device is remote from the virtual content system.

10. The media of claim 8 , wherein:

the plurality of radiation emitters is configured to emit infrared light; and

the instructions are further executable by the one or more hardware processors to determine the pose of the first user device with respect to the structure based on the sensed output of the plurality of radiation emitters.

11. The media of claim 8 , wherein the instructions are further executable by the one or more hardware processors to perform, responsive to user input received by the first user device, at least one of:

selecting the virtual content, and

modifying the virtual content.

12. The media of claim 8 , wherein the first user device may be worn on the head of a user.

13. The media of claim 8 , wherein the first user device further comprises at least one optical element which facilitates viewing of the image integrated with a view of a physical environment of the first user device.

14. The media of claim 8 , wherein the structure is attached to a physical object.

15. A computer-implemented method, comprising:

generate an image representing a physical environment of a first user device,

sensing outputs of a plurality of radiation emitters arranged in a structure in a predetermined pattern;

determining a pose of the first user device in a virtual content system configured to display virtual content with respect to the structure based on the sensed outputs of the plurality of radiation emitters;

selecting a physical location in the physical environment of the first user device relative to the structure,

placing the virtual content in a location in the image corresponding to the physical location in the physical environment of the first user device with a predetermined orientation relative to the structure,

generating an image of the virtual content based on the pose of the first user device with respect to the structure;

adding the image of the virtual content to the image representing the physical environment of the first user device,

generating a virtual proxy of a second user device that is viewing a second image of the virtual content in the second user device, with the position of the virtual proxy based on a pose of the second user device with respect to a second physical location in a physical environment of the second user device;

adding the virtual proxy to the image representing the physical environment of the first user device; and

causing one or more displays of the first user device to render the image representing the physical environment of the first user device with the image of the virtual content and the virtual proxy.

16. The method of claim 15 , wherein:

the second user device is remote from the virtual content system.

17. The method of claim 15 , wherein:

the plurality of radiation emitters is configured to emit infrared light; and

the method further comprises determining the pose of the first user device with respect to the structure based on the sensed output of the plurality of radiation emitters.

18. The method of claim 15 , further comprising, responsive to user input received by the first user device, at least one of:

selecting the virtual content, and

modifying the virtual content.

19. The method of claim 15 , wherein the first user device may be worn on the head of a user.

20. The method of claim 15 , wherein the first user device further comprises at least one optical element which facilitates viewing of the image integrated with a view of a physical environment of the first user device.

21. The method of claim 15 , wherein the structure is attached to a physical object.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2024
From: CAMPFIRE 3D, INC.
To: QUALCOMM INCORPORATED
Reel/Frame 067209/0066 →
CHANGE OF NAME Recorded Jun 5, 2023
From: META VIEW, INC.
To: CAMPFIRE 3D, INC.
Reel/Frame 063857/0919 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2022
From: BAR-ZEEV, AVI; TYURIN, ALEXANDER; WRIGHT, GERALD V., JR.
To: META VIEW, INC.
Reel/Frame 059744/0694 →
Continuity (3)
Continuation 17494256 · Oct 5, 2021
Continuation 17023220 · Sep 16, 2020
Related Publication 20220108538A1 · Apr 7, 2022
Cited By (2)
US 12,507,007 US 12,591,402