IP Library Granted Patent US 11,150,482
Granted Patent B2
US 11,150,482 · App. 15/043,959 · Granted Oct 19, 2021

Augmented reality content creation

Inventors: Brian Mullins (Sierra Madre, CA); Matthew Kammerait (West Hollywood, CA)
Assignee: FACEBOOK TECHNOLOGIES, LLC
G02B27/0179G02B27/017G06F3/011G06F3/012G02B2027/014G02B2027/0138G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,150,482
App. No.
15/043,959
Granted
Oct 19, 2021
Kind
B2
Abstract

A system and method for generating a virtual content to a physical object is described. A processor includes an augmented reality application. The augmented reality application creates virtual content at the head mounted device, and associates the virtual content with predefined conditions based on data from sensors embedded in the head mounted device at a time of creation of the virtual content. The virtual content is displayed in a display of the head mounted device in response to sensor data satisfying the predefined conditions.

Claims (58)

1. A method comprising:

receiving, by a server system, a first set of object identifiers and corresponding virtual content from a first head mounted device, the first set of object identifiers having been identified by the first head mounted device based on a first image of a physical object captured by an optical sensor of the first head mounted device, the virtual content having been generated by the first head mounted device based on video captured by the optical sensor of the first head mounted device, the video depicting a user of the first head mounted device performing a physical operation in relation to the physical object;

storing the first set of object identifiers and the corresponding virtual content in a data storage of the server system;

receiving a second set of object identifiers from a second head mounted device, the second set of object identifiers having been generated by the second head mounted device based on a second image of the physical object captured by an optical sensor of the second head mounted device;

comparing the second set of object identifiers received from the second head mounted device to sets of object identifies stored in the data storage of the server system, yielding a comparison;

in response to determining, based on the comparison, that at least one object identifier included in the second set of object identifiers matches at least one object identifier from the first set of object identifiers determining that the physical object in the second image matches the physical object in the first image;

accessing the virtual content corresponding to the first set of object identifiers from the data storage of the server system; and

providing the virtual content to the second head mounted device, the second head mounted device displaying the virtual content in a display of the second head mounted device.

2. The method of claim 1 , further comprising:

generating the virtual content based on a preset virtual content template, the preset virtual content template comprising preset attributes received from the first head mounted device at the time of creation of the virtual content.

3. The method of claim 2 , further comprising:

retrieving the preset virtual content template corresponding to the physical object viewed by the first head mounted device.

4. The method of claim 1 , further comprising:

generating the virtual content based on a custom virtual content template, the custom virtual content template comprising attributes selected by a user of the second head mounted device.

5. The method of claim 1 , wherein the second head mounted device:

records an operation on the physical object in response to receiving a recording instruction from a user of the second head mounted device,

applies a three-dimensional model to the physical object based on the identifiers, and

generates an animation of the three-dimensional model of the physical object based on the recorded operation.

6. The method of claim 5 , wherein the second head mounted device displays the animation of the three-dimensional model of the physical object in the display in response to satisfaction of predefined conditions.

7. The method of claim 6 , wherein the predefined conditions are related to a combination of at least the physical object, an operational status of the physical object, and a type of the physical object, ambient conditions, a geographic location, a position, and an orientation of the second head mounted device.

8. The method of claim 1 , wherein the second head mounted device displays the virtual content based on a position and orientation of the second head mounted device relative to the physical object, the virtual content associated with the physical object.

9. A server system comprising:

one or more computer processors; and

one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the server system to perform operations comprising:

receiving a first set of object identifiers and corresponding virtual content from a first head mounted device, the first set of object identifiers having been identified by the first head mounted device based on a first image of a physical object captured by an optical sensor of the first head mounted device, the virtual content having been generated by the first head mounted device based on video captured by the optical sensor of the first head mounted device, the video depicting a user of the first head mounted device performing a physical operation in relation to the physical object;

storing the first set of object identifiers and the corresponding virtual content in a data storage of the server system;

receiving a second set of object identifiers from a second head mounted device, the second set of object identifiers having been generated by the second head mounted device based on a second image of the physical object captured by an optical sensor of the second head mounted device;

comparing the second set of object identifiers received from the second head mounted device to sets of object identifies stored in the data storage of the server system, yielding a comparison;

in response to determining, based on the comparison, that at least one object identifier included in the second set of object identifiers matches at least one object identifier from the first set of object identifiers determining that the physical object in the second image matches the physical object in the first image;

accessing the virtual content corresponding to the first set of object identifiers from the data storage of the server system; and

providing the virtual content to the second head mounted device, the second head mounted device displaying the virtual content in a display of the second head mounted device.

10. The server system of claim 9 , the operations further comprising:

generating the virtual content based on a preset virtual content template, the preset virtual content template comprising preset attributes received from the first head mounted device at the time of creation of the virtual content.

11. The server system of claim 10 , the operations further comprising:

retrieving the preset virtual content template corresponding to the physical object viewed by the first head mounted device.

12. The server system of claim 9 , the operations further comprising:

generating the virtual content based on a custom virtual content template, the custom virtual content template comprising attributes selected by a user of the second head mounted device.

13. The server system of claim 9 , wherein the second head mounted device:

records an operation on the physical object in response to receiving a recording instruction from a user of the second head mounted device,

applies a three-dimensional model to the physical object based on the identifiers, and

generates an animation of the three-dimensional model of the physical object based on the recorded operation.

14. The server system of claim 13 , wherein the second head mounted device displays the animation of the three-dimensional model of the physical object in the display in response to satisfaction of predefined conditions.

15. The server system of claim 14 , wherein the predefined conditions are related to a combination of at least the physical object, an operational status of the physical object, and a type of the physical object, ambient conditions, a geographic location, a position, and an orientation of the second head mounted device.

16. The server system of claim 9 , wherein the second head mounted device displays the virtual content based on a position and orientation of the second head mounted device relative to the physical object, the virtual content associated with the physical object.

17. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a server system, cause the server system to perform operations comprising:

receiving a first set of object identifiers and corresponding virtual content from a first head mounted device, the first set of object identifiers having been identified by the first head mounted device based on a first image of a physical object captured by an optical sensor of the first head mounted device, the virtual content having been generated by the first head mounted device based on video captured by the optical sensor of the first head mounted device, the video depicting a user of the first head mounted device performing a physical operation in relation to the physical object;

storing the first set of object identifiers and the corresponding virtual content in a data storage of the server system;

receiving a second set of object identifiers from a second head mounted device, the second set of object identifiers having been generated by the second head mounted device based on a second image of the physical object captured by an optical sensor of the second head mounted device;

comparing the second set of object identifiers received from the second head mounted device to sets of object identifies stored in the data storage of the server system, yielding a comparison;

in response to determining, based on the comparison, that at least one object identifier included in the second set of object identifiers matches at least one object identifier from the first set of object identifiers determining that the physical object in the second image matches the physical object in the first image;

accessing the virtual content corresponding to the first set of object identifiers from the data storage of the server system; and

providing the virtual content to the second head mounted device, the second head mounted device displaying the virtual content in a display of the second head mounted device.

18. The non-transitory computer-readable medium of claim 17 , the operations further comprising:

generating the virtual content based on a preset virtual content template, the preset virtual content template comprising preset attributes received from the first head mounted device at the time of creation of the virtual content.

19. The non-transitory computer-readable medium of claim 17 , the operations further comprising:

retrieving the preset virtual content template corresponding to the physical object viewed by the first head mounted device.

20. The non-transitory computer-readable medium of claim 17 , the operations further comprising:

generating the virtual content based on a custom virtual content template, the custom virtual content template comprising attributes selected by a user of the second head mounted device.

Assignments (12)
CHANGE OF NAME Recorded Aug 3, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060936/0494 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2021
From: RPX CORPORATION
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 056777/0588 →
RELEASE OF SECURITY INTEREST Recorded Oct 26, 2020
From: JEFFERIES FINANCE LLC
To: RPX CORPORATION
Reel/Frame 054486/0422 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054198/0029 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054244/0566 →
RELEASE OF SECURITY INTEREST Recorded Aug 14, 2020
From: AR HOLDINGS I, LLC
To: DAQRI, LLC
Reel/Frame 053498/0580 →
PATENT SECURITY AGREEMENT Recorded Aug 14, 2020
From: RPX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 053498/0095 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2020
From: DAQRI, LLC
To: RPX CORPORATION
Reel/Frame 053413/0642 →
RELEASE OF SECURITY INTEREST Recorded Oct 23, 2019
From: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
To: DAQRI, LLC
Reel/Frame 050805/0606 →
LIEN Recorded Oct 8, 2019
From: DAQRI, LLC
To: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
Reel/Frame 050672/0601 →
SECURITY INTEREST Recorded Jun 26, 2019
From: DAQRI, LLC
To: AR HOLDINGS I LLC
Reel/Frame 049596/0965 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 10, 2016
From: MULLINS, BRIAN; KAMMERAIT, MATTHEW
To: DAQRI, LLC
Reel/Frame 039394/0840 →
Continuity (2)
Provisional Application 62120808 · Feb 25, 2015
Related Publication 20160247324A1 · Aug 25, 2016
Cited By (3)
US 12,266,057 US 12,443,288 US 12,469,310