IP Library › Granted Patent US 11,010,965
Granted Patent B2
US 11,010,965 · App. 16/925,728 · Granted May 18, 2021

Virtual object placement for augmented reality

Inventors: Szymon Piotr Stachniak (Seattle, WA); Hendrik Mark Langerak (Bellevue, WA); Michelle Brook (Redmond, WA)
Assignee: Microsoft Technology Licensing, LLC
G06T17/00G06T19/006G06T19/20G06T2200/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,010,965
App. No.
16/925,728
Granted
May 18, 2021
Kind
B2
Abstract

An augmented reality device includes a logic machine and a storage machine holding instructions executable by the logic machine to, for one or more real-world surfaces represented in a three-dimensional representation of a real-world environment of the augmented reality device, fit a virtual two-dimensional plane to the real-world surface. A request to place a virtual three-dimensional object on the real-world surface is received. For each of a plurality of candidate placement locations on the virtual two-dimensional plane, the candidate placement location is evaluated as a valid placement location or an invalid placement location for the virtual three-dimensional object. An invalidation mask is generated that defines the valid and invalid placement locations on the virtual two-dimensional plane.

Claims (35)

1. An augmented reality device, comprising:

a logic machine; and

a storage machine holding instructions executable by the logic machine to:

fit a virtual two-dimensional plane to a real-world surface represented in a three-dimensional representation of a real-world environment of the augmented reality device;

evaluate a candidate placement location on the virtual two-dimensional plane for placement of a virtual three-dimensional object at least by determining whether a length of the virtual three-dimensional object in a direction perpendicular to the virtual two-dimensional plane exceeds a distance between the candidate placement location and an adjacent real or virtual object; and

generate an invalidation mask defining an evaluation result for the candidate placement location.

2. The augmented reality device of claim 1 , where the evaluation result indicates the candidate placement location as a valid placement location at least by determining that the length of the virtual three-dimensional object in the direction perpendicular to the virtual two-dimensional plane is less than the distance between the candidate placement location and the adjacent real or virtual object.

3. The augmented reality device of claim 1 , where the evaluation result indicates the candidate placement location as an invalid placement location at least by determining that the length of the virtual three-dimensional object in the direction perpendicular to the virtual two-dimensional plane exceeds the distance between the candidate placement location and the adjacent real or virtual object.

4. The augmented reality device of claim 1 , where the invalidation mask defines evaluation results for each of a plurality of candidate placement locations, the evaluation results indicating whether each of the plurality of candidate placement locations is a valid placement location or an invalid placement location.

5. The augmented reality device of claim 4 , where the plurality of candidate placement locations is arranged in a regular grid on the virtual two-dimensional plane.

6. The augmented reality device of claim 4 , where the instructions are further executable to, via a display, visually indicate positions of the valid and invalid placement locations by displaying virtual imagery at positions on the real-world surface corresponding to the valid and invalid placement locations.

7. The augmented reality device of claim 6 , where visually indicating positions of the valid and invalid placement locations includes rasterizing the invalidation mask as an image and superimposing the rendered image on the real-world surface.

8. The augmented reality device of claim 6 , where valid placement locations are visually indicated using a different color than invalid placement locations.

9. The augmented reality device of claim 6 , where valid placement locations are indicated using a different visual pattern than invalid placement locations.

10. The augmented reality device of claim 1 , where the instructions are further executable to, based on the length of the three-dimensional object in the direction perpendicular to the virtual two-dimensional plane being less than the distance between the candidate placement location and the adjacent real or virtual object, place the virtual three-dimensional object at the candidate placement location by displaying, via a display, the virtual three-dimensional object at a position on the real-world surface corresponding to the candidate placement location.

11. The augmented reality device of claim 10 , where the candidate placement location is a most central placement location on the virtual two-dimensional plane at which the length of the virtual three-dimensional object in the direction perpendicular to the virtual two-dimensional plane is less than the distance between the candidate placement location and the real or virtual object.

12. The augmented reality device of claim 1 , where the real-world surface is detected by the augmented reality device by identifying one or more clusters of three-dimensional voxels in the three-dimensional representation of the physical environment that are substantially parallel to two axes of a three-axis coordinate system.

13. The augmented reality device of claim 1 , where every real-world surface represented in the three-dimensional representation having at least a minimum surface area is fit with a corresponding virtual two-dimensional plane.

14. The augmented reality device of claim 1 , where every real-world surface in the three-dimensional representation identified as belonging to one or more classes upon which virtual three-dimensional objects can be placed is fit with a corresponding virtual two-dimensional plane.

15. The augmented reality device of claim 1 , where the instructions are further executable to receive a second request to place a second virtual three-dimensional object on the real-world surface at a second candidate placement location, evaluate the second candidate placement location for placement of the second virtual three-dimensional object, and generate a second invalidation mask defining an evaluation result for the second candidate placement location.

16. The augmented reality device of claim 1 , where the three-dimensional representation of the real-world environment is generated by the augmented reality device based on one or both of visible light images and depth images of the real-world environment captured by the augmented reality device.

17. The augmented reality device of claim 1 , where the candidate placement location is evaluated for placement of the virtual three-dimensional object after receiving a user input requesting placement of the virtual three-dimensional object on the real-world surface.

18. A method for an augmented reality device, comprising:

fitting a virtual two-dimensional plane to a real-world surface represented in a three-dimensional representation of a real-world environment of the augmented reality device;

evaluating a candidate placement location on the virtual two-dimensional plane for placement of a virtual three-dimensional object at least by determining whether a length of the virtual three-dimensional object in a direction perpendicular to the virtual two-dimensional plane exceeds a distance between the candidate placement location and an adjacent real or virtual object; and

generating an invalidation mask defining an evaluation result for the candidate placement location.

19. The method of claim 18 , where the invalidation mask defines evaluation results for each of a plurality of candidate placement locations, the evaluation results indicating, for each of the plurality of candidate placement locations, whether the length of the virtual three-dimensional object in the direction perpendicular to the valid two-dimensional plane exceeds the distance between the candidate placement location and the adjacent real or virtual object.

20. A computing device, comprising:

a display;

a logic machine; and

a storage machine holding instructions executable by the logic machine to:

fit a virtual two-dimensional plane to a real-world surface represented in a three-dimensional representation of a real-world environment of the computing device;

evaluate a candidate placement location on the virtual two-dimensional plane for placement of a virtual three-dimensional object at least by determining whether a length of the virtual three-dimensional object in a direction perpendicular to the virtual two-dimensional plane exceeds a distance between the candidate placement location and an adjacent real or virtual object;

generate an invalidation mask defining an evaluation result for the candidate placement location; and

display, via the display, an indication of the evaluation result at a position on the real-world surface corresponding to the candidate placement location.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2020
From: STACHNIAK, SZYMON PIOTR; LANGERAK, HENDRIK MARK; BROOK, MICHELLE
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 053174/0208 →
Continuity (3)
Continuation 16294748 · Mar 6, 2019
Provisional Application 62791625 · Jan 11, 2019
Related Publication 20200342660A1 · Oct 29, 2020
Cited By (1)
US 12,423,881