IP Library › Granted Patent US 10,740,960
Granted Patent B2
US 10,740,960 · App. 16/294,748 · Granted Aug 11, 2020

Virtual object placement for augmented reality

Inventors: Szymon Piotr Stachniak (Seattle, WA); Hendrik Mark Langerak (Bellevue, WA); Michelle Brook (Redmond, WA)
Assignee: Microsoft Technology Licensing, LLC
G06T17/00G06T19/006G06T19/20G06T2200/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,740,960
App. No.
16/294,748
Granted
Aug 11, 2020
Kind
B2
Abstract

An augmented reality device includes a logic machine and a storage machine holding instructions executable by the logic machine to, for one or more real-world surfaces represented in a three-dimensional representation of a real-world environment of the augmented reality device, fit a virtual two-dimensional plane to the real-world surface. A request to place a virtual three-dimensional object on the real-world surface is received. For each of a plurality of candidate placement locations on the virtual two-dimensional plane, the candidate placement location is evaluated as a valid placement location or an invalid placement location for the virtual three-dimensional object. An invalidation mask is generated that defines the valid and invalid placement locations on the virtual two-dimensional plane.

Claims (39)

1. An augmented reality device, comprising:

a logic machine; and

a storage machine holding instructions executable by the logic machine to:

for one or more real-world surfaces represented in a three-dimensional representation of a real-world environment of the augmented reality device, fit a virtual two-dimensional plane to the real-world surface;

receive a request to place a virtual three-dimensional object on the real-world surface;

for each of a plurality of candidate placement locations on the virtual two-dimensional plane, evaluate whether the candidate placement location is a valid placement location or an invalid placement location for the virtual three-dimensional object at least by determining whether a length of the virtual three-dimensional object in a direction perpendicular to the virtual two-dimensional plane exceeds a distance between the candidate placement location and an adjacent real or virtual object; and

generate an invalidation mask defining the valid and invalid placement locations on the virtual two-dimensional plane.

2. The augmented reality device of claim 1 , further comprising a display configured to present virtual imagery that augments a user's view of the real-world environment.

3. The augmented reality device of claim 2 , where the instructions are further executable to, via the display, visually indicate positions of the valid and invalid placement locations by displaying virtual imagery at positions on the real-world surface corresponding to the valid and invalid placement locations.

4. The augmented reality device of claim 3 , where visually indicating positions of the valid and invalid placement locations includes rasterizing the invalidation mask as an image and superimposing the rendered image on the real-world surface.

5. The augmented reality device of claim 3 , where valid placement locations are visually indicated using a different color than invalid placement locations.

6. The augmented reality device of claim 3 , where valid placement locations are indicated using a different visual pattern than invalid placement locations.

7. The augmented reality device of claim 2 , where the instructions are further executable to place the virtual three-dimensional object on a valid placement location of the virtual two-dimensional plane by displaying the virtual three-dimensional object at a position on the real-world surface corresponding to the valid placement location.

8. The augmented reality device of claim 7 , where the virtual three-dimensional object is automatically displayed at a position on the real-world surface corresponding to a most central valid placement location on the virtual two-dimensional plane.

9. The augmented reality device of claim 1 , where the one or more real-world surfaces are detected by the augmented reality device by identifying one or more clusters of three-dimensional voxels in the three-dimensional representation of the physical environment that are substantially parallel to two axes of a three-axis coordinate system.

10. The augmented reality device of claim 1 , where every real-world surface represented in the three-dimensional representation having at least a minimum surface area is fit with a corresponding virtual two-dimensional plane.

11. The augmented reality device of claim 1 , where every real-world surface in the three-dimensional representation identified as belonging to one or more classes upon which virtual three-dimensional objects can be placed is fit with a corresponding virtual two-dimensional plane.

12. The augmented reality device of claim 1 , where the instructions are further executable to receive a second request to place a second virtual three-dimensional object on the real-world surface, evaluate valid and invalid placement locations for the second virtual three-dimensional object on the virtual two-dimensional plane, and generate a second invalidation mask defining the valid and invalid placement locations on the virtual two-dimensional plane for the second virtual three-dimensional object.

13. The augmented reality device of claim 12 , where the three-dimensional representation of the real-world environment is generated by the augmented reality device based on one or both of visible light images and depth images of the real-world environment captured by the augmented reality device.

14. The augmented reality device of claim 1 , where the request to place the virtual three-dimensional object is received from a user of the augmented reality device.

15. The augmented reality device of claim 1 , where the plurality of candidate placement locations is arranged in a regular grid on the virtual two-dimensional plane.

16. A method for an augmented reality computing device, comprising:

for one or more real-world surfaces represented in a three-dimensional representation of a real-world environment of the augmented reality device, fitting a virtual plane to the real-world surface;

receiving a request to place a virtual three-dimensional object on the real-world surface;

for each of a plurality of candidate placement locations on the virtual plane, evaluating whether the candidate placement location is a valid placement location or an invalid placement location for the virtual three-dimensional object at least by determining whether a length of the virtual three-dimensional object in a direction perpendicular to the virtual plane exceeds a distance between the candidate placement location and an adjacent real or virtual object; and

generating an invalidation mask defining the valid and invalid placement locations on the virtual plane.

17. The method of claim 16 , further comprising, via a display configured to present virtual imagery that augments a user's view of the real-world environment, visually indicating positions of the valid and invalid placement locations on the virtual plane by displaying virtual imagery at positions on the real-world surface corresponding to the valid and invalid placement locations.

18. The method of claim 17 , further comprising automatically displaying the virtual three-dimensional object at a position on the real-world surface corresponding to a most central valid placement location on the virtual two-dimensional plane.

19. The method of claim 17 , where the three-dimensional representation of the real-world environment is generated by the augmented reality device based on one or both of visual light images and depth images of the real-world environment captured by the augmented reality device.

20. An augmented reality device, comprising:

a display configured to present virtual imagery that augments a user's view of a real-world environment of the augmented reality device;

a logic machine; and

a storage machine holding instructions executable by the logic machine to:

generate a three-dimensional representation of a real-world environment of the augmented reality device based on one or both of visible light images and depth images of the real-world environment;

fit a virtual two-dimensional plane to a real-world surface detected in the three-dimensional representation;

receive a request to place a virtual three-dimensional object on the real-world surface;

for each of a plurality of candidate placement locations on the virtual two-dimensional plane, evaluate whether the candidate placement location is a valid placement location or an invalid placement location for the virtual three-dimensional object at least by determining whether a length of the virtual three-dimensional object in a direction perpendicular to the virtual two-dimensional plane exceeds a distance between the candidate placement location and an adjacent real or virtual object;

generate an invalidation mask defining the valid and invalid placement locations on the virtual two-dimensional plane; and

displaying, via the display, virtual imagery at positions on the real-world surface corresponding to the valid and invalid placement locations.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 6, 2019
From: STACHNIAK, SZYMON PIOTR; LANGERAK, HENDRIK MARK; BROOK, MICHELLE
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 048523/0590 →
Continuity (2)
Provisional Application 62791625 · Jan 11, 2019
Related Publication 20200226823A1 · Jul 16, 2020
Cited By (2)
US 12,361,476 US 12,620,178