IP Library Granted Patent US 11,073,690
Granted Patent B2
US 11,073,690 · App. 16/777,280 · Granted Jul 27, 2021

Surface modeling systems and methods

Inventor: Victor Ng-Thow-Hing (Los Altos, CA)
Assignee: Magic Leap, Inc.
G02B27/0093G02B27/017G02B27/0172G02B30/26G02B30/52G06F3/012G06F3/016G06F3/017G06F3/0308G06F3/0481G06F3/04812G06F3/04815G06K9/00671G06T17/20G06T19/003H04N13/194H04N13/239H04N13/344H04N13/383G02B2027/014G02B2027/0178G02B2027/0187G02F1/292G06K2209/40G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,073,690
App. No.
16/777,280
Granted
Jul 27, 2021
Kind
B2
Abstract

A method of generating a surface model of a physical environment includes obtaining an image of the physical environment. The method also includes generating a planar polygon mesh from at least the image. The method further includes extracting a boundary polygon of the planar polygon mesh. Moreover, the method includes generating a convex hull for the boundary polygon of the surface mesh. In addition, the method includes generating a minimal area oriented boundary polygon from the convex hull. The method may also include generating a maximal area oriented internal polygon inside of the boundary polygon of the planar polygon mesh.

Claims (37)

1. A computer-implemented method of generating a surface model of a physical environment, the method comprising:

a computer processor generating a planar polygon mesh from at least an image of the physical environment;

the computer processor extracting a boundary polygon from the planar polygon mesh;

the computer processor generating a convex hull for the boundary polygon; and

the computer processor generating a minimal area oriented boundary polygon (MAOBP) from the convex hull, the minimal area oriented boundary polygon (MAOBP) comprising a polygon that contains the convex hull and the boundary polygon.

2. The method of claim 1 , further comprising:

the computer processor obtaining the image of the physical environment by:

the computer processor obtaining a 3-D point cloud corresponding to the physical environment using an imaging device, and

the computer processor obtaining pose information for the imaging device; and

the computer processor computing a truncated signed distance function for the 3-D point cloud using the pose information, wherein generating the planar polygon mesh comprises tessellating the truncated signed distance function.

3. The method of claim 2 , further comprising the computer processor combining two smaller planar polygon meshes into one larger planar polygon mesh.

4. The method of claim 1 , further comprising the computer processor obtaining a gravity vector, wherein the generated planar polygon mesh is one of substantially parallel and orthogonal to the gravity vector.

5. The method of claim 1 , wherein generating the convex hull comprises the computer processor using a Graham-Scan algorithm.

6. The method of claim 1 , wherein generating the minimal area oriented boundary polygon (MAOBP) comprises the computer processor using a rotating calipers algorithm.

7. The method of claim 1 , further comprising the computer processor generating a maximal area oriented internal polygon (MAOIP) that is entirely contained inside of the boundary polygon.

8. The method of claim 7 , wherein generating the maximal area oriented internal polygon (MAOIP) comprises the computer processor performing a search in a search area defined by the boundary polygon, and

wherein generating the maximal area oriented internal polygon (MAOIP) comprises forming a grid in the search area,

the method further comprising adjusting a resolution of the grid based on a size of the search area.

9. The method of claim 7 , further comprising:

the computer processor receiving a selection of a point inside of the boundary polygon of the planar polygon mesh wherein generating the maximal area oriented internal polygon (MAOIP) comprises performing a search in a search area defined using the selected point and forming a grid in the search are; and

the computer processor adjusting a resolution of the grid based on a size of the search area,

wherein generating the maximal area oriented internal polygon (MAOIP) comprises the computer processor forming a grid in the search area.

10. The method of claim 7 , wherein the minimal area oriented boundary polygon (MAOBP) and the maximal area oriented internal polygon (MAOIP) have a same shape.

11. The method of claim 1 , wherein the planar polygon mesh is generated by the computer processor based on a marching cubes algorithm.

12. The method of claim 1 , wherein the minimal area oriented boundary polygon (MAOBP) is at least one of a rectangle, a triangle, and a circle.

13. The method of claim 1 , further comprising the computer processor determining a fit between the minimal area oriented boundary polygon (MAOBP) and the boundary polygon,

wherein determining the fit comprises the computer processor calculating a difference between a first area of the minimal area oriented boundary polygon (MAOBP) and a second area of the boundary polygon.

14. The method of claim 1 , further comprising the computer processor storing data representing the minimal area oriented boundary polygon (MAOBP),

wherein the minimal area oriented boundary polygon (MAOBP) is a rectangle,

wherein the data comprises four sets of coordinates corresponding to the rectangle, and

wherein each of the four sets of coordinates is a pair of coordinates.

15. The method of claim 14 , wherein the data further comprises a length of the rectangle, a width of the rectangle, and a center of the rectangle.

16. The method of claim 1 , wherein generating the planar polygon mesh comprises the computer processor capturing static portions of a series of images of the physical environment.

17. The method of claim 1 , wherein the minimal area oriented boundary polygon (MAOBP) comprises a smallest polygon that contains the convex hull and the boundary polygon, and

wherein the minimal area oriented boundary polygon (MAOBP) comprises a smallest rectangle that contains the convex hull and the boundary polygon.

18. The method of claim 17 , further comprising the computer processor generating a maximal area oriented internal polygon (MAOIP) that is contained inside of the boundary polygon.

19. The method of claim 18 , wherein the minimal area oriented boundary polygon (MAOBP) overestimates an extent of a surface being modeled, and the maximal area oriented internal polygon (MAOIP) underestimates the extent of the surface being modeled.

Assignments (2)
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2020
From: NG-THOW-HING, VICTOR
To: MAGIC LEAP, INC.
Reel/Frame 051675/0185 →
Continuity (3)
Continuation 15725801 · Oct 5, 2017
Provisional Application 62404617 · Oct 5, 2016
Related Publication 20200166745A1 · May 28, 2020
Cited By (1)
US 12,602,811