IP Library Granted Patent US 11,605,223
Granted Patent B2
US 11,605,223 · App. 17/814,494 · Granted Mar 14, 2023

Methods and apparatuses for corner detection

Inventor: Hélder Toshiro Suzuki (Mountain View, CA)
Assignee: Magic Leap, Inc.
G06V20/20G02B27/0093G02B27/0101G06F16/5866G06T7/73G06T19/006G06V10/44G06V10/758H04W4/021H04W4/029G02B2027/0147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,605,223
App. No.
17/814,494
Granted
Mar 14, 2023
Kind
B2
Abstract

An apparatus configured for head-worn by a user, includes: a screen configured to present graphics for the user; a camera system configured to view an environment in which the user is located; and a processing unit coupled to the camera system, the processing unit configured to: obtain a first image with a first resolution, the first image having a first corner, determine a second image with a second resolution, the second image having a second corner that corresponds with the first corner in the first image, wherein the second image is based on the first image, the second resolution being less than the first resolution, detect the second corner in the second image, determine a position of the second corner in the second image, and determine a position of the first corner in the first image based at least in part on the determined position of the second corner in the second image.

Claims (51)

1. An apparatus configured for head-worn by a user, comprising:

a screen configured to present graphics for the user;

a camera system configured to view an environment in which the user is located; and

a processing unit coupled to the camera system, the processing unit configured to:

obtain an image having a corner, and

detect corner in the image based on a constant k1,

wherein k1=R/(1+R){circumflex over ( )}2, where R is a number different from −1, and wherein a value of k1 is determined by selecting a number for R, and calculating k1 based on the above equation.

2. The apparatus of claim 1 , wherein the processing unit is configured to remove one or more duplicate detected corners in a set of corners detected from the image, the set of corners comprising the corner, before detecting the corner in the image.

3. The apparatus of claim 2 , wherein the processing unit is configured to detect the set of corners in the image based on one or more criteria; and

wherein the one or more criteria comprises a first criterion for excluding one or more objects in the image that is planar and/or that has insufficient texture, a second criterion for excluding one or more objects in the image that has a linear configuration, or both the first criterion and the second criterion.

4. The apparatus of claim 2 , wherein the processing unit is configured to perform spatial binning to select a subset of corners from the set of corners.

5. The apparatus of claim 4 , wherein the processing unit comprises hardware and software;

wherein the hardware of the processing unit is configured to detect the set of corners; and

wherein the software of the processing unit is configured to perform the spatial binning.

6. The apparatus of claim 2 , wherein the processing unit is configured to perform spatial binning by:

dividing the image into a plurality of image portions having a first image portion;

determining a score for each of the corners in the set of corners; and

selecting one or more corners from the corners in the first image portion based on the scores of the corners in the first image portion, and based on a prescribed maximum number of corners for the first image portion.

7. The apparatus of claim 2 , wherein the processing unit is configured to perform non-maxima suppression.

8. The apparatus of claim 7 , wherein the processing unit comprises hardware and software;

wherein the hardware of the processing unit is configured to detect the set of corners; and

wherein the software of the processing unit is configured to perform the non-maxima suppression.

9. The apparatus of claim 1 , wherein the processing unit is configured to detect a set of corners in the image that includes the corner; and

wherein the processing unit is also configured to detect the corner from the set of corners in the image.

10. The apparatus of claim 9 , wherein the processing unit comprises hardware configured to detect the set of corners in the image, and software configured to detect the corner from the set of corners in the image.

11. The apparatus of claim 1 , wherein the processing unit comprises a corner detector configured to operate based on a first criterion; and

wherein the first criterion excludes an object in the image that is planar and/or that has insufficient texture.

12. The apparatus of claim 11 , wherein the corner detector is further configured to operate based on a second criterion; and

wherein the second criterion excludes an object with a linear configuration in the image.

13. The apparatus of claim 1 , wherein the processing unit is configured to detect a second corner in the image based on a second constant k2.

14. The apparatus of claim 13 , wherein the second constant k2 is based on the constant k1.

15. The apparatus of claim 14 , wherein k2=(k1− 1/16)/16, and wherein a value of k2 is determined by obtaining a value for k1, and calculating k2 based on the above equation.

16. The apparatus of claim 1 , wherein the processing unit is configured to perform at least one of non-maxima suppression or spatial binning.

17. The apparatus of claim 16 , wherein the processing unit comprises software configured to perform the at least one of non-maxima suppression or spatial binning.

18. The apparatus of claim 1 , wherein the processing unit is configured to:

determine a position of the corner detected in the image; and

create a map based at least in part on the position of the corner; and

perform localization of the user based on the created map.

19. An apparatus configured for head-worn by a user, comprising:

a screen configured to present graphics for the user;

a camera system configured to view an environment in which the user is located; and

a processing unit coupled to the camera system, the processing unit configured to:

obtain an image having a corner,

determine a position of the corner in the image,

remove one or more duplicate detected corners in a set of corners detected from the image, the set of corners comprising the corner, and

remove a non-duplicate detected corner in the set of corners detected from the image based on corner scores and based on a prescribed maximum number of corners for an image portion that is geometrically smaller than the image.

20. A method performed by a head-worn image display device, comprising:

obtaining an image having a corner;

determining a position of the corner in the image;

removing one or more duplicate detected corners in a set of corners detected from the image, the set of corners comprising the corner; and

removing a non-duplicate detected corner in the set of corners detected from the image based on corner scores and based on a prescribed maximum number of corners for an image portion that is geometrically smaller than the image.

Assignments (3)
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0476 →
SECURITY INTEREST Recorded Sep 23, 2022
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 061197/0942 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2022
From: SUZUKI, HÉLDER TOSHIRO
To: MAGIC LEAP, INC.
Reel/Frame 060599/0930 →