IP Library Granted Patent US 11,640,677
Granted Patent B2
US 11,640,677 · App. 17/071,680 · Granted May 2, 2023

Navigation using selected visual landmarks

Inventors: Daniel Todd Kerzner (McLean, VA); Ahmad Seyfi (Tysons, VA); Timon Meyer (Tysons, VA); Donald Gerard Madden (Columbia, MD); Babak Rezvani (Tysons, VA); Glenn Tournier (Vienna, VA)
Assignee: Alarm.com Incorporated
G06T7/74B64C39/024G06F18/22G06V10/764G06V20/10G06V20/17G06V20/36G06V20/70B64U2101/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,640,677
App. No.
17/071,680
Granted
May 2, 2023
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for navigation using planar landmarks. In some implementations, images depicting an area of a property captured by a camera are obtained. Landmarks in the area are identified based on the images. An expected stability for each of the landmarks are determined. A map for the area is generated based on the expected stability for each of the landmarks. The map is transmitted to one or more electronic devices.

Claims (80)

1. A computer-implemented method comprising:

obtaining images captured by a camera, the images depicting an area of a property;

identifying landmarks in the area based on the images;

determining an expected stability for each of the landmarks including one or more values indicating a likelihood that a given landmark will move or stay in place;

generating a map for the area based on the expected stability for each of the landmarks; and

transmitting the map to one or more electronic devices.

2. The method of claim 1 , wherein obtaining images captured by the camera comprises obtaining a set of images captured by a camera of a drone flying in the property.

3. The method of claim 1 , wherein obtaining images captured by the camera comprises obtaining a set of images captured from a particular location in the property, the set of images including images corresponding to different perspectives of the camera.

4. The method of claim 1 , wherein identifying the landmarks in the area comprises identifying an object or a surface of an object using one or more algorithms,

wherein the object or the surface of the object is a landmark of the landmarks.

5. The method of claim 4 , wherein identifying the object or the surface of an object using the one or more algorithms comprises performing object localization on the images using at least one algorithm of the one or more algorithms to identify one or more of the following:

discrete objects in the area of the property based on the images;

discrete surfaces of particular objects in the area of the property based on the images; or

an area corresponding to discrete objects in the area of the property based on the images.

6. The method of claim 4 , wherein identifying the object or the surface of an object using the one or more algorithms comprises classifying the object or the surface of an object using at least one algorithm of the one or more algorithms, and

wherein:

classifying the object or the surface of an object comprises classifying the object or the surface of an object as belonging to a type of object, and

determining the expected stability for each of the landmarks comprises determining the expected stability for the object or the surface of an object based on the type of object.

7. The method of claim 4 , wherein identifying the object or the surface of an object using the one or more algorithms comprises classifying the object or the surface of an object using at least one algorithm of the one or more algorithms, and

wherein:

classifying the object or the surface of an object comprises classifying the object or the surface of an object as planar or non-planar, and

determining the expected stability for each of the landmarks comprises determining the expected stability for the object or the surface of an object based on the planar classification or the non-planar classification.

8. The method of claim 4 , wherein identifying the object or the surface of an object using the one or more algorithms comprises performing scene segmentation on the images using at least one algorithm of the one or more algorithms to identify one or more of the following:

discrete objects in the area of the property based on the images;

discrete surfaces of particular objects in the area of the property based on the images;

individual areas each corresponding to only a single discrete object in the area of the property based on the images; or

individual areas each corresponding to only a single discrete surface of particular objects in the area of the property based on the images.

9. The method of claim 1 , wherein identifying the landmarks in the area comprises identifying groupings of related surfaces.

10. The method of claim 9 , wherein identifying groupings of related surfaces comprises identifying one or more of the following:

surfaces that are within a threshold distance from one another;

surfaces that are adjacent to one another;

surfaces that are coplanar;

surfaces that are observed as having the same or substantially the same color; or

surfaces that are in contact with one another.

11. The method of claim 9 , wherein identifying groupings of related surfaces comprises identifying groupings of surfaces that each correspond to a particular object or a portion of a particular object in the area of the property.

12. The method of claim 1 , wherein determining the expected stability for each of the landmarks comprises:

determining a reliability score for each of the landmarks that corresponds to consistency in location of objects or surfaces of objects over time;

determining a stability score for each of the landmarks that corresponds to consistency in appearance of objects or surfaces of objects across viewpoint locations; and

calculating the expected stability using the reliability score and the stability score.

13. The method of claim 12 , wherein determining the reliability score for each of the landmarks comprises:

determining reliability scores for features of each of the landmarks; and

determining the reliability score from the reliability scores for the features of each of the landmarks.

14. The method of claim 12 , wherein determining the reliability score for each of the landmarks comprises determining the reliability score for each of the landmarks based one on or more of the following:

frequency that a landmark or a portion of a landmark changes position;

frequency that a landmark or a portion of a landmark changes appearance;

level of uncertainty associated with a landmark or a portion of a landmark; or

a classification of a landmark.

15. The method of claim 12 , wherein determining the stability score for each of the landmarks comprises determining the stability score for each of the landmarks based one on or more of the following:

number of vantage points that a landmark or a portion of a landmark is capable of being viewed from with respect to the camera;

variability in appearance of a landmark or a portion of a landmark from different perspectives of the camera or different vantage points;

maximum distance from which a landmark or a portion of a landmark is capable of being viewed from with respect to the camera;

magnitude of parallax observed with a landmark or a portion of a landmark;

level of uncertainty associated with a landmark or a portion of a landmark;

a classification of a landmark; or

prevalence of planar surfaces of a landmark or a portion of a landmark.

16. The method of claim 12 , wherein determining the stability score for each of the landmarks comprises determining that the landmark or a surface of the landmark is planar by:

identifying a set of interest points corresponding to the landmark or to a surface of the landmark; and

determining that a threshold percent of the interest points are coplanar.

17. The method of claim 1 , comprising comparing the expected stability for each of the landmarks to a minimum level of stability to identify a subset of landmarks that meet the minimum level of stability,

wherein generating the map for the area comprises:

generating a map for the area that includes representations of only the landmarks in the subset of landmarks; or

generating a map for the area that includes representations of the landmarks in the subset of landmarks, the representations tagged to indicate that the corresponding landmarks have a sufficient level of stability.

18. The method of claim 17 , wherein comparing the expected stability for each of the landmarks to the minimum level of stability comprises:

comparing reliability scores for the landmarks to a threshold reliability score to identify the subset of landmarks that meet the threshold reliability score; and

comparing stability scores for the landmarks to a threshold stability score to identify the subset of landmarks that meet the threshold stability score.

19. The method of claim 1 , wherein the one or more values indicating a likelihood that the given landmark will move or stay in place indicate one or more of the following: a number of vantage points that the given landmark or a portion of the given landmark is capable of being viewed; a variability in appearance of the given landmark or a portion of the given landmark; a maximum distance from which the given landmark or a portion of the given landmark is capable of being viewed; a magnitude of parallax observed with the given landmark or a portion of the given landmark; a level of uncertainty associated with the given landmark or a portion of the given landmark; a classification of the given landmark; or a prevalence of planar surfaces of the given landmark or a portion of the given landmark.

20. A system comprising:

one or more computers; and

one or more computer-readable media storing instructions that, when executed, cause the one or more computers to perform operations comprising:

obtaining images captured by a camera, the images depicting an area of a property;

identifying landmarks in the area based on the images;

determining an expected stability for each of the landmarks including one or more values indicating a likelihood that a given landmark will move or stay in place;

generating a map for the area based on the expected stability for each of the landmarks; and

transmitting the map to one or more electronic devices.

21. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

obtaining images captured by a camera, the images depicting an area of a property;

identifying landmarks in the area based on the images;

determining an expected stability for each of the landmarks including one or more values indicating a likelihood that a given landmark will move or stay in place;

generating a map for the area based on the expected stability for each of the landmarks; and

transmitting the map to one or more electronic devices.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 4, 2021
From: SEYFI, AHMAD; KERZNER, DANIEL TODD; MEYER, TIMON; MADDEN, DONALD GERARD; REZVANI, BABAK; TOURNIER, GLENN
To: ALARM.COM INCORPORATED
Reel/Frame 057082/0504 →
Continuity (3)
Provisional Application 62915278 · Oct 15, 2019
Provisional Application 63051611 · Jul 14, 2020
Related Publication 20210110137A1 · Apr 15, 2021
Cited By (1)
US 12,555,385