IP Library Granted Patent US 9,574,883
Granted Patent B2
US 9,574,883 · App. 14/667,655 · Granted Feb 21, 2017

Associating semantic location data with automated environment mapping

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,574,883
App. No.
14/667,655
Granted
Feb 21, 2017
Kind
B2
Abstract

Systems and methods are provided for generating maps with semantic labels. A computing device can determine a first map that includes features located at first positions and semantic labels located at semantic positions, and determine a second map that includes at least some of the features located at second positions. The computing device can identify a first region with fixed features located at first positions and corresponding equivalent second positions. The computing device can identify a second region with moved features located at first positions and corresponding non-equivalent second positions. The computing device can determine one or more transformations between first positions and second positions. The computing device can assign the semantic labels to the second map at second semantic positions, where the second semantic positions are the same in the first region, and where the second semantic positions in the second region are based on the transformation(s).

Claims (64)

1. A method, comprising:

determining a first map of an environment using a computing device, wherein the first map comprises a plurality of features located at a corresponding plurality of first positions and a plurality of semantic labels located at a corresponding plurality of semantic positions;

determining a second map of the environment using the computing device, wherein the second map comprises at least some of the plurality of features located at a corresponding plurality of second positions;

identifying a plurality of moved features using the computing device, wherein a moved feature in the plurality of moved features is located at a corresponding first position of the plurality of first positions and a different corresponding second position of the plurality of second positions, and wherein the corresponding first position of the moved feature is not equivalent to the different corresponding second position of the moved feature;

for each semantic label of the plurality of semantic labels, the computing device identifying a nearby subset of the plurality of moved features;

for each semantic label of the plurality of semantic labels, the computing device determining a transformation based on a coordinate mapping between the first positions and the second positions of the nearby subset of moved features identified for the semantic label;

for each semantic label of the plurality of semantic labels, the computing device assigning a semantic position in the second map to the semantic label using the transformation determined for the semantic label; and

generating an output of the computing device based on the second map.

2. The method of claim 1 , wherein the computing device is part of a robotic device, wherein determining the first map comprises determining a first map based on first data obtained by one or more sensor devices of the robotic device, wherein determining the second map comprises determining the second map based on second data obtained by the one or more sensor devices of the robotic device, and wherein generating the output comprises navigating the robotic device through the environment based on guidance provided by at least one semantic label of the second map.

3. The method of claim 1 , wherein determining the transformation based on the coordinate mapping comprises determining an affine transformation of an initial position to a transformed position.

4. The method of claim 3 , wherein the affine transformation comprises at least one of: a rotation related to the initial position, a translation related to the initial position, a reflection related to the initial position, and a combination of one or more rotations, translations, and reflections.

5. The method of claim 1 , wherein determining the second map comprises:

determining a current position of a particular feature of the plurality of features;

determining whether the current position of the particular feature is equivalent to a first position for the particular feature; and

after determining that the current position of the particular feature is not equivalent to a first position for the particular feature, determining the second map of the environment.

6. The method of claim 1 , wherein a particular semantic label of the plurality of semantic labels is associated with a particular object and the particular semantic label corresponds to a particular semantic position of the plurality of semantic positions, and wherein determining the second map comprises:

determining a current position of the particular object;

determining whether the current position of the particular object is equivalent to the particular semantic position; and

after determining that the current position of the particular object is not equivalent to the particular semantic position, determining the second map of the environment.

7. The method of claim 1 , wherein determining the second map comprises determining the second map at periodic intervals in time.

8. The method of claim 1 , wherein the environment comprises at least a third region that differs from the first region, wherein the third region comprises one or more third features, and wherein the method further comprises:

determining one or more moved third features of the one or more third features, wherein each moved third feature of the one or more moved third features is located at a corresponding first position of the plurality of first positions and a different corresponding second position of the plurality of second positions, and where the corresponding first position of the moved feature is not equivalent to the different corresponding second position of the moved feature; and

determining one or more second transformations for the third region between first positions and second positions of the one or more moved third features, wherein semantic positions in the second plurality of semantic positions for semantic labels in the third region are based on the one or more second transformations for the third region.

9. A computing device, comprising:

one or more processors; and

data storage including at least computer-executable instructions stored thereon that, when executed by the one or more processors, cause the computing device to perform functions comprising:

determining a first map of an environment, wherein the first map comprises a plurality of features located at a corresponding plurality of first positions and a plurality of semantic labels located at a corresponding plurality of semantic positions;

determining a second map of the environment, wherein the second map comprises at least some of the plurality of features located at a corresponding plurality of second positions;

identifying a plurality of moved features, wherein a moved feature in the plurality of moved features is located at a corresponding first position of the plurality of first positions and a different corresponding second position of the plurality of second positions, and wherein the corresponding first position of the moved feature is not equivalent to the different corresponding second position of the moved feature;

for each semantic label of the plurality of semantic labels, identifying a nearby subset of the plurality of moved features;

for each semantic label of the plurality of semantic labels, determining a transformation based on a coordinate mapping between the first positions and the second positions of the nearby subset of moved features identified for the semantic label;

for each semantic label of the plurality of semantic labels, assigning a semantic position in the second map to the semantic label using the transformation determined for the semantic label; and

generating an output based on the second map.

10. The computing device of claim 9 , wherein the computing device is part of a robotic device, wherein determining the first map comprises determining a first map based on first data obtained by one or more sensor devices of the robotic device, wherein determining the second map comprises determining the second map based on second data obtained by the one or more sensor devices of the robotic device, and wherein generating the output comprises navigating the robotic device through the environment based on guidance provided by at least one semantic label of the second map.

11. The computing device of claim 9 , wherein determining the transformation based on the coordinate mapping comprises determining an affine transformation of an initial position to a transformed position.

12. The computing device of claim 11 , wherein the affine transformation comprises at least one of: a rotation related to the initial position, a translation related to the initial position, a reflection related to the initial position, and a combination of one or more rotations, translations, and reflections.

13. The computing device of claim 9 , wherein determining the second map comprises:

determining a current position of a particular feature of the plurality of features;

determining whether the current position of the particular feature is equivalent to a first position for the particular feature; and

after determining that the current position of the particular feature is not equivalent to a first position for the particular feature, determining the second map of the environment.

14. The computing device of claim 9 , wherein a particular semantic label of the plurality of semantic labels is associated with a particular object and the particular semantic label corresponds to a particular semantic position of the plurality of semantic positions, and wherein determining the second map comprises:

determining a current position of the particular object;

determining whether the current position of the particular object is equivalent to the particular semantic position; and

after determining that the current position of the particular object is not equivalent to the particular semantic position, determining the second map of the environment.

15. The computing device of claim 9 , wherein determining the second map comprises determining the second map at periodic intervals in time.

16. The computing device of claim 9 , wherein the environment comprises at least a third region that differs from the first region, wherein the third region comprises one or more third features, and wherein the functions further comprise:

determining one or more moved third features of the one or more third features, wherein each moved third feature of the one or more moved third features is located at a corresponding first position of the plurality of first positions and a different corresponding second position of the plurality of second positions; and

determining one or more second transformations for the third region between first positions and second positions of the one or more moved third features, wherein semantic positions in the second plurality of semantic positions for semantic labels in the third region are based on the one or more second transformations for the third region.

17. A non-transitory computer readable medium having stored thereon instructions, that when executed by one or more processors of a computing device, cause the computing device to perform functions comprising:

determining a first map of an environment, wherein the first map comprises a plurality of features located at a corresponding plurality of first positions and a plurality of semantic labels located at a corresponding plurality of semantic positions;

determining a second map of the environment, wherein the second map comprises at least some of the plurality of features located at a corresponding plurality of second positions;

identifying a plurality of moved features, wherein a moved feature in the plurality of moved features is located at a corresponding first position of the plurality of first positions and a different corresponding second position of the plurality of second positions, and wherein the corresponding first position of the moved feature is not equivalent to the different corresponding second position of the moved feature;

for each semantic label of the plurality of semantic labels, identifying a nearby subset of the plurality of moved features;

for each semantic label of the plurality of semantic labels, determining a transformation based on a coordinate mapping between the first positions and the second positions of the nearby subset of moved features identified for the semantic label;

for each semantic label of the plurality of semantic labels, assigning a semantic position in the second map to the semantic label using the transformation determined for the semantic label; and

generating an output based on the second map.

18. The non-transitory computer readable medium of claim 17 , wherein the non-transitory computer readable medium is part of a robotic device, wherein determining the first map comprises determining a first map based on first data obtained by one or more sensor devices of the robotic device, wherein determining the second map comprises determining the second map based on second data obtained by the one or more sensor devices of the robotic device, and wherein generating the output comprises navigating the robotic device through the environment based on guidance provided by at least one semantic label of the second map.

19. The non-transitory computer readable medium of claim 17 , wherein determining the transformation based on the coordinate mapping comprises determining an affine transformation of an initial position to a transformed position, and wherein the affine transformation comprises at least one of: a rotation related to the initial position, a translation related to the initial position, a reflection related to the initial position, and a combination of one or more rotations, translations, and reflections.

20. The non-transitory computer readable medium of claim 17 , wherein determining the second map comprises:

determining a current position of a particular feature of the plurality of features;

determining whether the current position of the particular feature is equivalent to a first position for the particular feature; and

after determining that the current position of the particular feature is not equivalent to a first position for the particular feature, determining the second map of the environment.

21. The method of claim 1 , wherein identifying the plurality of moved features comprises:

determining whether a feature is in the plurality of moved features based on comparing movement of the feature to a corresponding threshold for at least one map dimension of a plurality of map dimensions.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2025
From: GOOGLE LLC
To: GDM HOLDING LLC
Reel/Frame 071109/0342 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE REMOVAL OF THE INCORRECTLY RECORDED APPLICATION NUMBERS 14/149802 AND 15/419313 PREVIOUSLY RECORDED AT REEL: 44144 FRAME: 1. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Mar 4, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 068092/0502 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2023
From: X DEVELOPMENT LLC
To: GOOGLE LLC
Reel/Frame 064658/0001 →
CHANGE OF NAME Recorded Oct 6, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044144/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2016
From: GOOGLE INC.
To: X DEVELOPMENT LLC
Reel/Frame 039900/0610 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2015
From: WATTS, KEVIN WILLIAM; MASON, JULIAN MAC NEILLE; ANDERSON-SPRECHER, PETER ELVING
To: GOOGLE INC.
Reel/Frame 035246/0672 →