IP Library Granted Patent US 10,852,740
Granted Patent B2
US 10,852,740 · App. 16/147,082 · Granted Dec 1, 2020

Determining the orientation of flat reflectors during robot mapping

Inventors: Elizabeth Murphy (San Jose, CA); Dirk Holz (Mountain View, CA)
Assignee: X Development LLC
G05D1/0236G01C21/367G05D1/0244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,852,740
App. No.
16/147,082
Granted
Dec 1, 2020
Kind
B2
Abstract

One method disclosed includes determining a map of markers fixed in an environment, where the map of markers includes a location and an orientation of each marker. The method further includes determining locations of a set of detected markers relative to the map based on a location of a robotic device relative to the map and based on sensor data from the robotic device. The method also includes associating a detected marker from the set of detected markers with a mapped marker based on the determined location of the detected marker relative to the map and based on a visibility constraint related to the orientation of the mapped marker. The method additionally includes adjusting, in the map, the orientation of the mapped marker based on the determined location of the detected marker relative to the map.

Claims (36)

1. A method comprising:

determining a map of markers fixed in an environment, wherein the map of markers includes a location and an orientation of each marker;

determining locations of a set of detected markers relative to the map based on a location of a robotic device relative to the map and based on sensor data from the robotic device;

associating a detected marker from the set of detected markers with a mapped marker based on the determined location of the detected marker relative to the map and based on a visibility constraint related to the orientation of the mapped marker;

adjusting, in the map, the orientation of the mapped marker based on the determined location of the detected marker relative to the map; and

controlling the robotic device based on the adjusted orientation of the mapped marker in the map of markers.

2. The method of claim 1 , further comprising initially determining the orientation of the mapped marker based on a plurality of previously determined locations of detected markers associated with the mapped marker.

3. The method of claim 2 , wherein determining the orientation of the mapped marker comprises fitting a confidence ellipse to the plurality of previously determined locations of detected markers associated with the mapped marker.

4. The method of claim 2 , wherein determining the orientation of the mapped marker comprises:

determining a covariance matrix based on the plurality of previously determined locations of detected markers associated with the mapped marker; and

determining the orientation of the mapped marker based on an eigenvector of the covariance matrix.

5. The method of claim 4 , further comprising determining a first and second eigenvalue of the covariance matrix, wherein determining the orientation of the mapped marker is based on the eigenvector associated with a smaller of the first and second eigenvalue.

6. The method of claim 4 , wherein adjusting the orientation of the mapped marker based on the determined location of the detected marker relative to the map comprises adjusting the covariance matrix.

7. The method of claim 1 , further comprising incrementally adjusting the orientation of the mapped marker after each scan of a sequence of scans by the robotic device.

8. The method of claim 1 , further comprising adjusting, in the map, the location of the mapped marker based on the determined location of the detected marker relative to the map.

9. The method of claim 8 , wherein adjusting the location of the mapped marker comprises determining a running mean of determined locations of detected markers associated with the mapped marker.

10. The method of claim 1 , wherein associating the detected marker from the set of detected markers with the mapped marker comprises determining that the detected marker and the mapped marker are within a threshold distance of each other.

11. The method of claim 1 , wherein each marker in the map is substantially flat, and wherein the orientation of each marker corresponds to a marker normal.

12. The method of claim 11 , further comprising evaluating the visibility constraint for the mapped marker by determining whether an angle between a measurement ray and the marker normal of the mapped marker is less than a threshold angle.

13. The method of claim 12 , wherein the measurement ray comprises a ray between a mapped position of the mapped marker and a current sensor position of a sensor on the robotic device.

14. The method of claim 12 , wherein determining whether the angle between the measurement ray and the marker normal of the mapped marker is less than the threshold angle comprises evaluating a dot product of the measurement ray and the marker normal.

15. The method of claim 1 , further comprising adjusting the location of the robotic device relative to the map based on associating the detected marker from the set of detected markers with the mapped marker.

16. The method of claim 1 , wherein each of the markers in the map is a retro-reflective marker, and wherein the sensor data is generated using a laser-based sensor on the robotic device.

17. The method of claim 1 , wherein each of the markers is fixed at substantially a same height within the environment.

18. A robotic device comprising a control system configured to:

determine a map of markers fixed in an environment, wherein the map of markers includes a location and an orientation of each marker;

determine locations of a set of detected markers relative to the map based on a location of a robotic device relative to the map and based on sensor data from the robotic device;

associate a detected marker from the set of detected markers with a mapped marker based on the determined location of the detected marker relative to the map and based on a visibility constraint related to the orientation of the mapped marker;

adjust, in the map, the orientation of the mapped marker based on the determined location of the detected marker relative to the map; and

control the robotic device based on the adjusted orientation of the mapped marker in the map of markers.

19. A non-transitory computer readable medium comprising program instructions executable by at least one processor to cause the at least one processor to perform functions comprising:

determining a map of markers fixed in an environment, wherein the map of markers includes a location and an orientation of each marker;

determining locations of a set of detected markers relative to the map based on a location of a robotic device relative to the map and based on sensor data from the robotic device;

associating a detected marker from the set of detected markers with a mapped marker based on the determined location of the detected marker relative to the map and based on a visibility constraint related to the orientation of the mapped marker;

adjusting, in the map, the orientation of the mapped marker based on the determined location of the detected marker relative to the map; and

controlling the robotic device based on the adjusted orientation of the mapped marker in the map of markers.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY ADDRESS PREVIOUSLY RECORDED AT REEL: 057650 FRAME: 0325. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 8, 2021
From: X DEVELOPMENT LLC
To: INTRINSIC INNOVATION LLC
Reel/Frame 057896/0807 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 3, 2021
From: X DEVELOPMENT LLC
To: INTRINSIC INNOVATION LLC
Reel/Frame 057650/0325 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: MURPHY, ELIZABETH; HOLZ, DIRK
To: X DEVELOPMENT LLC
Reel/Frame 047052/0589 →