IP Library Granted Patent US 12,138,812
Granted Patent B2
US 12,138,812 · App. 17/386,001 · Granted Nov 12, 2024

Systems and methods for laser and imaging odometry for autonomous robots

Inventors: Girish Bathala (San Diego, CA); Sahil Dhayalkar (San Diego, CA); Kirill Pirozhenko (San Diego, CA); Oleg Sinyavskiy (San Diego, CA)
Assignee: Brain Corporation
B25J9/1697B25J9/1664B25J13/089G06T7/20G06T7/579G06T7/70G06T2207/20221G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,138,812
App. No.
17/386,001
Granted
Nov 12, 2024
Kind
B2
Abstract

Systems and methods for laser and imaging odometry for autonomous robots are disclosed herein. According to at least one non-limiting exemplary embodiment, a robot may utilize images captured by a sensor and encoded with a depth parameter to determine its motion and localize itself. The determined motion and localization may then be utilized to verify calibration of the sensor based on a comparison between motion and localization data based on the images and motion and localization data based on data from other sensors and odometry units of the robot.

Claims (49)

1. A method for determining a pose of a sensor of a robot, comprising:

generating, via a controller configured to execute computer readable instructions, a composite image including overlaying of a first image and a second image captured by the sensor of the robot at different time intervals, the first image including a plurality of pixels representing an object at a first location, and the second image including a plurality of pixels representing the object at a different second location;

determining, via the controller, a first motion of the robot based on a discrepancy between the first and second images by evaluating at least one depth disparity measurement and at least one spatial disparity measurement of the object represented in the first and second images;

determining, via the controller, a second motion of the robot based on odometry data from one or more proprioceptive sensor of the robot;

determining, via the controller, a most probable pose of the sensor based at least in part on a plurality of possible poses of the sensor and a continuous distribution function of the plurality of possible poses by applying a maximum a posteriori estimation, wherein the continuous distribution function of the plurality of possible poses comprises a set of priors for the maximum a posteriori estimation, wherein each of the plurality of possible poses is determined based on a respective discrepancy of a plurality of discrepancies between the first motion and the second motion, and wherein the continuous distribution function of the plurality of possible poses is measured over a period of time beginning prior to the first motion and the second motion of the robot; and

localizing, via the controller, the object on a computer readable map based at least in part on the most probable pose of the sensor.

2. The method of claim 1 , further comprising:

determining, via the controller, discrepancy between the first and second images by translating and rotating the second image such that the translated and rotated second image matches the first image; and

determining, via the controller, the change in pose of the sensor based on the translations and rotations performed on the second image.

3. The method of claim 2 , wherein the translating and rotating of the second image is performed on a point cloud representation of the second image to match a point cloud representation of the first image.

4. The method of claim 2 , further comprising:

localizing, via the controller, the robot by utilizing the translations and rotations performed on the second image to determine a change in position and orientation of the sensor between a first time and a second time corresponding to a change in position of the robot.

5. The method of claim 1 , wherein

the sensor is configured to measure at least a distance parameter of objects within a visual scene, the sensor is positioned on the robot in a fixed pose, and

the pixels of the first image and the second image are encoded with the distance parameter.

6. The method of claim 1 , further comprising:

navigating the robot along a third motion on the computer readable map based at least in part on the most probable pose of the sensor.

7. A non-transitory computer readable storage medium comprising a plurality of computer readable instructions stored thereon, that when executed by a controller, configure the controller to

generate a composite image including overlaying of a first image and a second image captured by a sensor of a robot at different time intervals, the first image including a plurality of pixels representing an object at a first location, and the second image including a plurality of pixels representing the object at a different second location;

determine a first motion of the robot based on a discrepancy between the first and second images by evaluating at least one depth disparity measurement and spatial disparity measurement of the object represented in the first and second images;

determine a second motion of the robot based on odometry data from one or more proprioceptive sensor of the robot;

determine a most probable pose of the sensor based at least in part on a plurality of possible poses of the sensor and a continuous distribution function of the plurality of possible poses by applying a maximum a posteriori estimation, wherein the continuous distribution function of the plurality of possible poses comprises a set of priors for the maximum a posteriori estimation, wherein each of the plurality of possible poses is determined based on a respective discrepancy of a plurality of discrepancies between the first motion and the second motion, and wherein the continuous distribution function of the plurality of possible poses is measured over a period of time beginning prior to the first motion and the second motion of the robot; and

localize the object on a computer readable map based at least in part on the most probable pose of the sensor.

8. The non-transitory computer readable storage medium of claim 7 , wherein the controller is further configurable to execute the computer readable instructions to

determine discrepancy between the first and second images by translating and rotating the second image such that the translated and rotated second image matches the first image, and

determine the change in pose of the sensor based on the translations and rotations performed on the second image.

9. The non-transitory computer readable storage medium of claim 8 , wherein the translating and rotating of the second image is performed on a point cloud representation of the second image to match a point cloud representation of the first image.

10. The non-transitory computer readable storage medium of claim 8 , wherein the controller is further configurable to execute the computer readable instructions to

localize the robot by utilizing the translations and rotations performed on the second image to determine a change in position and orientation of the sensor between a first time and a second time corresponding to a change in position of the robot.

11. The non-transitory computer readable storage medium of claim 7 , wherein

the sensor is configured to measure at least a distance parameter of objects within a visual scene and positioned on the robot in a fixed pose, and

the pixels of the first image and the second image are encoded with the distance parameter.

12. A system for determining a pose of a sensor of a robot, comprising:

a memory having computer readable instructions stored thereon; and

a controller configurable to execute the computer readable instructions to

generate a composite image including overlaying of a first image and a second image captured by the sensor of the robot at different time intervals, the first image including a plurality of pixels representing an object at a first location, and the second image including a plurality of pixels representing the object at a different second location,

determine a first motion of the robot based on a discrepancy between the first and second images by evaluating depth disparity measurement and spatial disparity measurement of the object represented in the first and second images,

determine a second motion of the robot based on odometry data from one or more proprioceptive sensor of the robot;

a most probable pose of the sensor based at least in part on a plurality of possible poses of the sensor and a continuous distribution function of the plurality of possible poses by applying a maximum a posteriori estimation, wherein the continuous distribution function of the plurality of possible poses comprises a set of priors for the maximum a posteriori estimation, wherein each of the plurality of possible poses is determined based on a respective discrepancy of a plurality of discrepancies between the first motion and the second motion, and wherein the continuous distribution function of the plurality of possible poses is measured over a period of time beginning prior to the first motion and the second motion of the robot; and

localize the object on a computer readable map based at least in part on the most probable pose of the sensor.

13. The system of claim 12 , wherein the controller is further configurable to execute the computer readable instructions to

determine discrepancy between the first and second images by translating and rotating the second image such that the translated and rotated second image matches the first image, and

determine the change in pose of the sensor based on the translations and rotations performed on the second image.

14. The system of claim 13 , wherein the translating and rotating of the second image is performed on a point cloud representation of the second image to match a point cloud representation of the first image.

15. The system of claim 13 , wherein the controller is further configurable to

localize the robot by utilizing the translations and rotations performed on the second image to determine a change in position and orientation of the sensor between a first time and a second time corresponding to a change in position of the robot.

16. The robotic system of claim 12 , wherein

the sensor is configured to measure at least a distance parameter of objects within a visual scene, the sensor is positioned on the robot in a fixed pose, and

the pixels of the first image and the second image are encoded with the distance parameter.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2022
From: BATHALA, GIRISH; DHAYALKAR, SAHIL; PIROZHENKO, KIRILL; SINYAVSKIY, OLEG
To: BRAIN CORPORATION
Reel/Frame 061675/0355 →
SECURITY INTEREST Recorded Oct 8, 2021
From: BRAIN CORPORATION
To: HERCULES CAPITAL, INC.
Reel/Frame 057851/0574 →
Continuity (3)
Continuation PCTUS2020016070 · Jan 31, 2020
Provisional Application 62799142 · Jan 31, 2019
Related Publication 20210354302A1 · Nov 18, 2021