IP Library Granted Patent US 9,746,852
Granted Patent B1
US 9,746,852 · App. 14/828,437 · Granted Aug 29, 2017

Using laser sensors to augment stereo sensor readings for robotic devices

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,746,852
App. No.
14/828,437
Granted
Aug 29, 2017
Kind
B1
Abstract

An example system includes one or more laser sensors on a robotic device, where the one or more laser sensors are configured to produce laser sensor data indicative of a first area within a first distance in front of the robotic device. The system further includes one or more stereo sensors on the robotic device, where the stereo sensors on the robotic device are configured to produce stereo sensor data indicative of a second area past a second distance in front of the robotic device. The system also includes a controller configured to receive the laser sensor data, receive the stereo sensor data, detect one or more objects in front of the robotic device based on at least one of the laser sensor data and the stereo sensor data, and provide instructions for the robotic device to navigate based on the one or more detected objects.

Claims (48)

1. A system comprising:

one or more laser sensors on a robotic device, wherein the one or more laser sensors are configured to produce laser sensor data indicative of a first area within a first distance in front of the robotic device, and wherein the one or more laser sensors comprise a plurality of laser sensors spaced across the robotic device so that consecutive laser sensors are closer together than a predetermined minimum obstacle size;

one or more stereo sensors on the robotic device, wherein the one or more stereo sensors on the robotic device are configured to produce stereo sensor data indicative of a second area past a second distance in front of the robotic device, wherein the second distance is closer to the robotic device than the first distance; and

a controller configured to:

receive the laser sensor data indicative of the first area within the first distance in front of the robotic device from the one or more laser sensors;

receive the stereo sensor data indicative of the second area past the second distance in front of the robotic device from the one or more stereo sensors;

detect one or more objects in front of the robotic device based on at least one of the laser sensor data and the stereo sensor data; and

provide instructions for the robotic device to navigate based on the one or more detected objects.

2. The system of claim 1 , wherein the one or more laser sensors comprise a plurality of laser sensors that project respective narrow laser beams in front of the robotic device.

3. The system of claim 2 , wherein the plurality of laser sensors are aligned horizontally at a certain height on a front surface of the robotic device.

4. The system of claim 2 , wherein the plurality of laser sensors are oriented parallel to a direction of travel of the robotic device.

5. The system of claim 1 , wherein the second distance comprises a distance from the robotic device at which a field of view of the one or more stereo sensors encompasses a width larger than a width of the robotic device.

6. The system of claim 1 , wherein the one or more laser sensors comprise a plurality of laser sensors spaced across the front surface of the robotic device to detect obstacles of at least a predetermined minimum size at least up to the second distance from the robotic device.

7. The system of claim 1 , wherein the one or more laser sensors comprise a plurality of horizontal rows of laser sensors aligned at a plurality of different heights on the robotic device, and wherein each laser sensor projects respective narrow laser beams.

8. The system of claim 1 , wherein:

a first stereo sensor of the one or more stereo sensors comprises a first primary optical element and a secondary optical element such that the stereo sensor data from the first stereo sensor is indicative of a third area within the second area;

a second stereo sensor of the one or more stereo sensors comprises a second primary optical element and the secondary optical element such that the stereo sensor data from the second stereo sensor is indicative of a fourth area within the second area; and

the first primary optical element and the secondary optical element are positioned closer together on the robotic device than the second primary optical element and the secondary optical element.

9. The system of claim 8 , wherein the optical elements of the first stereo sensor are spaced in a narrow baseline arrangement, and wherein the optical elements of the second stereo sensor are spaced in a wide baseline arrangement.

10. A method comprising:

receiving laser sensor data from a plurality of laser sensors Off spaced across a robotic device so that consecutive laser sensors are closer together than a predetermined minimum obstacle size, wherein the laser sensor data is indicative of a first area within a first distance of a direction of travel of the robotic device;

receiving stereo sensor data from one or more stereo sensors on the robotic device, wherein the stereo sensor data is indicative of a second area past a second distance in the direction of travel of the robotic device, wherein the second distance is closer to the robotic device than the first distance;

monitoring the first area based on the laser sensor data and the second area based on the stereo sensor data;

detecting one or more objects in at least one of the first area and second area; and

providing instructions for the robotic device to navigate based at least on the one or more detected objects.

11. The method of claim 10 , wherein the plurality of laser sensors are configured to project respective narrow laser beams at least up to the first distance in the direction of travel of the robotic device.

12. The method of claim 10 , wherein the second distance comprises a distance from the robotic device at which a field of view of the one or more stereo sensors encompasses a width larger than a width of the robotic device.

13. The method of claim 10 , wherein the plurality of laser sensors are spaced across the robotic device to detect obstacles of at least a predetermined minimum size at least up to the second distance from the robotic device.

14. The method of claim 10 , wherein:

a first stereo sensor of the one or more stereo sensors comprises a first primary optical element and a secondary optical element such that the stereo sensor data from the first stereo sensor is indicative of a third area within the second area;

a second stereo sensor of the one or more stereo sensors comprises a second primary optical element and the secondary optical element such that the stereo sensor data from the second stereo sensor is indicative of a fourth area within the second area; and

the first primary optical element and the secondary optical element are closer together on the robotic device than the second primary optical element and the secondary optical element.

15. A robotic device comprising:

at least one motion component configured to cause motion of the robotic device in a direction of travel;

a plurality of laser sensors on the robotic device, wherein the plurality of laser sensors are horizontally aligned at a certain height on the robotic device, wherein the plurality of laser sensors are oriented parallel to the direction of travel of the robotic device, and wherein the plurality of laser sensors are configured to:

project respective narrow laser beams in front of the robotic device in the direction of travel of the robotic device;

detect reflected laser beams at the robotic device from at least some of the projected narrow laser beams; and

detect objects within a first area up to a first distance in the direction of travel of the robotic device based on the reflected laser beams; and

one or more stereo sensors on the robotic device, wherein the one or more stereo sensors on the robotic device are configured to:

receive respective images captured by at least two optical elements from at least two respective viewpoints;

identify corresponding features between the respective images captured by the at least two optical elements; and

detect objects within a second area past a second distance in the direction of travel of the robotic device based on the corresponding features, wherein the second distance is closer to the robotic device than the first distance.

16. The robotic device of claim 15 , wherein the second distance comprises a distance from the robotic device at which a field of view of the one or more stereo sensors encompasses a width larger than the width of the robotic device.

17. The robotic device of claim 15 , wherein the plurality of laser sensors are spaced across a front surface of the robotic device to detect obstacles of at least a predetermined minimum size at least up to the second distance from the robotic device.

18. The robotic device of claim 15 , wherein:

a first stereo sensor of the one or more stereo sensors comprises a first primary optical element and a secondary optical element such that the stereo sensor data from the first stereo sensor is indicative of a third area within the second area;

a second stereo sensor of the one or more stereo sensors comprises a second primary optical element and the secondary optical element such that the stereo sensor data from the second stereo sensor is indicative of a fourth area within the second area; and

the first primary optical element and the secondary optical element are closer together on the robotic device than the second primary optical element and the secondary optical element.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2025
From: GOOGLE LLC
To: GDM HOLDING LLC
Reel/Frame 071109/0342 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE REMOVAL OF THE INCORRECTLY RECORDED APPLICATION NUMBERS 14/149802 AND 15/419313 PREVIOUSLY RECORDED AT REEL: 44144 FRAME: 1. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Mar 4, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 068092/0502 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2023
From: X DEVELOPMENT LLC
To: GOOGLE LLC
Reel/Frame 064658/0001 →
CHANGE OF NAME Recorded Oct 6, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044144/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2016
From: GOOGLE INC.
To: X DEVELOPMENT LLC
Reel/Frame 039900/0610 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2015
From: WATTS, KEVIN WILLIAM; WILLIAMS, KURT
To: GOOGLE INC.
Reel/Frame 036345/0452 →