IP Library Granted Patent US 10,488,523
Granted Patent B2
US 10,488,523 · App. 15/642,530 · Granted Nov 26, 2019

Using laser sensors to augment stereo sensor readings for robotic devices

Inventors: Kevin William Watts (Palo Alto, CA); Kurt Konolige (Menlo Park, CA)
Assignee: X Development LLC
G01S17/936G01S17/023G01S17/87G05D1/0248G05D1/0251G05D2201/0216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,488,523
App. No.
15/642,530
Granted
Nov 26, 2019
Kind
B2
Abstract

An example system includes one or more laser sensors on a robotic device, where the one or more laser sensors are configured to produce laser sensor data indicative of a first area within a first distance in front of the robotic device. The system further includes one or more stereo sensors on the robotic device, where the stereo sensors on the robotic device are configured to produce stereo sensor data indicative of a second area past a second distance in front of the robotic device. The system also includes a controller configured to receive the laser sensor data, receive the stereo sensor data, detect one or more objects in front of the robotic device based on at least one of the laser sensor data and the stereo sensor data, and provide instructions for the robotic device to navigate based on the one or more detected objects.

Claims (42)

1. A system comprising:

a first stereo sensor coupled to a robotic device, the first stereo sensor comprising a first primary optical element and a secondary optical element;

a plurality of lasers arranged on the robotic device such that at a minimum range of the first stereo sensor, each distance between diverging laser beams from adjacent lasers of the plurality of lasers is less than a predetermined minimum obstacle size; and

a controller configured to:

receive, from the first stereo sensor, stereo sensor data indicative of a first scanned area, wherein the first scanned area extends from the minimum range of the first stereo sensor;

receive, from the plurality of lasers, laser sensor data indicative of a second scanned area, wherein the second scanned area extends from the robotic device past the minimum range of the first stereo sensor;

based on the stereo sensor data or the laser sensor data, detect an object in the first scanned area or the second scanned area; and

based on the detected object, provide navigation instructions to the robotic device.

2. The system of claim 1 , further comprising a second stereo sensor coupled to the robotic device, the second stereo sensor comprising a second primary optical element and the secondary optical element.

3. The system of claim 2 , wherein the first primary optical element and the secondary optical element are positioned closer together than the second primary optical element and the secondary optical element.

4. The system of claim 2 , wherein the optical elements of the first stereo sensor are spaced in a narrow baseline arrangement, and wherein the optical elements of the second stereo sensor are spaced in a wide baseline arrangement.

5. The system of claim 1 , wherein the first scanned area starts at a distance from the robotic device at which a field of view of the first stereo sensor is at least as wide as the robotic device.

6. A method comprising:

receiving stereo sensor data from a first stereo sensor coupled to a robotic device,

wherein the first stereo sensor comprises a first primary optical element and a secondary optical element,

wherein the stereo sensor data is indicative of a first scanned area, and

wherein the first scanned area extends from a minimum range of the first stereo sensor;

receiving laser sensor data from a plurality of lasers coupled to the robotic device,

wherein the plurality of lasers are arranged on the robotic device such that at the minimum range of the first stereo sensor, each distance between diverging laser beams from adjacent lasers of the plurality of lasers is less than a predetermined minimum obstacle size,

wherein the laser sensor data is indicative of a second scanned area, and

wherein the second scanned area extends from the robotic device past the minimum range of the first stereo sensor;

monitoring the first scanned area and the second scanned area;

detecting an object in the first scanned area or the second scanned area; and

providing navigation instructions to the robotic device such that the robotic device avoids colliding with the object.

7. The method of claim 6 , further comprising receiving stereo sensor data from a second stereo sensor coupled to the robotic device, wherein the second stereo sensor comprises a second primary optical element and the secondary optical element.

8. The method of claim 7 , wherein the first primary optical element and the secondary optical element are positioned closer together than the second primary optical element and the secondary optical element.

9. The method of claim 6 , wherein monitoring the first scanned area and the second scanned area comprises:

monitoring the first scanned area and the second scanned area for objects located in a path of the robotic device.

10. The method of claim 6 , wherein providing navigation instructions to the robotic device comprises:

determining a distance of the object from the robotic device;

based on the distance, determining a margin of error in the stereo sensor data; and

based on the distance and the margin of error, determining the navigation instructions.

11. A robotic device comprising:

at least one motion component configured to cause motion of the robotic device;

a first stereo sensor comprising a first primary optical element and a secondary optical element;

a plurality of lasers arranged on the robotic device such that at a minimum range of the first stereo sensor, each distance between diverging laser beams from adjacent lasers of the plurality of lasers is less than a predetermined minimum obstacle size; and

a controller configured to:

receive, from the first stereo sensor, stereo sensor data indicative of a first scanned area in front of the robotic device, wherein the first scanned area extends from the minimum range of the first stereo sensor;

receive, from the plurality of lasers, laser sensor data indicative of a second scanned area, wherein the second scanned area extends from the robotic device past the minimum range of the first stereo sensor;

based on the stereo sensor data or the laser sensor data, detect an object in the first scanned area or the second scanned area; and

control the at least one motion component such that the robotic device avoids colliding with the detected object.

12. The robotic device of claim 11 , further comprising a second stereo sensor, wherein the second stereo sensor comprises a second primary optical element and the secondary optical element, and wherein the optical elements of the first stereo sensor are spaced in a narrow baseline arrangement, and wherein the optical elements of the second stereo sensor are spaced in a wide baseline arrangement.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2025
From: GOOGLE LLC
To: GDM HOLDING LLC
Reel/Frame 071109/0342 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2023
From: X DEVELOPMENT LLC
To: GOOGLE LLC
Reel/Frame 064658/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2017
From: GOOGLE INC.
To: X DEVELOPMENT LLC
Reel/Frame 044413/0733 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 6, 2017
From: WATTS, KEVIN WILLIAM; KONOLIGE, KURT
To: GOOGLE INC.
Reel/Frame 044043/0447 →
Continuity (2)
Continuation 14828437 · Aug 17, 2015
Related Publication 20170308086A1 · Oct 26, 2017
Cited By (2)
US 12,300,477 US 12,400,888