IP Library › Granted Patent US 11,675,084
Granted Patent B2
US 11,675,084 · App. 16/856,275 · Granted Jun 13, 2023

Determining yaw error from map data, lasers, and cameras

Inventors: John Tisdale (Oakland, CA); Michael Montemerlo (Mountain View, CA); Andrew Chatham (San Francisco, CA)
Assignee: Waymo LLC
G01S17/89G01S7/4808G01S7/497G06V10/751G06V20/56H04N7/185
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,675,084
App. No.
16/856,275
Granted
Jun 13, 2023
Kind
B2
Abstract

The present disclosure relates to methods and systems that facilitate determination of a pose of a vehicle based on various combinations of map data and sensor data received from light detection and ranging (LIDAR) devices and/or camera devices. An example method includes receiving point cloud data from a (LIDAR) device and transforming the point cloud data to provide a top-down image. The method also includes comparing the top-down image to a reference image and determining, based on the comparison, a yaw error. An alternative method includes receiving camera image data from a camera and transforming the camera image data to provide a top-down image. The method also includes comparing the top-down image to a reference image and determining, based on the comparison, a yaw error.

Claims (40)

1. A method comprising:

capturing point cloud data using a light detection and ranging (LIDAR) device, wherein the point cloud data comprises information indicative of an environment around a vehicle;

transforming the point cloud data to provide a top-down image;

comparing the top-down image to a reference image;

determining, based on the comparison, a yaw error, wherein the yaw error comprises an adjustment signal;

receiving, at a pose estimator, the adjustment signal;

determining, with the pose estimator, a yaw orientation of the vehicle; and

adjusting the yaw orientation of the vehicle based on the adjustment signal, wherein adjusting the yaw orientation of the vehicle comprises updating a driving path of the vehicle.

2. The method of claim 1 , wherein capturing point cloud data comprises capturing information indicative of a 360 degree environment.

3. The method of claim 1 , wherein transforming the point cloud data to provide a top-down image comprises a scale-invariant feature transform, image flattening, image perspective rotation, or a 3D-to-2D image transform.

4. The method of claim 1 , wherein transforming the point cloud data into a top-down image comprises culling or ignoring at least a portion of the point cloud data.

5. The method of claim 1 , wherein the reference image comprises at least a portion of at least one of: one or more prior camera images, a prior LIDAR point cloud, prior RADAR data, satellite imagery, map data, or other overhead imagery.

6. The method of claim 1 , further comprising forming an aggregate image, wherein the aggregate image comprises a combination, overlay, superposition, or image comparison between the top-down image and the reference image.

7. The method of claim 6 , wherein the aggregate image comprises an indication of the yaw error and wherein the yaw error comprises an angle difference between an estimated heading and an actual heading.

8. A method comprising:

capturing camera image data using a camera, wherein the camera image data comprises information indicative of an environment around a vehicle;

transforming the camera image data to provide a top-down image;

comparing the top-down image to a reference image;

determining, based on the comparison, a yaw error, wherein the yaw error comprises an adjustment signal;

receiving, at a pose estimator, the adjustment signal;

determining, with the pose estimator, a yaw orientation of the vehicle; and

adjusting the yaw orientation of the vehicle based on the adjustment signal, wherein adjusting the yaw orientation of the vehicle comprises updating a driving path of the vehicle.

9. The method of claim 8 , wherein capturing camera image data comprises capturing the camera image data using a plurality of cameras configured to capture a plurality of images comprising information indicative of a 360 degree field of view of the environment.

10. The method of claim 9 , wherein transforming the camera image data comprises combining the plurality of images to form the top-down image, wherein the combining comprises one or more of projective geometry transforms or perspective morphing techniques applied to the plurality of images.

11. The method of claim 8 , further comprising forming a forward aggregate image based on a combination, overlay, or superposition image comparison between a forward reference image and forward camera image data.

12. The method of claim 11 , wherein determining the yaw error comprises comparing an actual yaw direction in the forward camera image data to an estimated yaw direction in the forward reference image.

13. A method comprising:

capturing information indicative of a representation of a three dimensional environment around a vehicle;

transforming the information to provide a top-down representation of the three dimensional environment;

comparing the top-down representation to reference information;

determining, based on the comparison, a rotational angle difference, wherein the rotational angle difference comprises an adjustment signal;

receiving, at a pose estimator, the adjustment signal;

determining, with the pose estimator, a yaw orientation of the vehicle; and

adjusting the yaw orientation of the vehicle based on the adjustment signal, wherein adjusting the yaw orientation of the vehicle comprises updating a driving path of the vehicle.

14. The method of claim 13 , wherein transforming the information to provide a top-down representation of the three dimensional environment comprises transforming the information with at least one of: a scale-invariant feature transform, image flattening, image perspective rotation, or 3D-to-2D image transform.

15. The method of claim 13 , wherein the reference information comprises at least one of: a previously captured camera image, previously captured LIDAR point cloud data, previously captured RADAR data, map data, satellite imagery, or other overhead imagery.

16. The method of claim 13 , wherein transforming the information to provide the top-down representation comprises forming an aggregated image based on at least two top-down representations, wherein forming the aggregated image comprises applying at least one of: a combination, an overlay, a superposition, a juxtaposition, or a comparison of the at least two top-down representations.

17. The method of claim 13 , wherein comparing the top-down representation to the reference information comprises performing at least one of: a normalized cross-correlation, grayscale matching, gradient matching, histogram matching, edge detection, scale-invariant feature transform (SIFT), or speeded-up robust features (SURF).

18. The method of claim 13 , further comprising

determining a refined yaw orientation based on the adjustment signal and the determined yaw orientation of the vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2020
From: TISDALE, JOHN; MONTEMERLO, MICHAEL; CHATHAM, ANDREW
To: WAYMO LLC
Reel/Frame 052522/0904 →
Continuity (2)
Continuation 15658800 · Jul 25, 2017
Related Publication 20200249359A1 · Aug 6, 2020