IP Library Granted Patent US 11,714,193
Granted Patent B1
US 11,714,193 · App. 16/136,006 · Granted Aug 1, 2023

Method for registering distance scan data

Inventors: Paul Foster (Mountain View, CA); Joel Pazhayampallil (Mountain View, CA); Lei Liu (Mountain View, CA)
G01S17/08G01S17/89G06T7/50
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,714,193
App. No.
16/136,006
Granted
Aug 1, 2023
Kind
B1
Abstract

One variation of a method for registering distance scan data includes: accessing a first distance scan recorded, at a first time, by a first depth sensor defining a first field of view; accessing a second distance scan recorded, at approximately the first time, by a second depth sensor defining a second field of view overlapping a portion of the first field of view; calculating a first set of lines represented by points in a first portion of the first distance scan overlapping a second portion of the second distance scan; calculating a second set of lines represented by points in the second portion of the second distance scan; calculating skew distances between each pair of corresponding lines in the first and second sets of lines; and calculating an alignment transformation that aligns the first distance scan and the second distance scan to minimize skew distances between corresponding pairs of lines.

Claims (44)

1 . One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:

accessing a first distance scan recorded by a first depth sensor during a time period, the first depth sensor defining a first field of view of a scene;

accessing a second distance scan recorded by a second depth sensor during the time period, the second depth sensor defining a second field of view overlapping at least a portion of the first field of view of the scene;

determining a first point set corresponding to a first portion of the first distance scan;

determining a second point set corresponding to a second portion of the second distance scan, the first portion of the first distance scan overlapping the second portion of the second distance scan;

determining a correspondence between the first set of points and the second set of points, the correspondence including a pair of point sets;

determining a skew distance between the pair of point sets by generating a weighted skew distance for the skew distance based on a perpendicularity of the pair of point sets;

generating an alignment transformation for the first depth sensor and the second depth sensor, the alignment transformation minimizing the weighted skew distance; and

generating a composite image of the scene based on the alignment transformation.

2 . The one or more tangible non-transitory computer-readable storage media of claim 1 , wherein a ground uncertainty region is identified in the composite image.

3 . The one or more tangible non-transitory computer-readable storage media of claim 2 , wherein at least one action of a vehicle is modified based on a location of at least one object being detected relative to the ground uncertainty region.

4 . The one or more tangible non-transitory computer-readable storage media of claim 3 , wherein the at least one object has a profile that is low relative to a height of the vehicle.

5 . The one or more tangible non-transitory computer-readable storage media of claim 2 , wherein the ground uncertainty region is generated based on an estimated profile of an object in the scene.

6 . The one or more tangible non-transitory computer-readable storage media of claim 5 , wherein the object is a road surface.

7 . The one or more tangible non-transitory computer-readable storage media of claim 5 , wherein the ground uncertainty region is further generated based on a vertical distance corresponding to at least one of noise or an error of known magnitude.

8 . One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:

accessing a first distance scan recorded by a first depth sensor, the first depth sensor defining a first field of view;

accessing a second distance scan recorded by a second depth sensor contemporaneously with the first distance scan, the second depth sensor physically offset from the first depth sensor and defining a second field of view, the second field of view overlapping at least a portion of the first field of view;

determining a pair of point sets, the pair of point sets including a first point set corresponding to a first portion of the first distance scan and a second point set corresponding to a second point set in a second portion of the second distance scan, the first portion of the first distance scan overlapping the second portion of the second distance scan;

determining a skew distance between the pair of point sets;

generating a weighted skew distance for the skew distance based on a magnitude of cross product between the first point set and the second point set; and

generating an alignment transformation for the first depth sensor and the second depth sensor, the alignment transformation minimizing the weighted skew distance.

9 . The one or more tangible non-transitory computer-readable storage media of claim 8 , wherein the first distance scan and the second distance scan are accessed in response to a calibration trigger.

10 . The one or more tangible non-transitory computer-readable storage media of claim 9 , wherein the calibration trigger is based on a threshold period of time elapsing from a generation of a previous alignment transformation.

11 . The one or more tangible non-transitory computer-readable storage media of claim 8 , wherein the first point set corresponds to a first line and the second point set corresponds to a second line.

12 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein each of the first line and the second line is formed by:

selecting a point within a row of pixels;

extracting a group of points around the point;

extracting a principal axis of inertia in the group of points; and

defining a line extending in a direction parallel to the principal axis of inertia.

13 . The one or more tangible non-transitory computer-readable storage media of claim 8 , further comprising:

causing a generation of motion planning for navigating along a movement path based on the alignment transformation.

14 . One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:

accessing a first LIDAR image recorded by a first LIDAR sensor; and

accessing a LIDAR image recorded by a second LIDAR sensor physically offset from the first LIDAR sensor;

determining a first set of points in a first region of the first LIDAR image and a second set of points in a second region of the second LIDAR image, the first region overlapping the second region;

determining a pair of point sets from the first set of points and the second set of points, the pair of point sets determined based on a spatial proximity and an initial alignment transformation defining relative positions of the first LIDAR sensor and the second LIDAR sensor;

determining a weight factor based on a perpendicularity of the pair of point sets; and

generating an alignment transformation for the first LIDAR image and the second LIDAR image based on the weight factor for the pair of point sets.

15 . The one or more tangible non-transitory computer-readable storage media of claim 14 , further comprising:

determining a difference between the alignment transformation and a previous alignment transformation.

16 . The one or more tangible non-transitory computer-readable storage media of claim 15 , wherein a prompt for transitioning from a first mode of operation to a second mode of operation is generated when the difference exceeds a threshold.

17 . The one or more tangible non-transitory computer-readable storage media of claim 14 , wherein a composite image of the scene is generated based on the alignment transformation.

18 . The one or more tangible non-transitory computer-readable storage media of claim 17 , wherein a ground uncertainty region is identified in the composite image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2024
From: DRIVE.AI, INC.
To: DIRECT CURRENT CAPITAL LLC
Reel/Frame 070510/0930 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2019
From: FOSTER, PAUL; PAZHAYAMPALLIL, JOEL; LIU, LEI
To: DRIVE.AI
Reel/Frame 048987/0818 →
Continuity (1)
Provisional Application 62560325 · Sep 19, 2017
Cited By (5)
US 12,384,404 US 12,535,318 US 12,664,780 US 12,688,547 US 12,694,695