Online sensor calibration for autonomous vehicles

1. A computer-implemented method, comprising:

capturing, at a vehicle as the vehicle travels, LIDAR scans and camera images;

selecting, at the vehicle as the vehicle travels, at least a subset of the LIDAR scans and at least a subset of the camera images for calibration;

computing, at the vehicle as the vehicle travels, LIDAR-to-camera transformations for the subset of the LIDAR scans and the subset of camera images using an optimization algorithm, one or more of the LIDAR-to-camera transformations being based at least on an unwinding, at the vehicle, of one or more LIDAR scans of the subset of LIDAR scans with respect to movement of the vehicle during capture of the one or more LIDAR scans, the unwinding being determined based at least on pose information for the vehicle as the vehicle travels; and

calibrating, at the vehicle as the vehicle travels, one or more sensors of the vehicle based at least on the LIDAR-to-camera transformations.

2. The computer-implemented method of claim 1 , wherein the capturing, selecting, computing, and calibrating are performed online at the vehicle as the vehicle travels without requiring the vehicle to stop traveling.

3. The computer-implemented method of claim 1 , further comprising validating, prior to the vehicle beginning to travel, that an initial calibration of the one or more sensors of the vehicle is within a threshold.

4. The computer-implemented method of claim 1 , wherein the subset of the LIDAR scans is selected for calibration based at least on detecting straight line features in the subset of LIDAR scans by at least:

separating ground points from non-ground points;

profiling intensity of the ground points; and

detecting intensity edges in the ground points.

5. The computer-implemented method of claim 1 , wherein the subset of the LIDAR scans is selected for calibration based at least on:

dividing a field of view of the subset of LIDAR scans into multiple regions; and

determining that each of the regions has a threshold number of straight line features.

6. A processor comprising processing circuitry to cause performance of operations, the operations comprising:

capturing, at a vehicle as the vehicle travels, LIDAR scans and camera images;

selecting, at the vehicle as the vehicle travels, at least a subset of the LIDAR scans and at least a subset of the camera images for calibration;

computing, at the vehicle as the vehicle travels, LIDAR-to-camera transformations for the subset of the LIDAR scans and the subset of the camera images using an optimization algorithm, one or more of the LIDAR-to-camera transformations being based at least on an adjustment, at the vehicle, of one or more LIDAR scans of the subset of LIDAR scans to compensate for movement of the vehicle during capture of the one or more LIDAR scans, the adjustment being based at least on changes in pose information corresponding to the vehicle as the vehicle travels; and

calibrating, at the vehicle as the vehicle travels, one or more sensors of the vehicle based at least on the LIDAR-to-camera transformations.

7. The processor of claim 6 , wherein the capturing, selecting, computing, and calibrating are performed online at the vehicle as the vehicle travels without requiring the vehicle to stop traveling.

8. The processor of claim 6 , further comprising validating, prior to the vehicle beginning to travel, that an initial calibration of the one or more sensors of the vehicle is within a threshold.

9. The processor of claim 6 , wherein the subset of the LIDAR scans is selected for calibration based at least on detecting straight line features in the subset of LIDAR scans by at least:

separating ground points from non-ground points;

profiling intensity of the ground points; and

detecting intensity edges in the ground points.

10. The processor of claim 6 , wherein the subset of the LIDAR scans is selected for calibration based at least on:

dividing a field of view of the subset of LIDAR scans into multiple regions; and

determining that each of the regions has a threshold number of straight line features.

11. A system comprising:

one or more processors to cause the system to perform operations, the operations comprising:

capturing, at a vehicle as the vehicle travels, one or more LIDAR scans and one or more camera images;

selecting, at the vehicle as the vehicle travels, at least one LIDAR scan of the one or more LIDAR scans and at least one camera image of the one or more camera images for calibration;

computing, at the vehicle as the vehicle travels, one or more LIDAR-to-camera transformations for the at least one LIDAR scan and the at least one camera image based at least on an unwinding of the at least one LIDAR scan according to pose information of the vehicle; and

calibrating, at the vehicle as the vehicle travels, one or more sensors of the vehicle based at least on the one or more LIDAR-to-camera transformations.

12. The system of claim 11 , wherein the capturing, selecting, computing, and calibrating are performed online at the vehicle as the vehicle travels without requiring the vehicle to stop traveling.

13. The system of claim 11 , further comprising validating, prior to the vehicle beginning to travel, that an initial calibration of the one or more sensors of the vehicle is within a threshold.

14. The system of claim 11 , wherein the at least one LIDAR scan is selected for calibration based at least on detecting straight line features in the at least one LIDAR scan by at least:

separating ground points from non-ground points;

profiling intensity of the ground points; and

detecting intensity edges in the ground points.

15. The system of claim 11 , wherein the at least one LIDAR scan is selected for calibration based at least on:

dividing a field of view of the at least one LIDAR scan into multiple regions; and

determining that each of the regions has a threshold number of straight line features.