Vehicular sensor system calibration

1 . A method, comprising:

receiving, at a device within a vehicle, visual data captured by a camera that is affixed to the vehicle;

detecting, by the device and based on the visual data, a plurality of instances of a same object and a position of the object in the field of view of the camera for an environment outside of the vehicle;

for each instance of the plurality of instances, determining a ray angle from a vehicle position to the object;

identifying, using a distribution of a plurality of ray angles, a central ray angle and a deviation from the central ray angle;

determining an object class to which the detected object belongs and represents a recognition of a type of the detected object;

generating a plurality of calibration data points corresponding to the ray angle based on the position of the object in the field of view to the vehicle position, wherein a ray angle exhibiting greater deviation from the central ray angle contributes more to calibration of a vehicle sensor system than a ray angle closer to the central ray angle; and

determining at least one calibration parameter of the vehicular sensor system based at least in part on a plurality of calibration data points in a calibration database, wherein the calibration database is configured to adjust the at least one calibration parameter for a sensor of the vehicle sensor system.

2 . The method of claim 1 , further comprising determining whether to store the calibration data points based on a determination that an object class to which the detected object belongs is one of a predetermined set of object classes having an attribute within the environment.

3 . The method of claim 2 , wherein the predetermined set of object classes comprises object classes that tend to be stationary and on or near a road.

4 . The method of claim 2 , wherein the predetermined set of object classes comprises traffic lights, traffic signs, and road markings.

5 . The method of claim 1 , further comprising:

receiving, at the device, the vehicle position from a positional sensor affixed to the vehicle, wherein the vehicular sensor system comprises the camera and the positional sensor;

determining a plurality of vehicle locations, wherein each vehicle location of the plurality of vehicle locations corresponds to a location of the vehicle at a time of a plurality of times when a visual data frame was captured by the camera; and

determining a plurality of object detections, wherein each object detection comprises a position of the object in the field of view of the camera at a time of the plurality of times.

6 . The method of claim 5 , wherein determining whether to store the data point is further based on a property of the plurality of object detections.

7 . The method of claim 6 ,

wherein the property of the plurality of object detections is a property of the plurality of ray angles.

8 . The method of claim 7 , wherein the property of the plurality of ray angles is a range of ray angles to the object, and wherein a calibration data point corresponding to the position of the object is stored in response to a determination that the range of ray angles is above a threshold.

9 . The method of claim 7 ,

wherein the property of the plurality of ray angles is the central ray angle.

10 . The method of claim 9 , wherein a calibration data point corresponding to the position of the object is stored in response to a determination that the corresponding ray angle deviates from the central ray angle.

11 . The method of claim 10 , further comprising:

determining a plurality of deviation values, wherein each deviation value corresponds to a difference between a ray angle of the plurality of ray angles and the central ray angle.

12 . The method of claim 11 ,

wherein each ray angle of the plurality of ray angles is stored along with a weighting parameter, wherein the weighting parameter is based on a corresponding deviation value, and wherein the weighting parameter is increased for ray angles of the plurality of ray angles that have an increased corresponding deviation value.

13 . The method of claim 11 , further comprising:

selecting a subset of ray angles from the plurality of ray angles; wherein a probability of selecting a ray angle is increased for ray angles that have an increased corresponding deviation value; and wherein a data point corresponding to the position of the object is stored in response to a determination that the ray angle belongs to the selected subset.

14 . The method of claim 6 , further comprising:

determining a vanishing point;

determining a distance from each object detection of the plurality of object detections to the vanishing point;

storing a data point corresponding to a position of each object detection in the calibration database, wherein each data point is stored along with a weighting parameter, and wherein the weighting parameter is increased for object detections that are more distant from the vanishing point.

15 . The method of claim 6 , further comprising:

determining a plurality of aspect ratios, each aspect ratio based on a bounding box associated with an object detection of the plurality of object detections; and

determining a variance in the plurality of aspect ratios; wherein the property of the plurality of object detections is the variance in the plurality of aspect ratios, and wherein a data point corresponding to the position of the object is stored in response to a determination that the variance is below a threshold.

16 . The method of claim 6 , wherein the property of the plurality of object detections is a hit rate of object detections, wherein the hit rate is a ratio of a number of object detections to a number of times in the plurality of times when a visual data frame was captured by the camera, and wherein the calibration data point is stored in response to a determination that the hit rate is above a threshold.

17 . The method of claim 5 , wherein determining whether to store the calibration data point is further based on a property of the plurality of vehicle locations.

18 . The method of claim 17 :

wherein the property of the plurality of vehicle locations corresponds to a distance travelled by the vehicle, and wherein the calibration data point corresponding to the position of the object is stored in response to a determination that the distance travelled is above a threshold.

19 . The method of claim 17 :

wherein the property of the plurality of vehicle locations corresponds to a curvature of a trajectory of the vehicle, and wherein the calibration data point corresponding to the position of the object is stored in response to a determination that the curvature is above a threshold.

20 . The method of claim 1 , further comprising:

filtering the calibration database; and wherein the plurality of stored calibration data points on which the determination of the at least one calibration parameter of the vehicular sensor system is based is a plurality of data points in the filtered calibration database.

21 . The method of claim 20 , further comprising:

determining a plurality of uncertainty values; wherein each uncertainty value corresponds to an extent of locations in an extra-vehicular reference frame that are consistent with a plurality of ray angles and vehicular positions that are stored in the calibration database and that correspond to a detected object; and wherein filtering the calibration database comprises ignoring calibration data points associated with uncertainty values that are atypically large with respect to the plurality of uncertainty values.

22 . The method of claim 1 , further comprising:

determining one or more keypoints within the detected object; and wherein the calibration data point corresponds to a keypoint of the one or more determined keypoints.

23 . The method of claim 1 , wherein a calibration parameter of the at least one calibration parameters is an offset between the camera and a positioning sensor, wherein the positioning sensor is affixed to the vehicle.

24 . The method of claim 1 , wherein the at least one calibration parameter is at least one of:

a distance and a direction between a camera and a positional sensor;

a height of the positional sensor when the vehicle is at rest;

a height of the camera when the vehicle is at rest;

a pitch of the camera when the vehicle is at rest;

a roll of the camera when the vehicle is at rest;

a yaw of the camera when the vehicle is at rest; or

an intrinsic parameter of camera.

25 . A computer program product, the computer program product comprising:

a non-transitory computer-readable medium having program code recorded thereon, the program code, when executed by one or more processors, causes the one or more processors to:

detect, by the first processor and based on the visual data, a plurality of instances of a same object and a position of the object in the field of view of the camera for an environment outside of the vehicle;

for each instance of the plurality of instances, determining a ray angle from a vehicle position to the object;

identifying, using a distribution of a plurality of ray angles, a central ray angle and a deviation from the central ray angle;

determining an object class to which the detected object belongs and represents a recognition of a type of the detected object;

generating a plurality of calibration data points corresponding to the ray angle based on the position of the object in the field of view to the vehicle position, wherein a ray angle exhibiting greater deviation from the central ray angle contributes more to calibration of a vehicle sensor system than a ray angle closer to the central ray angle; and

determine at least one calibration parameter of the vehicular sensor system based at least in part on a plurality of calibration data points in a calibration database, wherein the calibration database is configured to adjust the at least one calibration parameter for a sensor of the vehicle sensor system.

26 . The computer program product of claim 25 , wherein the object is an object belonging to a class of objects that tends to be stationary and situated on or near a road.