Granted Patent B2

US 12,423,866 · App. 17/893,732 · Granted Sep 23, 2025

Inventors: Peter Leeman (Pittsburgh, PA); Nijumudheen Muhassin (Jefferson Hills, PA); Jayesh Dwivedi (Oakmont, PA)

Assignee: Motional AD LLC

G06T7/80H04N23/6842

Abstract

Claims (53)

1. A method comprising:

obtaining, by at least one processor, a first camera sensor position at which a movable sensor of a camera is positioned when the camera captures a first image;

selecting, by the at least one processor and from at least one database comprising a plurality of intrinsic calibration data sets indexed by a respective plurality of second camera sensor positions, one of a plurality of second camera sensor positions based on the first camera sensor position of the movable sensor, wherein respective intrinsic parameters of an intrinsic calibration data set comprise a focal length, an optical center, and distortion parameters;

retrieving, from the at least one database and for the camera, a first intrinsic calibration data set comprising intrinsic parameters, of the plurality of intrinsic calibration data sets, that is indexed by the selected one of the plurality of second camera sensor positions;

rectifying, by the at least one processor, the first image using the first intrinsic calibration data set comprising the intrinsic parameters of the camera retrieved from the at least one database;

obtaining, by the at least one processor, a second camera sensor position at which the movable sensor of the camera is positioned when the camera captures a second image;

determining, by the at least one processor, that a distance between the second camera sensor position and the first camera sensor position satisfies a threshold;

selecting, by the at least one processor and from the at least one database, another one of the plurality of second camera sensor positions that is closest to the second camera sensor position in response to the determining;

retrieving, from the at least one database and for the camera, a second intrinsic calibration data set, of the plurality of intrinsic calibration data sets, that is indexed by the selected another one of the plurality of second camera sensor positions; and

rectifying, by the at least one processor, the second image using the second intrinsic calibration data set of the camera retrieved from the at least one database.

2. The method of claim 1 , further comprising:

classifying, by the at least one processor, at least one object represented in the first image that is rectified.

3. The method of claim 1 , wherein the camera is configured on a vehicle.

4. The method of claim 3 , further comprising:

providing, by the at least one processor, the first image that is rectified to a perception system of the vehicle to classify at least one object represented in the first image that is rectified.

5. The method of claim 1 , wherein the camera includes an in-body image stabilization system capable of causing the movable sensor to move within the camera.

6. The method of claim 5 , wherein:

the camera is configured on a vehicle; and

the in-body image stabilization system causes the movable sensor to move in response to a movement of the vehicle.

7. The method of claim 1 , wherein the selecting the one of the plurality of second camera sensor positions based on the first camera sensor position includes selecting the one of the plurality of second camera sensor positions that is closest to the first camera sensor position.

8. The method of claim 1 , wherein the respective intrinsic parameters in the plurality of intrinsic calibration data sets are generated by a diffractive optical element (DOE) geometrical camera calibration technique.

9. The method of claim 1 , wherein the respective intrinsic parameters in the plurality of intrinsic calibration data sets are generated by a Zhang camera calibration method.

10. The method of claim 1 , further comprising:

rectifying, by the at least one processor and in response to the determining, the second image using the first intrinsic calibration data set of the camera without retrieving any other intrinsic calibration data set from the at least one database.

11. A system, comprising:

at least one processor, and

at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

obtaining, by the at least one processor, a first camera sensor position at which a movable sensor of a camera is positioned when the camera captures a first image;

selecting, by the at least one processor and from at least one database comprising a plurality of intrinsic calibration data sets indexed by a respective plurality of second camera sensor positions, one of a plurality of second camera sensor positions based on the first camera sensor position of the movable sensor, wherein respective intrinsic parameters of an intrinsic calibration data set comprise a focal length, an optical center, and distortion parameters;

retrieving, from the at least one database and for the camera, a first intrinsic calibration data set comprising intrinsic parameters, of the plurality of intrinsic calibration data sets, that is indexed by the selected one of the plurality of second camera sensor positions;

rectifying, by the at least one processor, the first image using the first intrinsic calibration data set comprising the intrinsic parameters of the camera retrieved from the at least one database;

obtaining, by the at least one processor, a second camera sensor position at which the movable sensor of the camera is positioned when the camera captures a second image;

determining, by the at least one processor, that a distance between the second camera sensor position and the first camera sensor position satisfies a threshold;

selecting, by the at least one processor and from the at least one database, another one of the plurality of second camera sensor positions that is closest to the second camera sensor position in response to the determining;

retrieving, from the at least one database and for the camera, a second intrinsic calibration data set, of the plurality of intrinsic calibration data sets, that is indexed by the selected another one of the plurality of second camera sensor positions; and

rectifying, by the at least one processor, the second image using the second intrinsic calibration data set of the camera retrieved from the at least one database.

12. The system of claim 11 , wherein the operations further comprise:

classifying, by the at least one processor, at least one object represented in the first image that is rectified.

13. The system of claim 11 , wherein the camera is configured on a vehicle.

14. The system of claim 13 , wherein the operations further comprise providing, by the at least one processor, the first image that is rectified to a perception system of the vehicle to classify at least one object represented in the first image that is rectified.

15. The system of claim 11 , wherein the selecting the one of the plurality of second camera sensor positions based on the first camera sensor position includes selecting the one of the plurality of second camera sensor positions that is closest to the first camera sensor position.

16. The system of claim 11 , wherein the respective intrinsic parameters in the plurality of intrinsic calibration data sets are generated by a diffractive optical element (DOE) geometrical camera calibration technique.

17. The system of claim 11 , wherein the respective intrinsic parameters in the plurality of intrinsic calibration data sets are generated by a Zhang camera calibration method.

18. At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

obtaining, by the at least one processor, a first camera sensor position at which a movable sensor of a camera is positioned when the camera captures a first image;

selecting, by the at least one processor and from at least one database comprising a plurality of intrinsic calibration data sets indexed by a respective plurality of second camera sensor positions, one of a plurality of second camera sensor positions based on the first camera sensor position of the movable sensor, wherein respective intrinsic parameters of an intrinsic calibration data set comprise a focal length, an optical center, and distortion parameters;

retrieving, from the at least one database and for the camera, a first intrinsic calibration data set comprising intrinsic parameters, of the plurality of intrinsic calibration data sets, that is indexed by the selected one of the plurality of second camera sensor positions;

rectifying, by the at least one processor, the first image using the first intrinsic calibration data set comprising the intrinsic parameters of the camera retrieved from the at least one database;

obtaining, by the at least one processor, a second camera sensor position at which the movable sensor of the camera is positioned when the camera captures a second image;

determining, by the at least one processor, that a distance between the second camera sensor position and the first camera sensor position satisfies a threshold;

selecting, by the at least one processor and from the at least one database, another one of the plurality of second camera sensor positions that is closest to the second camera sensor position in response to the determining;

retrieving, from the at least one database and for the camera, a second intrinsic calibration data set, of the plurality of intrinsic calibration data sets, that is indexed by the selected another one of the plurality of second camera sensor positions; and

rectifying, by the at least one processor, the second image using the second intrinsic calibration data set of the camera retrieved from the at least one database.

Assignments (1)

Continuity (1)

References Cited (30)