Cascade camera tuning
Provided are methods for cascade camera image signal processing (ISP) tuning, which can include receiving first image data at a first time associated with a first image sensor of a camera, generating at least one tuned first ISP block parameter for at least one first ISP block based on the first image data and first tuning criterion, receiving second image data at a second time, and generating at least one tuned second ISP block parameter for at least one second ISP block based on the second image data and second tuning criterion. Some methods described also include tuning parameters of at least one ISP block of a camera after receiving tuned parameters. Some methods describe also include generating an image using ISP blocks tuned using various tuning criteria. Systems and computer program products are also provided.
1. A method, comprising:
receiving first image data at a first time, wherein the first image data is associated with a first image sensor of a first camera;
generating at least one tuned first image signal processing (ISP) block parameter for at least one first ISP block included in the first camera from at least one first ISP block parameter of the at least one first ISP block based on the first image data and at least one first tuning criterion of the first camera;
receiving second image data at a second time, wherein the second image data is associated with the first image sensor;
generating at least one tuned second ISP block parameter for at least one second ISP block included in the first camera from at least one second ISP block parameter of the at least one second ISP block based on the second image data and at least one second tuning criterion of the first camera; and
communicating the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to a second camera, wherein the second camera uses the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to process at least one third image data generated by a second image sensor of the second camera and generate images for a perception system of an autonomous vehicle.
2. The method of claim 1 , wherein a first image signal processor of the first camera uses the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to process the first image data and the second image data, and generate first images for a perception system of an autonomous vehicle,
wherein the perception system detects objects in the first images, and
wherein the autonomous vehicle uses the detected objects to navigate.
3. The method of claim 1 , further comprising communicating the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to a third camera, wherein the third camera uses the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to process at least one fourth image data generated by a third image sensor of the third camera and generate images for a perception system of an autonomous vehicle.
4. The method of claim 1 , wherein the at least one second tuning criterion is based on at least one of a mounting location for the first camera on an autonomous vehicle or objects to be detected from images generated by the first camera.
5. The method of claim 1 , wherein the at least one first ISP block comprises a plurality of first ISP blocks, wherein generating the at least one tuned first ISP block parameter comprises generating a plurality of tuned first ISP parameters for the plurality of first ISP blocks, and wherein a particular tuned first ISP block parameter of the plurality of tuned first ISP parameters is generated for a particular first ISP block of the plurality of first ISP blocks.
6. The method of claim 1 , wherein the at least one second ISP block comprises a plurality of second ISP blocks, wherein generating the at least one tuned second ISP block parameter comprises generating a plurality of tuned second ISP parameters for the plurality of second ISP blocks, and wherein a particular tuned second ISP block parameter of the plurality of tuned second ISP parameters is generated for a particular second ISP block of the plurality of second ISP blocks.
7. The method of claim 1 , wherein the at least one first tuning criterion includes at least one of a sharpness threshold, noise threshold, dynamic range threshold, tone reproduction threshold, contrast threshold, color accuracy threshold, distortion threshold, vignetting threshold, lateral chromatic aberration (LCA) threshold, lens flare threshold, color mode threshold, artifacts threshold, quality factor, or a modulation transfer function (MTF).
8. The method of claim 7 , further comprising communicating the at least one tuned first ISP block parameter to a third camera, wherein at least one fourth ISP block parameter of a fourth ISP block of the third camera is adjusted based on at least one fourth tuning criterion associated with the third camera to form at least one tuned fourth ISP block parameter for the fourth ISP block of the third camera.
9. The method of claim 1 , wherein the at least one first ISP block comprises at least one of a high dynamic range (HDR) merge, a bad pixel correction, an image black level correction, an image shading compensation, an achromatic aberration correction, an image white balance modification, an image demosaic, an image color correction, an image tone mapping, an image de-noising, an image sharpening, or an image motion compensation.
10. The method of claim 1 , wherein generating the at least one tuned first ISP block parameter comprises adjusting the at least one first ISP block parameter until the at least one first ISP block of the first camera outputs an image that satisfies an image quality threshold.
11. The method of claim 1 , further comprising adjusting at least one fourth ISP block parameter of a fourth ISP block of the first camera based on the at least one second tuning criterion to generate at least one tuned fourth ISP block parameter for the fourth ISP block of the first camera.
12. The method of claim 1 , further comprising communicating the at least one tuned first ISP block parameter to a third camera, wherein at least one third ISP block parameter of at least one third ISP block included in the third camera is adjusted based on at least one third tuning criterion associated with the third camera to form at least one tuned third ISP block parameter for the at least one third ISP block included in the third camera.
13. The method of claim 12 , wherein the at least one third ISP block comprises a plurality of third ISP blocks, wherein generating the at least one tuned third ISP block parameter comprises generating a plurality of tuned third ISP parameters for the plurality of third ISP blocks, and wherein a particular tuned third ISP block parameter of the plurality of tuned third ISP parameters is generated for a particular third ISP block of the plurality of third ISP blocks.
14. The method of claim 12 , further comprising communicating the at least one tuned first ISP block parameter and the at least one tuned third ISP block parameter to a fourth camera, wherein the fourth camera uses the at least one tuned first ISP block parameter and the at least one tuned third ISP block parameter to process at least one fourth image data generated by the fourth camera and generate images for a perception system of an autonomous vehicle.
15. The method of claim 12 , wherein the at least one third tuning criterion is at least one of a mounting location for the second camera on an autonomous vehicle or objects to be detected from images generated by the second camera.
16. The method of claim 12 , wherein the at least one first tuning criterion is based on hardware of the first camera and the second camera.
17. The method of claim 1 , wherein the first camera and the second camera include a same type of image signal processor and a same type of image sensor.
18. The method of claim 1 ,
wherein a first image signal processor of the first camera uses the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to process the first image data and generate first images for a perception system of an autonomous vehicle,
wherein the perception system detects objects in the images generated by the second camera and the first images, wherein the autonomous vehicle uses the detected objects to navigate.
19. A system, comprising:
a data store storing computer-executable instructions; and
a processor configured to:
receive first image data at a first time, wherein the first image data is associated with a first image sensor of a first camera;
generate at least one tuned first image signal processing (ISP) block parameter for at least one first ISP block included in the first camera from at least one first ISP block parameter of the at least one first ISP block based on the first image data and at least one first tuning criterion of the first camera;
receive second image data at a second time, wherein the second image data is associated with the first image sensor;
generate at least one tuned second ISP block parameter for at least one second ISP block included in the first camera from at least one second ISP block parameter of the at least one second ISP block based on the second image data and at least one second tuning criterion of the first camera; and
communicating the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to a second camera, wherein the second camera uses the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to process at least one third image data generated by a second image sensor of the second camera and generate images for a perception system of an autonomous vehicle.
20. Non-transitory computer-readable media comprising computer-executable instructions that, when executed by a computing system, causes the computing system to:
receive first image data at a first time, wherein the first image data is associated with a first image sensor of a first camera;
generate at least one tuned first image signal processing (ISP) block parameter for at least one first ISP block included in the first camera from at least one first ISP block parameter of the at least one first ISP block based on the first image data and at least one first tuning criterion of the first camera;
receive second image data at a second time, wherein the second image data is associated with the first image sensor;
generate at least one tuned second ISP block parameter for at least one second ISP block included in the first camera from at least one second ISP block parameter of the at least one second ISP block based on the second image data and at least one second tuning criterion of the first camera; and
communicating the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to a second camera, wherein the second camera uses the at least one tuned first ISP block parameter and the at least one tuned second ISP block parameter to process at least one third image data generated by a second image sensor of the second camera and generate images for a perception system of an autonomous vehicle.