Lens shading using non-radial image correction
In various examples, lens shading image correction systems and applications using non-radial correction of residual radial shading error are provided. In some embodiments, lens shading image correction may be implemented using calibration parameters corresponding to radial lens shading correction, and calibration parameters corresponding to non-radial lens shading correction. In some embodiments, sensor data comprising an image frame may be captured using a sensor. Radial lens shading correction may be applied to the image frame to produce a residual shading profile, and non-radial lens shading correction may be applied to the residual shading profile to produce a calibrated image frame. Parameters for radial lens shading correction may be computed from a lens shading profile associated with the sensor, and parameters for non-radial lens shading correction may be computed based a residual shading profile produced from the radial lens shading correction.
1. A processor comprising:
one or more processing units to:
generate a calibrated image frame from sensor data captured using a sensor based at least on applying a radial lens shading correction to the sensor data and applying a non-radial lens shading correction to a residual shading profile resulting from applying the radial lens shading correction, wherein applying the non-radial lens shading correction to the residual shading profile resulting from applying the radial lens shading correction reduces a residual non-symmetric lens shading effect resulting from applying the radial lens shading correction.
2. The processor of claim 1 , the one or more processing units further to:
remove a symmetrical component of a lens shading effect using a radial transfer function, the radial transfer function calibrated using a set of radial transfer function parameters of a set of lens shading calibration parameters.
3. The processor of claim 2 , wherein the set of radial transfer function parameters represent a curve fitting of a plurality of radial lines radiating from an optical center of a lens shading profile for the sensor.
4. The processor of claim 1 , the one or more processing units further to:
remove a non-symmetrical component of a lens shading effect using a non-radial lens shading correction model, the non-radial lens shading correction model calibrated using a set of non-radial surface model parameters of a set of lens shading calibration parameters.
5. The processor of claim 4 , wherein the set of non-radial surface model parameters represent a surface fitting of a patch array comprising a plurality of rectangular patch regions.
6. The processor of claim 1 , the one or more processing units further to:
perform one or more operations based at least on the calibrated image frame.
7. The processor of claim 1 , wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing real-time streaming;
a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
8. A system comprising:
one or more processing units to:
generate data comprising a representation of a calibrated image frame from sensor data captured using a sensor based at least on applying a radial lens shading correction to the sensor data and applying a non-radial lens shading correction to a residual shading profile resulting from applying the radial lens shading correction, wherein applying the non-radial lens shading correction to the residual shading profile resulting from applying the radial lens shading correction reduces a residual non-symmetric lens shading effect resulting from applying the radial lens shading correction.
9. The system of claim 8 , the one or more processing units further to:
receive the sensor data captured using the sensor;
receive one or more first parameters for the radial lens shading correction, the radial lens shading correction associated with the sensor;
receive one or more second parameters for the non-radial lens shading correction, the non-radial lens shading correction associated with the sensor; and
generate the data comprising the representation of the calibrated image frame from the sensor data based at least on the one or more first parameters and the one or more second parameters.
10. The system of claim 9 , the one or more processing units further to:
reduce non-symmetrical lens shading in the sensor data using a non-radial surface model based on the one or more second parameters.
11. The system of claim 8 , the one or more processing units further to:
compute a path for an ego-machine to travel that avoids collisions with obstacles based at least on the data comprising the representation of the calibrated image frame.
12. The system of claim 8 , the one or more processing units further to:
generate a display for a human machine interface, the display comprising the calibrated image frame stitched with at least one other image frame.
13. The system of claim 8 , the one or more processing units further to:
control a movement of an ego-machine within an environment based at least on the data comprising the representation of the calibrated image frame.
14. The system of claim 8 , wherein the sensor includes at least one of a monocular camera, a surround camera, wide-view camera, a fisheye camera, a long-range camera, a mid-range camera, a stereo camera, or a LIDAR sensor.
15. The system of claim 9 , wherein the one or more first parameters represent a curve fitting of symmetrical lens shading gain for a plurality of radial lines from an optical center of a lens shading profile of the sensor.
16. The system of claim 9 , wherein the one or more first parameters represent a curve fitting for a plurality of radial lines from an optical center of a lens shading profile of the sensor using a cubic Hermite spline.
17. The system of claim 9 , wherein the one or more second parameters represent a surface fitting of a surface model of non-symmetrical lens shading gain.
18. The system of claim 9 , wherein the one or more second parameters represent a Bezier surface fitting of a surface model of non-symmetrical lens shading gain.
19. The system of claim 8 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system for performing deep learning operations;
a system for performing real-time streaming;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
20. A method comprising:
generating a calibrated image frame from sensor data captured by an image sensor based at least on applying a radial lens shading correction to the sensor data and applying a non-radial lens shading correction to a residual shading profile resulting from applying the radial lens shading correction, wherein applying the non-radial lens shading correction to the residual shading profile resulting from applying the radial lens shading correction reduces a residual non-symmetric lens shading effect resulting from applying the radial lens shading correction.
21. The processor of claim 1 , wherein at least one of the radial lens shading correction or the non-radial lens shading correction are calibrated based at least on a set of lens shading calibration parameters associated with the sensor.
22. The system of claim 9 , the one or more processing units further to:
reduce symmetrical lens shading in the sensor data using a radial transfer function based on the one or more first parameters.