Automatic exposure metering for regions of interest that tracks moving subjects using artificial intelligence
An apparatus comprising an interface and a processor. The interface may be configured to receive pixel data. The processor may be configured to process the pixel data arranged as video frames, generate statistics and a timestamp for the video frames, perform computer vision operations on the video frames to determine one or more regions of interest within the video frames at the timestamp, track a movement of the regions of interest over a sequence of the video frames, evaluate a current luma value of the regions of interest, calculate auto-exposure parameters in response to the current luma value and a target luma value and generate an output signal comprising the auto-exposure parameters. The auto-exposure parameters may be calculated to prioritize an exposure of the regions of interest in the video frames and dynamically change the auto-exposure parameters based on the movement of the regions of interest.
1. An apparatus comprising:
an interface configured to receive pixel data; and
a processor configured to (i) process said pixel data arranged as video frames, (ii) generate statistics and a timestamp for said video frames, (iii) perform computer vision operations on said video frames to determine one or more regions of interest within said video frames at said timestamp, (iv) track a movement of said regions of interest over a sequence of said video frames, (v) evaluate a current luma value of said regions of interest, (vi) calculate auto-exposure parameters in response to said current luma value and a target luma value and (vii) generate an output signal comprising said auto-exposure parameters, wherein said auto-exposure parameters are calculated to
(a) prioritize an exposure of said regions of interest in said video frames, and
(b) dynamically change said auto-exposure parameters based on said movement of said regions of interest.
2. The apparatus according to claim 1 , wherein dynamically changing said auto-exposure parameters enables said regions of interest in said video frames to have said exposure over said sequence of said video frames be insensitive to said current luma value changing as a result of light conditions.
3. The apparatus according to claim 1 , wherein said statistics comprise information from said video frames about auto-white balance, auto-focus and auto-exposure.
4. The apparatus according to claim 1 , wherein
said computer vision operations are configured to generate information about a list of said regions of interest and a modified timestamp, and
said modified timestamp comprises said timestamp plus an amount of time to perform said computer vision operations.
5. The apparatus according to claim 4 , wherein each entry of said list of said regions of interest comprises a position, an identification value and a weight value.
6. The apparatus according to claim 5 , wherein said computer vision operations are configured to track said movement of said regions of interest across said sequence of said video frames based on said identification value.
7. The apparatus according to claim 5 , wherein said weight value
comprises a value between 0.0 and 1.0, and
provides a priority of said regions of interest for said auto-exposure parameters.
8. The apparatus according to claim 5 , wherein said current luma value of said regions of interest is evaluated in response to said information about said regions of interest, said modified timestamp, said timestamp and said statistics.
9. The apparatus according to claim 8 , wherein said current luma value of said regions of interest is further evaluated in response to
a track loop number,
a metering weight table, and
an initial settings weighting value for balancing said exposure between areas of said video frames.
10. The apparatus according to claim 4 , wherein said processor is further configured to perform a prediction of a location of said regions of interest in response to determining that said timestamp and said modified timestamp do not match.
11. The apparatus according to claim 10 , wherein said prediction of said location of said regions of interest is determined in response to a current position of said regions of interest at said modified timestamp and historical information about said regions of interest stored in a track list queue.
12. The apparatus according to claim 1 , wherein said computer vision operations comprise one or more of person detection, vehicle detection, motion detection, and face detection.
13. The apparatus according to claim 1 , wherein each of said regions of interest correspond to an object detected as defined by said computer vision operations.
14. The apparatus according to claim 1 , wherein said output signal is provided to an image sensor of a capture device.
15. The apparatus according to claim 14 , wherein said auto-exposure parameters comprise one or more of a shutter time, an iris value and an automatic gain control of said image sensor.
16. The apparatus according to claim 1 , wherein said processor is configured to calculate said auto-exposure parameters for an entire field of view of said video frames in response to
said computer vision operations not detecting at least one of said regions of interest, and
having no prior history of said regions of interest.
17. The apparatus according to claim 1 , wherein said processor is configured to re-use said current luma value from a previous one of said video frames in response to
said computer vision operations not detecting at least one of said regions of interest in a current one of said video frames, and
having a prior history of said regions of interest in said previous one of said video frames.
18. The apparatus according to claim 17 , wherein said current luma value from said previous one of said video frames is used if said current luma value corresponds to said prior history of said regions of interest within a track loop number of said video frames.
19. The apparatus according to claim 17 , wherein a predefined number of said prior history of said regions of interest is stored in a track list queue.
20. The apparatus according to claim 1 , wherein said statistics comprise a table of tile luma values that corresponds to a plurality of tiles of said video frames.