Dynamic ROI sizes for camera auto exposure
A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Raw region of interest (ROI) information is obtained, identifying a raw ROI having a raw ROI size, within a first video frame captured by a camera. Motion information representative of motion of the raw ROI relative to a field of view (FOV) of the camera is obtained. The raw ROI information and the motion information are processed to generate a dynamic ROI having a dynamic ROI size larger than the raw ROI size. A second video frame is captured by the camera. A portion of the second video frame defined by the dynamic ROI is processed to generate autoexposure (AE) settings for the camera.
1 . A method, performed by a processor, comprising:
obtaining raw region of interest (ROI) information identifying a raw ROI within a first video frame captured by a camera, the raw ROI being generated by a client, the raw ROI having a raw ROI size, and the raw ROI being located at a location of an object tracked by the client in the first video frame;
obtaining motion information representative of motion of the raw ROI relative to a field of view (FOV) of the camera;
processing the raw ROI information and the motion information to generate a dynamic ROI having a dynamic ROI size larger than the raw ROI size, the dynamic ROI size being enlarged relative to the raw ROI size based at least partly on a degree of motion of the object as represented by the motion information;
capturing a second video frame by the camera; and
processing a portion of the second video frame at the location of the object, as defined by the dynamic ROI, to generate autoexposure (AE) settings for the camera to avoid underexposure or overexposure of the object in further video frames captured by the camera.
2 . The method of claim 1 , wherein:
obtaining the motion information comprises:
obtaining camera motion information representative of motion of the camera; and
processing the camera motion information to generate the motion information.
3 . The method of claim 2 , wherein:
the camera motion information comprises rotation information and translation information generated by an inertial measurement unit (IMU) configured to rotate and translate with the camera.
4 . The method of claim 2 , wherein:
the raw ROI information identifies the raw ROI within a first plurality of video frames captured by the camera, including the first video frame;
obtaining the motion information further comprises processing the raw ROI information to generate ROI motion information representative of motion of the raw ROI within the FOV of the camera in the first plurality of video frames; and
the motion information is generated by processing the ROI motion information and the camera motion information.
5 . The method of claim 1 , wherein:
the raw ROI information identifies the raw ROI within a first plurality of video frames captured by the camera, including the first video frame; and
obtaining the motion information comprises:
processing the raw ROI information to generate ROI motion information representative of motion of the raw ROI within the FOV of the camera in the first plurality of video frames; and
processing the ROI motion information to generate the motion information.
6 . The method of claim 1 , wherein:
at least one additional video frame is captured by the camera after the first video frame and before the second video frame.
7 . The method of claim 1 , wherein:
the AE settings comprise a gain and an exposure time for the camera.
8 . The method of claim 1 , wherein:
the raw ROI information further identifies one or more additional raw ROIs, each having a respective raw ROI size;
the motion information is further representative of motion of the one or more additional raw ROIs relative to the FOV of the camera;
the raw ROI information and the motion information are further processed to generate, for each of the one or more additional raw ROIs, a respective additional dynamic ROI; and
the AE settings are further generated by processing a respective portion of the second video frame defined by each respective dynamic ROI.
9 . The method of claim 1 , wherein:
the dynamic ROI is generated to have a dynamic ROI location different from a location of the raw ROI.
10 . A system comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the system to perform operations comprising:
obtaining raw region of interest (ROI) information identifying a raw ROI within a first video frame captured by a camera, the raw ROI being generated by a client, the raw ROI having a raw ROI size, and the raw ROI being located at a location of an object tracked by the client in the first video frame;
obtaining motion information representative of motion of the raw ROI relative to a field of view (FOV) of the camera;
processing the raw ROI information and the motion information to generate a dynamic ROI having a dynamic ROI size larger than the raw ROI size, the dynamic ROI size being enlarged relative to the raw ROI size based at least partly on a degree of motion of the object as represented by the motion information;
capturing a second video frame by the camera; and
processing a portion of the second video frame at the location of the object, as defined by the dynamic ROI, to generate autoexposure (AE) settings for the camera to avoid underexposure or overexposure of the object in further video frames captured by the camera.
11 . The system of claim 10 , wherein:
obtaining the motion information comprises:
obtaining camera motion information representative of motion of the camera; and
processing the camera motion information to generate the motion information.
12 . The system of claim 11 , wherein:
the camera motion information comprises rotation information and translation information generated by an inertial measurement unit (IMU) configured to rotate and translate with the camera.
13 . The system of claim 11 , wherein:
the raw ROI information identifies the raw ROI within a first plurality of video frames captured by the camera, including the first video frame;
obtaining the motion information further comprises processing the raw ROI information to generate ROI motion information representative of motion of the raw ROI within the FOV of the camera in the first plurality of video frames; and
the motion information is generated by processing the ROI motion information and the camera motion information.
14 . The system of claim 10 , wherein:
the raw ROI information identifies the raw ROI within a first plurality of video frames captured by the camera, including the first video frame; and
obtaining the motion information comprises:
processing the raw ROI information to generate ROI motion information representative of motion of the raw ROI within the FOV of the camera in the first plurality of video frames; and
processing the ROI motion information to generate the motion information.
15 . The system of claim 10 , wherein:
at least one additional video frame is captured by the camera after the first video frame and before the second video frame.
16 . The system of claim 10 , wherein:
the AE settings comprise a gain and an exposure time for the camera.
17 . The system of claim 10 , wherein:
the raw ROI information further identifies one or more additional raw ROIs, each having a respective raw ROI size;
the motion information is further representative of motion of the one or more additional raw ROIs relative to the FOV of the camera;
the raw ROI information and the motion information are further processed to generate, for each of the one or more additional raw ROIs, a respective additional dynamic ROI; and
the AE settings are further generated by processing a respective portion of the second video frame defined by each respective dynamic ROI.
18 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that, when executed by a processor of a system, cause the system to perform operations comprising:
obtaining raw region of interest (ROI) information identifying a raw ROI within a first video frame captured by a camera, the raw ROI being generated by a client, the raw ROI having a raw ROI size, and the raw ROI being located at a location of an object tracked by the client in the first video frame;
obtaining motion information representative of motion of the raw ROI relative to a field of view (FOV) of the camera;
processing the raw ROI information and the motion information to generate a dynamic ROI having a dynamic ROI size larger than the raw ROI size, the dynamic ROI size being enlarged relative to the raw ROI size based at least partly on a degree of motion of the object as represented by the motion information;
capturing a second video frame by the camera; and
processing a portion of the second video frame at the location of the object, as defined by the dynamic ROI, to generate autoexposure (AE) settings for the camera to avoid underexposure or overexposure of the object in further video frames captured by the camera.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein:
obtaining the motion information comprises:
obtaining camera motion information representative of motion of the camera; and
processing the camera motion information to generate the motion information.
20 . The non-transitory computer-readable storage medium of claim 19 , wherein:
the camera motion information comprises rotation information and translation information generated by an inertial measurement unit (IMU) configured to rotate and translate with the camera.