Video compression using region of interest of moving objects
Described are systems and processes for compressing image data, such as video data, based on detection of region of interests (ROIs) in the image data. Image data may be analyzed to determine moving objects and stationary objects (e.g., background) depicted in the image data. To compress the image data, the stationary objects are refreshed less frequently than moving objects. ROIs are identified in the moving objects, such as a face of a person, hands of a person, and/or objects moved by the person. Some of these ROI regions/pixels are maintained at high quality, while non-ROI regions/pixels are reduced in quality to further compress the image data. Faces are reduced in quality to obfuscate a person's identity. Multiple ROI regions may be located and then tracked simultaneously. The resulting compression may reduce storage space to store the image data and/or may reduce bandwidth needed to transmit the image data to remote devices while maintaining high quality in ROI regions.
1 . A system, comprising:
a video camera having an overhead field of view of a surface within a materials handling facility;
one or more processors; and
a memory storing program instructions that when executed by the one or more processors cause the one or more processors to at least:
receive video data from the video camera, the video data having a first frame rate and including a high resolution video and a low resolution video of the surface within the materials handling facility;
decode the low resolution video to identify first pixels representing moving objects and second pixels representing stationary objects;
process the second pixels in the high resolution video at a second frame rate that is lower than the first frame rate;
analyze the first pixels representing the moving objects to identify:
third pixels representing regions of interest (ROI) within the moving objects, the ROI including at least hands of a person; and
fourth pixels selected from the first pixels and excluding the third pixels, the fourth pixels representing regions of disinterest (ROD) within the moving objects;
generate high resolution frames by compositing the third pixels and the fourth pixels at the first frame rate from the high resolution video and the fourth pixels refreshed at the second frame rate;
encode the high resolution frames to create compressed video at the first frame rate that includes:
the fourth pixels representing the ROD compressed to a low quality;
the third pixels representing the ROI with a high quality; and
the second pixels processed at the second frame rate; and
send the compressed video from the materials handling facility to an external site for analysis of the ROI in context of the ROD and the stationary objects.
2 . The system of claim 1 , wherein the program instructions that when executed by the one or more processors further cause the one or more processors to at least:
analyze the first pixels representing the moving objects to detect fifth pixels representing a face of a person; and
encode the fifth pixels at a lower quality to obfuscate the face.
3 . The system of claim 1 , wherein the second frame rate includes one new frame no more frequently than per one second.
4 . The system of claim 1 , wherein the third pixels representing the ROI within the moving objects includes a first ROI of the hands of the person and a second ROI of an object adjacent to the hands.
5 . The system of claim 1 , wherein the program instructions that when executed by the one or more processors further cause the one or more processors to at least:
determine a compression target for the encoding; and
wherein the third pixels are selected based at least in part on the compression target for the encoding.
6 . A method, comprising:
receiving image data including at least a high resolution video and a low resolution video at a first frame rate from one or more image capture devices located within a materials handling facility;
processing the low resolution video to identify at least a first region as including moving objects and at least a second region as including stationary objects;
processing the at least the second region at a second frame rate that is lower than the first frame rate;
analyzing the at least the first region to identify:
at least one region of interest (ROI) within the moving objects, the at least one ROI including at least one of a portion of a person or an object; and
a region of disinterest (ROD), the ROD including remaining imagery in the first region while excluding the at least one ROI;
generating high resolution frames by compositing the at least one ROI and the ROD at the first frame rate from the high resolution video and the at least the second region refreshed at the second frame rate; and
encoding a compressed video at a high resolution at the first frame rate that includes:
the at least one ROI with a high quality;
the ROD compressed to a low quality; and
the at least the second region processed at a second frame rate that is lower than the first frame rate.
7 . The method of claim 6 , further comprising:
detecting a third region within the first region that includes a face of a person;
reducing a quality of the third region to obfuscate an identity of the person; and
wherein the generating the compressed video further includes the third region at the first frame rate while having the quality reduced.
8 . The method of claim 6 , wherein the at least one ROI includes a first ROI and a second ROI, the method further comprising:
identifying the first ROI in the first region; and
identifying the second ROI in the first region, the second ROI being distinct from the first ROI.
9 . The method of claim 8 , wherein the second ROI is identified based at least in part on a location of the first ROI.
10 . The method of claim 8 , wherein:
the first ROI is a hand of the person; and
the second ROI is an object adjacent to the hand of the person.
11 . The method of claim 8 , further comprising:
determining a target compression for the compressed video based at least in part on network attributes associated with the materials handling facility;
selecting a first size of a first boundary for the first ROI; and
selecting a second size of a second boundary for the second ROI based at least in part on the first size of the first boundary and the target compression for the compressed video.
12 . The method of claim 6 , further comprising:
selecting the second frame rate to include one frame for at least every one second.
13 . The method of claim 6 , further comprising:
generating a boundary to define the at least one ROI; and
associating metadata with the at least one ROI, the metadata including at least one of a description, a name, or a time.
14 . The method of claim 6 , further comprising:
tracking the at least one ROI over a plurality of frames included in the image data, wherein the tracking includes performing motion estimation.
15 . A computer-implemented method, comprising:
receiving image data including at least a high resolution video and a low resolution video at a first frame rate from the one or more image capture devices;
processing the low resolution video to identify first pixels as including moving objects and second pixels as including stationary objects;
processing the second pixels at a second frame rate that is lower than the first frame rate;
identifying, from the first pixels, a region of interest (ROI), the ROI depicting at least one of a portion of a person or an object;
designating, from the first pixels, third pixels that exclude the ROI; and
generating high resolution frames by compositing the ROI and the third pixels at the first frame rate from the high resolution video and the second pixels refreshed at the second frame rate;
generating a compressed video at a high resolution at the first frame rate that includes:
the ROI maintaining a high quality;
the third pixels compressed to a low quality; and
the second pixels processed at the second frame rate.
16 . The computer-implemented method of claim 15 , wherein:
identification of the ROI is performed, at least in part, using a deep neural network configured to locate predetermined objects based at least in part on feature recognition.
17 . The computer-implemented method of claim 15 , wherein:
identification of at least the first pixels is performed using a vision algorithm configured to detect changes in values at corresponding pixel locations between successive frames of the image data.
18 . The computer-implemented method of claim 15 , further comprising:
selecting, at a first time, the second frame rate based at least in part on a target compression rate for the compressed video;
determining, at a second time, that the ROI is no longer included in a field of view depicted in the image data; and
modifying, at or after the second time, the second frame rate to increase a refresh rate of the second pixels.
19 . The computer-implemented method of claim 15 , further comprising:
selecting the low quality based at least in part on a target compression for the compressed video; and
reducing a quality of the third pixels from the high quality to the low quality.
20 . The computer-implemented method of claim 15 , further comprising:
tracking the ROI over a plurality of frames included in the image data.