Privacy preserving camera
A privacy reserving camera including a frame buffer, an object detection and tracking module, an object identification module, an instance segmentation module and an object removal and inpainting module. The frame buffer stores frames of a video stream captured by the privacy preserving camera. The object detection and tracking module detects a moving object in the video stream. The object identification module determines whether the moving object is a privacy protecting object. The instance segmentation module parses the moving object in the video stream. The object removal and inpainting module removes at least a portion of the moving object from the video stream, fills an area of the removed moving object in the video stream with a compensated patch from a neighbor frame of the video stream, and outputs a modified video stream having at least a portion of the moving object replaced with the compensated patch.
1. A privacy preserving camera, comprising:
a frame buffer storing frames of a video stream captured by the privacy preserving camera;
a memory storing computer-executable instructions; and
a processor, when executing the computer-executable instructions stored in the memory, configured to perform:
detecting a moving object in the video stream;
determining whether the moving object is a privacy protecting object, wherein when the moving object is determined not to be the privacy protecting object, outputting the video stream;
parsing the moving object in the video stream when the moving object is determined as the privacy protecting object;
removing at least a portion of the moving object from the video stream, filling an area of the removed moving object in the video stream with a compensated patch from a neighbor frame or a background of a current frame of the video stream, and outputting a modified video stream having at least a portion of the moving object replaced with the compensated patch;
performing a skin area detection to the moving object to determine a nakedness level of the moving object;
determining an identity of the moving object; and
determining whether to allow a first frame at a first time instance of the moving object to appear according to the object identity of the moving object in a second frame at a second time instance and the nakedness level of the moving object in the second frame at the second time instance, the second time instance being a later time instance of the first time instance.
2. The privacy preserving camera according to claim 1 , wherein the processor is further configured to perform:
detecting the moving object and tracking the moving object through a video clip; and
outputting a number of bounding boxes of the moving object and a class of the moving object.
3. The privacy preserving camera according to claim 1 , wherein the processor is further configured to:
perform a face recognition process to determine whether the moving object is a family member, a privacy requesting member, or an intruder.
4. The privacy preserving camera according to claim 3 , wherein the processor is further configured to perform:
inputting photos of the family member and the privacy requesting member in an initial stage;
comparing the detected moving object with a database of the input photos of the family member and the privacy requesting member to confirm an identity of the moving object.
5. The privacy preserving camera according to claim 1 , wherein the processor is further configured to perform:
generating a segmentation mask for the moving object; and
painting an area of the generated segmentation mask with a color mask.
6. A method for privacy preserving comprising:
obtaining a video stream captured by a camera and storing frames of the video stream in a frame buffer;
detecting a moving object in the video stream;
determining whether the moving object is a privacy protecting object;
when the moving object is determined not to be the privacy protecting object, outputting the video stream;
when the moving object is determined as the privacy protecting object:
parsing the moving object in the video stream;
removing at least a portion of the moving object from the video stream;
filling an area of the removed moving object in the video stream with a compensated patch from a neighbor frame or a background of a current frame of the video stream;
outputting a modified video stream having at least a portion of the moving object replaced with the compensated patch;
performing a skin area detection to the moving object to determine a nakedness level of the moving object;
determining an identity of the moving object; and
determining whether to allow a first frame at a first time instance of the moving object to appear according to the object identity of the moving object in a second frame at a second time instance and the nakedness level of the moving object in the second frame at the second time instance, the second time instance being a later time instance of the first time instance.
7. The method according to claim 6 , wherein the detecting a moving object in the video stream, further comprises:
detecting the moving object and tracking the moving object through a video clip;
and outputting a number of bounding boxes of the moving object and a class of the moving object.
8. The method according to claim 6 , wherein the determining whether the moving object is the privacy protecting object, further comprises:
performing a face recognition process to determine whether the moving object is a family member, a privacy requesting member, or an intruder.
9. The method according to claim 8 , wherein the determining whether the moving object is the privacy protecting object, further comprises:
inputting photos of the family member and the privacy requesting member in an initial stage;
comparing the detected moving object with a database of the input photos of the family member and the privacy requesting member to confirm an identity of the moving object.
10. The method according to claim 6 ,
wherein performing the skin area detection includes determining a skin area of an image of the moving object using a skin tone detection process.
11. The method according to claim 6 , wherein the parsing the moving object in the video stream, further comprising:
generating a segmentation mask for the moving object; and
painting an area of the generated segmentation mask with a color mask.
12. A non-transitory computer readable medium storing executable computer program instructions for privacy preserving, the computer program instructions comprising instructions for:
obtaining a video stream captured by a camera and storing frames of the video stream in a frame buffer;
detecting a moving object in the video stream;
determining whether the moving object is a privacy protecting object;
when the moving object is determined not to be the privacy protecting object, outputting the video stream;
when the moving object is determined as the privacy protecting object:
parsing the moving object in the video stream;
removing at least a portion of the moving object from the video stream;
filling an area of the removed moving object in the video stream with a compensated patch from a neighbor frame or a background of a current frame of the video stream;
outputting a modified video stream having at least a portion of the moving object replaced with the compensated patch;
performing a skin area detection to the moving object to determine a nakedness level of the moving object;
determining an identity of the moving object; and
determining whether to allow a first frame at a first time instance of the moving object to appear according to the object identity of the moving object in a second frame at a second time instance and the nakedness level of the moving object in the second frame at the second time instance, the second time instance being a later time instance of the first time instance.
13. The non-transitory computer readable medium according to claim 12 , wherein the detecting a moving object in the video stream, further comprises:
detecting the moving object and tracking the moving object through a video clip;
and outputting a number of bounding boxes of the moving object and a class of the moving object.
14. The non-transitory computer readable medium according to claim 12 , wherein the determining whether the moving object is the privacy protecting object, further comprises:
performing a face recognition process to determine whether the moving object is a family member, a privacy requesting member, or an intruder.
15. The non-transitory computer readable medium according to claim 14 , wherein the determining whether the moving object is the privacy protecting object, further comprises:
inputting photos of the family member and the privacy requesting member in an initial stage;
comparing the detected moving object with a database of the input photos of the family member and the privacy requesting member to confirm an identity of the moving object.
16. The non-transitory computer readable medium according to claim 12 ,
wherein performing the skin area detection includes determining a skin area of an image of the moving object using a skin tone detection process.
17. The non-transitory computer readable medium according to claim 12 , wherein the parsing the moving object in the video stream, further comprising:
generating a segmentation mask for the moving object; and
painting an area of the generated segmentation mask with a color mask.