Computer-vision based security system using a depth camera

1. A method for securing an environment, comprising:

obtaining a two-dimensional (2D) representation of a three-dimensional (3D) environment, wherein the 2D representation comprises a 2D frame of pixels encoding depth values of the 3D environment;

detecting, probabilistically, a pixel in the 2D frame that is affected by persistent depth dropout, wherein the pixel is affected by persistent depth dropout when the pixel is associated with unreliable depth values, wherein depth values associated with the pixel are unreliable when a confidence threshold for depth values associated with the pixel is exceeded over a period of time; and

assigning the pixel affected by the persistent depth dropout to a background of the 2D representation;

identifying a plurality of foreground pixels in the 2D representation;

defining a foreground object based on the plurality foreground pixels;

assigning pixels of the 2D representation that are not associated with the foreground object and that are not already associated with the background to the background;

eliminating the background from the 2D representation to obtain an updated 2D representation; and

sending the updated 2D representation to a remote processing service.

2. The method of claim 1 , wherein identifying the plurality of foreground pixels comprises:

detecting pixels where the depth value changes between a prior 2D frame and the 2D frame; and

identifying the plurality of foreground pixels, by selecting, from the detected pixels, pixels where the depth values do not abruptly increase.

3. The method of claim 1 , wherein defining the foreground object comprises:

identifying a cluster of coherent foreground pixels in the plurality of foreground pixels;

determining that the cluster of coherent foreground pixels has at least a minimum geometric extent;

identifying related pixels in the vicinity of the cluster of coherent foreground pixels, wherein the related pixels have depth values substantially similar to depth values associated with at least one of the cluster of coherent foreground pixels; and

combining the cluster of coherent foreground pixels and the related pixels to obtain the foreground object.

4. The method of claim 3 , wherein identifying related pixels is performed using a bilateral filter.

5. The method of claim 1 , further comprising:

making a determination that the foreground object has not moved for at least a specified duration; and

based on the determination, assigning the foreground object pixels of the foreground object to the background.

6. The method of claim 1 , further comprising:

obtaining a two-dimensional (2D) color video representation of the three-dimensional (3D) environment; and

sending the 2D color video representation to the remote processing service.

7. The method of claim 1 , further comprising:

obtaining an acceleration sensor measurement from an acceleration sensor attached to a camera used to obtain the 2D representation of the 3D environment; and

detecting camera orientation and camera movement from the acceleration sensor measurement.

8. A non-transitory computer readable medium comprising instructions, that enable a system to

obtain a two-dimensional (2D) representation of a three-dimensional (3D) environment, wherein the 2D representation comprises a 2D frame of pixels encoding depth values of the 3D environment;

identify a plurality of foreground pixels in the 2D representation;

detect, probabilistically, a pixel in the 2D frame that is affected by persistent depth dropout, wherein the pixel is affected by persistent depth dropout when the pixel is associated with unreliable depth values, wherein depth values associated with the pixel are unreliable when a confidence threshold for depth values associated with the pixel is exceeded over a period of time; and

assign the pixel affected by the persistent depth dropout to a background of the 2D representation;

define a foreground object based on the plurality foreground pixels;

assign pixels of the 2D representation that are not associated with the foreground object and that are not already associated with the background to the background;

eliminate the background from the 2D representation to obtain an updated 2D representation; and

send the updated 2D representation to a remote processing service.

9. The non-transitory computer readable medium of claim 8 , further comprising instructions, that enable the system to:

detect pixels where the depth value changes between a prior 2D frame and the 2D frame; and

identify the plurality of foreground pixels, by selecting, from the detected pixels, pixels where the depth values do not abruptly increase.

10. The non-transitory computer readable medium of claim 8 , wherein the instructions that enable the system to define the foreground object comprises instructions to:

identify a cluster of coherent foreground pixels in the plurality of foreground pixels;

determine that the cluster of coherent foreground pixels has at least a minimum geometric extent;

identify related pixels in the vicinity of the cluster of coherent foreground pixels, wherein the related pixels have depth values substantially similar to depth values associated with at least one of the cluster of coherent foreground pixels; and

combine the cluster of coherent foreground pixels and the related pixels to obtain the foreground object.

11. The non-transitory computer readable medium of claim 10 , wherein the instructions that enable the system to identify related pixels comprise a bilateral filter.

12. The non-transitory computer readable medium of claim 8 , further comprising instructions, that enable the system to:

make a determination that the foreground object has not moved for at least a specified duration; and

based on the determination, assign the foreground object pixels of the foreground object to the background.