Methods and systems for classifying pixels as foreground using both short-range depth data and long-range depth data
Disclosed herein are methods and systems for classifying pixels as foreground using both short-range depth data and long-range depth data. One embodiment takes the form of a process that includes obtaining video data depicting at least a portion of a user. The process also includes obtaining short-range depth data associated with the video data. The process also includes obtaining long-range depth data associated with the video data. The video data, short-range depth data, and long-range depth data may be obtained via a single 3-D video camera. The process also includes classifying pixels of the video data as foreground based at least in part on both the short-range depth data and the long-range depth data. In some embodiments, classifying pixels of the video data as foreground comprises employing an alpha mask. The alpha mask may comprise binary foreground (hard) indicators. The alpha mask may comprise foreground-likelihood (soft) indicators.
1. A method comprising:
obtaining, from a three-dimensional (3-D) video camera, video data depicting at least a portion of a user;
obtaining, from the 3-D video camera operating in a short-range mode, short-range depth data associated with the video data;
obtaining, from the 3-D video camera operating in a long-range mode, long-range depth data associated with the video data;
detecting less than a threshold amount of motion in at least one of the obtained short-range depth data and the obtained video data during a motion-detection period;
classifying pixels of the video data as foreground based at least in part on a comparison between the short-range depth data and the long-range depth data; and
extracting a user-persona from the video data based at least in part on the pixels of the video data classified as foreground.
2. The method of claim 1 , wherein detecting less than a threshold amount of motion in at least one of the obtained short-range depth data and the obtained video data during a motion-detection period comprises detecting a mode-switching trigger, and responsively switching from obtaining the short-range depth data from the 3-D video camera operating in the short-range mode to obtaining the long-range depth data from the 3-D video camera operating in the long-range mode.
3. The method of claim 2 , wherein the mode-switching trigger is at least one of a periodic mode-switching trigger and an on-demand mode-switching trigger.
4. The method of claim 1 , wherein the motion-detection period is a periodic motion-detection period.
5. The method of claim 1 , further comprising:
identifying a short-range foreground region at least in part by using the short-range depth data; and
identifying a long-range foreground region at least in part by using the long-range depth data.
6. The method of claim 5 , wherein classifying pixels of the video data as foreground based at least in part on a comparison between the short-range depth data and the long-range depth data comprises classifying pixels of the video data as foreground based at least in part on a comparison between the identified short-range foreground region and the identified long-range foreground region.
7. The method of claim 5 , wherein:
identifying the short-range foreground region at least in part by using the short-range depth data comprises employing a threshold depth value; and
identifying the long-range foreground region at least in part by using the long-range depth data comprises employing the threshold depth value.
8. The method of claim 5 , further comprising determining a user-hair region of the video data at least in part by using both the short-range foreground region and the long-range foreground region.
9. The method of claim 8 , further comprising identifying a foreground-region delta at least in part by subtracting the short-range foreground region from the long-range foreground region, wherein determining the user-hair region of the video data comprises including the identified foreground-region delta in the user-hair region.
10. The method of claim 9 , wherein classifying pixels of the video data as foreground comprises classifying pixels in the identified foreground-region delta as foreground.
11. The method of claim 9 , further comprising updating a user-hair-color model using respective colors of pixels in the identified foreground-region delta, wherein classifying pixels of the video data as foreground comprises classifying pixels of the video data as foreground at least in part by using the updated user-hair-color model.
12. The method of claim 11 , wherein classifying pixels of the video data as foreground at least in part by using the updated user-hair-color model comprises performing a flood fill using the updated user-hair-color model.
13. The method of claim 1 , wherein classifying pixels of the video data as foreground comprises employing an alpha mask.
14. A system comprising:
a communication interface;
a processor; and
data storage containing instructions executable by the processor for causing the system to carry out a set of functions, the set of functions including:
obtaining, from a three-dimensional (3-D) video camera, video data depicting at least a portion of a user;
obtaining, from the 3-D video camera operating in a short-range mode, short-range depth data associated with the video data;
obtaining, from the 3-D video camera operating in a long-range mode, long-range depth data associated with the video data;
detecting less than a threshold amount of motion in at least one of the obtained short-range depth data and the obtained video data during a motion-detection period;
classifying pixels of the video data as foreground based at least in part on a comparison between the short-range depth data and the long-range depth data; and
extracting a user-persona from the video data based at least in part on the pixels of the video data classified as foreground.
15. A method comprising:
obtaining, from a three-dimensional (3-D) video camera, video data depicting at least a portion of a user;
obtaining, from the 3-D video camera operating in a short-range mode, short-range depth data associated with the video data;
obtaining, from the 3-D video camera operating in a long-range mode, long-range depth data associated with the video data;
classifying pixels of the video data as foreground based at least in part on a comparison between the short-range depth data and the long-range depth data;
extracting a user-persona from the video data based at least in part on the pixels of the video data classified as foreground;
identifying a short-range foreground region at least in part by using the short-range depth data;
identifying a long-range foreground region at least in part by using the long-range depth data; and
determining a user-hair region of the video data at least in part by using both the short-range foreground region and the long-range foreground region.
16. The method of claim 15 , wherein classifying pixels of the video data as foreground based at least in part on a comparison between the short-range depth data and the long-range depth data comprises classifying pixels of the video data as foreground based at least in part on a comparison between the identified short-range foreground region and the identified long-range foreground region.
17. The method of claim 15 , wherein:
identifying the short-range foreground region at least in part by using the short-range depth data comprises employing a threshold depth value; and
identifying the long-range foreground region at least in part by using the long-range depth data comprises employing the threshold depth value.
18. The method of claim 15 , further comprising identifying a foreground-region delta at least in part by subtracting the short-range foreground region from the long-range foreground region, wherein determining the user-hair region of the video data comprises including the identified foreground-region delta in the user-hair region.
19. The method of claim 18 , wherein classifying pixels of the video data as foreground comprises classifying pixels in the identified foreground-region delta as foreground.
20. The method of claim 18 , further comprising updating a user-hair-color model using respective colors of pixels in the identified foreground-region delta, wherein classifying pixels of the video data as foreground comprises classifying pixels of the video data as foreground at least in part by using the updated user-hair-color model.