Depth camera based image stabilization

1. A method comprising:

collecting, by a processing device, depth measurements for pixels of frames in a sequence of images of a video stream being provided by a source device to a target device as part of a communication session between a user of the source device and a user of the target device, the depth measurements being created by a depth aware camera of the source device;

mapping, using the depth measurements for the pixels of the frames, feature locations of one or more features of an object in a frame in the sequence of images to feature locations of the one or more features of the object in at least one other frame in the sequence of images;

determining one or more overlapping frame sections between the frame and the at least one other frame using the mapped feature locations;

modifying, in the sequence of images, a set of images corresponding to the frame and the at least another frame based on the overlapping frame sections to create a stabilized stream of images for the video stream; and

providing the stabilized stream of images in the video stream as part of the communication session,

wherein the overlapping sections comprise at least a portion of a person, and modifying the set of images to create the stabilized stream of images comprises:

creating a copy of the frame and the at least one other frame;

replacing a section of the copy of the frame that contains the portion of the person with the overlapping frame section without modifying a background portion of the copy of the frame; and

replacing a section of the copy of the at least one other frame that contains the portion of the person with the overlapping frame section without modifying a background portion of the copy of the at least one other frame.

2. The method of claim 1 , wherein the object comprises at least a portion of a face or a facial feature.

3. The method of claim 1 , wherein modifying the set of images comprises:

identifying the person in the images as a foreground object;

identifying one or more objects in the set of images, other than the person, as background objects; and

removing one or more sections of the frames that correspond to the set of images containing the background objects.

4. The method of claim 1 , wherein determining the one or more overlapping frame sections comprises:

aligning the frame and the at least one other frame using the mapped feature locations; and

identifying, as the overlapping frame sections, one or more sections in a foreground portion of the frame and one or more sections in a foreground portion of the at least one other frame comprising at least one of same objects or same portions of objects.

5. The method of claim 1 , wherein the communication session is a video chat via a mobile device.

6. A system comprising:

a memory; and

a processing device, coupled to the memory, to:

collect depth measurements for pixels of frames in a sequence of images of a video stream being provided by a source device to a target device as part of a communication session between a user of the source device and a user of the target device, the depth measurements being created by a depth aware camera of the source device;

map, using the depth measurements for the pixels of the frames, feature locations of one or more features of an object in a frame in the sequence of images to feature locations of the one or more features of the object in at least one other frame in the sequence of images;

determine one or more overlapping frame sections between the frame and the at least one other frame using the mapped feature locations;

modify, in the sequence of images, a set of images corresponding to the frame and the at least another frame based on the overlapping frame sections to create a stabilized stream of images for the video stream; and

provide the stabilized stream of images in the video stream as part of the communication session,

wherein the overlapping sections comprise at least a portion of a person, and to modify the set of images to create the stabilized stream of images, the processing device is to:

create a copy of the frame and the at least one other frame;

replace a section of the copy of the frame that contains the portion of the person with the overlapping frame section without modifying a background portion of the copy of the frame; and

replace a section of the copy of the at least one other frame that contains the portion of the person with the overlapping frame section without modifying a background portion of the copy of the at least one other frame.

7. The system of claim 6 , wherein the object comprises at least a portion of a face or a facial feature.

8. The system of claim 6 , wherein to modify the set of images, the processing device is to:

identify the person in the images as a foreground object;

identify one or more objects in the set of images, other than the person, as background objects; and

remove one or more sections of the frames that correspond to the set of images containing the background objects.

9. The system of claim 6 , wherein to determine the one or more overlapping frame sections, the processing device is to:

align the frame and the at least one other frame using the mapped feature locations; and

identify, as the overlapping frame sections, one or more sections in a foreground portion of the frame and one or more sections in a foreground portion of the at least one other frame comprising at least one of same objects or same portions of objects.

10. The system of claim 6 , wherein the communication session is a video chat via a mobile device.

11. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processing device, cause the processing device to perform operations comprising:

collecting, by the processing device, depth measurements for pixels of frames in a sequence of images of a video stream being provided by a source device to a target device as part of a communication session between a user of the source device and a user of the target device, the depth measurements being created by a depth aware camera of the source device;

mapping, using the depth measurements for the pixels of the frames, feature locations of one or more features of an object in a frame in the sequence of images to feature locations of the one or more features of the object in at least one other frame in the sequence of images;

determining one or more overlapping frame sections between the frame and the at least one other frame using the mapped feature locations;

modifying, in the sequence of images, a set of images corresponding to the frame and the at least another frame based on the overlapping frame sections to create a stabilized stream of images for the video stream; and

providing the stabilized stream of images in the video stream as part of the communication session,

wherein the overlapping sections comprise at least a portion of a person, and modifying the set of images to create the stabilized stream of images comprises:

creating a copy of the frame and the at least one other frame;

replacing a section of the copy of the frame that contains the portion of the person with the overlapping frame section without modifying a background portion of the copy of the frame; and

replacing a section of the copy of the at least one other frame that contains the portion of the person with the overlapping frame section without modifying a background portion of the copy of the at least one other frame.

12. The non-transitory computer readable medium of claim 11 , wherein the object comprises at least a portion of a face or a facial feature.

13. The non-transitory computer readable medium of claim 11 , wherein modifying the set of images comprises:

identifying the person in the images as a foreground object;

identifying one or more objects in the set of images, other than the person, as background objects; and

removing one or more sections of the frames that correspond to the set of images containing the background objects.

14. The non-transitory computer readable medium of claim 11 , wherein determining the one or more overlapping frame sections comprises:

aligning the frame and the at least one other frame using the mapped feature locations; and

identifying, as the overlapping frame sections, one or more sections in a foreground portion of the frame and one or more sections in a foreground portion of the at least one other frame comprising at least one of same objects or same portions of objects.