Video conference background cleanup using reference image

1. A method, comprising:

storing a reference image representing a physical background within a field of view of a camera of a client device;

receiving, via the camera and during a video conference to which the client device is connected, camera-generated visual data for output to at least one remote device connected to the video conference;

identifying, based on facial recognition applied to the camera-generated visual data, foreground imagery representing at least one person and background imagery representing content of the camera-generated visual data other than the foreground imagery;

identifying a difference between the background imagery and the reference image by identifying an item in the background imagery and determining that the item is not present at a co-located part of the reference image, the co-located part of the reference image being identified based on non-movable fixtures depicted in the background imagery and the reference image;

generating a composite image by replacing, within the background imagery of the camera-generated visual data, an item represented within the background imagery and within the identified difference with a co-located part of the reference image; and

transmitting the composite image to the at least one remote device during the video conference.

2. The method of claim 1 , wherein the camera-generated visual data comprises at least one frame in a video stream generated by the camera.

3. The method of claim 1 , wherein, when the at least one person moves in front of the item, the composite image depicts the at least one person obscuring the co-located part of the reference image.

4. The method of claim 1 , wherein generating the composite image comprises:

adding a virtual sticker to overlay a part of the background imagery; and

detecting, via the camera, that the at least one person has moved in front of a first part of the virtual sticker, wherein the composite image comprises the at least one person obscuring the part of the virtual sticker, wherein the composite image comprises a second part of the virtual sticker that is not obscured by the at least one person.

5. The method of claim 1 , wherein storing the reference image comprises:

obtaining the reference image via the camera;

receiving a user prompt to remove an extraneous item from the reference image, the user prompt comprising drawing a shape around the extraneous item; and

modifying the reference image to replace the reference item with predicted visual data that would exist if the extraneous item were absent.

6. The method of claim 1 , wherein identifying the difference between the background imagery and the reference image comprises:

identifying that the item represented within the background imagery is different from the co-located part of the reference image.

7. The method of claim 1 , wherein the composite image maintains at least a portion of the background imagery from the camera-generated visual data.

8. The method of claim 1 , comprising:

storing multiple reference images, including the reference image representing the physical background; and

identifying the reference image representing the physical background based on a geographic location of the client device.

9. The method of claim 1 , comprising:

determining whether to generate the composite image at the client device or at a server based on capabilities of the client device, wherein the capabilities comprise at least one of processing capabilities, memory capabilities, network access capabilities, software capabilities, and hardware capabilities.

10. A non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:

storing a reference image representing a physical background within a field of view of a camera of a client device;

receiving, via the camera and during a video conference to which the client device is connected, camera-generated visual data for output to at least one remote device connected to the video conference;

identifying, based on facial recognition applied to the camera-generated visual data, foreground imagery representing at least one person and background imagery representing content of the camera-generated visual data other than the foreground imagery;

identifying a difference between the background imagery and the reference image by identifying an item in the background imagery and determining that the item is not present at a co-located part of the reference image, the co-located part of the reference image being identified based on non-movable fixtures depicted in the background imagery and the reference image;

generating a composite image by replacing, within the background imagery of the camera-generated visual data, an item represented within the background imagery and within the identified difference with a co-located part of the reference image; and

transmitting the composite image to the at least one remote device during the video conference.

11. The computer readable medium of claim 10 , wherein the camera-generated visual data comprises a frame in a video stream.

12. The computer readable medium of claim 10 , wherein, when the at least one person moves in front of the item, the composite image depicts the at least one person in front of the co-located part of the reference image.

13. The computer readable medium of claim 10 , wherein generating the composite image comprises:

adding a sticker to overlay a part of the background imagery; and

detecting, via the camera, that the at least one person has moved in front of a first part of the sticker, wherein the composite image comprises the at least one person standing in front of the part of the sticker, wherein the composite image comprises a second part of the sticker in front of which the at least one person is not standing.

14. The computer readable medium of claim 10 , wherein storing the reference image comprises:

receiving a user prompt to remove an extraneous item from the reference image, the user prompt comprising drawing a shape around the extraneous item; and

modifying the reference image to replace the reference item with predicted visual data that would exist if the extraneous item were absent.

15. The computer readable medium of claim 10 , wherein identifying the difference between the background imagery and the reference image comprises:

identifying that the item represented within the background imagery is absent from the co-located part of the reference image.

16. The computer readable medium of claim 10 , wherein the composite image maintains a portion of the background imagery.

17. The computer readable medium of claim 10 , the operations comprising:

storing multiple reference images, wherein the multiple reference images comprise the reference image representing the physical background; and

identifying the reference image representing the physical background based on a geographic location associated with a global positioning system of the client device.

18. The computer readable medium of claim 10 , the operations comprising:

determining whether to generate the composite image at the client device or at a server based on capabilities of the client device.

19. An apparatus, comprising:

a memory; and

a processor configured to execute instructions stored in the memory to:

store a reference image representing a physical background within a field of view of a camera of a client device;

receive, via the camera and during a video conference to which the client device is connected, camera-generated visual data for output to at least one remote device connected to the video conference;

identify, based on facial recognition applied to the camera-generated visual data, foreground imagery representing at least one person and background imagery representing content of the camera-generated visual data other than the foreground imagery;

identify a difference between the background imagery and the reference image by identifying an item in the background imagery and determining that the item is not present at a co-located part of the reference image, the co-located part of the reference image being identified based on non-movable fixtures depicted in the background imagery and the reference image;

generate a composite image by replacing, within the background imagery of the camera-generated visual data, an item represented within the background imagery and within the identified difference with a co-located part of the reference image; and

transmit the composite image to the at least one remote device during the video conference.

20. The apparatus of claim 19 , the processor configured to execute the instructions stored in the memory to:

determine whether to generate the composite image at the client device or at a server based on capabilities of the server.