Background replacement based on attribute of remote user or endpoint
A telecommunication device includes an image capture system that captures an image of a local participant in a telecommunication session, the image comprising foreground and background images defined by plural pixels, each of the pixels having a pixel magnitude related to a sample of the image at a spatial location of the respective pixel and a background modifier that segments plural pixels of the captured image into foreground and background sets of pixels, replaces the background set of pixels with a template set of pixels to form a new background set of pixels, selected pixels in the template set of pixels having a different magnitude than a magnitude of the corresponding pixel in the background set of pixels replaced by the selected pixel, and combines the new background set of pixels with the foreground set of pixels to form modified image information for transmission to a remote endpoint. A background selector selects the template set of pixels from among multiple template sets of pixels based on an attribute of a remote endpoint or remote participant associated with the remote endpoint.
1. A telecommunication device, comprising:
a microprocessor; and
a memory coupled with the processor and storing therein a set of instructions which, when executed by the microprocessor, cause the microprocessor to:
receive an image of a local participant in a telecommunication session, the image being captured by an image capture device and comprising foreground and background images defined by plural pixels, each of the plural pixels having a pixel magnitude related to a sample of the image at a spatial location of the respective pixel;
segment the plural pixels of the captured image into foreground and background sets of pixels;
select a template set of pixels from among multiple template sets of pixels based on an attribute of a remote endpoint or remote participant associated with the remote endpoint;
replace the background set of pixels with the selected template set of pixels to form a new background set of pixels, selected pixels in the template set of pixels having a different magnitude than a magnitude of the corresponding pixel in the background set of pixels replaced by the template set of pixels,
combine the new background set of pixels with the foreground set of pixels to form modified image information; and
provide the modified image information to the remote endpoint and/or to a display to display the modified image information to the local participant.
2. The device of claim 1 , wherein the image capture device is a camera, wherein the instructions cause the microprocessor to segment the plural pixels of the captured image into foreground and background sets of pixels based on spatial coordinates of the pixels and independent of pixel magnitudes, wherein the pixel magnitude is one or more of a pixel value, color plane, and colormap index of the corresponding pixel, wherein the foreground image comprises pixels defining an image of the local participant, wherein the background image comprises pixels defining one or more background objects, wherein the foreground set of pixels comprises pixels defining the image of the local participant and part of the one or more background objects, and wherein the background set of pixels comprises pixels defining the other part of the one or more background objects.
3. The device of claim 2 , wherein the image capture device is a camera, wherein the segmentation is based on a selected boundary dividing the background image into first and second subsets of background image information, pixels in the first subset of background image information being in the foreground set of pixels and pixels in the second subset of background image information being in the background set of pixels, and wherein a spatial position of the boundary is related to a dimension of a detected face of the local participant.
4. The device of claim 3 , wherein a spatial position of the boundary spatially moves across multiple frames based on movement of the local participant, wherein movement of the local participant is tracked by the microprocessor tracking movement of a selected facial feature of the local participant, and wherein the boundary spatially moves only when a degree of spatial displacement of the local participant image from a selected position is equal to or greater than a selected threshold.
5. The device of claim 3 , wherein the instructions cause the microprocessor to modify magnitudes of the pixels at the selected boundary to provide a desired visual effect and wherein at least some of the pixels having modified magnitudes are in the foreground set of pixels and/or at least some of the pixels having modified magnitudes are in the new background set of pixels.
6. The device of claim 3 , wherein input of the local participant is received to alter a spatial position of the boundary from a first position selected automatically by the background modifier to a second position selected manually by the local participant.
7. The device of claim 1 , wherein the attribute of a remote endpoint or remote participant associated with the remote endpoint is an identity of the remote participant, an association of the remote participant to the local participant or another entity, an electronic address associated with the remote endpoint, or a combination thereof, wherein the instructions causes the microprocessor to determine the attribute from a signal exchanged between the local and remote endpoint, input received by the local endpoint from the local or remote participant, face recognition based on an image of the remote participant, content analysis of audio information and/or video information of the telecommunication session, content analysis of a presentation displayed during the telecommunication session, or a combination thereof, and wherein the instructions cause the microprocessor to select the template set of pixels from among multiple template sets of pixels by mapping the determined attribute against associations of sets of one or more attributes against a corresponding template set of pixels.
8. A method, comprising:
automatically capturing, by an image capture device, an image of a local participant in a telecommunication session, the image comprising foreground and background images defined by plural pixels, each of the plural pixels having a pixel magnitude related to a sample of the image at a spatial location of the respective pixel;
automatically segmenting, by a microprocessor and in substantial real time with image capture, the plural pixels of the captured image into foreground and background sets of pixels;
automatically selecting, by the microprocessor, a template set of pixels from among multiple template sets of pixels based on an attribute of a remote endpoint or remote participant associated with the remote endpoint;
automatically replacing, by the microprocessor, the background set of pixels with the selected template set of pixels to form a new background set of pixels, pixels in the selected template set of pixels having a different magnitude than a magnitude of the corresponding pixel in the background set of pixels replaced by the pixel in the selected template set of pixels;
automatically combining, by the microprocessor, the new background set of pixels with the foreground set of pixels to form modified image information; and
providing, by an output, the modified image information to the remote endpoint and/or to a display to display the modified image information to the local participant.
9. The method of claim 8 , wherein the microprocessor automatically segments, in substantial real time with image capture, the plural pixels of the captured image into foreground and background sets of pixels based on spatial coordinates of the pixels and independent of pixel magnitudes, wherein the image capture device is a camera, wherein the pixel magnitude is one or more of a pixel value, color plane, and colormap index of the corresponding pixel, wherein the foreground image comprises pixels defining an image of the local participant, wherein the background image comprises pixels defining one or more background objects, wherein the foreground set of pixels comprises pixels defining the image of the local participant and part of the one or more background objects, and wherein the background set of pixels comprises pixels defining the other part of the one or more background objects.
10. The method of claim 9 , wherein the segmentation is based on a selected boundary dividing the background image information into first and second subsets of background image, pixels in the first subset of background image information being in the foreground set of pixels and pixels in the second subset of background image information being in the background set of pixels, and wherein a spatial position of the boundary is related to a dimension of a detected face of the local participant.
11. The method of claim 10 , wherein a spatial position of the boundary spatially moves across multiple frames based on movement of the local participant, wherein movement of the local participant is tracked by tracking, by the microprocessor, movement of a selected facial feature of the local participant, and wherein the boundary spatially moves only when a degree of spatial displacement of the local participant image from a selected position is equal to or greater than a selected threshold.
12. The method of claim 10 , wherein the microprocessor modifies magnitudes of the pixels at the selected boundary to provide a desired visual effect and wherein at least some of the pixels having modified magnitudes are in the foreground set of pixels and/or at least some of the pixels having modified magnitudes are in the new background set of pixels.
13. The method of claim 8 , wherein the attribute of a remote endpoint or remote participant associated with the remote endpoint is an identity of the remote participant, an association of the remote participant to the local participant or another entity, an electronic address associated with the remote endpoint, or a combination thereof, wherein the the microprocessor determines the attribute from a signal exchanged between the local and remote endpoint, input received by the local endpoint from the local or remote participant, face recognition based on an image of the remote participant, content analysis of audio information and/or video information of the telecommunication session, content analysis of a presentation displayed during the telecommunication session, or a combination thereof, and wherein the microprocessor selects the template set of pixels from among multiple template sets of pixels by mapping the determined attribute against associations of sets of one or more attributes against a corresponding template set of pixels.
14. A contact center, comprising:
a microprocessor; and
a non-transitory computer readable medium, coupled to the microprocessor, that comprises instructions which, when executed by the microprocessor, cause the microprocessor to:
assign work items, the work items comprising video calls with customer communication devices, to selected ones of agent communication devices to service the assigned work items;
receive a captured image of an agent from an agent communication device, the agent communication device selected for a telecommunication session with a selected customer communication device, the image comprising foreground and background images defined by plural pixels, each of the plural pixels having a pixel magnitude related to a sample of the image at a spatial location of the respective pixel;
select a template set of pixels from among multiple template sets of pixels based on an attribute of the selected customer communication device or a customer associated with the selected customer communication device;
segment the plural pixels of the captured image into foreground and background sets of pixels based on spatial coordinates of the pixels;
replace the background set of pixels with the selected template set of pixels to form a new background set of pixels;
combine the new background set of pixels with the foreground set of pixels to form modified image information; and
provide the modified image information to the selected customer communication device.
15. The contact center of claim 14 , wherein the captured image is a 2-dimensional image, wherein the pixel magnitude is one or more of a pixel value, color plane, and colormap index of the corresponding pixel, wherein the foreground image comprises pixels defining an image of a local participant, wherein the background image comprises pixels defining one or more background objects, wherein the foreground set of pixels comprises pixels defining the image of the local participant and part of the one or more background objects, wherein each selected pixel in the template set of pixels having a different magnitude than a magnitude of the corresponding pixel in the background set of pixels replaced by the selected pixel, and wherein the background set of pixels comprises pixels defining the other part of the one or more background objects.
16. The contact center of claim 15 , wherein the segmentation of the plural pixels into foreground and background sets of pixels is independent of pixel magnitudes and wherein the captured image is captured by a camera, wherein the segmentation is based on a selected boundary dividing the background image into first and second subsets of background image information, pixels in the first subset of background image information being in the foreground set of pixels and pixels in the second subset of background image information being in the background set of pixels, and wherein a spatial position of the boundary being related to a dimension of a detected face of the agent.
17. The contact center of claim 16 , wherein a spatial position of the boundary spatially moves across multiple frames based on movement of the agent, wherein movement of the agent is tracked by tracking movement of a selected facial feature of the local participant, and wherein the boundary spatially moves only when a degree of spatial displacement of the agent image from a selected position is equal to or greater than a selected threshold.
18. The contact center of claim 17 , wherein the instructions cause the microprocessor to modify magnitudes of the pixels at the selected boundary to provide a desired visual effect and wherein at least some of the pixels having modified magnitudes are in the foreground set of pixels and/or at least some of the pixels having modified magnitudes are in the new background set of pixels.
19. The contact center of claim 14 , wherein the attribute of the selected customer communication device or customer associated with the selected customer communication device is an identity of the customer, an electronic address received from the selected customer communication device, an electronic address associated with the customer communication device, or a combination thereof, wherein the instructions causes the microprocessor to determine the attribute from a signal exchanged with the selected customer communication device, input received from the customer communication device, face recognition based on an image of the customer, content analysis of audio information and/or video information of the video call, content analysis of a presentation displayed during the video calls, or a combination thereof, and wherein the instructions cause the microprocessor to select the template set of pixels from among multiple template sets of pixels by mapping the determined attribute against associations of sets of one or more attributes against a corresponding template set of pixels.
20. The contact center of claim 14 , wherein, when the attribute of the selected customer communication device or customer associated with the selected customer communication device is not associated with a template set of pixels in the multiple template sets of pixels, the instructions cause the microprocessor to query the selected agent communication device for a template set of pixels to be used in forming the modified image information and wherein the microprocessor uses, in forming the modified image information, the template set of pixels identified in a response received from the selected agent communication device.