Real time sign language conversion for communication in a contact center
Interactions between agents of a contact center and customers provide the bulk of many communications and generally work well, except when the customer is deaf or mute. Customers that sign would often prefer to conduct interactions in sign language but the pool of agents having such skills is limited. By providing systems and methods to alter an image of a non-signing agent, a customer may be presented with an image of a signing agent. Additionally, systems and methods herein enable a non-signing agent to be presented with generated speech and/or text translated from the signing gestures provided by the image of the signing customer.
1. A system for real-time sign language translation, comprising:
a communication interface configured to receive a video image of a human agent engaged in an interaction via a network with a customer utilizing a customer communication device;
a processor having an accessible memory; and
the processor configured to:
receive the video image of the human agent;
receive an audio input comprising speech of the human agent;
determine a sign language gesture of the speech;
modify the video image of the human agent to comprise the sign language gesture; and
present the customer communication device with the modified video image of the human agent comprising the sign language gesture.
2. The system of claim 1 , further comprising:
a data storage configured to maintain data records accessible to the processor; and
wherein the processor is further configured to select a record from the data storage comprising a pre-recorded gesture and selected in accordance with the determined sign language gesture of the speech and wherein the processor modifies the video image of the human agent comprising overlaying the pre-recorded gesture on the video image of the human agent.
3. The system of claim 1 , wherein the processor is further configured to map portions of the video image comprising associated portions of the human agent's anatomy to the sign language gesture and wherein modifying the video image of the human agent to comprise the sign language gesture, further comprises, placing the mapped portions of the human agent's anatomy to coincide with the sign language gesture.
4. The system of claim 1 , wherein the processor is further configured to generate images of portions of the human agent's anatomy to cause the generated images to produce the sign language gesture and wherein the processor modifies the video image of the human agent comprising the generated images overlaying the video image of the human agent.
5. The system of claim 1 , wherein the processor modifies the video image of the human agent to comprise the sign language gesture, comprising applying alterations to at least one polygons mapping portions of the video image of the human agent in accordance with alteration data maintained in a data storage.
6. The system of claim 1 , wherein the modification of the video image of the human agent to comprise the sign language gesture further comprises modification of a portion of the video image of the human agent chroma key masked and omitting modification of the video image of the human agent that are not chroma key masked.
7. The system of claim 1 , wherein the modification of the video image of the human agent to comprise the sign language gesture further comprises modification of a portion of the video image comprising the mouth of the human agent and applying a synchronization modification to cause the speech of the human agent to be synchronized with the sign language gesture.
8. The system of claim 1 , wherein the processor is further configured to:
receive a video image of the customer;
determine a sign language gesture from the video image of the customer;
translate the determined sign language gesture into speech; and
present the speech to the human agent.
9. The system of claim 1 , wherein the processor is further configured to:
receive a video image of the customer;
determine a sign language gesture from the video image of the customer;
translate the determined sign language gesture into text; and
present the text to the human agent.
10. The system of claim 1 , wherein the sign language gesture comprises a plurality of sign language gestures conforming to a previously selected sign language grammar.
11. A method, comprising:
receiving a video image of a human agent engaged in an interaction, via a network, with a customer utilizing a customer communication device;
receiving the video image of the human agent;
receiving an audio input comprising speech of the human agent;
determining a sign language gesture of the speech;
modifying the video image of the human agent to comprise the sign language gesture; and
presenting the customer communication device with the modified video image of the human agent comprising the sign language gesture.
12. The method of claim 11 , further comprising:
selecting a record, from a data storage comprising a pre-recorded gesture, the selection being in accordance with the determined sign language gesture of the speech of the human agent; and
wherein modifying the video image of the human agent comprises overlaying the pre-recorded gesture on the video image of the human agent.
13. The method of claim 11 , further comprising:
mapping portions of the video image comprising associated portions of the human agent's anatomy to the sign language gesture; and
wherein modifying the video image of the human agent to comprise the sign language gesture, further comprises, placing the mapped portions of the human agent's anatomy to coincide with the sign language gesture.
14. The method of claim 11 , further configured:
generating images of portions of the human agent's anatomy to cause the generated images to produce the sign language gesture; and
wherein modifying the video image of the human agent further comprises overlaying the generated images of the portions of the human agent's anatomy onto the video image of the human agent.
15. The method of claim 11 , wherein the modifying of the video image of the human agent to comprise the sign language gesture, further comprises applying alterations to at least one polygons mapping portions of the video image of the human agent in accordance with alteration data maintained in a data storage.
16. The method of claim 11 , wherein the modifying of the video image of the human agent to comprise the sign language gesture further comprises modifying a portion of the video image of the human agent that is chroma key masked and omitting modification of the video image of the human agent that are not chroma key masked.
17. The method of claim 11 , wherein the modifying of the video image of the human agent to comprise the sign language gesture further comprises modifying a portion of the video image comprising the mouth of the human agent and applying a synchronization modification to cause the speech of the human agent to be synchronized with the sign language gesture.
18. The method of claim 11 , further comprising:
receiving a video image of the customer;
determining a sign language gesture from the video image of the customer;
translating the determined sign language gesture into speech; and
presenting the speech to the human agent via an agent communication device.
19. The method of claim 11 , further comprising:
receiving a video image of the customer;
determining a sign language gesture from the video image of the customer;
translating the determined sign language gesture into text; and
present the text to the human agent via an agent communication device.
20. A system, comprising:
means to receive a video image of a human agent engaged in an interaction, via a network, with a customer utilizing a customer communication device;
means to receive the video image of the human agent;
means to receive an audio input comprising speech of the human agent;
means to determine a sign language gesture of the speech;
means to modify the video image of the human agent to comprise the sign language gesture; and
means to present the customer communication device with the modified video image of the human agent comprising the sign language gesture.