Sign language control for a virtual meeting
The present disclosure provides systems and methods for delivering assistance to impaired users within a virtual conferencing system, and more particularly, to providing assistance to hearing-impaired users to control the virtual conferencing system while using sign language. The systems and methods for providing assistance to impaired users includes initiating a virtual meeting, detecting one or more hand gestures within a predefined area, interpreting the one or more hand gestures to determine one or more commands, and updating one or more controls for the virtual meeting based on the one or more commands. These steps can be implemented to assist hearing-impaired users to manage controls when communicating within a virtual meeting.
1. A method, comprising:
initiating, by a client device, a virtual meeting comprising a plurality of client devices, wherein the virtual meeting includes a designation of a user of the client device as a hearing-impaired user;
capturing video from a first camera, the captured video comprising a predefined area initialized in response to the designation of the user of the client device as a hearing-impaired user;
detecting, by the client device, one or more hand gestures within the predefined area, comprising tracking at least one hand of the user of the client device within the predefined area;
interpreting, by the client device, the one or more hand gestures to determine one or more commands; and
updating, by the client device, one or more controls for the virtual meeting based on the one or more commands.
2. The method of claim 1 , wherein the predefined area is at least one of a layer within a first viewing area and a second viewing area.
3. The method of claim 1 , further comprising loading an overlay over a main viewing area of the virtual meeting, the overlay including a plurality of controls selectable by the one or more hand gestures to determine the one or more commands.
4. The method of claim 3 , further comprising capturing video from a second camera.
5. The method of claim 4 , further comprising:
detecting a gaze of the user of the client device into one of the first camera and the second camera; and
determining a conversation mode or a control command mode based on the detected gaze and a first mode of the first camera and a second mode of the second camera.
6. The method of claim 4 , further comprising:
detecting a presence of a hand of the user of the client device within a field of view of the second camera, wherein the interpreting the one or more hand gestures to determine the one or more commands is based on the presence of the hand of the user within the field of view of the second camera.
7. The method of claim 1 , wherein the designation of the user of the client device as the hearing-impaired user is based on a second designation of another user as a sign language interpreter.
8. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to:
initiate, by a client device, a virtual meeting comprising a plurality of client devices, wherein the virtual meeting includes a designation of a user of the client device as a hearing-impaired user;
capture video from a first camera, the captured video comprising a predefined area initialized in response to the designation of the user of the client device as a hearing-impaired user;
detect, by the client device, one or more hand gestures within the predefined area, comprising tracking at least one hand of the user of the client device within the predefined area;
interpret, by the client device, the one or more hand gestures to determine one or more commands; and
update, by the client device, one or more controls for the virtual meeting based on the one or more commands.
9. The system of claim 8 , wherein the predefined area is at least one of a layer within a first viewing area and a second viewing area.
10. The system of claim 8 , further comprising loading an overlay over a main viewing area of the virtual meeting, the overlay including a plurality of controls selectable by the one or more hand gestures to determine the one or more commands.
11. The system of claim 10 , further comprising capturing video from a second camera.
12. The system of claim 11 , further comprising:
detecting a gaze of the user of the client device into one of the first camera and the second camera; and
determining a conversation mode or a control command mode based on the detected gaze and a first mode of the first camera and a second mode of the second camera.
13. The system of claim 11 , further comprising:
detecting a presence of a hand of the user of the client device within a field of view of the second camera, wherein the interpreting the one or more hand gestures to determine the one or more commands is based on the presence of the hand of the user within the field of view of the second camera.
14. The method of claim 3 , wherein the plurality of controls comprise controls for use by hearing-impaired users and or sign language interpreters.
15. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to:
initiate, by a client device, a virtual meeting comprising a plurality of client devices, wherein the virtual meeting includes a designation of a user of the client device as a hearing-impaired user;
capture video from a first camera, the captured video comprising a predefined area initialized in response to the designation of the user of the client device as a hearing-impaired user;
detect, by the client device, one or more hand gestures within the predefined area, comprising tracking at least one hand of the user of the client device within the predefined area;
interpret, by the client device, the one or more hand gestures to determine one or more commands; and
update, by the client device, one or more controls for the virtual meeting based on the one or more commands.
16. The non-transitory computer-readable memory of claim 15 , wherein the predefined area is at least one of a layer within a first viewing area and a second viewing area.
17. The non-transitory computer-readable memory of claim 15 , further comprising loading an overlay over a main viewing area of the virtual meeting, the overlay including a plurality of controls selectable by the one or more hand gestures to determine the one or more commands.
18. The non-transitory computer-readable memory of claim 17 , further comprising capturing video from a second camera.
19. The non-transitory computer-readable memory of claim 18 , further comprising:
detecting a gaze of the user of the client device into one of the first camera and the second camera; and
determining a conversation mode or a control command mode based on the detected gaze and a first mode of the first camera and a second mode of the second camera.
20. The non-transitory computer-readable memory of claim 18 , further comprising:
detecting a presence of a hand of the user of the client device within a field of view of the second camera, wherein the interpreting the one or more hand gestures to determine the one or more commands is based on the presence of the hand of the user within the field of view of the second camera.