IP Library › Granted Patent US 11,507,758
Granted Patent B2
US 11,507,758 · App. 16/668,516 · Granted Nov 22, 2022

Vehicle-based sign language communication systems and methods

Inventors: Omar Makke (Lyon Township, MI); Oleg Gusikhin (Commerce Township, MI); Ayush Shah (Belleville, MI)
Assignee: Ford Global Technologies, LLC
G06F40/56G01C21/3438G06F3/016G06F3/017G06Q50/30G06T13/40G06V20/597G06V40/28G09B21/009G10L15/22H04N21/4856G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,507,758
App. No.
16/668,516
Granted
Nov 22, 2022
Kind
B2
Abstract

Vehicle-based sign language communication systems and methods are provided herein. An example device can be configured to determine a sign language protocol used by the first user, determine a target language used by a second user, obtain a translation library based on the sign language protocol and the target language, receive spoken word input from a second user through a microphone, convert the spoken word input into sign language output using the translation library, and provide the sign language output using a sign language output device.

Claims (59)

1. A method, comprising:

receiving sign language input from a first user using a camera of a vehicle;

translating the sign language input into a first target language translation;

outputting the first target language translation as audio output through an audio output to a second user;

receiving spoken word input from the second user;

translating the spoken word input into sign language output;

determining that the first user is paying attention to a sign language output device; and

providing the sign language output to the first user using the sign language output device based on the first user paying attention to the sign language output device.

2. The method according to claim 1 , further comprising translating the first target language translation into a second target language translation.

3. The method according to claim 1 , wherein the first target language translation is output in a natural language format.

4. The method according to claim 1 , wherein providing the sign language output to the first user comprises displaying a sign language avatar based on a sign language protocol, wherein the sign language protocol comprises a type of sign language spoken by the first user.

5. The method according to claim 1 , further comprising activating a haptic feedback on a seat associated with the first user based on the first user not paying attention to the sign language output device.

6. The method according to claim 1 , further comprising utilizing output of the camera to determine when the first user is looking at the sign language output device.

7. The method according to claim 1 , further comprising:

receiving a ride-share request comprising a selection of sign language service from the first user and a selection of a sign language protocol, wherein the selection of the sign language protocol indicates a type of sign language spoken by the first user; and

selecting the vehicle in response to the ride-share request.

8. The method according to claim 7 , further comprising:

determining a language type for the second user; and

receiving a translation library based on the language type and the sign language protocol.

9. A system, comprising:

a camera;

a microphone;

an audio output device;

a sign language output device; and

a vehicle controller comprising a processor and memory, the processor executing instructions stored in the memory to:

receive sign language input from a first user using the camera;

translate the sign language input into a first target language translation;

output the first target language translation as audio output through the audio output device to a second user;

receive spoken word input from the second user through the microphone;

convert the spoken word input into sign language output;

determine that the first user is paying attention to the sign language output device; and

provide the sign language output using the sign language output device based on the first user paying attention to the sign language output device.

10. The system according to claim 9 , further comprising obtaining a sign language protocol for the first user and a language library for the second user from a service provider, wherein the sign language protocol comprises a type of sign language spoken by the first user.

11. The system according to claim 10 , wherein the first target language translation is output through the audio output device in a natural language format.

12. The system according to claim 11 , wherein the vehicle controller is configured to provide the sign language output to the first user by generating an animated avatar that performs the sign language output according to the sign language protocol.

13. The system according to claim 9 , further comprising a haptic feedback mechanism on a seat associated with the first user, wherein the vehicle controller is configured to activate the haptic feedback mechanism based on the first user not paying attention to the sign language output device.

14. The system according to claim 9 , wherein the vehicle controller is configured to utilize output of the camera to determine when the first user is looking at the sign language output device.

15. The system according to claim 9 , further comprising a service provider that is configured to:

receive a ride-share request comprising a selection of sign language service from the first user and a selection of a sign language protocol, wherein the selection of the sign language protocol indicates a type of sign language spoken by the first user; and

select the vehicle in response to the ride-share request.

16. The system according to claim 15 , wherein the vehicle controller is further configured to:

determine a language type for the first target language translation; and

obtain a translation engine from the service provider based on the language type.

17. A device, comprising:

a processor; and

a memory for storing instructions, the processor executing the instructions to:

determine a sign language protocol used by a first user;

determine a target language used by a second user;

obtain a translation library based on the sign language protocol and the target language;

receive spoken word input from a second user through a microphone; and

convert the spoken word input into sign language output using the translation library;

determine that the first user is paying attention to a sign language output device; and

provide the sign language output using the sign language output device based on the first user paying attention to the sign language output device.

18. The device according to claim 17 , wherein the sign language output device comprises a mobile device of the first user.

19. The device according to claim 17 , wherein the processor is configured to:

receive sign language input from a first user using a camera;

translate the sign language input into a spoken or natural language output using the translation library; and

output the spoken or natural language output through an audio device.

20. The device according to claim 17 , wherein the processor is configured to utilize output of a camera to determine when the first user is looking at the sign language output device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2019
From: MAKKE, OMAR; GUSIKHIN, OLEG; SHAH, AYUSH
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 050945/0487 →
Continuity (1)
Related Publication 20210133290A1 · May 6, 2021
Cited By (1)
US 12,652,446