IP Library › Granted Patent US 11,521,612
Granted Patent B2
US 11,521,612 · App. 16/909,593 · Granted Dec 6, 2022

Vehicle control apparatus and method using speech recognition

Inventor: Jae Min Joh (Yongin-si, KR)
Assignees: HYUNDAI MOTOR COMPANY; KIA MOTORS CORPORATION
G10L15/22B60N2/002G01G19/52G06V40/172H04R1/403H04R1/406G06V40/178H04R2499/13
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,521,612
App. No.
16/909,593
Granted
Dec 6, 2022
Kind
B2
Abstract

A vehicle control apparatus and method use speech recognition and include: a passenger recognizing device configured to recognize passengers including a first passenger and at least one second passenger in a vehicle; a voice recognizing device configured to receive and to recognize a voice utterance by the first passenger or the at least one second passenger and to output a speech recognition result based on the received voice utterance; and a processor configured to additionally query the at least one second passenger or the first passenger based on the speech recognition result of the voice utterance of the first passenger or the at least one second passenger, respectively, to provide each of the first passenger and the at least one second passenger with a customized service.

Claims (44)

1. A vehicle control apparatus, the vehicle control apparatus comprising:

a passenger recognizing device configured to recognize passengers including a first passenger and at least one second passenger in a vehicle;

a voice recognizing device configured to receive and to recognize a voice utterance by the first passenger or the at least one second passenger and configured to output a speech recognition result based on the received voice utterance; and

a processor configured to output an additional query to the at least one second passenger or the first passenger based on the speech recognition result of the voice utterance of the first passenger or the at least one second passenger, respectively, to provide each of the first passenger and the at least one second passenger with a customized service,

wherein the processor is further configured to

generate and output the additional query for asking an intent of each of the at least one second passenger based on a control command matched with the speech recognition result of the voice utterance of the first passenger,

recognize a response of each of the at least one second passenger for the additional query through the voice recognizing device, and

perform an action according to the response of each of the at least one second passenger to provide a service matched with the intent of each of the at least one second passenger.

2. The vehicle control apparatus of claim 1 , wherein the passenger recognizing device is configured to recognize a presence and a location of the passengers through a weight sensor installed for each seat.

3. The vehicle control apparatus of claim 2 , wherein the passenger recognizing device is configured to recognize a face of each of the first passenger and the at least one second passenger through a camera to estimate an age of each of the first passenger and the at least one second passenger and to determine whether the first passenger and the at least one second passenger are capable of using speech recognition, based on the estimated age of each passenger.

4. The vehicle control apparatus of claim 3 , wherein the voice recognizing device includes a plurality of microphones, each of which is positioned for each seat and configured to receive a voice utterance of each passenger seated at each seat.

5. The vehicle control apparatus of claim 4 , further comprising:

a sound output device including a plurality of speakers, each of which is positioned for each seat and configured to independently output an acoustic signal.

6. The vehicle control apparatus of claim 5 , wherein the processor activates a microphone and a speaker for each seat based on the result of determining whether the first passenger and the at least one second passenger are capable of using speech recognition.

7. The vehicle control apparatus of claim 1 , wherein the processor processes an action matched with the speech recognition result to provide a service to the first passenger and the at least one second passenger when the speech recognition result is included in a common service domain.

8. The vehicle control apparatus of claim 1 , wherein the processor determines whether the speech recognition result is included in a domain requiring the additional query, when the speech recognition result is not included in a common service domain.

9. The vehicle control apparatus of claim 8 , wherein the processor determines whether to proceed with the additional query for each of the passengers, based on the location of each of the passengers, a voice input location, and an action control target state when the speech recognition result is included in the domain requiring the additional query.

10. A vehicle controlling method, the vehicle controlling method comprising:

recognizing passengers including a first passenger and at least one second passenger in a vehicle;

receiving and recognizing a voice utterance of the first passenger or the at least one second passenger;

outputting a speech recognition result based on the received voice utterance; and

additionally, outputting an additional query to the at least one second passenger or the first passenger based on the speech recognition result of the voice utterance of the first passenger or the at least one second passenger, respectively, to provide each of the first passenger and the at least one second passenger with a customized service,

wherein the outputting of the additional query to the at least one second passenger or the first passenger includes

generating and outputting the additional query for asking an intent of each of the at least one second passenger based on a control command matched with the speech recognition result of the voice utterance of the first passenger,

recognizing a response of each of the at least one second passenger for the additional query, and

performing an action according to the response of each of the at least one second passenger to provide a service matched with the intent of each of the at least one second passenger.

11. The vehicle controlling method of claim 10 , wherein the recognizing of the first passenger and the at least one second passenger includes:

recognizing a presence and a location of the passengers through a weight sensor installed for each seat.

12. The vehicle controlling method of claim 11 , wherein the recognizing of the first passenger and the at least one second passenger includes:

recognizing a face of each of the first passenger and the at least one second passenger through a camera to estimate an age of each of the first passenger and the at least one second passenger; and

determining whether the first passenger and the at least one second passenger are capable of using speech recognition, based on the estimated age of each passenger.

13. The vehicle controlling method of claim 12 , wherein the recognizing of the voice utterance of the first passenger or the at least one second passenger includes:

activating or deactivating a microphone and a speaker, which are mounted for each seat, based on the result of determining whether the first passenger and the at least one second passenger are capable of using speech recognition.

14. The vehicle controlling method of claim 10 , wherein the outputting of the additional query to the at least one second passenger or the first passenger includes:

receiving the speech recognition result of the voice utterance of the first passenger or the at least one second passenger;

determining whether the speech recognition result is included in a common service domain; and

processing an action matched with the speech recognition result to provide a service to the first passenger and the at least one second passenger, when the speech recognition result is included in the common service domain.

15. The vehicle controlling method of claim 10 , wherein the outputting of the additional query to the at least one second passenger or the first passenger includes:

determining whether the speech recognition result is included in a domain requiring the additional query, when the speech recognition result is not included in a common service domain; and

determining to proceed with the additional query for each of the passengers when the speech recognition result is included in the domain requiring the additional query.

16. The vehicle controlling method of claim 15 , wherein the determining to proceed with the additional query for each of the passengers includes:

determining to proceed with the additional query for each of the passengers based on the location of each of the passengers, a voice input location, and an action control target state.

17. The vehicle controlling method of claim 15 , further comprising:

performing an action according to the speech recognition result of each of the passengers, when it is not determined that the additional query for each of the passengers has progressed.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2020
From: JOH, JAE MIN
To: HYUNDAI MOTOR COMPANY; KIA MOTORS CORPORATION
Reel/Frame 053017/0034 →
Priority Claims (1)
KR 10-2019-0155512 · Nov 28, 2019 · national
Continuity (1)
Related Publication 20210166683A1 · Jun 3, 2021