IP Library Granted Patent US 12701315
Granted Patent B2
US 12701315 · App. 18/082,784 · Granted Aug 4, 2026

Vehicle component voice control based on image captures

Inventors: Minjae Park (Seongnam-Si, KR); Jaemin Moon (Yongin-Si, KR); Sungwang Kim (Seoul, KR)
Assignees: Hyundai Motor Company; Kia Corporation
G10L15/22B60K35/10B60K35/23B60K35/265B60K35/28B60K35/60B60K35/658B60K35/81B60R1/29G06F3/165G06F3/167G06T11/23G10L15/1815H04N23/695H04N23/90B60K35/22B60K35/26B60K35/90B60K2360/143B60K2360/148B60R1/22B60R2300/105B60R2300/20B60R2300/304B60R2300/8006G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12701315
App. No.
18/082,784
Granted
Aug 4, 2026
Kind
B2
Abstract

A vehicle may include: a display provided inside the vehicle; and a controller configured to control the display, based on a condition being satisfied, to display a vehicle image comprising a graphic object and a plurality of indicators, wherein the graphic object indicates a plurality of portions of the vehicle image, and wherein each of the plurality of indicators is respectively associated with one of the plurality of portions. Based on a user utterance associated with at least one indicator of the plurality of indicators, the controller may be configured to control a control target corresponding to the at least one indicator.

Claims (74)

1 . A vehicle comprising:

a camera configured to capture a vehicle image of the vehicle, wherein the vehicle image includes an image of a plurality of vehicle components of the vehicle, wherein the vehicle image comprises an image corresponding to a portion of a vehicle interior of the vehicle, and wherein the plurality of vehicle components of the vehicle comprise a plurality of vehicle components equipped in the vehicle interior of the vehicle;

a display provided inside the vehicle; and

a controller configured, based on a condition being satisfied, to control the camera to capture the vehicle image, and to control the display to display a guide image, wherein the guide image comprises the vehicle image, a graphic object, and a plurality of indicators, wherein the graphic object comprises a two-dimensional (2D) grid superimposed on the vehicle image, wherein the graphic object indicates a plurality of portions, of the vehicle image, defined by the 2D grid, wherein each of the plurality of indicators is respectively superimposed on one of the plurality of portions, wherein the plurality of vehicle components of the vehicle correspond to different indicators of the plurality of indicators, wherein the controller is configured to control, based on a user utterance associated with at least one indicator of the plurality of indicators, a control target corresponding to the at least one indicator, and wherein the control target is a controllable vehicle component of the vehicle.

2 . The vehicle of claim 1 , wherein the camera is configured to capture a second vehicle image comprising an image corresponding to a plurality of portions of a single vehicle, wherein the second vehicle image includes an image of a plurality of second vehicle components of the single vehicle,

wherein the plurality of second vehicle components of the single vehicle comprise at least one of:

a plurality of vehicle components equipped in a vehicle interior of the single vehicle, or

a plurality of exterior vehicle components corresponding to a plurality of exterior components of the single vehicle,

wherein the controller is configured, based on a condition being satisfied, to control the camera to capture the second vehicle image, and to control the display to display a second guide image, wherein the second guide image comprises the second vehicle image, a second graphic object, and a plurality of second indicators,

wherein the second graphic object comprises a contour indicating a boundary of each of a plurality of objects comprised in the second vehicle image,

wherein each of the plurality of second indicators is respectively superimposed on one of the plurality of portions of the single vehicle, and

wherein the plurality of portions of the single vehicle are defined by the contour.

3 . The vehicle of claim 1 , wherein the condition comprises at least one of:

a first condition that requires a subsequent user utterance to specify the control target; or

a second condition in which a context that requires a control for a specific control target occurs.

4 . The vehicle of claim 1 , further comprising a first camera configured to capture a rear interior of the vehicle and a second camera configured to capture a front interior of the vehicle,

wherein the vehicle image is an image corresponding to a portion of the captured rear interior or a portion of the captured front interior,

wherein the controller is further configured to:

based on a user utterance associated with a controllable object in the vehicle, identify the controllable object,

wherein the controllable object is a single object having a plurality of controllable portions, and

wherein the control target is associated with a controllable portion of the plurality of controllable portions.

5 . The vehicle of claim 4 , wherein the controller is configured to control at least one of:

based on a control target candidate being located in the rear interior of the vehicle, the first camera to capture the vehicle image, or

based on a control target candidate being located in the front interior of the vehicle, the second camera to capture the vehicle image.

6 . The vehicle of claim 1 , further comprising a rotatable camera configured to capture a rear interior of the vehicle at a first rotation angle and capture a front interior of the vehicle at a second rotation angle, wherein the vehicle image is an image corresponding to a portion of the captured rear interior or a portion of the captured front interior.

7 . The vehicle of claim 6 , wherein the controller is configured to control at least one of:

based on a control target candidate being located in the rear interior of the vehicle, a rotation of the rotatable camera at the first rotation angle to capture the vehicle image, or

based on a control target candidate being located in the front interior of the vehicle, a rotation of the rotatable camera at the second rotation angle to capture the vehicle image.

8 . The vehicle of claim 1 , further comprising:

a communication device configured to receive a result processing signal for controlling the vehicle, wherein the result processing signal indicates a user intention associated with the user utterance,

wherein the controller is configured to control the control target based on the result processing signal.

9 . The vehicle of claim 1 , further comprising:

a speaker,

wherein the controller is configured to:

determine, based on a plurality of control targets corresponding to the at least one indicator being identified, that the condition is satisfied; and

control the speaker to output a guide signal for specifying the control target as a control target intended by a user among the plurality of control targets.

10 . The vehicle of claim 1 , wherein the controller is configured to control the display to display an enlarged portion, of the plurality of portions, on which the at least one indicator is displayed, and

adjust the 2D grid and the plurality of indicators to correspond to the enlarged portion on which the at least one indicator is displayed.

11 . A vehicle system comprising:

a communication device configured to receive data indicating an utterance of a user in a vehicle;

a camera configured to capture a vehicle image of the vehicle, wherein the vehicle image includes an image of a plurality of vehicle components of the vehicle, wherein the vehicle image comprises an image corresponding to a portion of a vehicle interior of the vehicle, and wherein the plurality of vehicle components of the vehicle comprise a plurality of vehicle components equipped in the vehicle interior of the vehicle;

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the vehicle system to:

control, based on a condition being satisfied, the camera to capture the vehicle image;

determine, based on the data indicating the utterance of the user, an action and a control target intended by the user, wherein the control target is a controllable vehicle component of the plurality of vehicle components; and

generate, based on a determination of the action and the control target, a result processing signal for displaying a guide image on a display provided inside the vehicle,

wherein the guide image comprises the vehicle image, a graphic object, and a plurality of indicators,

wherein the graphic object comprises a contour indicating a boundary of each of a plurality of objects comprised in the vehicle image,

wherein the graphic object and the plurality of indicators are superimposed on the vehicle image,

wherein the graphic object indicates a plurality of portions, of the vehicle image defined by the contour,

wherein the plurality of vehicle components of the vehicle correspond to different indicators of the plurality of indicators,

wherein each of the plurality of indicators is respectively superimposed on one of the plurality of portions, and

wherein the instructions, when executed by the at least one processor, cause the vehicle system to: control, based on a user utterance associated with at least one indicator of the plurality of indicators, a control target that corresponds to the at least one indicator and is a controllable vehicle component of the plurality of vehicle components.

12 . The vehicle system of claim 11 , wherein the communication device is configured to receive data comprising at least one indicator of the plurality of indicators, and

wherein the instructions, when executed by the at least one processor, cause the vehicle system to determine the control target, wherein the control target corresponds to the at least one indicator.

13 . The vehicle system of claim 11 , wherein the instructions, when executed by the at least one processor, cause the vehicle system to map a component of the vehicle to:

at least one indicator superimposed on at least one portion of the plurality of portions; or

the at least one portion of the plurality of portions.

14 . The vehicle system of claim 12 , wherein the instructions, when executed by the at least one processor, cause the vehicle system to determine, as the control target, a component of the vehicle mapped to:

the at least one indicator; or

a portion, of the plurality of portions, on which the at least one indicator is displayed.

15 . The vehicle system of claim 12 , wherein the instructions, when executed by the at least one processor, cause the vehicle system to determine, as the control target, an object corresponding to the at least one indicator among a plurality of objects comprised in the vehicle image.

16 . An apparatus comprising:

a camera configured to capture an object image, wherein the object image includes an image of a plurality of vehicle components of a vehicle, wherein the object image comprises an image corresponding to a portion of a vehicle interior of the vehicle, and wherein the plurality of vehicle components of the vehicle comprise a plurality of vehicle components equipped in the vehicle interior of the vehicle;

a communication device configured to receive data indicating an utterance of a user of the vehicle; and

a controller configured to:

determine, based on the data indicating the utterance of the user, a plurality of indicators;

based on a condition being satisfied, control the camera to capture the object image, and control a display of the vehicle to display a guide image, wherein the guide image comprises the object image, a graphic object, and the plurality of indicators, wherein the graphic object comprises a two-dimensional (2D) grid superimposed on the object image, wherein the graphic object indicates a plurality of portions, of the object image, defined by the 2D grid, wherein each of the plurality of indicators is respectively superimposed on one of the plurality of portions, wherein the plurality of vehicle components of the vehicle correspond to different indicators of the plurality of indicators; and

generate, based on a subsequent user utterance input associated with at least one indicator of the plurality of indicators, a control signal to control a control target corresponding to the at least one indicator, wherein the control target is a controllable vehicle component of the plurality of vehicle components, and wherein the control target is associated with a controllable portion of a single object controllable by the user.

17 . The apparatus of claim 16 , wherein the single object is a single object, of a vehicle, having a plurality of controllable portions,

wherein the controller is further configured to identify, based on the utterance of the user, the single object, and

wherein the subsequent user utterance input comprises at least one of:

a further utterance of the user; or

an utterance of a second user.