IP Library Granted Patent US 12,361,937
Granted Patent B2
US 12,361,937 · App. 17/561,548 · Granted Jul 15, 2025

Method and system for assisting a user

Inventors: Keyvan Mohajer (Los Gatos, CA); Kaishin Kam (Palo Alto, CA); Christophe Pierret (Sèvres, FR)
Assignee: SoundHound AI IP, LLC
G10L15/22G01C21/3629G10L2015/223G10L2015/227G10L2015/228
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,361,937
App. No.
17/561,548
Granted
Jul 15, 2025
Kind
B2
Abstract

A method of assisting a user. The method including obtaining a plurality of rules having condition components and action components, the action components specifying conversation schemas, detecting, by a sensor, a fact related to an environment of the user, identifying a rule, of the plurality of rules, having a condition component that is satisfied by the detected fact, initiating a conversation with the user according to a conversation schema of the action component of the rule of the plurality of rules, and performing an action in response to a positive statement by the user.

Claims (45)

1. A method of assisting a user, the method comprising:

obtaining a plurality of rules having condition components and action components, the action components specifying conversation schemas;

detecting, by a sensor, a fact related to an environment of the user;

identifying a rule, of the plurality of rules, having a condition component that is satisfied by the detected fact;

determining one or more parameters in dependence on at least one of the identified rule, on the detected fact and on subject matter determined in dependence on at least one of the condition component of the identified rule, an action component of the identified rule, and the detected fact;

determining whether to initiate a conversation with the user according to a conversation schema of the action component of the identified rule and according to the one or more parameters, including (i) a conversation history according to which the conversation can be initiated with the user, and (ii) persistent data according to which the conversation can be initiated with the user, the persistent data indicating whether the user has enabled a quiet mode indicator according to which a determination can be made to not initiate the conversation with the user, despite the identification of the rule;

initiating the conversation according to the conversation schema and the determined subject matter; and

activating a motor in response to a positive statement by the user.

2. The method of claim 1 , wherein the identifying of the rule comprises:

identifying two or more rules having respective condition components that are satisfied in dependence on the detected fact;

ranking the identified two or more rules in dependence on priority; and

selecting a rule of the identified two or more rules having a highest priority, such that a conversation schema specified by an action component of the selected rule is used to initiate the conversation with the user.

3. The method of claim 1 , wherein the identifying of the rule comprises:

identifying two or more rules having respective condition components that are satisfied in dependence on the detected fact;

determining that the two or more rules are conflicting; and

applying an agenda to determine which of the conflicting rules is to be selected, such that a conversation schema specified by an action component of the selected rule is used to initiate the conversation with the user.

4. The method of claim 1 , wherein the identifying of the rule comprises:

identifying two or more action components included in the identified rule;

ranking the two or more action components in dependence on priority; and

selecting an action component of the two or more action components having a highest priority, such that a conversation schema specified by the selected action component is used to initiate the conversation with the user.

5. The method of claim 1 , wherein the specified conversation schema is at least one of a data structure, a script and a software component.

6. The method of claim 1 , wherein the conversation is initiated with the user in a written format.

7. The method of claim 1 , wherein the conversation is proactively initiated with the user without initial engagement from the user.

8. The method of claim 7 , further comprising determining an appropriate time for initiating the conversation with the user and initiating the conversation with the user at the appropriate time.

9. The method of claim 1 , wherein the method is performed by a virtual assistant that is at least partially included in a vehicle head unit of a vehicle.

10. The method of claim 1 , wherein the conversation schema specifies initiating a new conversation with the user that provides a reminder about sensor information.

11. The method of claim 1 , further comprising, in dependence on the identified rule having the condition component satisfied by the detected fact identifying an event on a roadway to be travelled by the user along a current navigation route, using the detected fact, the conversation schema and navigation information to determine that the user will arrive late to their destination because of the identified event on the roadway and initiate the conversation to ask the user whether or not to notify an individual regarding the late arrival.

12. A system including one or more processors coupled to memory, the memory loaded with computer instructions to assist a user, the computer instructions, when executed on the one or more processors, causing the one or more processors to implement actions comprising:

obtaining a plurality of rules having condition components and action components, the action components specifying conversation schemas;

detecting, by a sensor, a fact related to an environment of the user;

identifying a rule, of the plurality of rules, having a condition component that is satisfied by the detected fact;

determining one or more parameters in dependence on at least one of the identified rule, on the detected fact and on subject matter determined in dependence on at least one of the condition component of the identified rule, an action component of the identified rule, and the detected fact;

determining whether to initiate a conversation with the user according to a conversation schema of the action component of the identified rule and according to the one or more parameters, including (i) a conversation history according to which the conversation can be initiated with the user, and (ii) persistent data according to which the conversation can be initiated with the user, the persistent data indicating whether the user has enabled a quiet mode indicator according to which a determination can be made to not initiate the conversation with the user, despite the identification of the rule;

initiating the conversation according to the conversation schema and the determined subject matter; and

activating a motor in response to a positive statement by the user.

13. A method of assisting a user, the method comprising:

obtaining a plurality of rules having condition components and action components, the action components specifying conversation schemas;

detecting a fact related to the user;

identifying a rule, of the plurality of rules, having a condition component that is satisfied by the detected fact;

determining one or more parameters in dependence on at least one of the identified rule, on the detected fact and on subject matter determined in dependence on at least one of the condition component of the identified rule, an action component of the identified rule, and the detected fact;

determining whether to initiate a conversation with the user according to a conversation schema of the action component of the identified rule and according to the one or more parameters, including (i) a conversation history according to which the conversation can be initiated with the user, and (ii) persistent data according to which the conversation can be initiated with the user, the persistent data indicating whether the user has enabled a quiet mode indicator according to which a determination can be made to not initiate the conversation with the user, despite the identification of the rule;

initiating the conversation according to the conversation schema and the determined subject matter; and

performing an action in response to a positive statement by the user.

14. The method of claim 13 , wherein the detected fact is obtained from an electronic calendar of the user.

15. The method of claim 14 , wherein the conversation schema specifies initiating a new conversation with the user that provides a reminder about user information.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2025
From: SOUNDHOUND, INC.
To: SOUNDHOUND AI IP, LLC
Reel/Frame 071140/0809 →
RELEASE OF SECURITY INTEREST Recorded Jun 11, 2024
From: ACP POST OAK CREDIT II LLC, AS COLLATERAL AGENT
To: SOUNDHOUND, INC.; SOUNDHOUND AI IP, LLC
Reel/Frame 067698/0845 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2023
From: SOUNDHOUND AI IP HOLDING, LLC
To: SOUNDHOUND AI IP, LLC
Reel/Frame 064205/0676 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2023
From: SOUNDHOUND, INC.
To: SOUNDHOUND AI IP HOLDING, LLC
Reel/Frame 064083/0484 →
SECURITY INTEREST Recorded Apr 17, 2023
From: SOUNDHOUND, INC.; SOUNDHOUND AI IP, LLC
To: ACP POST OAK CREDIT II LLC
Reel/Frame 063349/0355 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2022
From: MOHAJER, KEYVAN; KAM, KAISHIN; PIERRET, CHRISTOPHE
To: SOUNDHOUND, INC.
Reel/Frame 059223/0313 →
Continuity (1)
Related Publication 20230206915A1 · Jun 29, 2023
References Cited (6)
US 20060074684A1 · Yoshida · 2006 [cited by examiner]
US 20190378515A1 · Kim · 2019 [cited by examiner]
US 20190392826A1 · Lee · 2019 [cited by examiner]
US 20190392827A1 · Park · 2019 [cited by examiner]
US 20200005778A1 · Kim · 2020 [cited by examiner]
US 20200013396A1 · Park · 2020 [cited by examiner]