Granted Patent B2

US 12,651,020 · App. 19/084,647 · Granted Jun 9, 2026

Inventors: Akshay Aggarwal (San Jose, CA); Shruti Bhargava (Seattle, WA); Karan M. Daryanani (San Francisco, CA); Saiyam Kohli (Cupertino, CA); Christine S. O'Mara (Cary, NC); Lewis N. Perkins (Cambridge, MA); Nidhi Rajshree (Los Gatos, CA); Chiraag Sumanth (San Francisco, CA); Nicholas L. Tzou (Sunnyvale, CA); Hong Yu (Santa Clara, CA)

Assignee: Apple Inc.

G06F16/632G06F16/24575G06F16/638G06N3/006

Abstract

Claims (133)

1 . A computer system configured to communicate with a display generation component and an input device comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

detecting a query;

in response to detecting the query:

retrieving contextual data related to the query;

in accordance with a determination that the query includes a request of a first type, wherein determining that the query includes the request of the first type includes determining that the query does not include a complex knowledge seeking intent:

converting the query to a rewritten query based on the contextual data related to the query; and

providing the rewritten query to a first digital assistant component; and

in accordance with a determination that the query includes a request of a second type different from the request of the first type, wherein determining that the query includes the request of the second type includes determining that the query includes a complex knowledge seeking intent;

providing the query and the contextual data related to the query to a second digital assistant component different from the first digital assistant component; and

generating, using a large language model of the second digital assistant component, a prompt with a format for a large language model of a third digital assistant component that is different from the second digital assistant component and the first digital assistant component, wherein the prompt is generated based on the query and the contextual data and includes a plurality of possible application intents and a plurality of possible entities for performing the complex knowledge seeking intent.

2 . The computer system of claim 1 , wherein the contextual data related to the query includes a set of possible entities available to the computer system that match the query.

3 . The computer system of claim 2 , wherein the set of possible entities includes an entity being displayed via a display generation component of the computer system.

4 . The computer system of claim 1 , wherein the contextual data includes an application intent related to the query.

5 . The computer system of claim 4 , the one or more programs further including instructions for:

selecting the application intent from a list of application intents associated with an application included in the query.

6 . The computer system of claim 4 , the one or more programs further including instructions for:

selecting the application intent from a list of application intents associated with an application displayed via a display generation component of the computer system.

7 . The computer system of claim 1 , the one or more programs further including instructions for:

determining a semantic comparison between the query and a set of possible application intents; and

selecting the application intent related to the query from the set of possible application intents.

8 . The computer system of claim 7 , wherein determining the semantic comparison between the query and the set of possible application intents includes:

comparing the query to a set of descriptions corresponding to the set of possible application intents; and

comparing the query to a set of examples corresponding to the set of possible application intents.

9 . The computer system of claim 1 , wherein the contextual data includes conversational history between a user and a digital assistant.

10 . The computer system of claim 1 , wherein determining that the query includes the request of the first type further comprises:

determining whether a domain of the query corresponds to a domain associated with the request of the first type.

11 . The computer system of claim 1 , wherein determining that the query includes the request of the first type further comprises:

determining a type of contextual data required to respond to the query; and

determining whether the type of contextual data required to respond to the query is a type of contextual data for the first digital assistant component.

12 . The computer system of claim 1 , wherein determining that the query includes the request of the first type further comprises:

determining a task specified in the query; and

in accordance with a determination that the task specified in the query is a task associated with the first digital assistant component, determining that the query includes the request of the first type.

13 . The computer system of claim 1 , wherein determining that the query includes the request of the first type further comprises:

in accordance with a determination that the query specifies the first digital assistant component, determining that the query includes the request of the first type.

14 . The computer system of claim 1 , wherein converting the query to the rewritten query based on the contextual data related to the query further comprises:

adding information from a subsequently received query to the query to create the rewritten query.

15 . The computer system of claim 1 , the one or more programs further including instructions for:

after providing the rewritten query to the first digital assistant component:

causing the first digital assistant component to perform a task determined from the rewritten query; and

providing an output of the task.

16 . The computer system of claim 1 , the one or more programs further including instructions for:

in response to detecting the query and after retrieving contextual data related to the query:

separating the query into a first query and a second query.

17 . The computer system of claim 16 , where in the first query is determined based on a first domain associated with a first request of the query and the second query is determined based on a second domain associated with a second request of the query.

18 . The computer system of claim 16 , the one or more programs further including instructions for:

after separating the query into the first query and the second query:

in accordance with a determination that the first query includes the request of the first type:

converting the first query to the rewritten query based on the contextual data related to the query; and

providing the rewritten query to the first digital assistant component; and

 in accordance with a determination that the first query includes the request of the second type different from the request of the first type, providing the first query and the contextual data related to the query to the second digital assistant component different from the first digital assistant component.

19 . The computer system of claim 16 , the one or more programs further including instructions for:

after separating the query into the first query and the second query:

providing the first query to the first digital assistant component; and

providing the second query to the second digital assistant component different from the first digital assistant component.

20 . The computer system of claim 19 , the one or more programs further including instructions for:

causing the first digital assistant component to perform a first task determined from the first query;

generating, using the large language model of the second digital assistant component, a prompt with a format for the large language model of a third digital assistant component that is different from the second digital assistant component and the first digital assistant component;

causing the third digital assistant component to perform a second task determined from the second query using the prompt; and

providing an output including results of the first task and the second task.

21 . The computer system of claim 1 , wherein the query with the complex knowledge seeking intent includes a query that requires an output of a first task determined from the request to be used as an input of a second task determined from the request.

22 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component and an input device, the one or more programs including instructions for:

detecting a query;

in response to detecting the query:

retrieving contextual data related to the query;

in accordance with a determination that the query includes a request of a first type, wherein determining that the query includes the request of the first type includes determining that the query does not include a complex knowledge seeking intent:

converting the query to a rewritten query based on the contextual data related to the query; and

providing the rewritten query to a first digital assistant component; and

in accordance with a determination that the query includes a request of a second type different from the request of the first type, wherein determining that the query includes the request of the second type includes determining that the query includes a complex knowledge seeking intent;

providing the query and the contextual data related to the query to a second digital assistant component different from the first digital assistant component; and

generating, using a large language model of the second digital assistant component, a prompt with a format for a large language model of a third digital assistant component that is different from the second digital assistant component and the first digital assistant component, wherein the prompt is generated based on the query and the contextual data and includes a plurality of possible application intents and a plurality of possible entities for performing the complex knowledge seeking intent.

23 . The non-transitory computer-readable subject matter of claim 22 , wherein the contextual data related to the query includes a set of possible entities available to the computer system that match the query.

24 . The non-transitory computer-readable subject matter of claim 22 , wherein the contextual data includes an application intent related to the query.

25 . The non-transitory computer-readable subject matter of claim 22 , the one or more programs further including instructions for:

determining a semantic comparison between the query and a set of possible application intents; and

selecting the application intent related to the query from the set of possible application intents.

26 . The non-transitory computer-readable subject matter of claim 22 , wherein the contextual data includes conversational history between a user and a digital assistant.

27 . The non-transitory computer-readable subject matter of claim 22 , wherein determining that the query includes the request of the first type further comprises:

determining whether a domain of the query corresponds to a domain associated with the request of the first type.

28 . The non-transitory computer-readable subject matter of claim 22 , wherein determining that the query includes the request of the first type further comprises:

determining a type of contextual data required to respond to the query; and

determining whether the type of contextual data required to respond to the query is a type of contextual data for the first digital assistant component.

29 . The non-transitory computer-readable subject matter of claim 22 , wherein determining that the query includes the request of the first type further comprises:

determining a task specified in the query; and

in accordance with a determination that the task specified in the query is a task associated with the first digital assistant component, determining that the query includes the request of the first type.

30 . The non-transitory computer-readable subject matter of claim 22 , wherein determining that the query includes the request of the first type further comprises:

in accordance with a determination that the query specifies the first digital assistant component, determining that the query includes the request of the first type.

31 . The non-transitory computer-readable subject matter of claim 22 , wherein converting the query to the rewritten query based on the contextual data related to the query further comprises:

adding information from a subsequently received query to the query to create the rewritten query.

32 . The non-transitory computer-readable subject matter of claim 22 , the one or more programs further including instructions for:

after providing the rewritten query to the first digital assistant component:

causing the first digital assistant component to perform a task determined from the rewritten query; and

providing an output of the task.

33 . The non-transitory computer-readable subject matter of claim 22 , the one or more programs further including instructions for:

in response to detecting the query and after retrieving contextual data related to the query:

separating the query into a first query and a second query.

34 . A method comprising:

at a computer system that is configured to communicate with a display generation component and an input device:

detecting a query;

in response to detecting the query:

retrieving contextual data related to the query;

in accordance with a determination that the query includes a request of a first type, wherein determining that the query includes the request of the first type includes determining that the query does not include a complex knowledge seeking intent:

converting the query to a rewritten query based on the contextual data related to the query; and

providing the rewritten query to a first digital assistant component; and

in accordance with a determination that the query includes a request of a second type different from the request of the first type, wherein determining that the query includes the request of the second type includes determining that the query includes a complex knowledge seeking intent;

providing the query and the contextual data related to the query to a second digital assistant component different from the first digital assistant component; and

generating, using a large language model of the second digital assistant component, a prompt with a format for a large language model of a third digital assistant component that is different from the second digital assistant component and the first digital assistant component, wherein the prompt is generated based on the query and the contextual data and includes a plurality of possible application intents and a plurality of possible entities for performing the complex knowledge seeking intent.

35 . The method of claim 34 , wherein the contextual data related to the query includes a set of possible entities available to the computer system that match the query.

36 . The method of claim 34 , wherein the contextual data includes an application intent related to the query.

37 . The method of claim 34 , further comprising:

determining a semantic comparison between the query and a set of possible application intents; and

selecting the application intent related to the query from the set of possible application intents.

38 . The method of claim 34 , wherein the contextual data includes conversational history between a user and a digital assistant.

39 . The method of claim 34 , wherein determining that the query includes the request of the first type further comprises:

determining whether a domain of the query corresponds to a domain associated with the request of the first type.

40 . The method of claim 34 , wherein determining that the query includes the request of the first type further comprises:

determining a type of contextual data required to respond to the query; and

determining whether the type of contextual data required to respond to the query is a type of contextual data for the first digital assistant component.

41 . The method of claim 34 , wherein determining that the query includes the request of the first type further comprises:

determining a task specified in the query; and

in accordance with a determination that the task specified in the query is a task associated with the first digital assistant component, determining that the query includes the request of the first type.

42 . The method of claim 34 , wherein determining that the query includes the request of the first type further comprises:

in accordance with a determination that the query specifies the first digital assistant component, determining that the query includes the request of the first type.

43 . The method of claim 34 , wherein converting the query to the rewritten query based on the contextual data related to the query further comprises:

adding information from a subsequently received query to the query to create the rewritten query.

44 . The method of claim 34 , further comprising:

after providing the rewritten query to the first digital assistant component:

causing the first digital assistant component to perform a task determined from the rewritten query; and

providing an output of the task.

45 . The method of claim 23 , further comprising:

in response to detecting the query and after retrieving contextual data related to the query:

separating the query into a first query and a second query.

Assignments (1)

Continuity (5)

References Cited (34)