IP Library › Granted Patent US 12,277,935
Granted Patent B2
US 12,277,935 · App. 18/607,239 · Granted Apr 15, 2025

Accessory device based authentication for digital assistant requests

Inventors: Ryan D. Webber (Vancouver, CA); Trungtin Tran (Sunnyvale, CA); Nikrouz Ghotbi (Seattle, WA); Naresh Kumar Chinnathambi Kailasam (San Jose, CA); Sue Wen Chiao (Redwood City, CA); Tyler C. Leppek (Santa Cruz, CA); Brandon J. Newendorp (San Francisco, CA)
Assignee: Apple Inc.
G10L15/22G10L15/18G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,277,935
App. No.
18/607,239
Granted
Apr 15, 2025
Kind
B2
Abstract

An example process includes at an electronic device with one or more processors and memory: while the electronic device is in a locked state and in communication with an external accessory device: receiving a natural language input; determining whether the natural language input corresponds to a secure task request; in accordance with a determination that the natural language input corresponds to a secure task request: determining whether one or more states of the external accessory device satisfy a set of authentication criteria; and in accordance with a determination that the one or more states of the external accessory device satisfy the set of authentication criteria: initiating, by the electronic device, a secure task corresponding to the natural language input; and providing an output indicative of the initiated secure task.

Claims (109)

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an accessory device that is in communication with an external device, cause the accessory device to:

receive, from the external device, a request for information about one or more states of the accessory device, wherein the request for information is received while the external device is in a locked state and in accordance with a determination that the external device received a natural language input corresponding to a secure task request; and

in response to receiving the request for information:

in accordance with a determination that the one or more states of the accessory device satisfy a set of authentication criteria, transmit, to the external device, an indication that the one or more states of the accessory device satisfies the set of authentication criteria, wherein the external device initiates the secure task corresponding to the natural language input in response to receiving the indication that the one or more states of the accessory device satisfies the set of authentication criteria.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

in accordance with a determination that the one or more states of the accessory device do not satisfy the set of authentication criteria:

transmit, to the external device, a second indication that the one or more states of the accessory device do not satisfy the set of authentication criteria, wherein the external device forgoes initiating the secure task corresponding to the natural language input; and

provide an output indicating that the external device is unable to perform the secure task.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

receive, from the external device, a second request for information about a second set of one or more states of the accessory device, wherein the second request for information is received while the external device is in the locked state and in accordance with a determination that satisfying a task corresponding to a second natural language input includes providing a secure output and that the external device received the second natural language input that does not correspond to a secure task request;

in response to receiving the second request for information:

in accordance with a determination that the second set of one or more states of the accessory device satisfy the set of authentication criteria, transmit, to the external device, a third indication that the second set of one or more states of the accessory device satisfies the set of authentication criteria, wherein the external device initiates the task corresponding to the second natural language input in response to receiving the third indication that the second set of one or more states of the accessory device satisfies the set of authentication criteria; and

in accordance with a determination that the second set of one or more states of the accessory device do not satisfy the set of authentication criteria, transmit, to the external device, a fourth indication that the second set of one or more states of the accessory device do not satisfy the set of authentication criteria, wherein the external device forgoes initiating the task corresponding to the second natural language input.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

receive, from the external device, a request for authentication, wherein the request for authentication is received while the external device is in the locked state and in accordance with a determination that the external device received a third natural language input corresponding to a second secure task request;

in response to receiving the request for authentication:

in accordance with a determination that a third set of one or more states of the accessory device satisfy the set of authentication criteria, transmit, to the external device, a fifth indication that the third set of one or more states of the accessory device satisfies the set of authentication criteria; and

in accordance with a determination that the third set of one or more states of the accessory device do not satisfy the set of authentication criteria, transmit, to the external device, a sixth indication that the third set of one or more states of the accessory device do not satisfy the set of authentication criteria.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

in accordance with a determination that the one or more states of the accessory device satisfy the set of authentication criteria, provide a second output indicating the initiation of the secure task by the external device.

6. The non-transitory computer-readable storage medium of claim 5 , wherein providing the second output includes indicating that the accessory device authenticated the external device.

7. The non-transitory computer-readable storage medium of claim 5 , wherein the second output includes a prompt for user cancellation of the initiated secure task, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

detect selection of the prompt; and

in accordance with detecting selection of the prompt, transmit, to the external device, a seventh indication of selection of the prompt, wherein the external device terminates the secure task in response to receiving the seventh indication of the selection of the prompt.

8. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

prior to receiving the natural language input, receive a user input corresponding to a request to initiate a digital assistant, wherein the user input is received while the external device is in the locked state; and

in response to receiving the user input, initiate a digital assistant session, wherein the natural language input is received in accordance with initiating the digital assistant session.

9. The non-transitory computer-readable storage medium of claim 8 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the accessory device to:

while the accessory device is in the locked state:

after transmitting, to the external device, the indication that the one or more states of the accessory device satisfies the set of authentication criteria, dismiss the initiated digital assistant session;

after dismissing the initiated digital assistant session, receive a third request for information about one or more states of the accessory device, wherein the third request for information is received while the external device is in a locked state and in accordance with a determination that the external device received a fourth natural language input corresponding to a third secure task request.

10. The non-transitory computer-readable storage medium of claim 1 , wherein satisfying the set of authentication criteria requires the accessory device to be physically associated with a user of the external device.

11. The non-transitory computer-readable storage medium of claim 1 , wherein satisfying the set of authentication criteria requires the accessory device to be in an unlocked state.

12. The non-transitory computer-readable storage medium of claim 1 , wherein satisfying the set of authentication criteria requires the accessory device to be within a predetermined distance of the external device.

13. The non-transitory computer-readable storage medium of claim 1 , wherein satisfying the set of authentication criteria requires that movement of the accessory device corresponds to a predetermined type of movement.

14. An accessory device configured to communicate with an external device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving, from the external device, a request for information about one or more states of the accessory device, wherein the request for information is received while the external device is in a locked state and in accordance with a determination that the external device received a natural language input corresponding to a secure task request; and

in response to receiving the request for information:

in accordance with a determination that the one or more states of the accessory device satisfy a set of authentication criteria, transmitting, to the external device, an indication that the one or more states of the accessory device satisfies the set of authentication criteria, wherein the external device initiates the secure task corresponding to the natural language input in response to receiving the indication that the one or more states of the accessory device satisfies the set of authentication criteria.

15. The accessory device configured to communicate with the external device of claim 14 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

in accordance with a determination that the one or more states of the accessory device do not satisfy the set of authentication criteria:

transmitting, to the external device, a second indication that the one or more states of the accessory device do not satisfy the set of authentication criteria, wherein the external device forgoes initiating the secure task corresponding to the natural language input; and

providing an output indicating that the external device is unable to perform the secure task.

16. The accessory device configured to communicate with the external device of claim 14 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

receiving, from the external device, a second request for information about a second set of one or more states of the accessory device, wherein the second request for information is received while the external device is in the locked state and in accordance with a determination that satisfying a task corresponding to a second natural language input includes providing a secure output and that the external device received the second natural language input that does not correspond to a secure task request;

in response to receiving the second request for information:

in accordance with a determination that the second set of one or more states of the accessory device satisfy the set of authentication criteria, transmitting, to the external device, a third indication that the second set of one or more states of the accessory device satisfies the set of authentication criteria, wherein the external device initiates the task corresponding to the second natural language input in response to receiving the third indication that the second set of one or more states of the accessory device satisfies the set of authentication criteria; and

in accordance with a determination that the second set of one or more states of the accessory device do not satisfy the set of authentication criteria, transmitting, to the external device, a fourth indication that the second set of one or more states of the accessory device do not satisfy the set of authentication criteria, wherein the external device forgoes initiating the task corresponding to the second natural language input.

17. The accessory device configured to communicate with the external device of claim 14 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

receiving, from the external device, a request for authentication, wherein the request for authentication is received while the external device is in the locked state and in accordance with a determination that the external device received a third natural language input corresponding to a second secure task request;

in response to receiving the request for authentication:

in accordance with a determination that a third set of one or more states of the accessory device satisfy the set of authentication criteria, transmitting, to the external device, a fifth indication that the third set of one or more states of the accessory device satisfies the set of authentication criteria; and

in accordance with a determination that the third set of one or more states of the accessory device do not satisfy the set of authentication criteria, transmitting, to the external device, a sixth indication that the third set of one or more states of the accessory device do not satisfy the set of authentication criteria.

18. The accessory device configured to communicate with the external device of claim 14 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

in accordance with a determination that the one or more states of the accessory device satisfy the set of authentication criteria, providing a second output indicating the initiation of the secure task by the external device.

19. The accessory device configured to communicate with the external device of claim 18 , wherein providing the second output includes indicating that the accessory device authenticated the external device.

20. The accessory device configured to communicate with the external device of claim 18 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

detecting selection of the prompt; and

in accordance with detecting selection of the prompt, transmitting, to the external device, a seventh indication of selection of the prompt, wherein the external device terminates the secure task in response to receiving the seventh indication of the selection of the prompt.

21. The accessory device configured to communicate with the external device of claim 14 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

prior to receiving the natural language input, receiving a user input corresponding to a request to initiate a digital assistant, wherein the user input is received while the external device is in the locked state; and

in response to receiving the user input, initiating a digital assistant session, wherein the natural language input is received in accordance with initiating the digital assistant session.

22. The accessory device configured to communicate with the external device of claim 21 , wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs further including instructions for:

while the accessory device is in the locked state:

after transmitting, to the external device, the indication that the one or more states of the accessory device satisfies the set of authentication criteria, dismissing the initiated digital assistant session;

after dismissing the initiated digital assistant session, receiving a third request for information about one or more states of the accessory device, wherein the third request for information is received while the external device is in a locked state and in accordance with a determination that the external device received a fourth natural language input corresponding to a third secure task request.

23. The accessory device configured to communicate with the external device of claim 14 , wherein satisfying the set of authentication criteria requires the accessory device to be physically associated with a user of the external device.

24. The accessory device configured to communicate with the external device of claim 14 , wherein satisfying the set of authentication criteria requires the accessory device to be in an unlocked state.

25. The accessory device configured to communicate with the external device of claim 14 , wherein satisfying the set of authentication criteria requires the accessory device to be within a predetermined distance of the external device.

26. The accessory device configured to communicate with the external device of claim 14 , wherein satisfying the set of authentication criteria requires that movement of the accessory device corresponds to a predetermined type of movement.

27. A method, comprising:

at an accessory device that is in communication with an external device with one or more processors and memory:

receiving, from the external device, a request for information about one or more states of the accessory device, wherein the request for information is received while the external device is in a locked state and in accordance with a determination that the external device received a natural language input corresponding to a secure task request; and

in response to receiving the request for information:

in accordance with a determination that the one or more states of the accessory device satisfy a set of authentication criteria, transmitting, to the external device, an indication that the one or more states of the accessory device satisfies the set of authentication criteria, wherein the external device initiates the secure task corresponding to the natural language input in response to receiving the indication that the one or more states of the accessory device satisfies the set of authentication criteria.

28. The method of claim 27 , further comprising:

in accordance with a determination that the one or more states of the accessory device do not satisfy the set of authentication criteria:

transmitting, to the external device, a second indication that the one or more states of the accessory device do not satisfy the set of authentication criteria, wherein the external device forgoes initiating the secure task corresponding to the natural language input; and

providing an output indicating that the external device is unable to perform the secure task.

29. The method of claim 27 , further comprising:

receiving, from the external device, a second request for information about a second set of one or more states of the accessory device, wherein the second request for information is received while the external device is in the locked state and in accordance with a determination that satisfying a task corresponding to a second natural language input includes providing a secure output and that the external device received the second natural language input that does not correspond to a secure task request;

in response to receiving the second request for information:

in accordance with a determination that the second set of one or more states of the accessory device satisfy the set of authentication criteria, transmitting, to the external device, a third indication that the second set of one or more states of the accessory device satisfies the set of authentication criteria, wherein the external device initiates the task corresponding to the second natural language input in response to receiving the third indication that the second set of one or more states of the accessory device satisfies the set of authentication criteria; and

in accordance with a determination that the second set of one or more states of the accessory device do not satisfy the set of authentication criteria, transmitting, to the external device, a fourth indication that the second set of one or more states of the accessory device do not satisfy the set of authentication criteria, wherein the external device forgoes initiating the task corresponding to the second natural language input.

30. The method of claim 27 , further comprising:

receiving, from the external device, a request for authentication, wherein the request for authentication is received while the external device is in the locked state and in accordance with a determination that the external device received a third natural language input corresponding to a second secure task request;

in response to receiving the request for authentication:

in accordance with a determination that a third set of one or more states of the accessory device satisfy the set of authentication criteria, transmitting, to the external device, a fifth indication that the third set of one or more states of the accessory device satisfies the set of authentication criteria; and

in accordance with a determination that the third set of one or more states of the accessory device do not satisfy the set of authentication criteria, transmitting, to the external device, a sixth indication that the third set of one or more states of the accessory device do not satisfy the set of authentication criteria.

31. The method of claim 27 , further comprising:

in accordance with a determination that the one or more states of the accessory device satisfy the set of authentication criteria, providing a second output indicating the initiation of the secure task by the external device.

32. The method of claim 31 , wherein providing the second output includes indicating that the accessory device authenticated the external device.

33. The method of claim 31 , wherein the second output includes a prompt for user cancellation of the initiated secure task, wherein the method further comprises

detecting selection of the prompt; and

in accordance with detecting selection of the prompt, transmitting, to the external device, a seventh indication of selection of the prompt, wherein the external device terminates the secure task in response to receiving the seventh indication of the selection of the prompt.

34. The method of claim 27 , further comprising:

prior to receiving the natural language input, receiving a user input corresponding to a request to initiate a digital assistant, wherein the user input is received while the external device is in the locked state; and

in response to receiving the user input, initiating a digital assistant session, wherein the natural language input is received in accordance with initiating the digital assistant session.

35. The method of claim 34 , further comprising:

while the accessory device is in the locked state:

after transmitting, to the external device, the indication that the one or more states of the accessory device satisfies the set of authentication criteria, dismissing the initiated digital assistant session;

after dismissing the initiated digital assistant session, receiving a third request for information about one or more states of the accessory device, wherein the third request for information is received while the external device is in a locked state and in accordance with a determination that the external device received a fourth natural language input corresponding to a third secure task request.

36. The method of claim 27 , wherein satisfying the set of authentication criteria requires the accessory device to be physically associated with a user of the external device.

37. The method of claim 27 , wherein satisfying the set of authentication criteria requires the accessory device to be in an unlocked state.

38. The method of claim 27 , wherein satisfying the set of authentication criteria requires the accessory device to be within a predetermined distance of the external device.

39. The method of claim 27 , wherein satisfying the set of authentication criteria requires that movement of the accessory device corresponds to a predetermined type of movement.

Continuity (3)
Continuation 17486670 · Sep 27, 2021
Provisional Application 63196842 · Jun 4, 2021
Related Publication 20240221746A1 · Jul 4, 2024
References Cited (66)
US 6677932B1 · Westerman · 2004 [cited by applicant]
US 9098931B2 · Shpunt et al. · 2015 [cited by applicant]
US 10102359B2 · Cheyer · 2018 [cited by applicant]
US 10127926B2 · James · 2018 [cited by examiner]
US 10185542B2 · Carson et al. · 2019 [cited by applicant]
US 10440574B2 · Ledvina et al. · 2019 [cited by applicant]
US 10496705B1 · Irani et al. · 2019 [cited by applicant]
US 10877637B1 · Antos · 2020 [cited by examiner]
US 20050190059A1 · Wehrenberg · 2005 [cited by applicant]
US 20060017692A1 · Wehrenberg et al. · 2006 [cited by applicant]
US 20060161870A1 · Hotelling et al. · 2006 [cited by applicant]
US 20060161871A1 · Hotelling et al. · 2006 [cited by applicant]
US 20070075965A1 · Huppi et al. · 2007 [cited by applicant]
US 20080140868A1 · Kalayjian et al. · 2008 [cited by applicant]
US 20080167834A1 · Herz et al. · 2008 [cited by applicant]
US 20080320391A1 · Lemay et al. · 2008 [cited by applicant]
US 20100159995A1 · Stallings et al. · 2010 [cited by applicant]
US 20110081889A1 · Gao et al. · 2011 [cited by applicant]
US 20110314539A1 · Horton · 2011 [cited by applicant]
US 20120015629A1 · Olsen et al. · 2012 [cited by applicant]
US 20130127854A1 · Shpunt et al. · 2013 [cited by applicant]
US 20130244615A1 · Miller · 2013 [cited by applicant]
US 20130332172A1 · Prakash et al. · 2013 [cited by applicant]
US 20140068751A1 · Last · 2014 [cited by applicant]
US 20140330560A1 · Venkatesha et al. · 2014 [cited by applicant]
US 20140337621A1 · Nakhimov · 2014 [cited by applicant]
US 20150341717A1 · Song et al. · 2015 [cited by applicant]
US 20150362977A1 · Doniwa · 2015 [cited by applicant]
US 20150363587A1 · Ahn et al. · 2015 [cited by applicant]
US 20160025993A1 · Mor et al. · 2016 [cited by applicant]
US 20160048666A1 · Dey et al. · 2016 [cited by applicant]
US 20160226713A1 · Dellinger · 2016 [cited by examiner]
US 20160294837A1 · Turgeman · 2016 [cited by applicant]
US 20170032168A1 · Kim · 2017 [cited by applicant]
US 20170039358A1 · Yuen et al. · 2017 [cited by applicant]
US 20170140644A1 · Hwang et al. · 2017 [cited by applicant]
US 20170357788A1 · Ledvina et al. · 2017 [cited by applicant]
US 20180247065A1 · Rhee et al. · 2018 [cited by applicant]
US 20190053739A1 · Inoue et al. · 2019 [cited by applicant]
US 20190394649A1 · Ledvina et al. · 2019 [cited by applicant]
US 20200159609A1 · Korotaev · 2020 [cited by examiner]
US 20200286472A1 · Newendorp et al. · 2020 [cited by applicant]
US 20200367827A1 · Min et al. · 2020 [cited by applicant]
US 20210012776A1 · Peterson et al. · 2021 [cited by applicant]
US 20210092128A1 · Leblang et al. · 2021 [cited by applicant]
US 20210210100A1 · Wang · 2021 [cited by examiner]
US 20220237274A1 · Paul · 2022 [cited by examiner]
US 20220375553A1 · Lasko · 2022 [cited by examiner]
US 20220392446A1 · Webber et al. · 2022 [cited by applicant]
KR 1020170006592A · 2017 [cited by applicant]
WO 2019033129A2 · 2019 [cited by applicant]
Corrected Notice of Allowance received for U.S. Appl. No. 17/486,670, mailed on Dec. 7, 2023, 2 pages. [cited by applicant]
International Preliminary Report on Patentability received for PCT Patent Application No. PCT/US2022/029062, mailed on Dec. 14, 2023, 9 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/486,670, mailed on Nov. 8, 2023, 11 pages. [cited by applicant]
Supplemental Notice of Allowance received for U.S. Appl. No. 17/486,670, mailed on Jan. 18, 2024, 9 pages. [cited by applicant]
MJ Tube, “How to Use Ok Google Assistant Even Phone is Locked”, Available online at:-https://www.youtube.com/watch?v=9B_gP4j_SP8, Mar. 12, 2018, 1 page. [cited by applicant]
Corrected Notice of Allowance received for U.S. Appl. No. 17/587,558, mailed on Apr. 16, 2024, 2 pages. [cited by applicant]
Extended European Search Report received for European Patent Application No. 22173658.0, mailed on Oct. 31, 2022, 6 pages. [cited by applicant]
International Preliminary Report on Patentability received for PCT Patent Application No. PCT/US2022/013730, mailed on Aug. 3, 2023, 15 pages. [cited by applicant]
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US2022/013730, mailed on Apr. 8, 2022, 19 pages. [cited by applicant]
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US2022/029062, mailed on Aug. 16, 2022, 12 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/587,558, mailed on Apr. 3, 2024, 19 pages. [cited by applicant]
Schürmann et al., “BANDANA—Body Area Network Device-to-Device Authentication Using Natural gAit”, Ambient Intelligence, Comnet, Aalto University, DOI: 10.1109/PERCOM.2017.7917865, Dec. 11, 2016, 11 pages. [cited by applicant]
Weiss et al., “Smartphone and Smartwatch-Based Biometrics using Activities of Daily Living”, IEEE Access, DOI: 10.1109/ACCESS.2019.2940729, vol. XX, 2017, 13 pages. [cited by applicant]
Zhang et al., “WristUnlock: Secure and Usable Smartphone Unlocking with Wrist Wearables”, IEEE Conference on Communications and Network Security (CNS), 2019, 9 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/587,558, mailed on Aug. 5, 2024, 10 pages. [cited by applicant]