IP Library Granted Patent US 11,189,280
Granted Patent B2
US 11,189,280 · App. 16/430,899 · Granted Nov 30, 2021

Securing voice assistant data

Inventors: Ramanandan Nambannor Kunnath (Bangalore, IN); Ramani Panchapakesan (Bangalore, IN)
Assignee: VMware, Inc.
G10L15/22G10L15/30H04L63/0442G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,189,280
App. No.
16/430,899
Granted
Nov 30, 2021
Kind
B2
Abstract

Disclosed are various approaches for encrypting a voice based response to a request through a voice assistant device. The request is associated with a network or federated service. A tunnel speaker containing a private key can decrypt the encrypted response using a private key provided to the tunnel speaker by a client application on a client device.

Claims (60)

1. A tunnel speaker for securing audio playback through a voice assistant device, comprising:

at least one processor;

a network interface for communicating with a client device;

at least one speaker configured to play back audio received from a voice assistant device having an integrated speaker, wherein the tunnel speaker is paired with the voice assistant device via a Bluetooth connection;

a storage system; and

a tunnel speaker application that, when executed by the at least one processor, causes the tunnel speaker to:

receive a request to link the tunnel speaker with a user account associated with the client device;

obtain a client private key from the client device, the client private key corresponding to a client public key, wherein the client device provides the client public key to a remotely executed assistant connection service that generates encrypted audio using the client public key;

store the client private key into an encrypted storage resource within the storage system without providing the client private key to the voice assistant device;

obtain encrypted audio from the voice assistant device with which the tunnel speaker is paired;

decrypt the encrypted audio using the client private key, wherein the voice assistant device is unable to decrypt the encrypted audio; and

causing the decrypted audio to be played back using the at least one speaker.

2. The system of claim 1 , wherein the tunnel speaker is paired to the voice assistant device as a peripheral speaker through which audio received by the voice assistant device is played, wherein the tunnel speaker is configured to play back encrypted audio and unencrypted audio on behalf of the voice assistant device.

3. The system of claim 1 , wherein the network interface for communicating with the client device further comprises a Bluetooth interface.

4. The tunnel speaker of claim 1 , wherein the instructions, when executed by the at least one processor, cause the tunnel speaker to obtain the client private key from the client device by further causing the tunnel speaker to at least:

send a tunnel speaker public key to the client device in response to the request to link with the client device, the tunnel speaker public key corresponding to a tunnel speaker private key stored in the storage system;

obtain an encrypted client private key that is encrypted using the tunnel speaker public key; and

decrypt the client private key using the tunnel speaker private key.

5. The tunnel speaker of claim 1 , wherein the instructions, when executed by the at least one processor, cause the tunnel speaker to at least:

detect the encrypted audio from the voice assistant device based upon a header associated with the encrypted audio.

6. The system of claim 1 , wherein the encrypted audio corresponds to enterprise data associated with a user account linked to the user account that is obtained by the assistant connection service on behalf of the user account.

7. The tunnel speaker of claim 1 , wherein the instructions, when executed by the at least one processor, cause the computing device to at least:

receive a command to terminate the link between the user account and the tunnel speaker; and

delete the client private key in response to receiving the command to terminate the link.

8. A non-transitory computer-readable medium comprising machine-readable instructions, wherein the instructions, when executed by at least one processor, cause a tunnel speaker to:

receive a request to link the tunnel speaker with a user account associated with the client device, the tunnel speaker paired with a voice assistant device having an integrated speaker via a Bluetooth connection;

obtain a client private key from the client device, the client private key corresponding to a client public key, wherein the client device provides the client public key to a remotely executed assistant connection service that generates encrypted audio using the client public key;

store the client private key into an encrypted storage resource within the storage system without providing the client private key to the voice assistant device;

obtain encrypted audio from the voice assistant device with which the tunnel speaker is paired;

decrypt the encrypted audio using the client private key, wherein the voice assistant device is unable to decrypt the encrypted audio; and

causing the decrypted audio to be played back using the at least one speaker.

9. The non-transitory computer-readable medium of claim 8 , wherein the tunnel speaker is paired to the voice assistant device as a peripheral speaker through which audio received by the voice assistant device is played, wherein the tunnel speaker is configured to play back encrypted audio and unencrypted audio on behalf of the voice assistant device.

10. The non-transitory computer-readable medium of claim 8 , wherein the network interface for communicating with the client device further comprises a Bluetooth interface.

11. The non-transitory computer-readable medium of claim 8 , wherein the instructions, when executed by the at least one processor, cause the tunnel speaker to at least:

send a tunnel speaker public key to the client device in response to the request to link with the client device, the tunnel speaker public key corresponding to a tunnel speaker private key stored in the storage system;

obtain an encrypted client private key that is encrypted using the tunnel speaker public key; and

decrypt the client private key using the tunnel speaker private key.

12. The non-transitory computer-readable medium of claim 8 , wherein the instructions, when executed by the at least one processor, cause the tunnel speaker to at least:

detect the encrypted audio from the voice assistant device based upon a header associated with the encrypted audio.

13. The non-transitory computer-readable medium of claim 8 , wherein the encrypted audio corresponds to enterprise data associated with a user account linked to the user account that is obtained by the assistant connection service on behalf of the user account.

14. The non-transitory computer-readable medium of claim 8 , wherein the instructions, when executed by the at least one processor, cause the tunnel speaker to at least:

receive a command to terminate the link between the user account and the tunnel speaker; and

delete the client private key in response to receiving the command to terminate the link.

15. A method comprising:

receiving a request to link a tunnel speaker with a user account associated with the client device, the tunnel speaker paired with a voice assistant device having an integrated speaker via a Bluetooth connection;

obtaining a client private key from the client device, the client private key corresponding to a client public key, wherein the client device provides the client public key to a remotely executed assistant connection service that generates encrypted audio using the client public key;

storing the client private key into an encrypted storage resource within the storage system without providing the client private key to the voice assistant device;

obtaining encrypted audio from the voice assistant device with which the tunnel speaker is paired;

decrypting the encrypted audio using the client private key, wherein the voice assistant device is unable to decrypt the encrypted audio; and

causing the decrypted audio to be played back using the at least one speaker.

16. The method of claim 15 , wherein the tunnel speaker is paired to the voice assistant device as a peripheral speaker through which audio received by the voice assistant device is played, wherein the tunnel speaker is configured to play back encrypted audio and unencrypted audio on behalf of the voice assistant device.

17. The method of claim 15 , further comprising:

sending a tunnel speaker public key to the client device in response to the request to link with the client device, the tunnel speaker public key corresponding to a tunnel speaker private key stored in the storage system;

obtaining an encrypted client private key that is encrypted using the tunnel speaker public key; and

decrypting the client private key using the tunnel speaker private key.

18. The method of claim 15 , further comprising detecting the encrypted audio from the voice assistant device based upon a header associated with the encrypted audio.

19. The method of claim 15 , wherein the encrypted audio corresponds to enterprise data associated with a user account linked to the user account that is obtained by the assistant connection service on behalf of the user account.

20. The method of claim 15 , further comprising:

receive a command to terminate the link between the user account and the tunnel speaker; and

delete the client private key in response to receiving the command to terminate the link.

Assignments (4)
PATENT ASSIGNMENT Recorded Aug 5, 2024
From: VMWARE LLC
To: OMNISSA, LLC
Reel/Frame 068327/0365 →
SECURITY INTEREST Recorded Jul 3, 2024
From: OMNISSA, LLC
To: UBS AG, STAMFORD BRANCH
Reel/Frame 068118/0004 →
CHANGE OF NAME Recorded Apr 15, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 067102/0314 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2019
From: KUNNATH, RAMANANDAN NAMBANNOR; PANCHAPAKESAN, RAMANI
To: VMWARE, INC.
Reel/Frame 049836/0448 →