IP Library Granted Patent US 10,210,863
Granted Patent B2
US 10,210,863 · App. 15/341,552 · Granted Feb 19, 2019

Reception of audio commands

Inventor: Gregory M. Garner (Saratoga, CA)
Assignee: Roku, Inc.
G10L15/22G10L15/08G10L15/30H04R1/406H04R3/005G10L21/02G10L2015/088G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,210,863
App. No.
15/341,552
Granted
Feb 19, 2019
Kind
B2
Abstract

Disclosed herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for controlling a media device and a display device using audio commands. In so doing, some embodiments operate to suppress noise from the display device, and enhance audio commands from users. Some embodiments operate by determining a position of the display device and de-enhancing audio from the display device based on the display device position. The position of the user is determined, and audio from the user based on the user position is enhanced. Then, a command in the enhanced user audio is identified, and the media device and/or the display device are caused to operate according to the command.

Claims (78)

1. A method of controlling a media device and a display device using audio commands, comprising:

detecting, at an audio responsive control device, a trigger word in audio from a source of audio commands;

determining, at the audio responsive control device, that a user is the source of audio commands based on the trigger word being associated with the audio responsive control device;

determining, at the audio responsive control device, a position of the display device;

de-enhancing, at the audio responsive control device, audio output by a speaker of the display device based on the position of the display device and the determination that the user is the source of audio commands;

determining, at the audio responsive control device, a position of the source of audio commands;

enhancing, at the audio responsive control device, the audio from the source of audio commands based on the determined position of the source of audio commands and the determination that the user is the source of audio commands;

identifying, at the audio responsive control device, a command in the enhanced audio from the source of audio commands; and

causing, at the audio responsive control device, at least one of the media device and the display device to operate according to the command.

2. The method of claim 1 , further comprising:

receiving, at the audio responsive control device, input identifying the position of the display device relative to a microphone array in the audio responsive control device, and the position of the source of audio commands relative to the microphone array.

3. The method of claim 1 , the determining the position of the source of audio commands further comprising:

identifying, at the audio responsive control device, a microphone in a microphone array in the audio responsive control device having a greatest signal strength when receiving the audio from the source of audio commands; and

determining, at the audio responsive control device, the position of the source of audio commands based on the identified microphone.

4. The method of claim 3 , wherein the enhancing the audio from the source of audio commands comprises:

beam forming, at the audio responsive control device, an audio reception pattern of the identified microphone to improve receipt of the audio from the source of audio commands.

5. The method of claim 1 , wherein the enhancing the audio from the source of audio commands comprises:

beam forming, at the audio responsive control device, an audio reception pattern of a microphone in a microphone array proximate to the position of the source of audio commands to improve receipt of the audio from the source of audio commands.

6. The method of claim 1 , wherein the de-enhancing the audio from the display device based on the position of the display device comprises at least one of:

deactivating, at the audio responsive control device, one or more microphones in a microphone array in the audio responsive control device that are proximate to the position of the display device; and

muting the audio from the speaker of the display device after receipt of the trigger word.

7. The method of claim 1 , wherein the de-enhancing the audio from the display device based on the position of the display device comprises:

beam forming, at the audio responsive control device, an audio reception pattern of a microphone in a microphone array that is proximate to the position of the display device to suppress receipt of the audio from the speaker of the display device.

8. The method of claim 1 , wherein the de-enhancing the audio output by the speaker of the display device based on the position of the display device comprises:

de-enhancing, at the audio responsive control device, the audio output by the speaker of the display device based on at least the trigger word in the audio from the source of audio commands.

9. The method of claim 1 , wherein the enhancing the audio from the source of audio commands based on the position of the source of audio commands and the trigger word in the audio from the source of audio commands comprises:

enhancing, at the audio responsive control device, the audio from the source of audio commands based on at least a trigger word type of the trigger word.

10. The method of claim 1 , wherein the de-enhancing comprises:

receiving, via a network, an audio stream that is to be output by the speaker of the display device;

determining the audio output by the speaker of the display device matches the audio stream that is to be output by the speaker of the display device; and

subtracting the audio stream that is to be output by the speaker of the display device from audio received by a microphone in a microphone array in the audio responsive control device.

11. An audio responsive control device to control a display device and a media server, comprising:

a memory; and

at least one processor operatively coupled to the memory, the at least one processor configured to:

detect a trigger word in audio from a source of audio commands;

determine that a user is the source of audio commands based on the trigger word being associated with the audio responsive control device;

determine a position of the display device;

de-enhance audio output by a speaker of the display device based on a position of the display device and the determination that the user is the source of audio commands;

determine a position of a source of audio commands;

enhance audio from the source of audio commands based on the determined position of the source of audio commands and the determination that the user is the source of audio commands;

identify a command in the enhanced audio from the source of audio commands; and

cause at least one of the media device and the display device to operate according to the command.

12. The audio responsive control device of claim 11 , the at least one processor further configured to:

receive input identifying the position of the display device relative to a microphone array, and the position of the source of audio commands relative to the microphone array.

13. The audio responsive control device of claim 11 , the at least one processor further configured to:

identify the trigger word in the audio from the source of audio commands; and

identify a microphone in a microphone array having a greatest signal strength when receiving the audio from the source of audio commands; and

determine the position of the source of audio commands based on the identified microphone.

14. The audio responsive control device of claim 13 , wherein to enhance the audio from the source of audio commands the at least one processor is configured to:

beam form an audio reception pattern of the identified microphone to improve receipt of the audio from the source of audio commands.

15. The audio responsive control device of claim 11 , wherein to enhance the audio from the source of audio commands the at least one processor is configured to:

beam form an audio reception pattern of a microphone in a microphone array proximate to the position of the source of audio commands to improve receipt of the audio from the source of audio commands.

16. The audio responsive control device of claim 11 , wherein to de-enhance the audio from the display device based on the display device position the at least one processor is configured to:

deactivate one or more microphones in the microphone array that are proximate to the position of the display device.

17. The audio responsive control device of claim 11 , wherein to de-enhance the audio from the display device based on the position of the display device the at least one processor is configured to:

beam form an audio reception pattern of a microphone in a microphone array that is proximate to the position of the display device to suppress receipt of the audio from the speaker of the display device.

18. A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

detecting a trigger word in audio from a source of audio commands;

determining that a user is the source of audio commands based on the trigger word being associated with the computing device;

determining a position of a display device;

de-enhancing audio output by a speaker of the display device based on a position of the display device and the determination that the user is the source of audio commands;

determining a position of a source of audio commands;

enhancing audio from the source of audio commands based on the determined position of the source of audio commands and the determination that the user is the source of audio commands;

identifying a command in the enhanced audio from the source of audio commands; and

causing at least one of a media device and the display device to operate according to the command.

19. The non-transitory, tangible computer-readable device of claim 18 , the operations further comprising:

identifying the trigger word in the audio from the source of audio commands;

identifying a microphone in a microphone array having a greatest signal strength when receiving the audio from the source of audio commands; and

determining the position of the source of audio commands based on the identified microphone.

20. A method of controlling a media device and a display device using audio commands, comprising:

identifying a trigger word;

determining the trigger word is associated with the display device;

determining a position of a user;

de-enhancing audio from the user based on the position of the user;

determining a position of the display device;

enhancing audio from the display device based on the position of the display device;

identifying at least one command in the enhanced display device audio; and

causing at least one of the media device and the display device to operate according to the at least one command.

Assignments (8)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 051300/0931) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0205 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 048385/0375) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.
Reel/Frame 062826/0001 →
SECURITY INTEREST Recorded Dec 16, 2019
From: ROKU, INC.; ROKU DX HOLDINGS, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 051300/0931 →
RELEASE OF SECURITY INTEREST Recorded Feb 25, 2019
From: SILICON VALLEY BANK, AS BANK
To: ROKU, INC.
Reel/Frame 048420/0841 →
PATENT SECURITY AGREEMENT Recorded Feb 20, 2019
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 048385/0375 →
AMENDED AND RESTATED INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jun 12, 2017
From: ROKU, INC.
To: SILICON VALLEY BANK
Reel/Frame 042768/0268 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 21, 2016
From: GARNER, GREGORY M.
To: ROKU, INC.
Reel/Frame 040384/0258 →
Continuity (1)
Related Publication 20180122373A1 · May 3, 2018
Cited By (3)
US 12,265,746 US 12,482,467 US 12,614,549