Granted Patent B2

US 12,265,746 · App. 17/481,141 · Granted Apr 1, 2025

Inventors: Ali Vassigh (San Jose, CA); Soren Riise (San Jose, CA); Robert Burdick (Los Altos, CA); Kevin Cooper (San Jose, CA)

Assignee: Roku, Inc.

G06F3/14G06F3/167G10L15/22G08C2201/31G10L2015/223

Abstract

Claims (69)

1. An audio responsive remote control, comprising:

a microphone array;

a beam-forming module configured for enhancement of an audio stream from a command source and de-enhancement of a background audio stream from a background source; and

at least one processor configured to:

receive a trigger command, wherein the trigger command comprises a first keyword or a second keyword, wherein the first keyword is associated with a user and the second keyword is associated with a display device;

identify, based on a keyword in the trigger command, the user as the identified command source and the display device as the identified background source responsive to determining that the trigger command comprises the first keyword;

identifying the display device as the identified command source and the user as the identified background source responsive to determining that the trigger command comprises the second keyword;

determine a first position of the identified background source relative to the audio responsive remote control;

perform, based on the first position of the identified background source, the de-enhancement of the background audio stream from the identified background source;

determine, based on receiving the trigger command, a second position of the identified command source; and

perform, based on the second position of the identified command source, the enhancement of the audio stream from the identified command source to form an enhanced audio stream.

2. The audio responsive remote control of claim 1 , wherein the first position of the identified background source is determined based on a first configuration setting and the second position of the identified command source is determined based on a second configuration setting, wherein the first configuration setting and the second configuration setting are stored in a memory of the audio responsive remote control.

3. The audio responsive remote control of claim 1 , wherein to perform the de-enhancement, the at least one processor is further configured to:

identify one or more microphones of the microphone array based on a proximity of the one or more microphones to the first position of the identified background source; and

deactivate the one or more microphones.

4. The audio responsive remote control of claim 1 , wherein to perform the de-enhancement, the at least one processor is further configured to:

receive the background audio stream from a media device over a network connection;

receive, by the microphone array, audio data;

match the background audio stream to the audio data;

identify one or more microphones of the microphone array based on the matching; and

adjust, by the beam-forming module, a reception pattern of the one or more microphones of the microphone array.

5. The audio responsive remote control of claim 4 , wherein to adjust the reception pattern, the beam-forming module is further configured to perform one of:

null audio reception of the one or more microphones; and

subtract the audio data from combined audio data received by the microphone array.

6. The audio responsive remote control of claim 1 , wherein to perform the enhancement, the at least one processor is further configured to:

identify one or more microphones of the microphone array based on a signal amplitude of the one or more microphones when the trigger command was received; and

adjust, by the beam-forming module, a reception pattern of the one or more microphones.

7. The audio responsive remote control of claim 1 , wherein when the identified command source is the display device, the audio stream is part of an audio track of content provided by the display device, and the audio responsive remote control is a media device.

8. The audio responsive remote control of claim 7 , wherein the at least one processor is further configured to:

detect, from the enhanced audio stream, an audio command; and

retrieve, based on the audio command, additional content associated with the audio track of content provided by the display device.

9. The audio responsive remote control of claim 7 , wherein the at least one processor is further configured to:

receive, by the audio responsive remote control, a second trigger command;

determine, based on content of the second trigger command, another command source based on the second trigger command, wherein the content of the second trigger command identifies the another command source; and

perform, based on the determining, another enhancement of another audio stream received from the another command source.

10. A non-transitory, tangible computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations for enhancement of an audio stream from a command source and de-enhancement of a background audio stream from a background source in an audio responsive remote control, the operations comprising:

receiving, by the audio responsive remote control from an application, a trigger command, wherein the trigger command comprises a first keyword or a second keyword, wherein the first keyword is associated with a user and the second keyword is associated with a display device, and wherein the trigger command is configured to cause the audio responsive remote control to record the audio stream and wherein the application is remote from the audio responsive remote control;

identifying, based on a keyword in the trigger command, the user as the identified command source and the display device as the identified background source responsive to determining that the trigger command comprises the first keyword;

identifying the display device as the identified command source and the user as the identified background source responsive to determining that the trigger command comprises the second keyword;

determining, by the audio responsive remote control, a first position of the identified background source relative to the audio responsive remote control;

performing, based on the first position of the identified background source, the de-enhancement of the background audio stream from the identified background source;

determining, by the audio responsive remote control and based on receiving the trigger command, a second position of the identified command source;

performing, based on the second position of the identified command source, the enhancement of the audio stream from the identified command source to form an enhanced audio stream;

recording, by the audio responsive remote control and responsive to receiving the trigger command, the enhanced audio stream; and

transmitting the recorded enhanced audio stream to the application.

11. The non-transitory, tangible computer-readable medium of claim 10 , wherein the first position of the identified background source is determined based on a first configuration setting and the second position of the identified command source is determined based on a second configuration setting, wherein the first configuration setting and the second configuration setting are stored in a memory of the audio responsive remote control.

12. The non-transitory, tangible computer-readable medium of claim 10 , wherein when the identified command source is the display device, the audio stream is part of an audio track of content provided by the display device and wherein the audio responsive remote control is a media device.

13. The non-transitory, tangible computer-readable medium of claim 12 , the operations further comprising:

detecting, from the enhanced audio stream, an audio command; and

retrieving, by the media device and based on the audio command, additional content associated with the audio track of content provided by the display device.

14. The non-transitory, tangible computer-readable medium of claim 10 , wherein the audio responsive remote control comprises a microphone array and wherein performing the de-enhancement further comprises:

identifying one or more microphones of the microphone array based on a proximity of the one or more microphones to the first position of the identified background source; and

deactivating the one or more microphones.

15. The non-transitory, tangible computer-readable medium of claim 10 , wherein the audio responsive remote control comprises a microphone array and a beam-forming module, and wherein performing the de-enhancement further comprises:

receiving, by the audio responsive remote control, the background audio stream from a media device over a network connection;

receiving, by the microphone array, audio data;

matching the background audio stream to the audio data;

identifying one or more microphones of the microphone array based on the matching; and

adjusting, by the beam-forming module, a reception pattern of the one or more microphones of the microphone array.

16. The non-transitory, tangible computer-readable medium of claim 15 , wherein adjusting the reception pattern further comprises one of:

nulling audio reception of the one or more microphones; and

subtracting the audio data from combined audio data received by the microphone array.

17. The non-transitory, tangible computer-readable medium of claim 10 , wherein the audio response remote control comprises a microphone array and a beam-forming module, and wherein performing the enhancement further comprises:

identifying one or more microphones of the microphone array based on a signal amplitude of the one or more microphones when the trigger command was received; and

adjusting, by the beam-forming module, a reception pattern of the one or more microphones.

18. The non-transitory, tangible computer-readable medium of claim 10 , the operations further comprising:

receiving, by the audio responsive remote control, a second trigger command;

determining, based on content of the second trigger command, another command source based on the second trigger command, wherein the content of the second trigger command identifies the another command source; and

performing, based on the determining, another enhancement of another audio stream received from the another command source.

Assignments (2)

Continuity (3)

References Cited (221)