IP Library Granted Patent US 10,083,006
Granted Patent B1
US 10,083,006 · App. 15/702,164 · Granted Sep 25, 2018

Intercom-style communication using multiple computing devices

Inventors: Sandro Feuz (Zurich, CH); Sebastian Millius (Zurich, CH); Jan Althaus (Zurich, CH)
Assignee: GOOGLE LLC
G06F3/167G10L15/22G10L2015/225H04L67/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,083,006
App. No.
15/702,164
Granted
Sep 25, 2018
Kind
B1
Abstract

Techniques are described related to improved intercom-style communication using a plurality of computing devices distributed about an environment. In various implementations, voice input may be received, e.g., at a microphone of a first computing device of multiple computing devices, from a first user. The voice input may be analyzed and, based on the analyzing, it may be determined that the first user intends to convey a message to a second user. A location of the second user relative to the multiple computing devices may be determined, so that, based on the location of the second user, a second computing device may be selected from the multiple computing devices that is capable of providing audio or visual output that is perceptible to the second user. The second computing device may then be operated to provide audio or visual output that conveys the message to the second user.

Claims (53)

1. A method comprising:

accessing a trained machine learning model, wherein the machine learning model is trained using a corpus of labeled voice inputs, and wherein labels applied to the voice inputs include:

a first label indicative of a user intent to convey a message to another user;

a second label indicative of a user intent to engage in a human-to-computer dialog with an automated assistant; and

a third label indicative of background conversation between multiple users;

receiving, at a microphone of a first computing device of a plurality of computing devices, from a first user, voice input;

analyzing the voice input, wherein the analyzing includes applying data indicative of an audio recording of the voice input as input across the trained machine learning model to generate output, wherein the output indicates that the first user intends to convey a message to a second user;

determining, based on the analyzing, that the first user intends to convey the message to the second user;

determining a location of the second user relative to the plurality of computing devices;

selecting, from the plurality of computing devices, based on the location of the second user, a second computing device that is capable of providing audio or visual output that is perceptible to the second user; and

causing the second computing device to exclusively provide audio or visual output that conveys the message to the second user.

2. The method of claim 1 , wherein the selecting is performed in response to a determination, based on the location of the second user, that the second user is not within earshot of the first user.

3. The method of claim 1 , wherein the location of the second user is determined based at least in part on one or more signals generated by a mobile computing device operated by the second user.

4. The method of claim 1 , wherein the location of the second user is determined based at least in part on one or more signals generated by one or more of the plurality of computing devices other than the first computing device.

5. The method of claim 4 , wherein the one or more signals include a signal indicative of the second user being detected by one or more of the plurality of computing devices other than the first computing device using passive infrared or ultrasound.

6. The method of claim 4 , wherein the one or more signals include a signal indicative of the second user being detected by one or more of the plurality of computing devices other than the first computing device using a camera or a microphone.

7. The method of claim 1 , wherein the analyzing includes determining that the voice input includes an explicit command to convey the message to the second user as an intercom message via one or more of the plurality of computing devices.

8. The method of claim 1 , wherein the analyzing includes performing speech-to-text processing on the audio recording of the voice input to generate, as the data indicative of the audio recording, textual input, wherein the textual input is applied as input across the trained machine learning model.

9. The method of claim 1 , further comprising:

identifying a search query issued by the second user after the audio or visual output is provided by the second computing device;

obtaining search results that are responsive to the search query, wherein the obtaining is based at least in part on the voice input from the first user; and

causing one or more of the plurality of computing devices to provide output indicative of at least some of the search results.

10. A system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:

receiving, at a microphone of a first computing device of a plurality of computing devices, from a first user, voice input;

analyzing the voice input;

determining, based on the analyzing, that the first user intends to convey a message to a second user;

determining a location of the second user relative to the plurality of computing devices;

selecting, from the plurality of computing devices, based on the location of the second user, a second computing device that is capable of providing audio or visual output that is perceptible to the second user;

causing the second computing device to exclusively provide audio or visual output that conveys the message to the second user;

identifying a search query issued by the second user after the audio or visual output is provided by the second computing device;

disambiguating at least part of the search query issued by the second user based at least in part on the voice input from the first user;

obtaining search results that are responsive to the disambiguated search query; and

causing one or more of the plurality of computing devices to provide output indicative of at least some of the search results.

11. The system of claim 10 , wherein the selecting is performed in response to a determination, based on the location of the second user, that the second user is not within earshot of the first user.

12. The system of claim 10 , wherein the location of the second user is determined based at least in part on one or more signals generated by a mobile computing device operated by the second user.

13. The system of claim 10 , wherein the location of the second user is determined based at least in part on one or more signals generated by one or more of the plurality of computing devices other than the first computing device.

14. The system of claim 13 , wherein the one or more signals include a signal indicative of the second user being detected by one or more of the plurality of computing devices other than the first computing device using passive infrared or ultrasound.

15. The system of claim 13 , wherein the one or more signals include a signal indicative of the second user being detected by one or more of the plurality of computing devices other than the first computing device using a camera or a microphone.

16. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:

accessing a trained machine learning model, wherein the machine learning model is trained using a corpus of labeled voice inputs, and wherein labels applied to the voice inputs include:

a first label indicative of a user intent to convey a message to another user;

a second label indicative of a user intent to engage in a human-to-computer dialog with an automated assistant; and

a third label indicative of background conversation between multiple users;

receiving, at a microphone of a first computing device of a plurality of computing devices, from a first user, voice input;

analyzing the voice input, wherein the analyzing includes applying data indicative of an audio recording of the voice input as input across the trained machine learning model to generate output, wherein the output indicates that the first user intends to convey a message to the second user;

determining, based on the analyzing, that the first user intends to convey the message to a second user;

determining a location of the second user relative to the plurality of computing devices;

selecting, from the plurality of computing devices, based on the location of the second user, a second computing device that is capable of providing audio or visual output that is perceptible to the second user; and

causing the second computing device to exclusively provide audio or visual output that conveys the message to the second user.

17. The at least one non-transitory computer-readable medium of claim 16 , wherein the selecting is performed in response to a determination, based on the location of the second user, that the second user is not within earshot of the first user.

18. The at least one non-transitory computer-readable medium of claim 16 , wherein the location of the second user is determined based at least in part on one or more signals generated by a mobile computing device operated by the second user.

19. The at least one non-transitory computer-readable medium of claim 16 , wherein the location of the second user is determined based at least in part on one or more signals generated by one or more of the plurality of computing devices other than the first computing device.

20. The at least one non-transitory computer-readable medium of claim 19 , wherein the one or more signals include a signal indicative of the second user being detected by one or more of the plurality of computing devices other than the first computing device using passive infrared or ultrasound.

Assignments (2)
CHANGE OF NAME Recorded Aug 22, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 046896/0555 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 14, 2017
From: FEUZ, SANDRO; MILLIUS, SEBASTIAN; ALTHAUS, JAN
To: GOOGLE INC.
Reel/Frame 043583/0870 →
Cited By (4)
US 12,334,069 US 12,483,652 US 12,531,066 US 12,682,706