IP Library › Granted Patent US 11,184,408
Granted Patent B2
US 11,184,408 · App. 16/788,956 · Granted Nov 23, 2021

Voice calling with a connected device

Inventors: Jeffrey Ching Wang (San Francisco, CA); Chien-Jung Kung (Los Alton, CA); Madhusudhan R. Adupala (Fremont, CA)
Assignee: GOOGLE LLC
H04L65/1006G06F9/45529G10L15/22G10L15/30H04L65/1069H04L65/608H04L67/02G10L2015/223H04L67/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,184,408
App. No.
16/788,956
Granted
Nov 23, 2021
Kind
B2
Abstract

Systems and techniques are provided for voice calling with a connected device that does not include a SIM card or telephone port. Outgoing audio data may be received at an embedded browser running on a connected device, may be sent using Web Real Time Communications (WebRTC) from the embedded browser to an integration layer panning within the embedded browser, and may be sent from the integration layer to a border controller for a voice call carrier over a Session Initiation Protocol (SIP) connection according to Secure Real Time Transport Protocol (SRTP). Incoming audio data may be received at the integration layer from the border controller for the voice call carrier over the SIP connection according to SRTP, may be sent using WebRTC from the integration layer to the embedded browser, and may be sent from the embedded browser to an audio output of the connected device which may output audio.

Claims (58)

1. A computer-implemented method performed by a data processing apparatus, the method comprising:

receiving outgoing audio data from an audio input of a connected device at an embedded browser running on the connected device, wherein the connected device comprises none of a SIM card slot, eSIM electronics, and a telephone port;

sending the outgoing audio data from the embedded browser to an integration layer running within the embedded browser, wherein the outgoing audio data is sent using Web Real Time Communications (WebRTC);

sending the outgoing audio data from the integration layer to a border controller for a voice call carrier over a Session Initiation Protocol (SIP) connection, wherein the outgoing audio data is sent according to Secure Real Time Transport Protocol (SRTP);

receiving incoming audio data at the integration layer from the border controller for the voice call carrier over the SIP connection, wherein the incoming audio data is received according to the SRTP;

sending the incoming audio data from the integration layer to the embedded browser, wherein the incoming audio data is sent using the WebRTC;

sending the incoming audio data from the embedded browser to an audio output of the connected device; and

outputting, by the audio output of the connected device, audio based on the incoming audio data.

2. The computer-implemented method of claim 1 , wherein the SIP connection is established by:

sending, to a voice call carrier system, a private token and a phone number of a specified party, wherein the private token is associated with a user account on the voice call carrier system;

receiving, from the voice call carrier system, configuration data and credentials, wherein the configuration data comprises the phone number of the specified party; and

sending, to the border controller for the voice call carrier, the configuration data and credentials.

3. The computer-implemented method of claim 2 , wherein the private token is obtained from the voice call carrier system based on sending the voice call carrier system credentials for a cloud computing account and a phone number associated with the user account on the voice call carrier system and sending a verification code sent by the voice call carrier system to a telephonic device using the phone number associated with the user account on the voice call carrier system to the voice call carrier system, and further comprising:

receiving the private token from the voice call carrier system or a cloud computing system that comprises the cloud computing account; and

storing the private token.

4. The computer-implemented method of claim 1 , further comprising:

receiving, at the audio input, audio for a voice command to place a call and converting the audio to audio data for the voice command to place the call;

sending the audio data for the voice command to place the call to a cloud computing system comprising a user account associated with the connected device; and

receiving a phone number for a specified party in the audio data for the voice command to place the call from the cloud computing system.

5. The computer-implemented of claim 1 , wherein the embedded browser is written in C++ and the integration layer is written in JavaScript.

6. The computer-implemented method of claim 1 , further comprising:

receiving, at the connected device from a cloud computing system or a voice call carrier system, feature code for a feature, wherein the feature is associated with phone calling; and

storing the feature code in a stub of the integration layer, wherein the stub is associated with the feature.

7. The computer-implemented method of claim 6 , wherein the feature code is written in a same programming language as the integration layer and includes at least one star code associated with systems of the voice call carrier.

8. The computer-implemented method of claim 6 , further comprising:

receiving, at the audio input, audio for a voice command invoking the feature and converting the audio to audio data for the voice command invoking the feature;

sending the audio data for the voice command invoking the feature to the cloud computing system comprising a user account associated with the connected device;

receiving an indication of the feature in the audio data for the voice command invoking the feature from the cloud computing system;

running, in the embedded browser, the feature code for the feature from the stub in the integration layer; and

sending, from the integration layer, at least one item of data for the feature to the border controller of the voice call carrier based on the running of the feature code.

9. The computer-implemented method of claim 1 , wherein the connected device is a smart speaker or a smart display.

10. A computer-implemented system for voice calling with a connected device comprising:

an audio input configured to receive audio and convert the audio to outgoing audio data;

an audio output configured to receive incoming audio data and convert the incoming audio data to audio;

a storage; and

a processor configured to receive the outgoing audio data from the audio input at an embedded browser running on the connected device, send the outgoing audio data from the embedded browser to an integration layer running within the embedded browser, wherein the outgoing audio data is sent using Web Real Time Communications (WebRTC), send the outgoing audio data from the integration layer to a border controller for a voice call carrier over a Session Initiation Protocol (SIP) connection, wherein the outgoing audio data is sent according to Secure Real Time Transport Protocol (SRTP), receive the incoming audio data at the integration layer from the border controller for the voice call carrier over the SIP connection, wherein the incoming audio data is received according to the SRTP, and send the incoming audio data from the integration layer to the embedded browser, wherein the incoming audio data is sent using the WebRTC, and send the incoming audio data from the embedded browser to the audio output,

and wherein the connected device comprises none of a SIM card slot, eSIM electronics, and a telephone port.

11. The computer-implemented system of claim 10 , wherein the processor is further configured to establish the SIP connection by sending, to a voice call carrier system, a private token and a phone number of a specified party, wherein the private token is associated with a user account on the voice call carrier system, receiving, from the voice call carrier system, configuration data and credentials, wherein the configuration data comprises the phone number of the specified party, and sending, to the border controller for the voice call carrier, the configuration data and credentials.

12. The computer-implemented system of claim 11 , wherein the private token is obtained from the voice call carrier system based on sending the voice call carrier system credentials for a cloud computing account and a phone number associated with the user account on the voice call carrier system and sending a verification code sent by the voice call carrier system to a telephonic device using the phone number associated with the user account on the voice call carrier system to the voice call carrier system, and wherein the processor is further configured to receive the private token from the voice call carrier system or a cloud computing system that comprises the cloud computing account and store the private token in the storage.

13. The computer-implemented system of claim of claim 10 , wherein the audio input is further configured to receive audio for a voice command to place a call and convert the audio for the voice command to place the call to audio data for the voice command to place the call, and wherein the processor is further configured send the audio data for the voice command to place the call to a cloud computing system comprising a user account associated with the connected device and receive a phone number for a specified party in the audio data for the voice command to place the call from the cloud computing system.

14. The computer-implemented system of claim 10 , wherein the embedded browser is written in C++ and the integration layer is written in JavaScript.

15. The computer-implemented system of claim 10 , wherein the processor is further configured to receive, from a cloud computing system or a voice call carrier system, feature code for a feature, wherein the feature is associated with phone calling, and—store the feature code in a stub of the integration layer, wherein the stub is associated with the feature.

16. The computer-implemented system of claim 15 , wherein the feature code is written in the same programming language as the integration layer and includes at least one star code associated with systems of the voice call carrier.

17. The computer-implemented system of claim 15 , wherein the audio input is further configured to receive audio for a voice command invoking the feature and convert the audio for the voice command invoking the feature to audio data for the voice command invoking the feature, and wherein the processor is further configured to send the audio data for the voice command invoking the feature to the cloud computing system comprising a user account associated with the connected device, receive an indication of the feature in the audio data for the voice command invoking the feature from the cloud computing system, run, in the embedded browser, the feature code for the feature from the stub in the integration layer,

and send, from the integration layer, at least one item of data for the feature to the border controller of the voice call carrier based on the running of the feature code.

18. The computer-implemented system of claim 10 , wherein the connected device is a smart speaker or a smart display.

19. A system comprising: one or more computers and one or more storage devices storing instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving outgoing audio data from an audio input of a connected device at an embedded browser running on the connected device, wherein the connected device comprises none of a SIM card slot, eSIM electronics, and a telephone port;

sending the outgoing audio data from the embedded browser to an integration layer running within the embedded browser, wherein the outgoing audio data is sent using Web Real Time Communications (WebRTC);

sending the outgoing audio data from the integration layer to a border controller for a voice call carrier over a Session Initiation Protocol (SIP) connection, wherein the outgoing audio data is sent according to Secure Real Time Transport Protocol (SRTP);

receiving incoming audio data at the integration layer from the border controller for the voice call carrier over the SIP connection, wherein the incoming audio data is received according to the SRTP;

sending the incoming audio data from the integration layer to the embedded browser, wherein the incoming audio data is sent using the WebRTC;

sending the incoming audio data from the embedded browser to an audio output of the connected device; and

outputting, by the audio output of the connected device, audio based on the incoming audio data.

20. The system of claim 19 , wherein the instructions further cause the one or more computers to perform operations to establish the SIP connection comprising:

sending, to a voice call carrier system, a private token and a phone number of a specified party, wherein the private token is associated with a user account on the voice call carrier system;

receiving, from the voice call carrier system, configuration data and credentials, wherein the configuration data comprises the phone number of the specified party; and

sending, to the border controller of the voice call carrier, the configuration data and credentials.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2020
From: WANG, JEFFREY CHING; KUNG, CHIEN-JUNG; ADUPALA, MADHUSUDHAN R.
To: GOOGLE LLC
Reel/Frame 051802/0343 →
Continuity (2)
Provisional Application 62962868 · Jan 17, 2020
Related Publication 20210227000A1 · Jul 22, 2021