IP Library Granted Patent US 10,979,236
Granted Patent B1
US 10,979,236 · App. 16/746,653 · Granted Apr 13, 2021

Systems and methods for smoothly transitioning conversations between communication channels

Inventors: Andrew Lovitt (Redmond, WA); Antonio John Miller (Woodinville, WA); Philip Robinson (Seattle, WA); Scott Selfon (Kirkland, WA)
Assignee: Facebook Technologies, LLC
H04L12/1818G06F3/165G10L15/30H04R3/005H04R3/12H04R5/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,979,236
App. No.
16/746,653
Granted
Apr 13, 2021
Kind
B1
Abstract

The disclosed computer-implemented method may include (1) establishing a communication channel to indirectly convey a conversation, (2) receiving, via the communication channel, a portion of the conversation, (3) presenting the portion of the conversation to a user, (4) receiving, via the communication channel, an additional portion of the conversation, (5) detecting an additional communication channel capable of conveying the conversation, (6) determining a human-perceivable difference between how the conversation has been conveyed via the communication channel and how the conversation will be conveyed via the additional communication channel, and (7) compensating for the human-perceivable difference when presenting the additional portion of the conversation to the user in order to smoothly transition the conversation from the communication channel to the additional communication channel. Various other methods, systems, and computer-readable media are also disclosed.

Claims (91)

1. A computer-implemented method comprising:

monitoring, via a first computing device, a conversation between a first user of the first computing device and a second user of a second computing device, a first portion of the conversation being conveyed via a first communication channel;

establishing, between the first computing device and the second computing device, a second communication channel capable of indirectly conveying the conversation between the first user and the second user;

receiving, at the first computing device, a second portion of the conversation between the second user and the first user;

determining a human-perceivable property of the conversation conveyed via the first communication channel; and

compensating for the human-perceivable property when presenting, via the first computing device, the second portion of the conversation to the first user in order to smoothly transition the conversation from the first communication channel to the second communication channel.

2. The computer-implemented method of claim 1 , wherein:

the first communication channel is a physical environment of the first user and the second user; and

the computer-implemented method further comprises:

detecting when the physical environment will be incapable of directly conveying the conversation between the first user and the second user; and

making, in response to detecting when the physical environment will be incapable of directly conveying the conversation between the first user and the second user, a transition to the second communication channel by using the second communication channel to indirectly convey the second portion of the conversation between the first user and the second user.

3. The computer-implemented method of claim 1 , wherein detecting when the physical environment will be incapable of directly conveying the conversation between the first user and the second user comprises detecting, using a simultaneous localization and mapping sensor of the first computing device, when the first user and the second user will not both be present within the physical environment.

4. The computer-implemented method of claim 3 , wherein detecting when the first user and the second user will not both be present within the physical environment comprises one of:

detecting the first user exiting a room occupied by the second user; or

detecting the second user exiting a room occupied by the first user.

5. The computer-implemented method of claim 2 , wherein determining when the physical environment will be incapable of directly conveying the conversation between the first user and the second user comprises detecting, using a microphone of the first computing device, a verbal cue from the second user or the first user.

6. The computer-implemented method of claim 2 , wherein determining when the physical environment will be incapable of directly conveying the conversation between the first user and the second user comprises detecting, using a microphone of the first computing device, a rise in a noise level in the physical environment.

7. The computer-implemented method of claim 2 , wherein determining when the physical environment will be incapable of directly conveying the conversation between the first user and the second user comprises detecting, using a microphone of the first computing device, user movements in the physical environment.

8. The computer-implemented method of claim 1 , wherein:

the first communication channel is a physical environment of the first user and the second user;

the second portion of the conversation was conveyed via the second communication channel;

determining the human-perceivable property comprises determining the first user's perceived spatial localization of the first portion of the conversation; and

compensating for the human-perceivable property when presenting the second portion of the conversation comprises presenting at least some of the second portion of the conversation to the first user such that the first user's perceived spatial localization of the second portion of the conversation is substantially similar to the first user's perceived spatial localization of the first portion of the conversation conveyed via the first communication channel.

9. The computer-implemented method of claim 1 , wherein:

the first communication channel is a physical environment of the first user and the second user;

the second portion of the conversation was conveyed via the second communication channel;

determining the human-perceivable property comprises determining an acoustic property of the first portion of the conversation conveyed via the first communication channel; and

compensating for the human-perceivable property when presenting the second portion of the conversation comprises presenting the second portion of the conversation to the first user such that the acoustic property of at least some of the second portion of the conversation conveyed via the second communication channel is substantially similar to the perceived acoustic property of the first portion of the conversation conveyed via the first communication channel.

10. The computer-implemented method of claim 1 , wherein:

the second portion of the conversation was conveyed via the first communication channel;

determining the human-perceivable property comprises determining that a temporal delay of a third portion of the conversation conveyed via the second communication channel will be more than a temporal delay of the first portion of the conversation conveyed via the first communication channel; and

compensating for the human-perceivable property when presenting the second portion of the conversation comprises temporally stretching the second portion of the conversation before presentation to the first user.

11. The computer-implemented method of claim 10 , wherein:

the first communication channel is a physical environment of the first user and the second user; and

the computer-implemented method further comprises:

receiving, at the first computing device, the third portion of the conversation;

preventing, at the first computing device while the second portion of the conversation is presented to the first user, the first user from perceiving the third portion of the conversation;

temporally compressing the third portion of the conversation; and

presenting, via the first computing device after presenting the second portion of the conversation, the third portion of the conversation to the first user.

12. The computer-implemented method of claim 1 , wherein:

the first communication channel is a physical environment of the first user and the second user;

the second portion of the conversation was conveyed via the second communication channel;

determining the human-perceivable property comprises determining that the first portion of the conversation conveyed via the physical environment comprises a first direct-path acoustical component and a first reflected-path acoustical component; and

compensating for the human-perceivable property when presenting the second portion of the conversation to the first user comprises:

estimating a second direct-path acoustical component for the second portion of the conversation;

estimating a second reflected-path acoustical component for the second portion of the conversation;

presenting, via the first computing device, the second direct-path acoustical component to the first user such that the first user's perceived spatial localization of the second direct-path acoustical component is substantially similar to the first user's perceived spatial localization of the first direct-path acoustical component; and

presenting, via the first computing device, the second reflected-path acoustical component to the first user such that the first user's perceived spatial localization of the second reflected-path acoustical component is substantially similar to the first user's perceived spatial localization of the first reflected-path acoustical component.

13. A communication system comprising:

at least one physical processor; and

physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the physical processor to:

monitor, via a first computing device, a conversation between a first user of the first computing device and a second user of a second computing device, a first portion of the conversation being conveyed via a first communication channel;

establish, between the first computing device and the second computing device, a second communication channel capable of indirectly conveying the conversation between the first user and the second user;

receive, at the first computing device, a second portion of the conversation between the second user and the first user;

determine a human-perceivable property of the conversation conveyed via the first communication channel; and

compensate for the human-perceivable property when presenting, via the first computing device, the second portion of the conversation to the first user in order to smoothly transition the conversation from the first communication channel to the second communication channel.

14. The communication system of claim 13 , wherein:

the first communication channel is a physical environment of the first user and the second user; and

the computer-executable instructions further cause the physical processor to:

detect when the physical environment will be incapable of directly conveying the conversation between the first user and the second user; and

make, in response to detecting when the physical environment will be incapable of directly conveying the conversation between the first user and the second user, a transition to the second communication channel by using the second communication channel to indirectly convey the second portion of the conversation between the first user and the second user.

15. The communication system of claim 14 , wherein detecting when the physical environment will be incapable of directly conveying the conversation between the first user and the second user comprises detecting, using a simultaneous localization and mapping sensor of the first computing device, when the first user and the second user will not both be present within the physical environment.

16. The communication system of claim 13 , wherein:

the first communication channel is a physical environment of the first user and the second user;

the second portion of the conversation was conveyed via the second communication channel;

determining the human-perceivable property comprises determining the first user's perceived spatial localization of the first portion of the conversation; and

compensating for the human-perceivable property when presenting the second portion of the conversation comprises presenting at least some of the second portion of the conversation to the first user such that the first user's perceived spatial localization of the second portion of the conversation is substantially similar to the first user's perceived spatial localization of the first portion of the conversation conveyed via the first communication channel.

17. The communication system of claim 13 , wherein:

the first communication channel is a physical environment of the first user and the second user;

the second portion of the conversation was conveyed via the second communication channel;

determining the human-perceivable property comprises determining an acoustic property of the first portion of the conversation conveyed via the first communication channel; and

compensating for the human-perceivable property when presenting the second portion of the conversation comprises presenting the second portion of the conversation to the first user such that the acoustic property of at least some of the second portion of the conversation conveyed via the second communication channel is substantially similar to the perceived acoustic property of the first portion of the conversation conveyed via the first communication channel.

18. The communication system of claim 13 , wherein:

the second portion of the conversation was conveyed via the first communication channel;

determining the human-perceivable property comprises determining that a temporal delay of a third portion of the conversation conveyed via the second communication channel will be more than a temporal delay of the first portion of the conversation conveyed via the first communication channel; and

compensating for the human-perceivable property when presenting the second portion of the conversation comprises temporally stretching the second portion of the conversation before presentation to the first user.

19. The communication system of claim 13 , wherein:

the first communication channel is a physical environment of the first user and the second user;

the second portion of the conversation was conveyed via the second communication channel;

determining the human-perceivable property comprises determining that the first portion of the conversation conveyed via the physical environment comprises a first direct-path acoustical component and a first reflected-path acoustical component; and

compensating for the human-perceivable property when presenting the second portion of the conversation to the first user comprises:

estimating a second direct-path acoustical component for the second portion of the conversation;

estimating a second reflected-path acoustical component for the second portion of the conversation;

presenting, via the first computing device, the second direct-path acoustical component to the first user such that the first user's perceived spatial localization of the second direct-path acoustical component is substantially similar to the first user's perceived spatial localization of the first direct-path acoustical component; and

presenting, via the first computing device, the second reflected-path acoustical component to the first user such that the first user's perceived spatial localization of the second reflected-path acoustical component is substantially similar to the first user's perceived spatial localization of the first reflected-path acoustical component.

20. A non-transitory computer-readable medium comprising computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

monitor, via a first computing device, a conversation between a first user of the first computing device and a second user of a second computing device, a first portion of the conversation being conveyed via a first communication channel;

establish, between the first computing device and the second computing device, a second communication channel capable of indirectly conveying the conversation between the first user and the second user;

receive, at the first computing device, a second portion of the conversation between the second user and the first user;

determine a human-perceivable property of the conversation conveyed via the first communication channel; and

compensate for the human-perceivable property when presenting, via the first computing device, the second portion of the conversation to the first user in order to smoothly transition the conversation from the first communication channel to the second communication channel.

Assignments (2)
CHANGE OF NAME Recorded May 26, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060199/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2020
From: LOVITT, ANDREW; MILLER, ANTONIO JOHN; ROBINSON, PHILIP; SELFON, SCOTT
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 052803/0627 →