IP Library Granted Patent US 11,947,871
Granted Patent B1
US 11,947,871 · App. 18/299,865 · Granted Apr 2, 2024

Spatially aware virtual meetings

Inventors: Steven Lee Fisher-Stawinski (Buffalo Grove, IL); Shikhar Kwatra (San Jose, CA); Moitreyee Mukherjee-Roy (San Jose, CA); Scott E. Schneider (Rolesville, NC)
Assignee: International Business Machines Corporation
G06F3/165G06F3/013G06F3/017G06T11/00G10L15/22H04S7/303H04S2400/11H04S2400/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,947,871
App. No.
18/299,865
Filed
Apr 13, 2023
Granted
Apr 2, 2024
Kind
B1
Art Unit
2624
USPC
345/156
Abstract

According to one embodiment, a method, computer system, and computer program product for spatially aware virtual meetings is provided. The embodiment may include establishing a virtual conference room and connections thereto by a speaking participant, an addressee participant, and a non-addressee participant. The embodiment may also include assigning the speaking participant, the addressee participant, and the non-addressee participant to positions in a virtual space. The embodiment may further include displaying a virtual output stream of the virtual space to a viewing participant on a user device display screen. The embodiment may also include determining that a speaking participant is directing a speech segment to an addressee participant. The embodiment may further include altering at least one of the visual output stream and an audio output stream for a viewing participant based at least on the positions of the speaking participant and the addressee participant.

Claims (43)

1. A processor-implemented method, the method comprising:

establishing a virtual conference room and connections thereto by a speaking participant, an addressee participant, and a non-addressee participant;

assigning the speaking participant, the addressee participant, and the non-addressee participant to positions in a virtual space;

displaying a virtual output stream of the virtual space to a viewing participant on a user device display screen, wherein the viewing participant is selected from a group consisting of the speaking participant, the addressee participant, and the non-addressee participant, and wherein the virtual output stream comprises a schematic visual representation of the virtual space from the viewing participant's spatial perspective that is configured to be displayed in a two-dimensional viewport from a fixed perspective;

determining that the speaking participant is directing a speech segment to the addressee participant; and

altering the virtual output stream and an audio output stream for the viewing participant based at least on the positions of the speaking participant and the addressee participant, wherein the altering of the virtual output stream modifies a viewing direction of a virtual representation of each participant based on a gaze direction of the respective participant towards a representation of another participant on the user device display screen.

2. The method of claim 1 , wherein the determining is based on at least one content element of speech of the speaking participant.

3. The method of claim 1 , wherein the determining is based on detecting at least one gesture of the speaking participant.

4. The method of claim 1 , wherein the determining is based on detecting a gaze of the speaking participant to the addressee participant.

5. The method of claim 1 , wherein the altering further comprises:

determining a first direction from the speaking participant to the addressee participant and a second direction from the speaking participant to the viewing participant based on each participant's position in the virtual space; and

generating a viewing participant output audio stream by, at least, altering a speaking participant audio stream of the speech segment to include at least one of a first directional feature based on the first direction and a second directional feature based on the second direction.

6. The method of claim 5 , wherein the generating is performed by altering the speaking participant audio stream based on at least one gesture of the speaking participant.

7. The method of claim 5 , wherein generating the viewing participant output audio stream for a non-addressee participant is performed by altering the speaking participant audio stream to further include at least one clarity feature or at least one indirectness feature based on determining that the speaking participant is directing the speech segment to the addressee participant.

8. A computer system, the computer system comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:

establishing a virtual conference room and connections thereto by a speaking participant, an addressee participant, and a non-addressee participant;

assigning the speaking participant, the addressee participant, and the non-addressee participant to positions in a virtual space;

displaying a virtual output stream of the virtual space to a viewing participant on a user device display screen, wherein the viewing participant is selected from a group consisting of the speaking participant, the addressee participant, and the non-addressee participant, and wherein the virtual output stream comprises a schematic visual representation of the virtual space from the viewing participant's spatial perspective that is configured to be displayed in a two-dimensional viewport from a fixed perspective;

determining that the speaking participant is directing a speech segment to the addressee participant; and

altering the virtual output stream and an audio output stream for the viewing participant based at least on the positions of the speaking participant and the addressee participant, wherein the altering of the virtual output stream modifies a viewing direction of a virtual representation of each participant based on a gaze direction of the respective participant towards a representation of another participant on the user device display screen.

9. The computer system of claim 8 , wherein the determining is based on at least one content element of speech of the speaking participant.

10. The computer system of claim 8 , wherein the determining is based on detecting at least one gesture of the speaking participant.

11. The computer system of claim 8 , wherein the determining is based on detecting a gaze of the speaking participant to the addressee participant.

12. The computer system of claim 8 , wherein the altering further comprises:

determining a first direction from the speaking participant to the addressee participant and a second direction from the speaking participant to the viewing participant based on each participant's position in the virtual space; and

generating a viewing participant output audio stream by, at least, altering a speaking participant audio stream of the speech segment to include at least one of a first directional feature based on the first direction and a second directional feature based on the second direction.

13. The computer system of claim 12 , wherein the generating is performed by altering the speaking participant audio stream based on at least one gesture of the speaking participant.

14. The computer system of claim 12 , wherein generating the viewing participant output audio stream for a non-addressee participant is performed by altering the speaking participant audio stream to further include at least one clarity feature or at least one indirectness feature based on determining that the speaking participant is directing the speech segment to the addressee participant.

15. A computer program product, the computer program product comprising:

one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor capable of performing a method, the method comprising:

establishing a virtual conference room and connections thereto by a speaking participant, an addressee participant, and a non-addressee participant;

assigning the speaking participant, the addressee participant, and the non-addressee participant to positions in a virtual space;

displaying a virtual output stream of the virtual space to a viewing participant on a user device display screen, wherein the viewing participant is selected from a group consisting of the speaking participant, the addressee participant, and the non-addressee participant, and wherein the virtual output stream comprises a schematic visual representation of the virtual space from the viewing participant's spatial perspective that is configured to be displayed in a two-dimensional viewport from a fixed perspective;

determining that the speaking participant is directing a speech segment to the addressee participant; and

altering the virtual output stream and an audio output stream for the viewing participant based at least on the positions of the speaking participant and the addressee participant, wherein the altering of the virtual output stream modifies a viewing direction of a virtual representation of each participant based on a gaze direction of the respective participant towards a representation of another participant on the user device display screen.

16. The computer program product of claim 15 , wherein the determining is based on at least one content element of speech of the speaking participant.

17. The computer program product of claim 15 , wherein the determining is based on detecting at least one gesture of the speaking participant.

18. The computer program product of claim 15 , wherein the determining is based on detecting a gaze of the speaking participant to the addressee participant.

19. The computer program product of claim 15 , wherein the altering further comprises:

determining a first direction from the speaking participant to the addressee participant and a second direction from the speaking participant to the viewing participant based on each participant's position in the virtual space; and

generating a viewing participant output audio stream by, at least, altering a speaking participant audio stream of the speech segment to include at least one of a first directional feature based on the first direction and a second directional feature based on the second direction.

20. The computer program product of claim 19 , wherein the generating is performed by altering the speaking participant audio stream based on at least one gesture of the speaking participant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 13, 2023
From: FISHER-STAWINSKI, STEVEN LEE; KWATRA, SHIKHAR; MUKHERJEE-ROY, MOITREYEE; SCHNEIDER, SCOTT E.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 063313/0763 →
Cited By (3)
US 12,299,718 US 12,675,955 US 12,712,930