IP Library Granted Patent US 11,228,731
Granted Patent B1
US 11,228,731 · App. 16/905,050 · Granted Jan 18, 2022

Artificial intelligence communication assistance in audio-visual composition

Inventors: Oleksiy Shevchenko (West Vancouver, CA); Ayan Mandal (Oakdale, CA); Bradley Jon Hoover (San Francisco, CA); Joel Tetreault (New York, NY); Maksym Lytvyn (West Vancouver, CA); Dmytro Lider (Kyiv, UA)
Assignee: GRAMMARLY, INC.
H04N7/147G06F9/453G06N20/00G10L15/197G10L15/22H04N7/148
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,228,731
App. No.
16/905,050
Granted
Jan 18, 2022
Kind
B1
Abstract

In embodiments of the present invention improved capabilities are described for artificial intelligence communication assistance for aiding in the audio-visual composition of electronic communications.

Claims (20)

1. A method of electronic communication assistance, the method comprising: receiving an audio-visual electronic communication at an artificial intelligence assistant computing platform from a first user, the audio-visual electronic communication comprising an audio communication content and a video communication content, wherein an intended recipient of the audio-visual electronic communication is a second user; extracting an audio communication information from the audio communication content; extracting a video communication information from the video communication content; processing the audio-visual electronic communication with a processor to generate a compositional change for the communication content of the audio-visual electronic communication based on non-verbal information determined from the extracted audio communication information or the extracted video communication information; generating a changed electronic communication from the audio-visual electronic communication and the compositional change, the changed electronic communication being a modified version of the audio-visual electronic communication, by directly making at least one change to the audio-visual electronic communication, wherein the at least one change includes a style transformation that is a vocabulary shift; and providing the changed electronic communication.

2. The method of claim 1 , wherein the non-verbal information comprises body language information.

3. The method of claim 2 , wherein the compositional change is made based on processing the audio-visual electronic communication for a psycho-emotional state of the first user based on the body language information.

4. The method of claim 3 , wherein the psycho-emotional state of the first user is an emotional state or a stress level state.

5. The method of claim 2 , wherein the compositional change is made based on processing the audio-visual electronic communication for an intent of the first user based on the body language information.

6. The method of claim 5 , wherein the body language information includes facial expression information, posture information, or gesture information.

7. The method of claim 1 , wherein the non-verbal information comprises voice tone information.

8. The method of claim 7 , wherein the compositional change is made based on processing the audio-visual electronic communication for a psycho-emotional state of the first user based on the voice tone information.

9. The method of claim 8 , wherein the psycho-emotional state of the first user is an emotional state or a stress level state.

10. The method of claim 1 , wherein the non-verbal information comprises environmental information.

11. The method of claim 10 , wherein the environmental information is background audio information, wherein the compositional change is made based on processing the audio-visual electronic communication for an environmental state based on the background audio information.

12. The method of claim 10 , wherein the environmental information is background of a video scene information, wherein the compositional change is made based on processing the audio-visual electronic communication for an environmental state based on the background of a video scene information.

13. The method of claim 1 , further comprising processing the audio-visual electronic communication with the processor to generate the compositional change for the communication content of the audio-visual electronic communication based on sensor information received from a wearable user device.

14. The method of claim 13 , wherein the sensor information comprises biometric sensor information, wherein the compositional change is made based on processing the audio-visual electronic communication for a psycho-emotional state of the first user based on the biometric sensor information.

15. The method of claim 14 , wherein the psycho-emotional state of the first user is an emotional state or a stress level state.

16. A system comprising: a server computer comprising a processor and a computer-readable storage device that stores instructions that, when executed by the processor, cause the processor to perform operations comprising: receiving an audio-visual electronic communication at an artificial intelligence assistant computing platform from a first user, the audio-visual electronic communication comprising an audio communication content and a video communication content, wherein an intended recipient of the audio-visual electronic communication is a second user; extracting an audio communication information from the audio communication content; extracting a video communication information from the video communication content; processing the audio-visual electronic communication with a processor to generate a compositional change for the communication content of the audio-visual electronic communication based on non-verbal information determined from the extracted audio communication information or the extracted video communication information; generating a changed electronic communication from the audio-visual electronic communication and the compositional change, the changed electronic communication being a modified version of the audio-visual electronic communication, by directly making at least one change to the audio-visual electronic communication, wherein the at least one change includes a style transformation that is a vocabulary shift; and providing the changed electronic communication.

17. The system of claim 16 , wherein the non-verbal information comprises body language information, wherein the compositional change is made based on processing the audio-visual electronic communication for a psycho-emotional state or an intent of the first user based on the body language information.

18. The system of claim 16 , wherein the non-verbal information comprises voice tone information, wherein the compositional change is made based on processing the audio-visual electronic communication for a psycho-emotional state of the first user based on the voice tone information.

19. The system of claim 16 , wherein the non-verbal information comprises environmental information, wherein the environmental information is background information, wherein the compositional change is made based on processing the audio-visual electronic communication for an environmental state based on the background information.

20. The system of claim 16 , further comprising processing the audio-visual electronic communication with the processor to generate the compositional change for the communication content of the audio-visual electronic communication based on sensor information received from a wearable user device, wherein the sensor information comprises biometric sensor information, and the compositional change is made based on processing the audio-visual electronic communication for a psycho-emotional state of the first user based on the biometric sensor information.

Assignments (2)
CHANGE OF NAME Recorded Nov 21, 2025
From: GRAMMARLY, INC.
To: SUPERHUMAN PLATFORM INC.
Reel/Frame 073655/0099 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: SHEVCHENKO, OLEKSIY; MANDAL, AYAN; HOOVER, BRADLEY JON; TETREAULT, JOEL; LYTVYN, MAKSYM; LIDER, DMYTRO
To: GRAMMARLY, INC.
Reel/Frame 053097/0268 →
Continuity (2)
Continuation 16055036 · Aug 4, 2018
Provisional Application 62541203 · Aug 4, 2017
Cited By (2)
US 12,204,852 US 12,333,247