IP Library Granted Patent US 11,010,645
Granted Patent B2
US 11,010,645 · App. 16/551,363 · Granted May 18, 2021

Interactive artificial intelligence analytical system

Inventors: JiaoJiao Xu (Pittsburgh, PA); Yi Xu (Pittsburgh, PA); Chenchen Zhu (Pittsburgh, PA); Matthew Thomas Spettel (Merrimack, NH)
Assignee: TalkMeUp
G06K9/629G06K9/00302G06K9/00744G06N3/02G09B19/00G10L15/26G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,010,645
App. No.
16/551,363
Granted
May 18, 2021
Kind
B2
Abstract

A method and system for an AI-based communication training system for individuals and organizations is disclosed. A video analyzer is used to convert a video signal into a plurality of human morphology features with an accompanying audio analyzer converting an audio signal into a plurality of human speech features. A transformation module transforms the morphology features and the speech features into a current multi-dimensional performance vector and combinatorial logic generates an integration of the current multi-dimensional performance vector and one or more prior multi-dimensional performance vectors to generate a multi-session rubric. Backpropagation logic applies a current multi-dimensional performance vector from the combinatorial logic to the video analyzer and the audio analyzer.

Claims (47)

1. A system comprising:

a video analyzer to convert a video signal into a plurality of human morphology features;

an audio analyzer to convert an audio signal into a plurality of human speech features;

a transformation module to transform the human morphology features and the human speech features into performance metrics for passion, content, and engagement in a current multi-feature performance vector;

combinatorial logic to generate an integration of the current multi-feature performance vector and one or more prior multi-feature performance vectors to generate a multi-session rubric;

a plurality of behavioral models each configurable as a scoring control on the combinatorial logic such that scores generated for the integration in the multi-session rubric by the combinatorial logic vary according to the behavioral model configured as a scoring control for the combinatorial logic;

wherein the combinatorial logic comprises a supervised and unsupervised machine learning system;

the multi-session rubric applied to generate visual and auditory behavioral change recommendations for a human operator; and

backpropagation logic to apply the current multi-feature performance vector from the combinatorial logic to the video analyzer and the audio analyzer.

2. The system of claim 1 , wherein the video analyzer comprises one or more models embodied in a neural network.

3. The system of claim 2 , wherein the neural network comprises at least one convolutional neural network.

4. The system of claim 2 , wherein the neural network comprises at least one convolutional neural network coupled to at least one recurrent neural network.

5. The system of claim 1 , further comprising a speech-to-text converter.

6. The system of claim 5 , further comprising a natural language processor coupled to receive text from the speech-to-text converter.

7. The system of claim 1 , the audio analyzer comprising a canny edge detector and a Fast Fourier Transform.

8. The system of claim 1 , wherein the multi-session rubric comprises scores for a plurality of second-level performance features grouped within top-level categories for passion, content, and engagement.

9. A method comprising:

converting a video signal into a plurality of human morphology features with a video analyzer;

converting an audio signal into a plurality of human speech features with an audio analyzer;

utilizing supervised and unsupervised machine learning models to transform the human morphology features and the human speech features into performance metrics for passion, content, and engagement in a current multi-feature performance vector;

generating an integration, by combinatorial logic, of the current multi-feature performance vector and one or more prior multi-feature performance vectors;

configuring one of a plurality of behavioral models as a scoring control on the combinatorial logic such that scores generated for the integration in a multi-session rubric by the combinatorial logic vary according to the behavioral features against model configured as a scoring control for combinatorial logic;

wherein the multi-session rubric comprises a plurality of second-level performance scores grouped within top-level categories of passion, content, and engagement;

applying the multi-session rubric to generate visual and auditory behavioral change recommendations for a human operator; and

applying the current multi-feature performance vector as an adaptive feedback signal to the video analyzer and the audio analyzer.

10. The method of claim 9 , wherein the video analyzer comprises one or more models embodied in a neural network.

11. The method of claim 10 , wherein the neural network comprises at least one convolutional neural network.

12. The method of claim 10 , wherein the neural network comprises at least one convolutional neural network coupled to at least one recurrent neural network.

13. The method of claim 9 , wherein transforming the human speech features is performed with a speech-to-text converter.

14. The method of claim 13 , further comprising the use of a natural language processor coupled to receive text from the speech-to-text converter.

15. The method of claim 9 , wherein the audio analyzer comprises a canny edge detector and a Fast Fourier Transform.

16. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor, cause a machine comprising the processor to:

convert a video signal into a plurality of human morphology features with a video analyzer;

convert an audio signal into a plurality of human speech features with an audio analyzer;

utilize supervised and unsupervised machine learning models to transform the human morphology features and the human speech features into performance metrics for passion, content, and engagement in a current multi-feature performance vector;

generate an integration, by combinatorial logic, of the current multi-feature performance vector and one or more prior multi-feature performance vectors;

configure one of a plurality of behavioral models as a scoring control on the combinatorial logic such that scores generated for the integration in a multi-session rubric by the combinatorial logic vary according to the behavioral model configured as a scoring control for combinatorial logic;

wherein the multi-session rubric comprises a plurality of second-level performance scores grouped within top-level categories of passion, content, and engagement;

apply the multi-session rubric to generate visual and auditory behavioral change recommendations for a human operator; and

apply the current multi-feature performance vector as an adaptive feedback signal to the video analyzer and the audio analyzer.

17. The non-transitory computer-readable storage medium of claim 16 , the computer-readable storage medium further including instructions that when executed by the processor, cause the machine comprising the processor to:

convert the video signal into the plurality of human morphology features using the video analyzer, wherein the video analyzer comprises one or more models embodied in a neural network;

transform the human speech features using a speech-to-text converter; and

convert the audio signal into the plurality of human speech features using the audio analyzer, wherein the audio analyzer comprises a canny edge detector and a Fast Fourier Transform.

18. The non-transitory computer-readable storage medium of claim 17 , wherein:

the neural network comprises at least one convolutional neural network; and

the neural network comprises the at least one convolutional neural network coupled to at least one recurrent neural network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 4, 2019
From: XU, JIAOJIAO; XU, YI; ZHU, CHENCHEN; SPETTEL, MATTHEW THOMAS
To: TALKMEUP INC.
Reel/Frame 050258/0147 →
Continuity (2)
Provisional Application 62723369 · Aug 27, 2018
Related Publication 20200065612A1 · Feb 27, 2020
Cited By (2)
US 12,455,636 US 12,609,107