IP Library › Granted Patent US 12,027,061
Granted Patent B2
US 12,027,061 · App. 17/838,236 · Granted Jul 2, 2024

Systems and methods for artificial intelligence (AI) virtual reality (VR) emotive conversation training

Inventors: Ryan M. Scanlon (East Hampton, CT); Hoa Ton-That (Glastonbury, CT); Douglas L. Roy (Plantsville, CT); Michael C. Kunkel (Glastonbury, CT); Pee T. Lim (Middletown, CT); Andrea Vazquez (Dallas, TX)
Assignee: The Travelers Indemnity Company
G09B19/00G06F40/289G09B5/065G10L25/63
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,027,061
App. No.
17/838,236
Granted
Jul 2, 2024
Kind
B2
Abstract

Systems and methods for Artificial Intelligence (AI) Virtual Reality (VR) emotive conversation training.

Claims (40)

1. A system for Artificial Intelligence (AI) Virtual Reality (VR) emotive conversation training, comprising:

a conversation training controller comprising a plurality of electronic processing devices; and

a non-transitory data storage device in communication with the conversation training controller, the non-transitory data storage device storing (i) an AI natural language intent model, (ii) a conversational state program, and (iii) instructions that when executed by the conversation training controller, result in:

generating, by an execution of the conversational state program by the conversation training controller, a first virtual conversational element of a simulated conversation, wherein the first virtual conversational element comprises a first version of a Virtual Reality (VR) avatar;

outputting, to a human participant of the simulated conversation and via a VR headset, an indication of the first virtual conversational element, wherein the outputting further comprises generating a first version of a VR environment and outputting the first version of the VR environment via the VR headset, and

wherein the first version of the VR environment comprises the first version of the VR avatar, and the first virtual conversational element further comprises a positioning of the VR avatar at a first location in the virtual environment that is offset by a first distance from a center of eye orientation of the human participant;

receiving, from a sensor device, input descriptive of a first human conversational element of the human participant of the simulated conversation;

computing, by an execution of the AI natural language intent model by the conversation training controller, and utilizing the first human conversational element of the human participant of the simulated conversation as input, a first human intent metric;

generating, by an execution of the conversational state program by the conversation training controller, and utilizing the first human intent metric as input, a second virtual conversational element of the simulated conversation,

wherein the second virtual conversational element comprises a second version of the VR avatar, and wherein the second virtual conversational element comprises a physical attribute of the VR avatar;

outputting, to the human participant of the simulated conversation and via the VR headset, an indication of the second virtual conversational element,

wherein the outputting further comprises generating a second version of the VR environment and outputting the second version of the VR environment via the VR headset, and wherein the second version of the VR environment comprises the second version of the VR avatar, and the physical attribute of the VR avatar comprises a positioning of the VR avatar at a second location in the virtual environment that is offset by a second distance from the center of eye orientation of the human participant;

identifying, by the conversation training controller, a score assigned to the second virtual conversational element of the simulated conversation;

computing, by the conversation training controller and utilizing the score assigned to the second virtual conversational element of the simulated conversation, an outcome of the simulated conversation; and

outputting, to the human participant of the simulated conversation, an indication of the outcome of the simulated conversation.

2. The system of claim 1 , wherein the input descriptive of the first human conversational element of the human participant of the simulated conversation is received in response to the outputting of the indication of the first virtual conversational element.

3. The system of claim 1 , wherein the indication of the first virtual conversational element comprises a first computer-generated audio segment comprising a first demeanor.

4. The system of claim 3 , wherein the indication of the second virtual conversational element further comprises a second computer-generated audio segment comprising a second demeanor.

5. The system of claim 1 , wherein the physical attribute of the VR avatar further comprises at least one of a mouth attribute, a lip attribute, a cheek attribute, an eye attribute, an eyebrow attribute, a head tilt attribute, and an additional avatar position attribute.

6. The system of claim 5 , wherein the physical attribute further comprises the additional avatar position attribute and wherein the additional avatar position attribute comprises a virtual distance of the VR avatar from a virtual position of the human participant.

7. The system of claim 1 , wherein the input descriptive of the first human conversational element of the human participant of the simulated conversation comprises audio input.

8. The system of claim 7 , wherein the computing of the first human intent metric by the AI natural language intent model comprises:

identifying at least one of a cadence, a volume, a pitch, and a tone of the audio input; and

computing, utilizing the identified at least one of the cadence, the volume, the pitch, and the tone of the audio input as input for trained AI logic, the first human intent metric.

9. The system of claim 1 , wherein the input descriptive of the first human conversational element of the human participant of the simulated conversation comprises sensor data descriptive of the human participant of the simulated conversation.

10. The system of claim 9 , wherein the computing of the first human intent metric by the AI natural language intent model comprises:

identifying, based on the sensor data, at least one of an eye gaze direction, a head angle, a shoulder angle, a hand gesture, a foot position, and a stance of the human participant of the simulated conversation; and

computing, utilizing the identified at least one of the eye gaze direction, the head angle, the shoulder angle, the hand gesture, the foot position, and the stance of the human participant of the simulated conversation as input for trained AI logic, the first human intent metric.

11. The system of claim 1 , wherein the first human intent metric comprises one of a plurality of emotion levels.

12. The system of claim 1 , wherein the first virtual conversational element further comprises a first demeanor of the VR avatar, and wherein the second virtual conversational element further comprises a second demeanor of the VR avatar.

13. The system of claim 1 , wherein the computing of the second virtual conversational element of the simulated conversation further utilizes a random number seed as input.

14. The system of claim 1 , wherein the conversational state program defines a simulated conversation decision tree, wherein each of the first and second virtual conversational elements comprise leaves of the decision tree, wherein the score assigned to the second virtual conversational element of the simulated conversation is assigned to a corresponding leaf of the decision tree, and wherein other scores are assigned to other leaves of the decision tree.

15. The system of claim 14 , wherein the conversational state program defines at least one target path through the decision tree and wherein the outcome of the simulated conversation is based at least in part on a comparison of a path associated with the simulated conversation and the at least one target path.

16. The system of claim 1 , wherein the instructions, when executed by the conversation training controller, further result in:

receiving, from the sensor device and in response to the outputting of the indication of the second virtual conversational element, input descriptive of a second human conversational element of the human participant of the simulated conversation;

computing, by an execution of the AI natural language intent model by the conversation training controller, and utilizing the second human conversational element of the human participant of the simulated conversation as input, a second human intent metric;

generating, by an execution of the conversational state program by the conversation training controller, and utilizing the second human intent metric as input, a third virtual conversational element of the simulated conversation;

identifying, by the conversation training controller, a score assigned to the third virtual conversational element of the simulated conversation; and

wherein the computing of the outcome of the simulated conversation by the conversation training controller further comprises utilizing the score assigned to the third virtual conversational element of the simulated conversation.

17. The system of claim 16 , wherein the computing of the outcome of the simulated conversation comprises adding the scores of the second and third virtual conversational elements of the simulated conversation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2023
From: SCANLON, RYAN M.; TON-THAT, HOA; ROY, DOUGLAS L.; KUNKEL, MICHAEL C.; LIM, PEE T.; VAZQUEZ, ANDREA
To: THE TRAVELERS INDEMNITY COMPANY
Reel/Frame 064821/0795 →
Continuity (1)
Related Publication 20230401976A1 · Dec 14, 2023