IP Library › Granted Patent US 11,501,657
Granted Patent B2
US 11,501,657 · App. 17/196,949 · Granted Nov 15, 2022

Systems and methods for virtual reality based driver training

Inventors: Anna Marie Madison (Morton, IL); Yixiang Zeng (Urbana, IL); Jeffrey Yin (Fresh Meadows, NY)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
G09B9/05A63F13/212A63F13/30A63F13/46A63F13/53A63F13/803G09B9/052A63F2300/8082
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,501,657
App. No.
17/196,949
Granted
Nov 15, 2022
Kind
B2
Abstract

Systems and methods for providing driver training in a virtual reality environment are disclosed. According to some aspects, an appropriate virtual reality driving simulation may be determined based on one or more input parameters provided by a user. The virtual reality driving simulation may include: (i) an instructional lesson, to be rendered in virtual reality, for teaching driving-related rules and/or skills to a user, and (ii) a driving scenario, to be rendered in virtual reality, for the user to practice the driving-related rules and/or skills taught by the instructional lesson. While the virtual reality driving simulation is rendered, user performance data may be recorded. Based on an analysis of the user performance data, a driving competency score and/or user feedback may be determined.

Claims (44)

1. A computer-implemented method for driver training, comprising:

receiving, via a computer network, at least one input parameter corresponding to at least one feature to be rendered during a virtual reality driving simulation;

determining, by one or more processors, the virtual reality driving simulation based on the at least one input parameter, wherein the virtual reality driving simulation comprises: (i) an instructional lesson, to be rendered in virtual reality, for teaching driving-related rules and/or skills to a user, and (ii) a driving scenario, to be rendered in virtual reality after the instructional lesson, for the user to practice the driving-related rules and/or skills taught by the instructional lesson, wherein the instructional lesson comprises displaying at least a portion of the driving scenario used for practice in combination with information for teaching the driving-related rules and/or skills, and wherein the at least a portion of the driving scenario displayed during the instructional lesson visually corresponds to the at least a portion of the driving scenario used for practice except that the at least a portion of the driving scenario used for practice does not display the information for teaching the driving-related rules and/or skills; and

transmitting, via the computer network, the virtual reality driving simulation for execution on at least one head-mounted virtual reality device.

2. The computer-implemented method of claim 1 , comprising:

receiving, via the computer network, user performance data recorded during execution of the virtual reality driving simulation; and

analyzing, by the one or more processors, the user performance data to determine a driving competency score, wherein the driving competency score measures user compliance with the driving-related rules and/or skills during the driving scenario of the virtual reality driving simulation; and

generating, by the one or more processors, feedback based on the driving competency score.

3. The computer-implemented method of claim 2 , comprising transmitting, via the computer network, the feedback for execution on the at least one head-mounted virtual reality device.

4. The computer-implemented method of claim 3 , comprising:

comparing, by the one or more processors, the driving competency score to a predetermined threshold; and

if the driving competency score is below the predetermined threshold, generating the feedback to include at least: (i) an updated instructional lesson, to be rendered in virtual reality, for re-teaching the driving-related rules and/or skills to the user, and (ii) an updated driving scenario, to be rendered in virtual reality, for the user to practice the driving-related rules and/or skills taught by the updated instructional lesson.

5. The computer-implemented method of claim 3 , comprising:

comparing, by the one or more processors, the driving competency score to a predetermined threshold; and

if the driving competency score is equal to or above the predetermined threshold, generating the feedback to include at least: (i) an updated instructional lesson, to be rendered in virtual reality, for teaching additional driving-related rules and/or skills to the user, and (ii) an updated driving scenario, to be rendered in virtual reality, for the user to practice the additional driving-related rules and/or skills taught by the updated instructional lesson.

6. The computer-implemented method of claim 1 , wherein the user performance data is representative of at least one of: controls operated by the user during the virtual reality driving simulation, movement of a head of the user during the virtual reality driving simulation, movement of one or more eyes of the user during the virtual reality driving simulation, movement of one or more arms of the user during the virtual reality driving simulation, movement of one or more hands of the user during the virtual reality driving simulation, movement of a torso of the user during the virtual reality driving simulation, a heart rate of the user during the virtual reality driving simulation, a blood pressure of the user during the virtual reality driving simulation, a body temperature of the user during the virtual reality driving simulation, or a breathing characteristic of the user during the virtual reality driving simulation.

7. The computer-implemented method of claim 1 , wherein the information for teaching the driving-related rules and/or skills comprises a visual cue superimposed on and identifying an aspect of the at least a portion of the driving scenario.

8. The computer-implemented method of claim 1 , wherein the at least a portion of the driving scenario displayed during the instructional lesson is visually identical to the at least a portion of the driving scenario used for practice except that the at least a portion of the driving scenario used for practice does not display the information for teaching the driving-related rules and/or skills.

9. A computer-implemented method for driver training, comprising:

receiving, via one or more client devices, at least one input parameter corresponding to at least one feature to be rendered during a virtual reality driving simulation, wherein the one or more client devices includes a head-mounted virtual reality device;

transmitting, via a computer network, the at least one input parameter;

receiving, via the computer network, a particular virtual reality driving simulation based on the at least one input parameter; and

executing, via the one or more client devices, the virtual reality driving simulation such that the head-mounted virtual reality device renders at least: (i) an instructional lesson for teaching driving-related rules and/or skills to a user, and (ii) after rendering the instructional lesson, a driving scenario for the user to practice the driving-related rules and/or skills taught by the instructional lesson, wherein the instructional lesson comprises displaying at least a portion of the driving scenario used for practice in combination with information for teaching the driving-related rules and/or skills, and wherein the at least a portion of the driving scenario displayed during the instructional lesson visually corresponds to the at least a portion of the driving scenario used for practice except that the at least a portion of the driving scenario used for practice does not display the information for teaching the driving-related rules and/or skills.

10. The computer-implemented method of claim 9 , comprising:

recording, via the one or more client devices, user performance data during execution of the virtual reality driving simulation;

transmitting, via the one or more client devices, the user performance data; and

receiving, via the one or more client devices, feedback based on a driving competency score, wherein the driving competency score measures user compliance with the driving-related rules and/or skills during the driving scenario of the virtual reality driving simulation.

11. The computer-implemented method of claim 10 , comprising executing, via the one or more client devices, the feedback such that, if the driving competency score is below a predetermined threshold, the head-mounted virtual reality device renders at least: (i) an updated instructional lesson for re-teaching the driving-related rules and/or skills to the user, and (ii) an updated driving scenario for the user to practice the driving-related rules and/or skills taught during the updated instructional lesson.

12. The computer-implemented method of claim 11 , comprising executing, via the one or more client devices, the feedback such that, if the driving competency score is equal to or above a predetermined threshold, the head-mounted virtual reality device renders at least: (i) an updated instructional lesson for teaching additional driving-related rules and/or skills to the user, and (ii) an updated driving scenario for the user to practice the additional driving-related rules and/or skills taught during the updated instructional lesson.

13. The computer-implemented method of claim 9 , comprising, recording, via the one or more client devices, user performance data during execution of the virtual reality driving simulation, wherein the user performance data is representative of at least one of: controls operated by the user during the virtual reality driving simulation, movement of a head of the user during the virtual reality driving simulation, movement of one or more eyes of the user during the virtual reality driving simulation, movement of one or more arms of the user during the virtual reality driving simulation, movement of one or more hands of the user during the virtual reality driving simulation, movement of a torso of the user during the virtual reality driving simulation, a heart rate of the user during the virtual reality driving simulation, a blood pressure of the user during the virtual reality driving simulation, a body temperature of the user during the virtual reality driving simulation, or a breathing characteristic of the user during the virtual reality driving simulation.

14. The computer-implemented method of claim 9 , wherein the at least a portion of the driving scenario displayed during the instructional lesson is visually identical to the at least a portion of the driving scenario used for practice except that the at least a portion of the driving scenario used for practice does not display the information for teaching the driving-related rules and/or skills.

15. A system for driver training, comprising:

one or more processors;

a head-mounted virtual reality display device; and

a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the system to:

receive at least one input parameter corresponding to at least one feature to be rendered during a virtual reality driving simulation,

transmit the at least one input parameter,

receive a particular virtual reality driving simulation based on the at least one input parameter, and

execute the virtual reality driving simulation such that the head-mounted virtual reality device renders at least: (i) an instructional lesson for teaching driving-related rules and/or skills to the user, and (ii) after rendering the instructional lesson, a driving scenario for the user to practice the driving-related rules and/or skills taught by the instructional lesson, wherein the instructional lesson comprises displaying at least a portion of the driving scenario used for practice in combination with information for teaching the driving-related rules and/or skills, and wherein the at least a portion of the driving scenario displayed during the instructional lesson visually corresponds to the at least a portion of the driving scenario used for practice except that the at least a portion of the driving scenario used for practice does not display the information for teaching the driving-related rules and/or skills.

16. The system of claim 15 , the non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the system to receive feedback based on a driving competency score, wherein the driving competency score measures user compliance with the driving-related rules and/or skills during the driving scenario of the virtual reality driving simulation.

17. The system of claim 16 , the non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the system to execute the feedback such that, if the driving competency score is below a predetermined threshold, the head-mounted virtual reality device renders at least: (i) an updated instructional lesson for re-teaching the driving-related rules and/or skills to the user, and (ii) an updated driving scenario for the user to practice the driving-related rules and/or skills taught during the updated instructional lesson.

18. The system of claim 16 , the non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the system to execute the feedback such that, if the driving competency score is equal to or above a predetermined threshold, the head-mounted virtual reality device renders at least: (i) an updated instructional lesson for teaching additional driving-related rules and/or skills to the user, and (ii) an updated driving scenario for the user to practice the additional driving-related rules and/or skills taught during the updated instructional lesson.

19. The system of claim 15 , wherein the information for teaching the driving-related rules and/or skills comprises a visual cue superimposed on and identifying an aspect of the at least a portion of the driving scenario.

20. The system of claim 15 , wherein the at least a portion of the driving scenario displayed during the instructional lesson is visually identical to the at least a portion of the driving scenario used for practice except that the at least a portion of the driving scenario used for practice does not display the information for teaching the driving-related rules and/or skills.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 11, 2021
From: MADISON, ANNA MARIE; ZENG, YIXIANG; YIN, JEFFREY
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 055568/0404 →
Continuity (2)
Continuation 15340646 · Nov 1, 2016
Related Publication 20210192968A1 · Jun 24, 2021
Cited By (1)
US 12,515,523