IP Library Granted Patent US 11,891,083
Granted Patent B2
US 11,891,083 · App. 17/544,066 · Granted Feb 6, 2024

System and method for virtual experience as a service with context-based adaptive control

Inventors: Alaa M. Khamis (Courtice, CA); Neeraj R. Gautama (Whitby, CA); Jarvis Chau (Markham, CA); Jomo Hanley (Bowmanville, CA); Kelley J. Kettenbeil (Milford, MI); Gina E. Griffith (Shelby Township, MI); Scott E. Jackson (Northville, MI); Ian Gibbs (Bloomfield Hills, MI); Mark T. Gaylord (Lake Orion, MI); Maria Alejandra Gonzalez (Birmingham, MI); Nicholis Festa (Novi, MI); Nathan Robinson (South Lyon, MI); Ute Winter (Petach Tikva, IL); Sachin Raviram (Novi, MI)
Assignee: GM Global Technology Operations LLC
B60W50/14B60W50/10G06V20/597G06V40/174H04W4/46B60W2050/146
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,891,083
App. No.
17/544,066
Granted
Feb 6, 2024
Kind
B2
Abstract

A system for context-based adaptive virtual experience control in a vehicle is provided. The system includes an output device configured for providing a sensory output to a user of the vehicle and a computerized virtual experience control module configured for controlling the output device based upon a virtual experience mode. The system further includes a computerized context-based adaptive control module configured for monitoring contextual data related to one of the user of the vehicle or operation of the vehicle, monitoring feedback from the user related to one of favor or disfavor related to the virtual experience mode, and utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode.

Claims (48)

1. A system for virtual experience as a service with context-based adaptive control in a vehicle, comprising:

an output device configured for providing a sensory output to a user of the vehicle;

a computerized virtual experience control module configured for controlling the output device based upon a virtual experience mode; and

a computerized context-based adaptive control module configured for:

monitoring contextual data related to one of the user of the vehicle or operation of the vehicle;

monitoring feedback from the user related to one of favor or disfavor related to the virtual experience mode; and

utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode.

2. The system of claim 1 , wherein the computerized context-based adaptive control module configured for monitoring the feedback from the user includes the computerized context-based adaptive control module being configured for one of monitoring user inputs, monitoring activation of a manual control override changing operation of the virtual experience mode, monitoring a frequency of the activation of the manual control override, monitoring a time elapsed between an initiation of the virtual experience mode and the activation of the manual control override, monitoring inputs to a microphone device related to operation of the virtual experience mode, or monitoring facial expressions of the user.

3. The system of claim 2 , further comprising a computerized quality of experience module configured for:

monitoring the feedback from the user; and

generating a feedback reward value including a discreet, scalar value based upon the monitored feedback; and

wherein the computerized context-based adaptive control module configured for monitoring the feedback from the user includes the computerized context-based adaptive control module being configured for monitoring the feedback reward value.

4. The system of claim 1 , wherein the computerized context-based adaptive control module configured for utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode includes the computerized context-based adaptive control module being configured for utilizing a learning algorithm to update the commanded activation.

5. The system of claim 4 , wherein the computerized context-based adaptive control module is further configured for utilizing the monitored feedback and the contextual data to generate contextualized feedback; and

wherein the computerized context-based adaptive control module configured for utilizing the learning algorithm to update the commanded activation includes the computerized context-based adaptive control module being configured for updating the learning algorithm based upon the contextualized feedback.

6. The system of claim 5 , wherein the computerized context-based adaptive control module configured for utilizing the learning algorithm includes the computerized context-based adaptive control module being configured for learning an optimal state-virtual experience activation policy.

7. The system of claim 5 , wherein the computerized context-based adaptive control module configured for utilizing the learning algorithm includes the computerized context-based adaptive control module being configured for utilizing an asynchronous advantage actor-critic algorithm.

8. The system of claim 1 , wherein the computerized context-based adaptive control module configured for utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode includes the computerized context-based adaptive control module being configured for referencing a look-up table.

9. The system of claim 1 , wherein the computerized context-based adaptive control module configured for monitoring the contextual data includes the computerized context-based adaptive control module being configured for monitoring one of an identity of the user, an identity of a passenger in the vehicle, or an identity of a driver of another vehicle.

10. The system of claim 1 , wherein the computerized context-based adaptive control module configured for monitoring the contextual data includes the computerized context-based adaptive control module being configured for monitoring one of an emotional state of the user, a distraction level of the user, a fatigue level of the user, a drowsiness level of the user, a weather condition, a road condition, or a vehicle condition.

11. The system of claim 1 , wherein the computerized context-based adaptive control module configured for monitoring the contextual data includes the computerized context-based adaptive control module being configured for monitoring one of a time of day, occurrence of a threshold social event, occurrence of a threshold sporting event, or occurrence of a vehicle anomaly event.

12. The system of claim 1 , wherein the computerized context-based adaptive control module configured for monitoring the contextual data includes the computerized context-based adaptive control module being configured for monitoring one of a user proximity to the vehicle, a proximity of the vehicle to a home location, a proximity of the vehicle to a workplace location, a proximity of the vehicle to a landmark, or a geographic location of the vehicle.

13. The system of claim 1 , further comprising a remote computerized server device including a virtual experience infrastructure system configured for receiving and publishing new virtual experience modes.

14. The system of claim 1 , further comprising a remote computerized server device including a distributed ledger-based system configured for employing blockchain methodology.

15. The system of claim 1 , further comprising a remote computerized server device including a virtual experience orchestration module enabling a common virtual experience within a plurality of vehicles.

16. A system for virtual experience as a service with context-based adaptive control in a vehicle, comprising:

an output device configured for providing a sensory output to a user of the vehicle;

a computerized virtual experience control module configured for controlling the output device based upon a virtual experience mode; and

a computerized context-based adaptive control module configured for:

monitoring contextual data related to one of the user of the vehicle or operation of the vehicle;

monitoring feedback from the user related to one of favor or disfavor related to the virtual experience mode; and

utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode; and

a computerized quality of experience module configured for:

monitoring the feedback from the user; and

generating a feedback reward value including a discreet, scalar value based upon the monitored feedback;

wherein the computerized quality of experience module configured for monitoring the feedback from the user includes the computerized quality of experience module being configured for one of monitoring user inputs, monitoring activation of a manual control override changing operation of the virtual experience mode, monitoring a frequency of the activation of the manual control override, monitoring a time elapsed between an initiation of the virtual experience mode and the activation of the manual control override, monitoring inputs to a microphone device related to operation of the virtual experience mode, or monitoring facial expressions of the user;

wherein the computerized context-based adaptive control module configured for monitoring the feedback from the user includes the computerized context-based adaptive control module being configured for monitoring the feedback reward value; and

wherein the computerized context-based adaptive control module configured for utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode includes the computerized context-based adaptive control module being configured for utilizing a learning algorithm to update the commanded activation.

17. A method for context-based adaptive virtual experience control in a vehicle, comprising:

operating an output device configured for providing a sensory output to a user of the vehicle;

within a computerized virtual experience control module, controlling the output device based upon a virtual experience mode; and

within a computerized context-based adaptive control module:

monitoring contextual data related to one of the user of the vehicle or operation of the vehicle;

monitoring feedback from the user related to one of favor or disfavor related to the virtual experience mode; and

utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode.

18. The method of claim 17 , wherein monitoring the feedback from the user includes one of monitoring user inputs, monitoring activation of a manual control override changing operation of the virtual experience mode, monitoring a frequency of the activation of the manual control override, monitoring a time elapsed between an initiation of the virtual experience mode and the activation of the manual control override, monitoring inputs to a microphone device related to operation of the virtual experience mode, or monitoring facial expressions of the user.

19. The method of claim 17 , wherein utilizing the contextual data and the feedback from the user to selectively, automatically command activation of the virtual experience mode includes utilizing a learning algorithm to update the commanded activation.

20. The method of claim 17 , wherein monitoring the contextual data includes monitoring one of an identity of the user, an identity of a passenger in the vehicle, or an identity of a driver of another vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2021
From: KHAMIS, ALAA M.; GAUTAMA, NEERAJ R.; CHAU, JARVIS; HANLEY, JOMO; KETTENBEIL, KELLEY J.; GRIFFITH, GINA E.; JACKSON, SCOTT E.; GIBBS, IAN; GAYLORD, MARK T.; GONZALEZ, MARIA ALEJANDRA; FESTA, NICHOLIS; ROBINSON, NATHAN; WINTER, UTE; RAVIRAM, SACHIN
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 058321/0894 →
Continuity (1)
Related Publication 20230174089A1 · Jun 8, 2023