IP Library › Granted Patent US 11,938,819
Granted Patent B2
US 11,938,819 · App. 17/574,726 · Granted Mar 26, 2024

Evaluation of a simulated vehicle-related feature

Inventors: Casper Wickman (Öckerö, SE); Mattias Wikenmalm (Torslanda, SE); Timotei Ghiurau (Gothenburg, SE); Jon Seneger (Boulder Creek, CA)
Assignee: Volvo Car Corporation
B60K35/00G02B27/0093G02B27/0172G02B27/0179G06F3/013G06T19/006B60K2370/149B60K2370/1529B60K2370/31G02B2027/0181G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,938,819
App. No.
17/574,726
Granted
Mar 26, 2024
Kind
B2
Abstract

A method performed by a vehicle feature evaluation system for enabling evaluation of a simulated vehicle-related feature. The vehicle feature evaluation system determines in relation to a road-driven vehicle, with support from a tracking system, an orientation of a head-mounted display, HMD, adapted to be worn by an occupant on-board the road-driven vehicle or position and gaze direction of the occupant. The vehicle feature evaluation system further determines a simulated vehicle design feature to be evaluated in the road-driven vehicle. Moreover, the vehicle feature evaluation system provides in real-time to a HMD display of the HMD or another display, taking into consideration the HMD orientation or position and gaze direction, a virtual representation of the simulated vehicle design feature superimposed on a real-time surrounding-showing image derived from real-world perception sensor data captured with support from one or more vehicle-attached perception sensors adapted to capture surroundings internal or external to the road-driven vehicle.

Claims (86)

1. A method performed by a vehicle feature evaluation system for enabling evaluation of a simulated vehicle-related feature, the method comprising:

determining in relation to a road-driven vehicle, with support from a tracking system, an orientation of a head mounted display, HMD, adapted to be worn by an occupant on-board the road-driven vehicle or a position and gaze direction of the occupant on-board the road-driven vehicle;

determining a simulated vehicle design feature to be evaluated in the road-driven vehicle; and

providing in real-time to a HMD display of the HMD or another display visible to the occupant, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated vehicle design feature superimposed on a real-time surrounding-showing image derived from real-world perception sensor data captured with support from one or more vehicle-attached perception sensors adapted to capture surroundings internal or external to the road-driven vehicle, wherein the virtual representation of the simulated vehicle design feature replaces an excluded portion of the real-time surrounding-showing image, and wherein the virtual representation of the simulated vehicle design feature and the replaced excluded portion of the real-time surrounding-showing image are varied based on the HMD orientation or the position and gaze direction.

2. The method according to claim 1 , further comprising receiving one or more of the real-world perception sensor data and an external dataset and enhancing one or more of the virtual representation of the simulated vehicle design feature and the real-time surrounding showing image based on the real-world perception sensor data and an external dataset.

3. The method according to claim 1 , further comprising receiving one or more of the real-world perception sensor data and an external dataset and supplementing one or more of the virtual representation of the simulated vehicle design feature and the real-time surrounding showing image using on the real-world perception sensor data and an external dataset by providing an audio supplementation, a haptic supplementation, a taste supplementation, and a smell supplementation.

4. The method according to claim 1 , wherein the simulated vehicle design feature comprises one or more of a simulated vehicle interior section and a simulated vehicle exterior section.

5. The method according to claim 1 , further comprising:

determining a simulated vehicle functionality feature to be evaluated in the road-driven vehicle;

wherein the providing further comprises additionally providing in real-time to the HMD display or the another display, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated vehicle functionality feature superimposed on the real-time surrounding-showing image, which simulated vehicle functionality feature comprises one or more of:

a simulated vehicle human machine interface, HMI;

a simulated vehicle display;

a simulated vehicle head-up display, HUD;

simulated light characteristics;

a simulated effect resulting from a simulation of the simulated vehicle functionality feature; and

a dynamic simulation.

6. The method according to claim 1 , further comprising:

determining a simulated object feature to be evaluated in the road-driven vehicle;

wherein the providing further comprises additionally providing in real-time to the HMD display or the another display, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated object feature superimposed on the real-time surrounding-showing image, which simulated object feature comprises one or more of:

a simulated other vehicle;

a simulated road user; and

a simulated moving or fixed obstacle.

7. The method according to claim 1 , further comprising:

determining one or more of:

a fictive design feature location of the simulated vehicle design feature relative the road-driven vehicle;

a fictive functionality feature location of the simulated vehicle functionality feature relative the road-driven vehicle; and

a fictive object feature location of the simulated object feature relative the road-driven vehicle;

wherein the providing taking into consideration the HMD orientation or the position and gaze direction comprises one or more of:

providing the virtual design feature representation superimposed on the real-time surrounding-showing image such that positioning of the virtual design feature representation corresponds to the fictive design feature location;

providing a virtual functionality feature representation superimposed on the real-time surrounding-showing image such that positioning of the virtual functionality feature representation corresponds to the fictive functionality feature location; and

providing the virtual object feature representation superimposed on the real-time surrounding-showing image such that positioning of the virtual object feature representation corresponds to the fictive object feature location.

8. The method according to claim 7 , further comprising one or more of:

updating the virtual functionality feature representation based on a user interaction with the simulated vehicle functionality feature, which user interaction is based on detection, by means of one or more user interaction sensors, of a user in and/or at the fictive functionality feature location; and

updating the virtual functionality feature representation based on a vehicle signal affecting the simulated vehicle functionality feature, derived from the road-driven vehicle.

9. The method according to claim 1 , further comprising updating one or more of:

the real-time surrounding-showing image;

the superimposed virtual design feature representation;

a superimposed virtual functionality feature representation; and

a superimposed virtual object feature representation based on updated captured perception sensor data and/or based on an updated determined HMD orientation or position and gaze direction.

10. A vehicle feature evaluation system for enabling evaluation of a simulated vehicle-related feature, the vehicle feature evaluation system comprising:

a HMD orientation or position and gaze direction determining unit comprising memory storing instructions executed by a processor for determining in relation to a road-driven vehicle, with support from a tracking system, an orientation of a head mounted display, HMD, adapted to be worn by an occupant on-board the road-driven vehicle or a position and gaze direction of the occupant on-board the road-driven vehicle;

a design feature determining unit comprising memory storing instructions executed by the processor for determining a simulated vehicle design feature to be evaluated in the road-driven vehicle; and

a HMD or display providing unit comprising memory storing instructions executed by the processor for providing in real-time to a HMD display of the HMD or another display visible to the occupant, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated vehicle design feature superimposed on a real-time surrounding-showing image derived from real-world perception sensor data captured with support from one or more vehicle-attached perception sensors adapted to capture surroundings internal or external to the road-driven vehicle, wherein the virtual representation of the simulated vehicle design feature replaces an excluded portion of the real-time surrounding-showing image, and wherein the virtual representation of the simulated vehicle design feature and the replaced excluded portion of the real-time surrounding-showing image are varied based on the HMD orientation or the position and gaze direction.

11. The vehicle feature evaluation system according to claim 10 , the HMD or display providing unit further comprising memory storing instructions executed by the processor for receiving one or more of the real-world perception sensor data and an external dataset and enhancing one or more of the virtual representation of the simulated vehicle design feature and the real-time surrounding showing image based on the real-world perception sensor data and an external dataset.

12. The vehicle feature evaluation system according to claim 10 , the HMD or display providing unit further comprising memory storing instructions executed by the processor for receiving one or more of the real-world perception sensor data and an external dataset and supplementing one or more of the virtual representation of the simulated vehicle design feature and the real-time surrounding showing image using on the real-world perception sensor data and an external dataset by providing an audio supplementation, a haptic supplementation, a taste supplementation, and a smell supplementation.

13. The vehicle feature evaluation system according to claim 10 , wherein the simulated vehicle design feature comprises one or more of a simulated vehicle interior section and a simulated vehicle exterior section.

14. The vehicle feature evaluation system according to claim 10 , further comprising:

a functionality feature determining unit comprising memory storing instructions executed by the processor for determining a simulated vehicle functionality feature to be evaluated in the road-driven vehicle;

wherein the HMD or display providing unit is adapted for additionally providing in real-time to the HMD display or the another display, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated vehicle functionality feature superimposed on the real-time surrounding-showing image, which simulated vehicle functionality feature comprises one or more of:

a simulated vehicle human machine interface, HMI;

a simulated vehicle display;

a simulated vehicle head-up display, HUD;

simulated light characteristics;

a simulated effect resulting from a simulation of the simulated vehicle functionality feature; and

a dynamic simulation.

15. The vehicle feature evaluation system according to claim 10 , further comprising:

an object feature determining unit comprising memory storing instructions executed by the processor for determining a simulated object feature to be evaluated in the road-driven vehicle;

wherein the HMD or display providing unit is adapted for additionally providing in real-time to the HMD display or the another display, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated object feature superimposed on the real-time surrounding-showing image, which simulated object feature comprises one or more of:

a simulated other vehicle;

a simulated road user; and

a simulated moving or fixed obstacle.

16. The vehicle feature evaluation system according to claim 10 , further comprising:

a location determining unit comprising memory storing instructions executed by the processor for determining one or more of:

a fictive design feature location of the simulated vehicle design feature relative the road-driven vehicle;

a fictive functionality feature location of the simulated vehicle functionality feature relative the road-driven vehicle; and

a fictive object feature location of the simulated object feature relative the road-driven vehicle;

wherein the HMD or display providing unit further is adapted for one or more of:

providing the virtual design feature representation superimposed on the real-time surrounding-showing image such that positioning of the virtual design feature representation corresponds to the fictive design feature location;

providing a virtual functionality feature representation superimposed on the real-time surrounding-showing image such that positioning of the virtual functionality feature representation corresponds to the fictive functionality feature location; and

providing the virtual object feature representation superimposed on the real-time surrounding-showing image such that positioning of the virtual object feature representation corresponds to the fictive object feature location.

17. The vehicle feature evaluation system according to claim 16 , further comprising one or more of:

a user interaction updating unit comprising memory storing instructions executed by the processor for updating the virtual functionality feature representation based on a user interaction with the simulated vehicle functionality feature, which user interaction for instance is based on detection, by means of one or more user interaction sensors, of a user in and/or at the fictive functionality feature location; and

a vehicle signal updating unit comprising memory storing instructions executed by the processor for updating the virtual functionality feature representation based on a vehicle signal affecting the simulated vehicle functionality feature, derived from the road-driven vehicle.

18. The vehicle feature evaluation system according to claim 10 , further comprising:

a continuous updating unit comprising memory storing instructions executed by the processor for updating one or more of:

the real-time surrounding-showing image;

the superimposed virtual design feature representation;

a superimposed virtual functionality feature representation; and

a superimposed virtual object feature representation based on updated captured perception sensor data and/or based on and updated determined HMD orientation or position and gaze direction.

19. A vehicle, comprising:

a vehicle feature evaluation system for enabling evaluation of a simulated vehicle-related feature, the vehicle feature evaluation system comprising:

a HMD orientation or position and gaze direction determining unit comprising memory storing instructions executed by a processor for determining in relation to a road-driven vehicle, with support from a tracking system, an orientation of a head mounted display, HMD, adapted to be worn by an occupant on-board the road-driven vehicle or a position and gaze direction of the occupant on-board the road-driven vehicle;

a design feature determining unit comprising memory storing instructions executed by the processor for determining a simulated vehicle design feature to be evaluated in the road-driven vehicle; and

a HMD or display providing unit comprising memory storing instructions executed by the processor for providing in real-time to a HMD display of the HMD or another display visible to the occupant, taking into consideration the HMD orientation or the position and gaze direction, a virtual representation of the simulated vehicle design feature superimposed on a real-time surrounding-showing image derived from real-world perception sensor data captured with support from one or more vehicle-attached perception sensors adapted to capture surroundings internal or external to the road-driven vehicle, wherein the virtual representation of the simulated vehicle design feature replaces an excluded portion of the real-time surrounding-showing image, and wherein the virtual representation of the simulated vehicle design feature and the replaced excluded portion of the real-time surrounding-showing image are varied based on the HMD orientation or the position and gaze direction.

20. The vehicle according to claim 19 , the HMD or display providing unit further comprising memory storing instructions executed by the processor for receiving one or more of the real-world perception sensor data and an external dataset and enhancing one or more of the virtual representation of the simulated vehicle design feature and the real-time surrounding showing image based on the real-world perception sensor data and an external dataset.

21. The vehicle according to claim 19 , the HMD or display providing unit further comprising memory storing instructions executed by the processor for receiving one or more of the real-world perception sensor data and an external dataset and supplementing one or more of the virtual representation of the simulated vehicle design feature and the real-time surrounding showing image using on the real-world perception sensor data and an external dataset by providing an audio supplementation, a haptic supplementation, a taste supplementation, and a smell supplementation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 13, 2022
From: WICKMAN, CASPER; WIKENMALM, MATTIAS; GHIURAU, TIMOTEI; SENEGER, JON
To: VOLVO CAR CORPORATION
Reel/Frame 058640/0954 →
Priority Claims (1)
EP 18210988 · Dec 7, 2018 · regional
Continuity (2)
Continuation In Part 16672756 · Nov 4, 2019
Related Publication 20220134881A1 · May 5, 2022
Cited By (2)
US 12,195,000 US 12,296,678