IP Library › Granted Patent US 11,257,390
Granted Patent B2
US 11,257,390 · App. 16/672,756 · Granted Feb 22, 2022

Evaluation of a simulated vehicle-related feature

Inventors: Casper Wickman (Ockerö, SE); Mattias Wikenmalm (Torslanda, SE); Timotei Ghiurau (Gothenburg, SE); Jon Seneger (Boulder Creek, CA)
Assignee: Volvo Car Corporation
G09B9/05G02B27/0172G02B27/0179G06F30/20G09B9/042G02B2027/0181G02B2027/0183G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,257,390
App. No.
16/672,756
Granted
Feb 22, 2022
Kind
B2
Abstract

A method performed by a vehicle feature evaluation system ( 1 ) for enabling evaluation of a simulated vehicle-related feature. The vehicle feature evaluation system determines ( 1001 ) in relation to a road-driven vehicle ( 2 ), with support from a tracking system ( 5 ), an orientation of a head-mounted display ( 4 ), HMD, adapted to be worn by an occupant ( 3 ) on-board the road-driven vehicle. The vehicle feature evaluation system further determines ( 1002 ) a simulated vehicle design feature to be evaluated in the road-driven vehicle. Moreover, the vehicle feature evaluation system provides ( 1006 ) in real-time to a HMD display ( 41 ) of the HMD, taking into consideration the HMD orientation, a virtual representation ( 7 ) of the simulated vehicle design feature superimposed on a real-time surrounding-showing video stream ( 6 ) derived from real-world image data ( 211 ) captured with support from one or more vehicle-attached cameras ( 21 ) adapted to capture surroundings external of the road-driven vehicle.

Claims (75)

1. A method performed by a vehicle feature evaluation system for enabling evaluation of a simulated vehicle-related feature, the method comprising:

determining in relation to a road-driven vehicle, with support from a tracking system, an orientation of a head mounted display, HMD, adapted to be worn by an occupant on-board the road-driven vehicle;

determining a simulated vehicle design feature to be evaluated in the road-driven vehicle; and

providing in real-time to a HMD display of the HMD, taking into consideration the HMD orientation, a virtual representation of the simulated vehicle design feature superimposed on a real-time surrounding-showing video stream derived from real-world image data captured with support from one or more vehicle-attached cameras adapted to capture surroundings external of the road-driven vehicle, wherein the virtual representation of the simulated vehicle design feature replaces an excluded portion of the real-time surrounding-showing video stream, and wherein the virtual representation of the simulated vehicle design feature and the replaced excluded portion of the real-time surrounding-showing video stream are varied based on the HMD orientation.

2. The method according to claim 1 , wherein the simulated vehicle design feature comprises one or more of a simulated vehicle interior section and a simulated vehicle exterior section.

3. The method according to claim 1 , further comprising:

determining a simulated vehicle functionality feature to be evaluated in the road-driven vehicle;

wherein the providing further comprises additionally providing in real-time to the HMD display, taking into consideration the HMD orientation, a virtual representation of the simulated vehicle functionality feature superimposed on the real-time surrounding-showing video stream, which simulated vehicle functionality feature comprises one or more of:

a simulated vehicle human machine interface, HMI;

a simulated vehicle display;

a simulated vehicle head-up display, HUD;

simulated light characteristics; and

a simulated effect resulting from a simulation of the simulated vehicle functionality feature.

4. The method according to claim 1 , further comprising:

determining a simulated object feature to be evaluated in the road-driven vehicle;

wherein the providing further comprises additionally providing in real-time to the HMD display, taking into consideration the HMD orientation, a virtual representation of the simulated object feature superimposed on the real-time surrounding-showing video stream, which simulated object feature comprises one or more of:

a simulated other vehicle;

a simulated road user; and

a simulated moving or fixed obstacle.

5. The method according to claim 4 , further comprising one or more of:

updating the virtual functionality feature representation based on a user interaction with the simulated vehicle functionality feature, which user interaction for instance is based on detection, by means of one or more user interaction sensors, of a user in and/or at the fictive functionality feature location; and/or

updating the virtual functionality feature representation based on a vehicle signal affecting the simulated vehicle functionality feature, derived from the road-driven vehicle.

6. The method according to claim 1 , further comprising:

determining one or more of:

a fictive design feature location of the simulated vehicle design feature relative the road-driven vehicle;

a fictive functionality feature location of the simulated vehicle functionality feature relative the road-driven vehicle; and

a fictive object feature location of the simulated object feature relative the road-driven vehicle;

wherein the providing taking into consideration the HMD orientation comprises one or more of:

providing the virtual design feature representation superimposed on the real-time surrounding-showing video stream such that positioning of the virtual design feature representation corresponds to the fictive design feature location;

providing a virtual functionality feature representation superimposed on the real-time surrounding-showing video stream such that positioning of the virtual functionality feature representation corresponds to the fictive functionality feature location; and

providing the virtual object feature representation superimposed on the real-time surrounding-showing video stream such that positioning of the virtual object feature representation corresponds to the fictive object feature location.

7. The method according to claim 1 , further comprising one or more of:

updating:

the real-time surrounding-showing video stream;

the superimposed virtual design feature representation;

a superimposed virtual functionality feature representation; and

a superimposed virtual object feature representation based on updated captured image data and/or based on updated determined HMD orientation.

8. A vehicle feature evaluation system for enabling evaluation of a simulated vehicle-related feature, the vehicle feature evaluation system comprising:

a HMD orientation determining unit comprising memory storing instructions executed by a processor for determining in relation to a road-driven vehicle, with support from a tracking system, an orientation of a head mounted display, HMD, adapted to be worn by an occupant on-board the road-driven vehicle;

a design feature determining unit comprising memory storing instructions executed by the processor for determining a simulated vehicle design feature to be evaluated in the road-driven vehicle; and

a HMD providing unit comprising memory storing instructions executed by the processor for providing in real-time to a HMD display of the HMD, taking into consideration the HMD orientation, a virtual representation of the simulated vehicle design feature superimposed on a real-time surrounding-showing video stream derived from real-world image data captured with support from one or more vehicle-attached cameras adapted to capture surroundings external of the road-driven vehicle, wherein the virtual representation of the simulated vehicle design feature replaces an excluded portion of the real-time surrounding-showing video stream, and wherein the virtual representation of the simulated vehicle design feature and the replaced excluded portion of the real-time surrounding-showing video stream are varied based on the HMD orientation.

9. The vehicle feature evaluation system according to claim 8 , wherein the simulated vehicle design feature comprises one or more of a simulated vehicle interior section and a simulated vehicle exterior section.

10. The vehicle feature evaluation system according to claim 8 , further comprising:

a functionality feature determining unit comprising memory storing instructions executed by the processor for determining a simulated vehicle functionality feature to be evaluated in the road-driven vehicle;

wherein the HMD providing unit is adapted for additionally providing in real-time to the HMD display, taking into consideration the HMD orientation, a virtual representation of the simulated vehicle functionality feature superimposed on the real-time surrounding-showing video stream, which simulated vehicle functionality feature comprises one or more of:

a simulated vehicle human machine interface, HMI;

a simulated vehicle display;

a simulated vehicle head-up display, HUD;

simulated light characteristics; and

a simulated effect resulting from a simulation of the simulated vehicle functionality feature.

11. The vehicle feature evaluation system according to claim 8 , further comprising:

an object feature determining unit comprising memory storing instructions executed by the processor for determining a simulated object feature to be evaluated in the road-driven vehicle;

wherein the HMD providing unit is adapted for additionally providing in real-time to the HMD display, taking into consideration the HMD orientation, a virtual representation of the simulated object feature superimposed on the real-time surrounding-showing video stream, which simulated object feature comprises one or more of:

a simulated other vehicle;

a simulated road user; and

a simulated moving or fixed obstacle.

12. The vehicle feature evaluation system according to claim 11 , further comprising one or more of:

a user interaction updating unit comprising memory storing instructions executed by the processor for updating the virtual functionality feature representation based on a user interaction with the simulated vehicle functionality feature, which user interaction for instance is based on detection, by means of one or more user interaction sensors, of a user in and/or at the fictive functionality feature location; and

a vehicle signal updating unit comprising memory storing instructions executed by the processor for updating the virtual functionality feature representation based on a vehicle signal affecting the simulated vehicle functionality feature, derived from the road-driven vehicle.

13. The vehicle feature evaluation system according to claim 8 , further comprising:

a location determining unit comprising memory storing instructions executed by the processor for determining one or more of:

a fictive design feature location of the simulated vehicle design feature relative the road-driven vehicle;

a fictive functionality feature location of the simulated vehicle functionality feature relative the road-driven vehicle; and

a fictive object feature location of the simulated object feature relative the road-driven vehicle;

wherein the HMD providing unit further is adapted for one or more of:

providing the virtual design feature representation superimposed on the real-time surrounding-showing video stream such that positioning of the virtual design feature representation corresponds to the fictive design feature location;

providing a virtual functionality feature representation superimposed on the real-time surrounding-showing video stream such that positioning of the virtual functionality feature representation corresponds to the fictive functionality feature location; and

providing the virtual object feature representation superimposed on the real-time surrounding-showing video stream such that positioning of the virtual object feature representation corresponds to the fictive object feature location.

14. The vehicle feature evaluation system according to claim 8 , further comprising one or more of:

a continuous updating unit comprising memory storing instructions executed by the processor for updating:

the real-time surrounding-showing video stream;

the superimposed virtual design feature representation;

a superimposed virtual functionality feature representation; and

a superimposed virtual object feature representation based on updated captured image data and/or based on updated determined HMD orientation.

15. A vehicle comprising a vehicle feature evaluation system according to claim 8 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2019
From: WICKMAN, CASPER; WIKENMALM, MATTIAS; GHIURAU, TIMOTEI; SENEGER, JON
To: VOLVO CAR CORPORATION
Reel/Frame 050903/0605 →
Priority Claims (1)
EP 18210988 · Dec 7, 2018 · regional
Continuity (1)
Related Publication 20200184841A1 · Jun 11, 2020
Cited By (2)
US 12,296,678 US 12,585,847