Remote driving with a virtual reality system
View Patent ↗A method and system for facilitating a user drive an automobile remotely in real-time through a virtual reality system is disclosed. The virtual reality environment may present real-time road conditions, weather condition, automobile information, and/or any other information regarding driving of the automobile in real time.
1. A computer-implemented method for facilitating a user to remotely drive an automobile through a virtual reality system, the method comprising:
receiving, by a server, from an automobile, a first geo-location of the automobile at a first time instance;
based on the first geo-location received, rendering, by the server, a first view of a virtual reality environment reflecting an environment the automobile travels in at the first time instance, wherein the rendering of the first view of the virtual reality environment includes:
obtaining, from a scene file database, pre-rendered scene files corresponding to the first geo-location, the scene files representing the environment the automobile travels in at the first geo-location; and
constructing the first view of the virtual reality environment using the pre-rendered scene files;
transmitting the first view of the virtual reality environment to a display device associated with the user;
receiving, by the server, a first control input from a first interactive control device associated with the user;
based on the first control input, determining, by the server, a second view of the virtual reality environment; and
rendering, by the server, one or more images representing the second view.
2. The computer-implemented method of claim 1 , wherein the first control input includes information indicating a translational or a rotational movement of the first interactive control device.
3. The computer-implemented method of claim 1 , wherein the first interactive control device is a virtual steering wheel, a physical steering wheel, a physical brake pad, or a physical gas pedal.
4. The computer-implemented method of claim 1 , further comprising receiving view information regarding a first view point of the first user into the environment of the road at the first geo-location; and wherein, the rendering of the first view of the virtual reality environment is further based on the view point information.
5. The computer-implemented method of claim 1 , wherein the determination of the second view includes determining a change of the first view based on the first control input.
6. The computer-implemented method of claim 1 , wherein the determination of the second view includes determining driving information regarding the automobile at the time instance.
7. The computer-implemented method of claim 1 , wherein rendering the second view includes determining driving information regarding the automobile for presentation on a virtual instrument panel in the VR environment.
8. The computer-implemented method of claim 1 , further comprising receiving, by the server, from a head mount device (HMD) neck movement information indicating a neck movement of a user that remotely drives the automobile, and wherein the second view is determined further based on the neck movement.
9. The computer-implemented method of claim 1 , further comprising
receiving, by the server, from a second automobile, a second geo-location of the second automobile at a first time instance;
based on the second geo-location received, rendering, by the server, a third view of a virtual reality environment reflecting a environment the second automobile travels in at the first time instance;
receiving, by the server, a second control input from a first interactive control device associated with the automobile;
based on the first control input, determining, by the server, a fourth view of the virtual reality environment; and
rendering one or more images representing the second view.
10. A virtual reality system for facilitating a user to remotely drive an automobile, the virtual reality system comprising a processor configured to execute machine-readable instruction, when executed, causing the processor to perform:
receiving, from an automobile, a first geo-location of the automobile at a first time instance;
based on the first geo-location received, rendering, by the server, a first view of a virtual reality environment reflecting an environment the automobile travels in at the first time instance, wherein the rendering of the first view of the virtual reality environment includes:
obtaining, from a scene file database, pre-rendered scene files corresponding to the first geo-location, the scene files representing the environment the automobile travels in at the first geo-location; and
constructing the first view of the virtual reality environment using the pre-rendered scene files and the condition information;
transmitting the first view of the virtual reality environment to a display device associated with the user;
receiving, by the server, a first control input from a first interactive control device associated with the user;
based on the first control input, determining, by the server, a second view of the virtual reality environment; and
rendering one or more images representing the fourth view.
11. The system of claim 10 , wherein the first control input includes information indicating a translational or a rotational movement of the first interactive control device.
12. The system of claim 10 , wherein the first interactive control device is a virtual steering wheel, a physical steering wheel, a physical brake pad, or a physical gas pedal.
13. The system of claim 10 , wherein the machine-readable instructions are further configured to cause the processor to perform receiving view information regarding a first view point of the first user into the environment of the road at the first geo-location; and wherein, the rendering of the first view of the virtual reality environment is further based on the view point information.
14. The system of claim 10 , wherein the determination of the second view includes determining a change of the first view based on the first control input.
15. The system of claim 10 , wherein the determination of the second view includes determining driving information regarding the automobile at the time instance.
16. The system of claim 10 , wherein rendering the second view includes determining driving information regarding the automobile for presentation on a virtual instrument panel in the VR environment.
17. The system of claim 10 , wherein the machine-readable instructions further causing the processor to perform receiving from a head mount device (HMD) neck movement information indicating a neck movement of a user that remotely drives the automobile, and wherein the second view is determined further based on the neck movement of the user.
18. The system of claim 10 , wherein the machine-readable instructions further cause the processor to perform
receiving, by the server, from a second automobile, a second geo-location of the second automobile at a first time instance;
based on the second geo-location received, rendering, by the server, a third view of a virtual reality environment reflecting an environment the second automobile travels in at the first time instance;
receiving, by the server, a second control input from a first interactive control device associated with the automobile;
based on the first control input, determining, by the server, a fourth view of the virtual reality environment; and
rendering one or more images representing the fourth view.