IP Library Granted Patent US 11,295,132
Granted Patent B2
US 11,295,132 · App. 16/516,065 · Granted Apr 5, 2022

Method, a device for assisting driving, an unmanned device and a readable storage medium

Inventor: Youhan Chen (Haidian District Beijing, CN)
Assignee: Baidu Online Network Technology(Beijing) Co., Ltd.
G06K9/00671B64C39/024G05D1/0022G05D1/0044G06F3/011G06K9/00791B64C2201/141
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,295,132
App. No.
16/516,065
Granted
Apr 5, 2022
Kind
B2
Abstract

The present disclosure provides a method, a device for assisting driving, an unmanned device and a readable storage medium, where the method includes: the unmanned device triggers remote assisting automatic driving function, receiving the scene information captured by the unmanned device; processing the scene information, generating a VR scene; displaying the VR scene for a user, so that the user can trigger a remote assisting driving operation according to the VR scene; where the remote assisting driving operation is used to control the unmanned device to execute a driving operation. Thus providing a VR scene of the environment where the unmanned device is located, improving effectively user's environment engagement and perception of actual environment, which is helpful for the user to trigger an accurate remote assisting driving instruction, thus to ensure the driving safety of the unmanned device.

Claims (67)

1. A method for assisting driving, comprising:

receiving scene information captured by an unmanned device, and receiving, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, when the unmanned device triggers a remote assisting automatic driving function, wherein the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located are transmitted by the unmanned device, and the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located;

processing the scene information, generating a virtual reality (VR) scene; and

displaying the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, so that the user can trigger a remote assisting driving operation according to the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located; wherein the remote assisting driving operation is used to control the unmanned device to execute the driving operation.

2. The method according to claim 1 , wherein the scene information comprises image information of multiple environment angles of the unmanned device under the current environment; wherein, the image information of multiple environment angles is captured by the unmanned device at the same time;

correspondingly, the processing the scene information generating a VR scene comprises:

synthesizing and obtaining, according to the environment angles corresponding to respective image information, environment image information of the current environment;

performing visual image processing to the environment image information, so as to obtain the VR scene comprising left eye environment image information and right eye environment image information.

3. The method according to claim 1 , wherein the scene information comprises video information of multiple environment angles of the unmanned device under the current environment;

correspondingly, the processing the scene information generating a VR scene comprises:

performing framing processing to the video information of multiple environment angles, so as to obtain image data of multiple environment angles corresponding to one frame;

performing visual image processing to the image data of multiple environment angles of every frame, so as to obtain left eye environment image data and right eye environment image data of every frame;

obtaining the VR scene, according to the left eye environment image data and the right eye environment image data of every frame.

4. The method according to claim 1 , wherein after the displaying the VR scene for a user, the method further comprising:

receiving the remote assisting driving operation triggered by the user;

generating a corresponding operation instruction according to the remote assisting driving operation, and transmitting it to the unmanned device, so that the unmanned device executes the driving operation according to the operation instruction.

5. The method according to claim 1 , wherein receiving the scene information transmitted by the unmanned device via a 5G mobile communication network.

6. A method for assisting driving, comprising:

capturing scene information, when a remote assisting automatic driving function is triggered;

transmitting the scene information to a device for assisting driving, and transmitting a current configuration parameter of an unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, enabling the device for assisting driving to process the scene information, generate a virtual reality (VR) scene, and display the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, wherein the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located.

7. The method according to claim 6 , wherein after the transmitting the scene information to the device for assisting driving, the method further comprises:

receiving and executing a remote assisting driving operation instruction returned by the device for assisting driving, wherein the remote assisting driving operation instruction is triggered by the user according to the VR scene.

8. A device for assisting driving, comprising: a processor; and

a computer-readable medium for storing program codes, which, when executed by the processor, the processor is caused to:

receive scene information captured by an unmanned device, and receive, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, when the unmanned device triggers remote assisting automatic driving function, wherein the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located are transmitted by the unmanned device, and the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located;

process the scene information and generate a virtual reality (VR) scene; and

display the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, so that the user can trigger a remote assisting driving operation according to the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device; wherein the remote assisting driving operation is used to control the unmanned device to execute the driving operation.

9. The device according to claim 8 , wherein the scene information comprises image information of multiple environment angles of the unmanned device under the current environment; wherein, the image information of multiple environment angles is captured by the unmanned device at the same time;

wherein the processor is further caused to:

synthesize and obtain, according to the environment angles corresponding to respective image information, environment image information of the current environment;

perform visual image processing to the environment image information, so as to obtain the VR scene comprising left eye environment image information and right eye environment image information.

10. The device according to claim 8 , wherein the scene information comprises video information of multiple environment angles of the unmanned device under the current environment;

wherein the processor is further caused to:

perform framing processing to the video information of multiple environment angles, so as to obtain image data of multiple environment angles corresponding to one frame;

perform visual image processing to the image data of multiple environment angles of every frame, so as to obtain left eye environment image data and right eye environment image data of every frame;

obtain the VR scene, according to the left eye environment image data and the right eye environment image data of every frame.

11. The device according to claim 8 , wherein the processor is further caused to:

receive a remote assisting driving operation triggered by the user, after the displaying the VR scene for a user;

generate a corresponding operation instruction according to the remote assisting driving operation, and transmit the operation instruction to the unmanned device, so that the unmanned device executes the driving operation according to the operation instruction.

12. The device according to claim 8 , wherein the processor is further caused to: receive the scene information transmitted by the unmanned device via a 5G mobile communication network.

13. An unmanned device, comprising:

a processor; and

a computer-readable medium for storing program codes, which, when executed by the processor, the processor is caused to:

capture scene information, when remote assisting automatic driving function is triggered; and

transmit the scene information to a device for assisting driving, and transmit, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, enabling the device for assisting driving to process the scene information, generate a virtual reality (VR) scene, and display the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, wherein the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located.

14. The unmanned device according to claim 13 , wherein the processor is further caused to:

receive a remote assisting driving operation instruction returned by the device for assisting driving, the remote assisting driving operation instruction is triggered by the user according to the VR scene;

execute the remote assisting driving operation instruction.

15. A device for assisting driving, comprising: a display device, a memory, a processor coupled to the memory, and a computer program stored in the memory and executable on the processor,

the processor is configured to execute a computer program;

the display device is coupled to the memory and the processor, and configured to display a virtual reality (VR) scene for a user by means of VR technology; and

wherein the computer program:

receives scene information captured by an unmanned device, and receives, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, when the unmanned device triggers a remote assisting automatic driving function, wherein the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located are transmitted by the unmanned device, and the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located;

processes the scene information, generating a VR scene; and

displays the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, so that the user can trigger a remote assisting driving operation according to the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located; wherein the remote assisting driving operation is used to control the unmanned device to execute the driving operation.

16. An unmanned device, comprising: a memory, a processor coupled to the memory, and a computer program stored in the memory and executable on the processor,

the processor is configured to execute the computer program;

wherein the computer program:

captures scene information, when a remote assisting automatic driving function is triggered; and

transmits the scene information to a device for assisting driving, and transmits, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, enabling the device for assisting driving to process the scene information, generate a virtual reality (VR) scene, and display the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, wherein the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located.

17. A non-transitory computer-readable medium, comprising a program, and when the program is executed by a terminal, the program performing the steps of:

receiving scene information captured by an unmanned device, and receiving, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, when the unmanned device triggers a remote assisting automatic driving function, wherein the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located are transmitted by the unmanned device, and the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located;

processing the scene information, generating a virtual reality (VR) scene; and

displaying the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, so that the user can trigger a remote assisting driving operation according to the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located; wherein the remote assisting driving operation is used to control the unmanned device to execute the driving operation.

18. A non-transitory computer-readable medium, comprising a program, and when the program is executed by a terminal, the program performs the steps of:

capturing scene information, when a remote assisting automatic driving function is triggered; and

transmitting the scene information to a device for assisting driving, and transmitting, a current configuration parameter of the unmanned device, an operating parameter of the unmanned device and a measurement parameter of an environment where the unmanned device is located, enabling the device for assisting driving to process the scene information, generate a virtual reality (VR) scene, and display the VR scene, the current configuration parameter of the unmanned device, the operating parameter of the unmanned device and the measurement parameter of the environment where the unmanned device is located for a user at the same time, wherein the current configuration parameter of the unmanned device comprises a model number, an automatic driving mode and a currently ongoing task of the unmanned device, the operating parameter of the unmanned device comprises a speed, an acceleration, a vehicle processing operating status and a vehicle inclination of the unmanned device, the measurement parameter of the environment where the unmanned device is located comprises a distance between the unmanned device and an obstacle in the environment where the unmanned device is located, a weather parameter of the environment where the unmanned device is located, and the ground surface or topography status of the environment where the unmanned device is located.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2019
From: CHEN, YOUHAN
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 050470/0944 →
Priority Claims (1)
CN 201811045954.9 · Sep 7, 2018 · national
Continuity (1)
Related Publication 20200082171A1 · Mar 12, 2020