Method, apparatus and device for identifying passenger state in unmanned vehicle, and storage medium

1. A method for identifying a passenger state in an unmanned vehicle, comprising:

obtaining monitoring data of different dimensions in a process where a passenger takes the unmanned vehicle, wherein the monitoring data of different dimensions comprises expression monitoring data, limb movement monitoring data, sound monitoring data, vital sign monitoring data, and collision data of colliding a vehicle body or a seat;

performing feature extraction on the monitoring data of different dimensions and forming feature data of different dimensions;

identifying the passenger state according to the feature data of different dimensions, wherein the passenger state comprises a safety state and a danger state; and

executing at least one step of:

reporting the danger state, identity information of the passenger and contact information of the passenger's family to a server, if the passenger state is the danger state, for the server to notify the passenger's family according to the identity information of the passenger and the contact information of the passenger's family, wherein the identity information of the passenger and the contact information of the passenger's family are pre-stored in the unmanned vehicle; and

reporting position of the unmanned vehicle to the server, if the passenger's state is the danger state, for the server to obtain the position of the unmanned vehicle that the passenger takes, wherein the unmanned vehicle is located by a GPS module provided on the unmanned vehicle,

wherein the identifying the passenger state according to the feature data of different dimensions comprises:

inputting feature data of each dimension into a corresponding first identification algorithm, and outputting passenger state probability data corresponding to the feature data of each dimension;

obtaining a weight value corresponding to the passenger state probability data of each dimension;

performing a weighted summation calculation on passenger state probability data of respective dimensions to obtain general passenger state probability data; and

determining the passenger state according to the general passenger state probability data and a preset threshold.

2. The method according to claim 1 , wherein the obtaining monitoring data of different dimensions in a process where the passenger takes the unmanned vehicle comprises:

collecting in real time, by different sensors provided in the unmanned vehicle, monitoring data of corresponding dimensions in the process where the passenger takes the unmanned vehicle.

3. The method according to claim 2 , wherein the sensors comprise at least: an internal camera, a microphone, a vital sign sensor, and a collision sensor.

4. The method according to claim 3 , wherein the collecting in real time, by different sensors provided in the unmanned vehicle, monitoring data of corresponding dimensions in the process where the passenger takes the unmanned vehicle, comprises:

collecting, by the internal camera, the expression monitoring data and the limb movement monitoring data in the process where the passenger takes the unmanned vehicle;

collecting, by the microphone, the sound monitoring data in the process where the passenger takes the unmanned vehicle;

collecting, by the vital sign sensor, the vital sign monitoring data in the process where the passenger takes the unmanned vehicle; and

collecting, by the collision sensor, the collision data of colliding a vehicle body or a seat in the process where the passenger takes the unmanned vehicle.

5. The method according to claim 1 , wherein the performing feature extraction on the monitoring data of different dimensions and forming feature data of different dimensions comprises:

performing the feature extraction on the monitoring data of each dimension by means of a corresponding feature extraction algorithm and forming feature data of a corresponding dimension.

6. An apparatus for identifying a passenger state in an unmanned vehicle, comprising:

a processor and a computer-readable medium for storing program codes, which, when executed by the processor, cause the processor to: obtain monitoring data of different dimensions in a process where a passenger takes the unmanned vehicle, wherein the monitoring data of different dimensions comprises expression monitoring data, limb movement monitoring data, sound monitoring data, vital sign monitoring data, and collision data of colliding a vehicle body or a seat;

perform feature extraction on the monitoring data of different dimensions and form feature data of different dimensions;

identify the passenger state according to the feature data of different dimensions, wherein the passenger state comprises a safety state and a danger state; and

execute at least one step of:

reporting the danger state, identity information of the passenger and contact information of the passenger's family to a server, if the passenger state is the danger state, for the server to notify the passenger's family according to the identity information of the passenger and the contact information of the passenger's family, wherein the identity information of the passenger and the contact information of the passenger's family are pre-stored in the unmanned vehicle; and

reporting position of the unmanned vehicle to the server, if the passenger's state is the danger state, for the server to obtain the position of the unmanned vehicle that the passenger takes, wherein the unmanned vehicle is located by a GPS module provided on the unmanned vehicle,

wherein the program codes further cause the processor to:

input feature data of each dimension into a corresponding first identification algorithm, and output passenger state probability data corresponding to the feature data of each dimension;

obtain a weight value corresponding to the passenger state probability data of each dimension;

perform a weighted summation calculation on passenger state probability data of respective dimensions to obtain general passenger state probability data; and

determine the passenger state according to the general passenger state probability data and a preset threshold.

7. The apparatus according to claim 6 , wherein the program codes further cause the processor to:

collect in real time, by different sensors provided in the unmanned vehicle, monitoring data of corresponding dimensions in the process where the passenger takes the unmanned vehicle.

8. The apparatus according to claim 7 , wherein the sensors comprise at least:

an internal camera, a microphone, a vital sign sensor, a collision sensor.

9. The apparatus according to claim 8 , wherein the program codes further cause the processor to:

collect, by the internal camera, the expression monitoring data and the limb movement monitoring data in the process where the passenger takes the unmanned vehicle;

collect, by the microphone, the sound monitoring data in the process where the passenger takes the unmanned vehicle;

collect, by the vital sign sensor, the vital sign monitoring data in the process where the passenger takes the unmanned vehicle; and

collect, by the collision sensor, the collision data of colliding a vehicle body or a seat in the process where the passenger takes the unmanned vehicle.

10. The apparatus according to claim 6 , wherein the program codes further cause the processor to:

perform the feature extraction on the monitoring data of each dimension by means of a corresponding feature extraction algorithm and form feature data of a corresponding dimension.

11. A terminal device, comprising:

one or more processors;

wherein one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method according to claim 1 .

12. A non-transitory computer readable storage medium, having stored thereon a computer program, wherein the program is executed by a processor to implement the steps of:

obtaining monitoring data of different dimensions in a process where a passenger takes the unmanned vehicle, wherein the monitoring data of different dimensions comprises expression monitoring data, limb movement monitoring data, sound monitoring data, vital sign monitoring data, and collision data of colliding a vehicle body or a seat;

performing feature extraction on the monitoring data of different dimensions and forming feature data of different dimensions;

identifying the passenger state according to the feature data of different dimensions, wherein the passenger state comprises a safety state and a danger state; and

executing at least one step of:

reporting the danger state, identity information of the passenger and contact information of the passenger's family to a server, if the passenger state is the danger state, for the server to notify the passenger's family according to the identity information of the passenger and the contact information of the passenger's family, wherein the identity information of the passenger and the contact information of the passenger's family are pre-stored in the unmanned vehicle; and

reporting position of the unmanned vehicle to the server, if the passenger's state is the danger state, for the server to obtain the position of the unmanned vehicle that the passenger takes, wherein the unmanned vehicle is located by a GPS module provided on the unmanned vehicle,

wherein the identifying the passenger state according to the feature data of different dimensions comprises:

inputting feature data of each dimension into a corresponding first identification algorithm, and outputting passenger state probability data corresponding to the feature data of each dimension;

obtaining a weight value corresponding to the passenger state probability data of each dimension;

performing a weighted summation calculation on passenger state probability data of respective dimensions to obtain general passenger state probability data; and

determining the passenger state according to the general passenger state probability data and a preset threshold.