System and method for providing support to user of autonomous vehicle (AV) based on sentiment analysis
Assistance can be provided to users of AVs based on user sentiments. A system may receive a request for assistance from a user of an AV (e.g., a passenger of the AV). The system may also receive sensor data that is captured by a sensor suite of the AV from detecting the user, the AV, or another object. The system determines a sentiment of the user based on the sensor data. The system may input the sensor data into a machine learning model and the model outputs information indicating the user's sentiment. The system can determine who, when, or how to service the user's request based on the user's sentiment. The system may assign the user's request to an agent who can provide the assistance. The system can further provide guidance (e.g., suggested content of a conversation with the user) to the agent based on the user's sentiment.
1. A method, comprising:
receiving, from a first device associated with a user receiving a service from an autonomous vehicle, a request for assistance;
receiving sensor data from the vehicle, the sensor data captured by one or more sensors of the vehicle;
determining a state of a user based on the sensor data;
generating, based on the state of the user, an instruction for an interaction between the user and an agent to provide the assistance to the user; and
providing the instruction to a second device associated with the agent, the agent to make the interaction with the user in accordance with the instruction.
2. The method of claim 1 , wherein determining the state of the user based on the sensor data comprises:
inputting the sensor data into a model, the model trained to receive sensor data associated with users and to generate outputs indicating states of the users; and
determining the state of the user based on an output of the model.
3. The method of claim 2 , wherein the output of the model comprises a sentiment score indicating a likelihood of the user having a sentiment.
4. The method of claim 1 , wherein the sensor data is generated by the one or more sensors detecting the user, the vehicle, or an object in an environment surrounding the user or vehicle.
5. The method of claim 1 , wherein determining the state of the user comprises:
determining the state of the user further based on information associated with a historical request for assistance that was made by the user for a historical service that was provided to the user by the vehicle or another vehicle.
6. The method of claim 1 , wherein receiving the sensor data from the vehicle comprises:
in response to receiving the request for assistance, sending a request for sensor data to the vehicle,
wherein the sensor data is provided by the vehicle in response to the request for sensor data.
7. The method of claim 1 , wherein the instruction specifies content of a conversation between the agent and the user.
8. The method of claim 1 , further comprising:
determining a waiting time based on the state of the user, the waiting time specifying a duration of time before the assistance is provided to the user.
9. The method of claim 1 , further comprising:
identifying the agent from a plurality of agents based on the state of the user.
10. The method of claim 1 , further comprising:
instructing the vehicle to change a setting of a component of the vehicle based on the state of the user.
11. One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:
receiving, from a first device associated with a user receiving a service from an autonomous vehicle, a request for assistance;
receiving sensor data from the vehicle, the sensor data captured by one or more sensors of the vehicle;
determining a state of a user based on the sensor data;
generating, based on the state of the user, an instruction for an interaction between the user and an agent to provide the assistance to the user; and
providing the instruction to a second device associated with the agent, the agent to make the interaction with the user in accordance with the instruction.
12. The one or more non-transitory computer-readable media of claim 11 , wherein determining the state of the user based on the sensor data comprises:
inputting the sensor data into a model, the model trained to receive sensor data associated with users and to generate outputs indicating states of the users; and
determining the state of the user based on an output of the model.
13. The one or more non-transitory computer-readable media of claim 12 , wherein the output of the model comprises a sentiment score indicating a likelihood of the user having a sentiment.
14. The one or more non-transitory computer-readable media of claim 11 , wherein the sensor data is generated by the one or more sensors detecting the user, the vehicle, or an object in an environment surrounding the user or vehicle.
15. The one or more non-transitory computer-readable media of claim 11 , wherein determining the state of the user comprises:
determining the state of the user further based on information associated with a historical request for assistance that was made by the user for a historical service that was provided to the user by the vehicle or another vehicle.
16. The one or more non-transitory computer-readable media of claim 11 , wherein the instruction specifies content of a conversation between the agent and the user.
17. The one or more non-transitory computer-readable media of claim 11 , wherein the operations further comprise:
instructing the vehicle to change a setting of a component of the vehicle based on the state of the user.
18. A computer system, comprising:
a computer processor for executing computer program instructions; and
one or more non-transitory computer-readable media storing computer program instructions executable by the computer processor to perform operations comprising:
receiving, from a first device associated with a user receiving a service from an autonomous vehicle, a request for assistance
receiving sensor data from the vehicle, the sensor data captured by one or more sensors of the vehicle
determining a state of a user based on the sensor data
generating, based on the state of the user, an instruction for an interaction between the user and an agent to provide the assistance to the user and
providing the instruction to a second device associated with the agent, the agent to make the interaction with the user in accordance with the instruction.
19. The computer system of claim 18 , wherein determining the state of the user based on the sensor data comprises:
inputting the sensor data into a model, the model trained to receive sensor data associated with users and to generate outputs indicating states of the users; and
determining the state of the user based on an output of the model.
20. The computer system of claim 18 , wherein the operations further comprise:
instructing the vehicle to change a setting of a component of the vehicle based on the state of the user.