System and method of predicting human interaction with vehicles
Systems and methods for predicting user interaction with vehicles. A computing device receives an image and a video segment of a road scene, the first at least one of an image and a video segment being taken from a perspective of a participant in the road scene and then generates stimulus data based on the image and the video segment. Stimulus data is transmitted to a user interface and response data is received, which includes at least one of an action and a likelihood of the action corresponding to another participant in the road scene. The computing device aggregates a subset of the plurality of response data to form statistical data and a model is created based on the statistical data. The model is applied to another image or video segment and a prediction of user behavior in the another image or video segment is generated.
1. A computer-implemented method comprising:
receiving a plurality of training images of an environment, the plurality of training images including one or more persons;
sending the plurality of training images to annotators via a user interface requesting user responses describing an attribute of the one or more persons;
for the plurality of training images, receiving a plurality of user responses from the annotators, each user response describing the attribute of the one or more persons displayed in the plurality of training images;
generating a training dataset comprising summary statistics of the plurality of user responses describing the attribute of the one or more persons displayed in the plurality of training images;
training, using the training dataset, a machine learning based model configured to receive input images and predict summary statistics describing an attribute of one or more persons displayed in the input images;
receiving a plurality of new images of a new environment, the plurality of new images including one or more new persons;
predicting, using the machine learning based model, summary statistics describing an attribute of the one or more new persons in the plurality of new images; and
determining an action to be performed based on the predicted summary statistics of the one or more new persons.
2. The computer-implemented method of claim 1 , wherein the one or more persons represent a group of people and the attribute of the one or more persons is an aggregate state of mind of the group of people.
3. The computer-implemented method of claim 1 , wherein the action to be performed is an action associated with managing movement of the one or more new persons in the plurality of new images.
4. The computer-implemented method of claim 1 , wherein the summary statistics predicted by the machine learning based model includes a likelihood of the one or more persons performing a common predicted action, the action including one of:
staying in a current location, and
moving from the current location to another location.
5. The computer-implemented method of claim 4 , wherein responsive to the likelihood of the one or more persons moving from the current location to another location being greater than a threshold, sending instructions to dispatch ground transportation to the current location.
6. The computer-implemented method of claim 1 , wherein the summary statistics predicted by the machine learning based model includes a likelihood of the one or more persons being aware of an event associated with a current location.
7. The computer-implemented method of claim 6 , wherein responsive to the likelihood of the one or more persons being aware of the abnormal even being greater than a threshold, dispatching personnel in response to the event.
8. The computer-implemented method of claim 1 , wherein the new environment is a road intersection and the summary statistics predicted by the machine learning based model includes intentions of the one or more persons to cross the road intersection.
9. The computer-implemented method of claim 8 , wherein responsive to the intentions of the one or more persons to cross the road intersection being greater than a threshold, generating instructions to adjust signal timing at the road intersection.
10. The computer-implemented method of claim 8 , wherein responsive to the intentions of the one or more persons to cross the intersection being greater than a threshold, sending a message including an image of the one or more persons.
11. The computer-implemented method of claim 1 , wherein the new environment is a store and the summary statistics predicted by the machine learning based model includes a likelihood of the one or more persons performing a suspicious activity.
12. The computer-implemented method of claim 11 , wherein responsive to the likelihood of the one or more persons performing a suspicious activity being greater than a threshold, sending an alert message.
13. The computer-implemented method of claim 1 , wherein the new environment is a store and the summary statistics predicted by the machine learning based model includes intention of the one or more persons to purchase a product.
14. The computer-implemented method of claim 13 , wherein the action to be performed is determining a promotion plan for the one or more persons.
15. A non-transitory computer readable storage medium storing instructions that when executed by a computer processor cause the computer processor to perform steps of a computer-implemented method for determining an attribute of one or more persons, the steps comprising:
receiving a plurality of training images of an environment, the plurality of training images including one or more persons;
sending the plurality of training images to annotators via a user interface requesting user responses describing an attribute of the one or more persons;
for the plurality of training images, receiving a plurality of user responses from the annotators, each user response describing the attribute of the one or more persons displayed in the plurality of training images;
generating a training dataset comprising summary statistics of the plurality of user responses describing the attribute of the one or more persons displayed in the plurality of training images;
training, using the training dataset, a machine learning based model configured to receive input images and predict summary statistics describing an attribute of one or more persons displayed in the input images;
receiving a plurality of new images of a new environment, the plurality of new images including one or more new persons;
predicting, using the machine learning based model, summary statistics describing an attribute of the one or more new persons in the plurality of new images; and
determining an action to be performed based on the predicted summary statistics of the one or more new persons.
16. The non-transitory computer readable storage medium of claim 15 , wherein the attribute of the one or more persons is an aggregate state of mind of the one or more persons.
17. The non-transitory computer readable storage medium of claim 15 , wherein the action to be performed is an action for managing movement of the one or more new persons in the plurality of new images.
18. A computer system comprising:
a computer processor; and
a non-transitory computer readable storage medium storing instructions that when executed by a computer processor cause the computer processor to perform steps of a computer-implemented method for determining an attribute of one or more persons, the steps comprising:
receiving a plurality of training images of an environment, the plurality of training images including one or more persons;
sending the plurality of training images to annotators via a user interface requesting user responses describing an attribute of the one or more persons;
for the plurality of training images, receiving a plurality of user responses from the annotators, each user response describing the attribute of the one or more persons displayed in the plurality of training images;
generating a training dataset comprising summary statistics of the plurality of user responses describing the attribute of the one or more persons displayed in the plurality of training images;
training, using the training dataset, a machine learning based model configured to receive input images and predict summary statistics describing an attribute of one or more persons displayed in the input images;
receiving a plurality of new images of a new environment, the plurality of new images including one or more new persons;
predicting, using the machine learning based model, summary statistics describing an attribute of the one or more new persons in the plurality of new images; and
determining an action to be performed based on the predicted summary statistics of the one or more new persons.
19. The computer system of claim 18 , wherein the attribute of the one or more persons is an aggregate state of mind of the one or more persons.
20. The computer system of claim 18 , wherein the action to be performed is an action for managing movement of the one or more new persons in the plurality of new images.