IMAGE-BASED PEDESTRIAN SPEED ESTIMATION
This document discloses system, method, and computer program product embodiments for image-based pedestrian speed estimation. For example, the method includes receiving an image of a scene, wherein the image includes a pedestrian and predicting a speed of the pedestrian by applying a machine-learning model to at least a portion of the image that includes the pedestrian. The machine-learning model is trained using a data set including training images of pedestrians, the training images associated with corresponding known pedestrian speeds. The method further includes providing the predicted speed of the pedestrian to a motion-planning system that is configured to control a trajectory of an autonomous vehicle in the scene.
1 . A method comprising, by one or more electronic devices:
receiving an image of a scene, wherein the image includes a pedestrian;
predicting a speed of the pedestrian by applying a machine-learning model to at least a portion of the image that includes the pedestrian, wherein the machine-learning model has been trained using a data set comprising training images of pedestrians, the training images associated with corresponding known pedestrian speeds; and
providing the predicted speed of the pedestrian to a motion-planning system that is configured to control a trajectory of an autonomous vehicle in the scene.
2 . The method of claim 1 , wherein predicting the speed of the pedestrian is performed by applying the machine-learning model to the image and no additional images.
3 . The method of claim 1 , wherein predicting the speed of the pedestrian further comprises:
determining a confidence level associated with the predicted speed; and
providing the confidence level to the motion-planning system.
4 . The method of claim 3 , wherein determining the confidence level associated with the predicted speed comprises:
predicting a speed of the pedestrian in a second image by applying the machine-learning model to at least a portion of the second image, and
comparing the predicted speed of the pedestrian in the second image to the predicted speed of the pedestrian in the received image.
5 . The method of claim 1 , further comprising, by one or more sensors of the autonomous vehicle moving in the scene, capturing the image.
6 . The method of claim 1 , wherein predicting the speed of the pedestrian is done in response to detecting the pedestrian within a threshold distance of the autonomous vehicle.
7 . The method of claim 1 , wherein detecting the pedestrian in the portion of the captured image comprises:
extracting one or more features from the image;
associating a bounding box or cuboid with the extracted features, the bounding boxes or cuboids defining a portion of the image containing the extracted features; and
applying a classifier to the portion of the image within the bounding box or cuboid, the classifier configured to identify images of pedestrians.
8 . A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
receive an image of a scene, wherein the image includes a pedestrian;
predict a speed of the pedestrian by applying a machine-learning model to at least a portion of the image that includes the pedestrian, wherein the machine-learning model has been trained using a data set comprising training images of pedestrians, the training images associated with corresponding known pedestrian speeds; and
provide the predicted speed of the pedestrian to a motion-planning system that is configured to control a trajectory of an autonomous vehicle in the scene.
9 . The system of claim 8 , wherein the at least one processor is configured to predict the speed of the pedestrian by applying the machine-learning model to the image and no additional images.
10 . The system of claim 8 , wherein the at least one processor is further configured to:
determine a confidence level associated with the predicted speed; and
provide the confidence level to the motion-planning system.
11 . The system of claim 10 , wherein the at least one processor is configured to determine the confidence level associated with the predicted speed by:
predicting a speed of the pedestrian in a second image by applying the machine-learning model to at least a portion of the second image, and
comparing the predicted speed of the pedestrian in the second image to the predicted speed of the pedestrian in the received image.
12 . The system of claim 8 , further comprising one or more sensors configured to capture the image.
13 . The system of claim 8 , wherein the at least one processor is configured to predict the speed of the pedestrian in response to detecting the pedestrian within a threshold distance of the autonomous vehicle.
14 . A non-transitory computer-readable medium that stores instructions that are configured to, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
receiving an image of a scene, wherein the image includes a pedestrian;
predicting a speed of the pedestrian by applying a machine-learning model to at least a portion of the image that includes the pedestrian, wherein the machine-learning model has been trained using a data set comprising training images of pedestrians, the training images associated with corresponding known pedestrian speeds; and
providing the predicted speed of the pedestrian to a motion-planning system that is configured to control a trajectory of an autonomous vehicle in the scene.
15 . The non-transitory computer-readable medium of claim 14 , wherein predicting the speed of the pedestrian is performed by applying the machine-learning model to the image and no additional images.
16 . The non-transitory computer-readable medium of claim 14 , wherein predicting the speed of the pedestrian further comprises:
determining a confidence level associated with the predicted speed; and
providing the confidence level to the motion-planning system.
17 . The non-transitory computer-readable medium of claim 14 , wherein:
determining the confidence level associated with the predicted speed comprises:
predicting a speed of the pedestrian in a second image by applying the machine-learning model to at least a portion of the second image, and
comparing the predicted speed of the pedestrian in the second image to the predicted speed of the pedestrian in the received image.
18 . The non-transitory computer-readable medium of claim 14 , wherein the instructions cause the at least one computing device to perform operations further comprising capturing the image by one or more sensors of the autonomous vehicle.
19 . The non-transitory computer-readable medium of claim 14 , wherein predicting the speed of the pedestrian is done in response to detecting the pedestrian within a threshold distance of the autonomous vehicle.
20 . The non-transitory computer-readable medium of claim 14 , wherein detecting the pedestrian in the portion of the captured image comprises:
extracting one or more features from the image;
associating a bounding box or cuboid with the extracted features, the bounding boxes or cuboids defining a portion of the image containing the extracted features; and
applying a classifier to the portion of the image within the bounding box or cuboid, the classifier configured to identify images of pedestrians.