Enhanced sensor operation
A two-dimensional image of a vehicle occupant in a vehicle is collected. The collected two-dimensional image is input to a machine learning program trained to output one or more reference points of the vehicle occupant, each reference point being a landmark of the vehicle occupant. One or more reference points of the vehicle occupant in the two-dimensional image is output from the machine learning program. A location of the vehicle occupant in an interior of the vehicle is determined based on the one or more reference points. A vehicle component is actuated based on the determined location. For each of the one or more reference points, a similarity measure is determined between the reference point and a three-dimensional reference point, the similarity measure based on a distance between the reference point and the three-dimensional reference point.
1. A system, comprising:
a three-dimensional image sensor;
a two-dimensional image sensor; and
a computer communicatively coupled to the three-dimensional image sensor and the two-dimensional image sensor, the computer including a processor and a memory, the memory storing instructions executable by the processor to:
receive a two-dimensional image of an object from the two-dimensional image sensor;
output, from a machine learning program trained to identify one or more reference points in an input two-dimensional image, a reference point of the object in the two-dimensional image;
when a similarity measure based on a distance between the reference point and a three-dimensional reference point is below a threshold, collect three-dimensional image data of the object from the three-dimensional image sensor;
output a new three-dimensional reference point based on the collected three-dimensional image data; and
deactivate the three-dimensional image sensor upon outputting the new three-dimensional reference point.
2. The system of claim 1 , wherein the three-dimensional image sensor is one of a stereo image sensor or a time-of-flight image sensor.
3. The system of claim 1 , wherein the instructions further include instructions to identify the three-dimensional reference point from a reference three-dimensional image captured by the three-dimensional image sensor before the two-dimensional image sensor captured the two-dimensional image.
4. The system of claim 1 , wherein the image includes an occupant, and the reference point in the two-dimensional image is associated to a joint of the occupant.
5. The system of claim 1 , wherein the reference point is an object keypoint, and the similarity measure is an object keypoint similarity measure.
6. The system of claim 1 , wherein the distance is a Euclidean distance between the reference point and the three-dimensional reference point.
7. The system of claim 1 , wherein the instructions further include instructions to actuate a vehicle component based on the output reference point.
8. A system, comprising a computer including a processor and a memory, the memory storing instructions executable by the processor to:
collect a two-dimensional image of a vehicle occupant in a vehicle;
input the collected two-dimensional image to a machine learning program trained to output one or more reference points of the vehicle occupant, each reference point being a landmark of the vehicle occupant;
output, from the machine learning program, one or more reference points of the vehicle occupant in the two-dimensional image;
determine a location of the vehicle occupant in an interior of the vehicle based on the one or more reference points;
actuate a vehicle component based on the determined location;
for each of the one or more reference points, determine a similarity measure between the reference point and a three-dimensional reference point, the similarity measure based on a distance between the reference point and the three-dimensional reference point;
actuate a three-dimensional image sensor to collect three-dimensional image data; and
deactivate the three-dimensional image sensor upon generating a new three-dimensional reference point based on the three-dimensional image data.
9. The system of claim 8 , wherein the three-dimensional image sensor is one of a stereo image sensor or a time-of-flight image sensor.
10. The system of claim 8 , wherein the instructions further include instructions to identify a plurality of reference points in the two-dimensional image and to identify an action performed by the vehicle occupant based on the plurality of reference points.
11. The system of claim 10 , wherein the instructions further include instructions to actuate one or more vehicle components based on the identified action.
12. The system of claim 8 , wherein the instructions further include instructions to identify the three-dimensional reference point from a previously collected three-dimensional image.
13. The system of claim 8 , wherein at least one of the one or more reference points in the two-dimensional image is associated to a joint of the vehicle occupant.
14. The system of claim 8 , wherein each reference point is an object keypoint, and the similarity measure is an object keypoint similarity measure.
15. The system of claim 8 , wherein the instructions further include instructions to actuate a safety device based on the location of the vehicle occupant in the vehicle.
16. The system of claim 8 , wherein the instructions further include instructions to rotate an air vent toward the vehicle occupant based on the location of the vehicle occupant in the vehicle.
17. A system, comprising:
a three-dimensional image sensor;
a two-dimensional image sensor; and
a computer communicatively coupled to the three-dimensional image sensor and the two-dimensional image sensor, the computer including a processor and a memory, the memory storing instructions executable by the processor to:
receive a two-dimensional image of an object from the two-dimensional image sensor;
output, from a machine learning program trained to identify one or more reference points in an input two-dimensional image, a reference point of the object in the two-dimensional image;
when a similarity measure based on a distance between the reference point and a three-dimensional reference point is below a threshold, collect three-dimensional image data of the object from the three-dimensional image sensor;
output a new three-dimensional reference point based on the collected three-dimensional image data; and
reactivate the three-dimensional image sensor when the similarity measure is below the threshold.
18. The system of claim 17 , wherein the three-dimensional image sensor is one of a stereo image sensor or a time-of-flight image sensor.
19. The system of claim 17 , wherein the image includes an occupant, and the reference point in the two-dimensional image is associated to a joint of the occupant.
20. The system of claim 17 , wherein the reference point is an object keypoint, and the similarity measure is an object keypoint similarity measure.
21. The system of claim 17 , wherein the instructions further include instructions to actuate a vehicle component based on the output reference point.
22. A system, comprising a computer including a processor and a memory, the memory storing instructions executable by the processor to:
collect a two-dimensional image of a vehicle occupant in a vehicle;
input the collected two-dimensional image to a machine learning program trained to output one or more reference points of the vehicle occupant, each reference point being a landmark of the vehicle occupant;
output, from the machine learning program, one or more reference points of the vehicle occupant in the two-dimensional image;
determine a location of the vehicle occupant in an interior of the vehicle based on the one or more reference points;
actuate a vehicle component based on the determined location;
for each of the one or more reference points, determine a similarity measure between the reference point and a three-dimensional reference point, the similarity measure based on a distance between the reference point and the three-dimensional reference point; and
reactivate a three-dimensional image sensor when a similarity measure between a new three-dimensional reference point and one of the one or more reference points of the vehicle occupant in the image is below a threshold.
23. The system of claim 22 , wherein the three-dimensional image sensor is one of a stereo image sensor or a time-of-flight image sensor.
24. The system of claim 22 , wherein the instructions further include instructions to identify a plurality of reference points in the two-dimensional image and to identify an action performed by the vehicle occupant based on the plurality of reference points.
25. The system of claim 24 , wherein the instructions further include instructions to actuate one or more vehicle components based on the identified action.
26. The system of claim 22 , wherein the instructions further include instructions to identify the three-dimensional reference point from a previously collected three-dimensional image.