Vehicle pose management
A computer includes a processor and a memory, and the memory stores instructions executable by the processor to receive an image frame from a camera, the image frame showing a vehicle, determine a two-dimensional bounding box around the vehicle in the image frame, determine a first three-dimensional point that is an intersection between (a) a ray extending from the camera through a center of the two-dimensional bounding box and (b) a first plane that is parallel to a ground plane, determine a second three-dimensional point on the vehicle, and determine an orientation of the vehicle based on the first three-dimensional point, the second three-dimensional point, and the ground plane. The ground plane represents a surface on which the vehicle sits.
1. A computer comprising a processor and a memory, the memory storing instructions executable by the processor to:
receive an image frame from a camera, the image frame showing a vehicle;
determine a two-dimensional bounding box around the vehicle in the image frame;
determine a first three-dimensional point that is an intersection between (a) a ray extending from the camera through a center of the two-dimensional bounding box and (b) a first plane that is parallel to a ground plane, the ground plane representing a surface on which the vehicle sits;
determine a second three-dimensional point on the vehicle; and
determine an orientation of the vehicle based on the first three-dimensional point, the second three-dimensional point, and the ground plane.
2. The computer of claim 1 , wherein the instructions further include instructions to transmit a message to the vehicle for actuating the vehicle based on the first three-dimensional point and the orientation.
3. The computer of claim 1 , wherein a distance between the first plane and the ground plane is half of a height of the vehicle.
4. The computer of claim 1 , wherein determining the second three-dimensional point includes identifying a landmark of the vehicle in the image frame and determining the second three-dimensional point corresponding to the landmark.
5. The computer of claim 4 , wherein the ray is a first ray, the intersection is a first intersection, and determining the second three-dimensional point includes determining a second intersection between (a) a second ray extending from the camera through the landmark and (b) the first plane.
6. The computer of claim 4 , wherein identifying the landmark includes receiving an input designating the landmark.
7. The computer of claim 1 , wherein the orientation includes a vehicle roll, a vehicle pitch, and a vehicle yaw, and determining the orientation includes setting the vehicle roll equal to a ground-plane roll of the ground plane and setting the vehicle pitch equal to a ground-plane pitch of the ground plane.
8. The computer of claim 7 , wherein determining the orientation includes determining the vehicle yaw based on a direction from the first three-dimensional point to the second three-dimensional point.
9. The computer of claim 1 , wherein determining the two-dimensional bounding box includes executing a neural network.
10. The computer of claim 1 , wherein the ground plane is preset in the memory.
11. The computer of claim 1 , wherein the instructions further include instructions to determine the ground plane based on range data received from a depth sensor.
12. The computer of claim 11 , wherein the range data is generated by the depth sensor when the vehicle is absent.
13. The computer of claim 12 , wherein determining the ground plane includes fitting a plane to the range data.
14. The computer of claim 1 , wherein determining the first three-dimensional point and determining the second three-dimensional point are performed repeatedly, each repetition for one of a plurality of image frames.
15. The computer of claim 14 , wherein determining the first three-dimensional point and determining the second three-dimensional point include minimizing an error across the repetitions.
16. The computer of claim 15 , wherein the error includes a reprojection error between pixel coordinates of the first three-dimensional point and the second three-dimensional point and reprojections of the first three-dimensional point and the second three-dimensional point to pixel coordinates.
17. The computer of claim 14 , wherein determining the first three-dimensional point and determining the second three-dimensional point include performing factor-graph optimization of the first three-dimensional point and the second three-dimensional point.
18. The computer of claim 14 , wherein determining the second three-dimensional point includes identifying a landmark of the vehicle in a first of the image frames, determining the second three-dimensional point corresponding to the landmark, and tracking the landmark over the repetitions.
19. The computer of claim 18 , wherein tracking the landmark over the repetitions is based on optical flow.
20. A method comprising:
receiving an image frame from a camera, the image frame showing a vehicle;
determining a two-dimensional bounding box around the vehicle in the image frame;
determining a first three-dimensional point that is an intersection between (a) a ray extending from the camera through a center of the two-dimensional bounding box and (b) a first plane that is parallel to a ground plane, the ground plane representing a surface on which the vehicle sits;
determining a second three-dimensional point on the vehicle; and
determining an orientation of the vehicle based on the first three-dimensional point, the second three-dimensional point, and the ground plane.