Self-supervised estimation of observed vehicle pose
A method may include: receiving a first image captured by a camera at a first time instance, wherein the first image includes at least a portion of an observed vehicle; determining a first ray angle based on a coordinate system of an ego-vehicle and a coordinate system of the observed vehicle corresponding to the first image; receiving a second image captured by the camera at a second time instance, wherein the second image includes at least a portion of the observed vehicle oriented at a different viewpoint; determining a second ray angle based on a coordinate system of the ego-vehicle and the coordinate system of the observed vehicle corresponding to the second image; determining a local angle difference based on the first ray angle and the second ray angle; and training a deep neural network using the local angle difference, the first image, and the second image.
1. A system comprising a computer including a processor and a memory, the memory including instructions such that the processor is programmed to:
receive a first image captured by a camera at a first time instance, wherein the first image includes at least a portion of an observed vehicle;
determine a first ray angle based on a coordinate system of an ego-vehicle and a first coordinate system of the observed vehicle based on the first image;
receive a second image captured by the camera at a second time instance, wherein the second image includes at least a portion of the observed vehicle oriented at a different viewpoint;
determine a second ray angle based on the coordinate system of the ego-vehicle and a second coordinate system of the observed vehicle based on the second image;
receive vehicle odometry data;
determine a local angle difference based on the first ray angle, the second ray angle, and the vehicle odometry data; and
train a deep neural network using the local angle difference, the first image, and the second image to output a position of the ego-vehicle with respect to the observed vehicle for operating the ego-vehicle,
wherein the deep neural network comprises a Siamese neural network and wherein the Siamese neural network determines a first local angle based on a first color image and a second local angle based on a second color image and determines a contrastive loss based on a difference between the first local angle and the second local angle.
2. The system of claim 1 , wherein the deep neural network comprises a convolutional neural network.
3. The system of claim 1 , wherein the Siamese neural network comprises a first convolutional neural network and a second convolutional neural network.
4. The system of claim 1 , wherein the processor is further programmed to:
update the deep neural network based on the contrastive loss.
5. The system of claim 4 , wherein the processor is further programmed to:
update at least one weight associated with a neuron of the deep neural network based on the contrastive loss.
6. The system of claim 1 , wherein the camera comprises a monocular camera.
7. The system of claim 1 , wherein the processor is further programmed to:
calculate, via the trained deep neural network, a yaw component corresponding to the observed vehicle based on at least one of the first image or the second image.
8. The system of claim 7 , wherein the processor is further programmed to:
calculate, via the trained deep neural network, a pitch component, a roll component, an x-component, a y-component, and a z-component corresponding to the observed vehicle based on at least one of the first image or the second image.
9. A method comprising:
receiving a first image captured by a camera at a first time instance, wherein the first image includes at least a portion of an observed vehicle;
determining a first ray angle based on a coordinate system of an ego-vehicle and a first coordinate system of the observed vehicle based on the first image;
receiving a second image captured by the camera at a second time instance, wherein the second image includes at least a portion of the observed vehicle oriented at a different viewpoint;
determining a second ray angle based on the coordinate system of the ego-vehicle and a second coordinate system of the observed vehicle based on the second image;
receiving vehicle odometry data;
determining a local angle difference based on the first ray angle, the second ray angle, and the vehicle odometry data; and
training a deep neural network using the local angle difference, the first image, and the second image to output a position of the ego-vehicle with respect to the observed vehicle for operating the ego-vehicle,
wherein the deep neural network comprises a Siamese neural network and wherein the Siamese neural network determines a first local angle based on a first color image and a second local angle based on a second color image and determines a contrastive loss based on a difference between the first local angle and the second local angle.
10. The method of claim 9 , wherein the deep neural network comprises a convolutional neural network.
11. The method of claim 9 , wherein the Siamese neural network comprises a first convolutional neural network and a second convolutional neural network.
12. The method of claim 9 , further comprising:
updating the deep neural network based on the contrastive loss.
13. The method of claim 12 , further comprising:
updating at least one weight associated with a neuron of the deep neural network based on the contrastive loss.
14. The method of claim 9 , wherein the camera comprises a monocular camera.
15. The method of claim 9 , wherein the processor is further programmed to:
calculate, via the trained deep neural network, a yaw component corresponding to the observed vehicle based on at least one of the first image or the second image.
16. The method of claim 9 , wherein the processor is further programmed to:
calculate, via the trained deep neural network, a pitch component, a roll component, an x-component, a y-component, and a z-component corresponding to the observed vehicle based on at least one of the first image or the second image.