System and method for estimating relative trailer angle
A method and system are disclosed for estimating and using a trailer angle of a trailer relative to a vehicle connected to thereto. The method includes receiving image data from at least one first camera disposed on a vehicle and from at least one second camera disposed on a trailer coupled to the vehicle, and identifying matched point pairs by matching points in the image data from the at least one first camera with points in the image data from the at least one second camera, the points matched not being points of a representation of the vehicle or a representations of the trailer in the image data. The trailer angle of the trailer relative to the vehicle is estimated based upon the matched point pairs.
1. A method for estimating and using a trailer angle of a trailer relative to a vehicle connected to thereto, comprising:
receiving, by data processing hardware including at least one processor, image data from at least one first camera disposed on a vehicle and from at least one second camera disposed on a trailer coupled to the vehicle, the image data from the at least one second camera being two dimensional image data;
identifying, by the data processing hardware, matched point pairs by matching points in the image data from the at least one first camera with points in the image data from the at least one second camera, the points matched not being points of a representation of the vehicle in the image data and not being points of a representation of the trailer in the image data;
estimating, by the data processing hardware, a trailer angle of the trailer relative to the vehicle based upon the matched point pairs;
providing, by the data processing hardware, at least one command to control at least part of the vehicle based upon the estimated trailer angle;
generating, by the data processing hardware, a three dimensional (3D) point cloud map from the image data from the at least one first camera;
determining, by the data processing hardware, a pose of the vehicle within the 3D point cloud map, wherein the trailer angle is estimated based upon the pose of the vehicle within the 3D point cloud map; and
based on the 3D point cloud map and the matched point pairs, determining, by the data processing hardware, a pose of the trailer within the 3D point cloud map, wherein estimating the trailer angle is based upon the pose of the vehicle and the pose of the trailer,
wherein matching points in the image data from the at least one first camera with points in the image data from the at least one second camera comprises matching 3D points in the 3D point cloud map with points of the image data of the at least one second camera,
wherein the trailer angle is estimated using the method without identifying features of the tow vehicle or the trailer in the image data.
2. The method of claim 1 , further comprising:
determining, by the data processing hardware, a pose of the at least one first camera within the 3D point cloud map, wherein the pose of the vehicle is determined based upon the pose of the at least one first camera and intrinsic parameters thereof; and
determining, by the data processing hardware, a pose of the at least one second camera within the 3D point cloud map, wherein the pose of the trailer is determined based upon the pose of the at least one second camera and intrinsic parameters thereof.
3. The method of claim 1 , wherein the at least one second camera captures a view rearward of the trailer.
4. The method of claim 1 , further comprising moving the vehicle so that the image data from the at least one first camera captures a scene which at least partly overlaps a scene captured by the at least one second camera.
5. A trailer system of a vehicle coupled to a trailer, the trailer assist system comprising:
non-transitory memory hardware, the non-transitory memory hardware storing instructions that when executed on data processing hardware cause the data processing hardware to perform operations comprising:
receiving image data from at least one first camera disposed on a vehicle and from at least one second camera disposed on a trailer coupled to the vehicle, the image data from the at least one second camera comprising two-dimensional image data;
identifying matched point pairs by matching points in the image data from the at least one first camera with points in the image data from the at least one second camera, the matched points not being points of a representation of the vehicle in the image data and not being points of a representations of the trailer in the image data, the identifying of the matched point pairs comprising using a bag-of-words model to match the point pairs;
estimating a trailer angle of the trailer relative to the vehicle based upon the matched point pairs; and
providing at least one signal or command to control the vehicle based upon the estimated trailer angle,
generating a three dimensional (3D) point cloud map from the image data from the at least one first camera; and
determining a pose of the vehicle within the 3D point cloud map,
wherein the trailer angle is estimated based upon the pose of the vehicle w thin the 3D point cloud map, and
wherein matching the points comprises matching 3D points in the 3D point cloud map with points in the image data of the at least one second camera, and
wherein the trailer angle is estimated by the trailer system without identifying features of the tow vehicle or the trailer in the image data.
6. The trailer assist system of claim 5 , wherein the instructions stored in non-transitory memory, when executed by the data processing hardware, cause the data processing hardware to perform operations further comprising, based the 3D point cloud map and the matched point pairs, determining a pose of the trailer within the 3D point cloud map, wherein estimating the trailer angle is based upon the pose of the vehicle and the pose of the trailer.
7. The trailer assist system of claim 6 , wherein the instructions stored in non-transitory memory, when executed by the data processing hardware, cause the data processing hardware to perform operations further comprising:
determining a pose of the at least one first camera within the 3D point cloud map, wherein the pose of the vehicle is determined based upon the pose of the at least one first camera and intrinsic parameters of the at least one first camera; and
determining a pose of the at least one second camera within the 3D point cloud map, wherein the pose of the trailer is determined based upon the pose of the at least one second camera and intrinsic parameters of the at least one second camera.
8. The trailer assist system of claim 5 , wherein the instructions stored in non-transitory memory, when executed by the data processing hardware, cause the data processing hardware to perform operations further comprising determining a pose of the at least one first camera within the 3D point cloud map, wherein the pose of the vehicle is determined based upon the pose of the at least one first camera.
9. The trailer assist system of claim 5 , wherein the instructions stored in non-transitory memory, when executed by the data processing hardware, cause the data processing hardware to perform operations further comprising moving the vehicle so that the image data from the at least one first camera captures a scene which at least partly overlaps a scene captured by the at least one second camera.
10. The method of claim 1 , wherein the received image data used to generate the 3D point cloud map and used in the matching is received only from the at least one first camera and the at least one second camera, without image data from sources that are not disposed on the vehicle and not disposed on the trailer.
11. The trailer assist system of claim 5 , wherein the received image data used to generate the 3D point cloud map and used in the matching is received only from the at least one first camera and the at least one second camera, without image data from sources that are not disposed on the vehicle or the trailer.
12. A program code product for a vehicle coupled to a trailer, the trailer assist program code product maintained in a non-transitory memory and including instructions which, when executed by at least one processor, causes the at least one processor to perform operations comprising:
receiving image data from at least one first camera disposed on a vehicle and from at least one second camera disposed on a trailer coupled to the vehicle;
generating a three dimensional (3D) point cloud map from the image data from the at least one first camera;
determining a pose of the vehicle within the 3D point cloud map;
identifying matched point pairs by matching points in the image data of the at least one second camera with 3D points in the 3D point cloud map, the matched points not being points of a representation of the vehicle in the image data and not being points of a representations of the trailer in the image data, wherein identifying the matched point pairs comprising using a bag-of-words model to match the point pairs;
estimating a trailer angle of the trailer relative to the vehicle based upon the matched point pairs; and
providing at least one signal to at least partly control movement of the vehicle based in part upon the estimated trailer angle,
wherein the operations performed by the at least one processor further include determining a pose of the trailer within the 3D point cloud map based the 3D point cloud map and the matched point pairs, wherein estimating the trailer angle is based upon the pose of the vehicle and the pose of the trailer, and
wherein the trailer angle is estimated by program code product without identifying features of the tow vehicle or the trailer in the image data.
13. The program code product of claim 12 , wherein the received image data used to generate the 3D point cloud map and used in the matching is received only from the at least one first camera and the at least one second camera, without image data from cameras that are not disposed on the vehicle or on the trailer.
14. The method of claim 1 , wherein identifying of the matched point pairs comprises using a bag-of-words model to match the point pairs.
15. The method of claim 1 , wherein the at least one first camera is a front vehicle camera that captures an environment forwardly of the vehicle, and the at least one second camera is a rear trailer camera that captures an environment rearwardly of the trailer.
16. The trailer system of claim 5 , wherein the at least one first camera is a front vehicle camera that captures an environment forwardly of the vehicle, and the at least one second camera is a rear trailer camera that captures an environment rearwardly of the trailer.
17. The program code product of claim 12 , wherein the at least one first camera is a front vehicle camera that captures an environment forwardly of the vehicle, and the at least one second camera is a rear vehicle camera that captures an environment rearwardly of the trailer.
18. The program code product of claim 12 , wherein the image data from the at least one second camera is two dimensional image data.