System and method for detecting a trailer coupler and planning a trajectory
A system and method for locating a trailer coupler on a trailer to assist with the coupling of the trailer coupler to a vehicle hitch on a vehicle. The system includes a human-machine interface, a vehicle camera and a trailer camera, a mobile device application (App) with access to camera and mobile network a vehicle-to-infrastructure communication network, controllers, a memory, sensors, and a trailer detection and trajectory planning application. The application includes: initializing a data stream from the vehicle and trailer camera, capturing the data stream from the vehicle camera and from the trailer camera, determining key landmarks in the data stream, combining the key landmarks, determining a coupler location, computing a joint reference trajectory using the coupler location, and providing navigation assistance in real time based on the joint reference trajectory.
1. A system for detecting a trailer coupler on a trailer to assist with coupling the trailer coupler to a vehicle hitch, the system comprising:
one or more human-machine interfaces (HMIs) disposed within a vehicle;
a vehicle camera disposed on the vehicle and detecting an environment surrounding the vehicle;
a trailer camera disposed on the trailer and detecting an environment surrounding the trailer;
a vehicle-to-infrastructure (V2I) communication network;
one or more controllers, each of the one or more controllers having a processor, a memory, and one or more input/output (I/O) ports, the one or more I/O ports in communication with the vehicle camera and the trailer camera, and in communication the one or more HMIs via the V2I communication network, the memory storing instructions including a trailer detection and trajectory planning application (DTPA) which, when executed by the processor, perform the following:
initialize a data stream from the vehicle camera and the trailer camera;
simultaneously capture the data stream from the vehicle camera and from the trailer camera;
determine key landmarks in the data stream from the vehicle camera and in the data stream from the trailer camera;
combine the key landmarks;
determine a coupler location with respect to the vehicle hitch using the combined key landmarks;
compute a joint reference trajectory using the coupler location with respect to a hitch location; and
provide navigation assistance in real time based on the joint reference trajectory.
2. The system of claim 1 , wherein the trailer camera is a cell phone camera in communication with the mobile application.
3. The system of claim 1 , wherein the one or more controllers is located in the vehicle and at the trailer.
4. The system of claim 1 , wherein the one or more controllers is located at a remote computing system.
5. The system of claim 1 , further comprising instructions which, when executed by the processor, requests an operator to provide an input to the HMI to confirm initialization of the DTPA.
6. The system of claim 1 , wherein determining the key landmarks in the data stream from the vehicle camera and the data stream from the trailer camera includes performing key landmark detection and dimensional calculations on the captured data stream to determine the coupler location in real time.
7. The system of claim 1 , further comprising instructions which, when executed by the processor, present navigation assistance including at least one of verbal or visual navigation assistance to the HMI to guide the vehicle hitch to the trailer coupler.
8. The system of claim 1 , further comprising instructions which, when executed by the processor, provide automated steering assistance.
9. A method for detecting location of a trailer coupler on a trailer and assisting with coupling of the trailer coupler to a vehicle hitch on a vehicle, the method comprising:
initializing a data stream from a vehicle camera and a trailer camera;
capturing the data stream from the vehicle camera and from the trailer camera;
determining key landmarks in the data stream from the vehicle camera and in the data stream from the trailer camera;
combining the key landmarks;
determining a coupler location with respect to the vehicle hitch using the combined key landmarks;
computing a joint reference trajectory using the coupler location with respect to a hitch location; and
providing navigation assistance in real time based on the joint reference trajectory.
10. The method of claim 9 , further comprising:
requesting an operator to provide an input to one or more human-machine interfaces (HMIs).
11. The method of claim 9 , further comprising:
providing a remote computing system, wherein one or more input/output (I/O) ports are in communication with the remote computing system via a vehicle-to-infrastructure (V2I) communication network.
12. The method of claim 9 , wherein capturing the data stream includes simultaneously capturing video stream from the vehicle camera and from the trailer camera.
13. The method of claim 9 , wherein presenting the navigation assistance in real time includes presenting at least one of verbal or visual assistance using one or more human-machine interfaces.
14. The method of claim 9 , wherein presenting the navigation assistance in real time includes providing steering assistance.
15. The method of claim 9 , wherein determining the key landmarks in the data stream from the vehicle camera and in the data stream from the trailer camera includes using a processor in the vehicle.
16. The method of claim 9 , wherein determining the key landmarks in the data stream from the vehicle camera and in the data stream from the trailer camera includes using a processor in the vehicle and a processor in the trailer.
17. The method of claim 9 , wherein determining the key landmarks in the data stream from the vehicle camera and in the data stream from the trailer camera includes using a controller in a remote computer.
18. The method of claim 9 , wherein the trailer camera includes a mobile device camera located at the trailer coupler.
19. A method for detecting location of a trailer coupler on a trailer and assisting with coupling of the trailer coupler to a vehicle hitch on a vehicle, the method comprising:
requesting an operator to provide an input to one or more human-machine interfaces (HMIs);
initializing a data stream from a vehicle camera and a trailer camera;
capturing the data stream from the vehicle camera and from the trailer camera;
determining key landmarks in the data stream from the vehicle camera and in the data stream from the trailer camera in real time;
combining the key landmarks determined from the data stream from the vehicle camera and from the data stream from the trailer camera;
determining a coupler location with respect to the vehicle hitch using combined key landmarks;
actively and dynamically computing a joint reference trajectory using the coupler location with respect to a hitch location; and
actively and continuously presenting navigation assistance in real time based on the joint reference trajectory, wherein the navigation assistance includes automated steering assistance.
20. The method of claim 19 , wherein the navigation assistance includes at least one of verbal or visual instruction displayed on the one or more human-machine interfaces (HMIs).