Granted Patent B2

US 12,384,295 · App. 17/646,902 · Granted Aug 12, 2025

Inventors: Jyothi P. Gali (Rochester Hills, MI); Harold E. Joseph (Brampton, CA); Prasanna Ananthakrishnan (Tamilnadu, IN); Alexander Velichko (Toronto, CA); Guruprasad Mani Iyer Shankaranarayanan (Mississauga, CA)

Assignee: Magna Electronics Inc.

B60R1/002B60R2300/105B60R2300/307B60R2300/607B60R2300/80

Abstract

Claims (71)

1. A vehicular trailer assist system, the vehicular trailer assist system comprising:

a rearward viewing camera disposed at a vehicle equipped with the vehicular trailer assist system, the rearward viewing camera viewing at least a portion of a trailer hitched at a fifth wheel hitch at a bed of the vehicle, wherein the rearward viewing camera captures fisheye-view frames of image data;

wherein the rearward viewing camera comprises a CMOS imaging array having at least one million photosensors arranged in rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor operable to process fisheye-view frames of image data captured by the rearward viewing camera;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system, via image processing at the ECU of fisheye-view frames of image data captured by the rearward viewing camera, transforms frames of image data captured by the rearward viewing camera from fisheye-view frames of image data into bird's-eye view frames of image data;

wherein the vehicular trailer assist system determines the fifth wheel hitch within the transformed bird's-eye view frames of image data;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system warps the transformed bird's-eye view frame of image data, and wherein the warped transformed bird's-eye view frame of image data comprises (i) an X-axis where movement of the trailer within the frame of image data along the X-axis indicates change in trailer angle of the trailer relative to the vehicle and (ii) a Y-axis where position of the trailer within the frame of image data along the Y-axis indicates distance from the fifth wheel hitch; and

wherein the vehicular trailer assist system, using the X-axis of the warped transformed bird's-eye view frame of image data, determines the trailer angle of the trailer relative to the vehicle.

2. The vehicular trailer assist system of claim 1 , wherein the rearward viewing camera is disposed at a center high-mounted stop lamp (CHMSL) of the vehicle.

3. The vehicular trailer assist system of claim 1 , wherein the vehicular trailer assist system warps the transformed bird's-eye view frame of image data based on edges output by a Canny edge detection algorithm.

4. The vehicular trailer assist system of claim 1 , wherein the vehicular trailer assist system determines the trailer angle of the trailer relative to the vehicle using template matching.

5. The vehicular trailer assist system of claim 4 , wherein the vehicular trailer assist system dynamically generates a template during template matching.

6. The vehicular trailer assist system of claim 5 , wherein the vehicular trailer assist system dynamically generates a template during template matching at predetermined intervals of change of the trailer angle.

7. A vehicular trailer assist system, the vehicular trailer assist system comprising:

a rearward viewing camera disposed at a vehicle equipped with the vehicular trailer assist system, the rearward viewing camera viewing at least a portion of a trailer hitched at a fifth wheel hitch at a bed of the vehicle, wherein the rearward viewing camera captures fisheye-view frames of image data;

wherein the rearward viewing camera comprises a CMOS imaging array having at least one million photosensors arranged in rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor operable to process fisheye-view frames of image data captured by the rearward viewing camera;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system, via image processing at the ECU of fisheye-view frames of image data captured by the rearward viewing camera, transforms frames of image data captured by the rearward viewing camera from fisheye-view frames of image data into bird's-eye view frames of image data;

wherein the vehicular trailer assist system determines the fifth wheel hitch within the transformed bird's-eye view frames of image data;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system warps the transformed bird's-eye view frame of image data, and wherein the warped transformed bird's-eye view frame of image data comprises (i) an X-axis that indicates change in trailer angle of the trailer relative to the vehicle and (ii) a Y-axis that indicates distance from the fifth wheel hitch;

wherein the vehicular trailer assist system, using the X-axis of the warped transformed bird's-eye view frame of image data, determines the trailer angle of the trailer relative to the vehicle;

wherein the vehicular trailer assist system determines the trailer angle of the trailer relative to the vehicle using template matching;

wherein the vehicular trailer assist system dynamically generates a template during template matching;

wherein the vehicular trailer assist system dynamically generates a template during template matching at predetermined intervals of change of the trailer angle; and

wherein the predetermined intervals of change are each five degrees.

8. A vehicular trailer assist system, the vehicular trailer assist system comprising:

a rearward viewing camera disposed at a vehicle equipped with the vehicular trailer assist system, the rearward viewing camera viewing at least a portion of a trailer hitched at a fifth wheel hitch at a bed of the vehicle, wherein the rearward viewing camera captures fisheye-view frames of image data;

wherein the rearward viewing camera comprises a CMOS imaging array having at least one million photosensors arranged in rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor operable to process fisheye-view frames of image data captured by the rearward viewing camera;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system, via image processing at the ECU of fisheye-view frames of image data captured by the rearward viewing camera, transforms frames of image data captured by the rearward viewing camera from fisheye-view frames of image data into bird's-eye view frames of image data;

wherein the vehicular trailer assist system determines the fifth wheel hitch within the transformed bird's-eye view frames of image data;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system warps the transformed bird's-eye view frame of image data, and wherein the warped transformed bird's-eye view frame of image data comprises (i) an X-axis that indicates change in trailer angle of the trailer relative to the vehicle and (ii) a Y-axis that indicates distance from the fifth wheel hitch;

wherein the vehicular trailer assist system, using the X-axis of the warped transformed bird's-eye view frame of image data, determines the trailer angle of the trailer relative to the vehicle;

wherein the vehicular trailer assist system determines the trailer angle of the trailer relative to the vehicle using template matching; and

wherein the vehicular trailer assist system dynamically determines thresholds for the template matching based on at least one selected from the group consisting of (i) a level of brightness in the warped transformed bird's-eye view frame of image data and (ii) a level of contrast in the warped transformed bird's-eye view frame of image data.

9. The vehicular trailer assist system of claim 1 , wherein the vehicular trailer assist system tracks the fifth wheel hitch in consecutive frames of image data captured by the rearward viewing camera.

10. The vehicular trailer assist system of claim 1 , wherein the vehicular trailer assist system tracks the trailer angle in consecutive frames of image data captured by the rearward viewing camera.

11. The vehicular trailer assist system of claim 1 , wherein the trailer hitched to the fifth wheel hitch at the bed of the vehicle comprises a gooseneck trailer or a fifth wheel trailer.

12. A vehicular trailer assist system, the vehicular trailer assist system comprising:

a rearward viewing camera disposed at a vehicle equipped with the vehicular trailer assist system, the rearward viewing camera viewing at least a portion of a trailer hitched at a fifth wheel hitch at a bed of the vehicle, wherein the rearward viewing camera captures fisheye-view frames of image data;

wherein the rearward viewing camera comprises a CMOS imaging array having at least one million photosensors arranged in rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor operable to process fisheye-view frames of image data captured by the rearward viewing camera;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system, via image processing at the ECU of fisheye-view frames of image data captured by the rearward viewing camera, transforms frames of image data captured by the rearward viewing camera from fisheye-view frames of image data into bird's-eye view frames of image data;

wherein the vehicular trailer assist system determines the fifth wheel hitch within the transformed bird's-eye view frames of image data;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system warps the transformed bird's-eye view frame of image data, and wherein the warped transformed bird's-eye view frame of image data comprises (i) an X-axis that indicates change in trailer angle of the trailer relative to the vehicle and (ii) a Y-axis that indicates distance from the fifth wheel hitch;

wherein the vehicular trailer assist system, using the X-axis of the warped transformed bird's-eye view frame of image data, determines the trailer angle of the trailer relative to the vehicle; and

wherein, when the trailer angle increases, the trailer moves along the X-axis of the warped transformed bird's-eye view frame of image data in a first direction, and wherein, when the trailer angle decreases, the trailer moves along the X-axis of the warped transformed bird's-eye view frame of image data in a second direction.

13. A vehicular trailer assist system, the vehicular trailer assist system comprising:

a rearward viewing camera disposed at a vehicle equipped with the vehicular trailer assist system, the rearward viewing camera viewing at least a portion of a trailer hitched at a hitch at a bed of the vehicle, wherein the rearward viewing camera captures fisheye-view frames of image data;

wherein the rearward viewing camera comprises a CMOS imaging array having at least one million photosensors arranged in rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor operable to process frames of image data captured by the rearward viewing camera;

wherein, with the trailer hitched to the hitch at the bed of the vehicle, the vehicular trailer assist system, via image processing at the ECU of frames of image data captured by the rearward viewing camera, transforms frames of image data captured by the rearward viewing camera into bird's-eye view frames of image data;

wherein the vehicular trailer assist system determines a hitch point of the hitch within transformed bird's-eye view frames of image data;

wherein the vehicular trailer assist system warps the transformed bird's-eye view frame of image data, and wherein the warped transformed bird's-eye view frame of image data comprises (i) an X-axis where movement of the trailer within the frame of image data along the X-axis indicates change in trailer angle of the trailer relative to the vehicle and (ii) a Y-axis where position of the trailer within the frame of image data along the Y-axis indicates distance from the hitch point; and

wherein the vehicular trailer assist system, using the X-axis of the warped transformed bird's-eye view frame of image data, determines the trailer angle of the trailer relative to the vehicle.

14. The vehicular trailer assist system of claim 13 , wherein the rearward viewing camera is disposed at a center high-mounted stop lamp (CHMSL) of the vehicle.

15. The vehicular trailer assist system of claim 13 , wherein the vehicular trailer assist system warps the transformed bird's-eye view frame of image data based on edges output by a Canny edge detection algorithm.

16. The vehicular trailer assist system of claim 13 , wherein the vehicular trailer assist system determines the trailer angle of the trailer relative to the vehicle using template matching.

17. The vehicular trailer assist system of claim 16 , wherein the vehicular trailer assist system dynamically generates a template during template matching.

18. A vehicular trailer assist system, the vehicular trailer assist system comprising:

a rearward viewing camera disposed at a vehicle equipped with the vehicular trailer assist system, the rearward viewing camera viewing at least a portion of a trailer hitched at a fifth wheel hitch at a bed of the vehicle, wherein the rearward viewing camera captures fisheye-view frames of image data;

wherein the rearward viewing camera comprises a CMOS imaging array having at least one million photosensors arranged in rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software, wherein the electronic circuitry comprises an image processor operable to process fisheye-view frames of image data captured by the rearward viewing camera;

wherein, with the trailer hitched to the fifth wheel hitch at the bed of the vehicle, the vehicular trailer assist system, via image processing at the ECU of fisheye-view frames of image data captured by the rearward viewing camera, transforms frames of image data captured by the rearward viewing camera from fisheye-view frames of image data into bird's-eye view frames of image data;

wherein the vehicular trailer assist system determines the fifth wheel hitch within a first transformed bird's-eye view frame of image data;

wherein the vehicular trailer assist system warps the transformed bird's-eye view frame of image data, and wherein the warped transformed bird's-eye view frame of image data comprises (i) an X-axis where movement of the trailer within the frame of image data along the X-axis indicates change in trailer angle of the trailer relative to the vehicle and (ii) a Y-axis where position of the trailer within the frame of image data along the Y-axis indicates distance from the fifth wheel hitch;

wherein the vehicular trailer assist system, using the X-axis of the warped transformed bird's-eye view frame of image data, determines the trailer angle of the trailer relative to the vehicle;

wherein, after determining the trailer angle of the trailer relative to the vehicle, the vehicular trailer assist system determines the fifth wheel hitch within a second transformed bird's-eye view frame of image data; and

wherein the vehicular trailer assist system, using the second transformed-bird's-eye view frame of image data, updates the trailer angle of the trailer relative to the vehicle.

19. The vehicular trailer assist system of claim 18 , wherein the first transformed bird's-eye view frame of image data and the second transformed bird's-eye view frame of image data are consecutive frames of image data captured by the rearward viewing camera.

20. The vehicular trailer assist system of claim 18 , wherein the trailer hitched to the fifth wheel hitch at the bed of the vehicle comprises a gooseneck trailer or a fifth wheel trailer.

Assignments (1)

Continuity (2)

References Cited (88)

Cited By (1)

US 12,522,222