IP Library Granted Patent US 11,220,216
Granted Patent B2
US 11,220,216 · App. 16/704,856 · Granted Jan 11, 2022

Vehicle-trailer rearview vision system and method

Inventors: Mohannad Murad (Warren, MI); Fred W. Huntzicker (Warren, MI); Caroline Chung (Warren, MI)
Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
B60R1/003B60R1/00B60R1/04B60R1/12B60R2001/1253B60R2300/105B60R2300/303B60R2300/8046B60R2300/8066
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,220,216
App. No.
16/704,856
Granted
Jan 11, 2022
Kind
B2
Abstract

A method and system for generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method including: capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle; capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer; determining an obstructed viewing region within the first image, the obstructed viewing region being a region of the first image in which the trailer resides; overlaying the second image on the first image and at least partially within the obstructed viewing region; and inserting graphics in the first image, in the second image, and/or in a patch region within the obstructed viewing region and between the second image and the first image.

Claims (40)

1. A method of generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method comprising:

capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle;

capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer;

determining an articulation angle ϕ that corresponds to a relative alignment between the vehicle and the trailer, when the vehicle and the trailer are aligned in the same direction the articulation angle ϕ is zero, when the vehicle and the trailer are not aligned in the same direction the articulation angle ϕ is nonzero;

determining an obstructed viewing region within the first image based at least in part on the articulation angle ϕ, the obstructed viewing region being a region of the first image in which the trailer resides;

overlaying at least a portion of the second image at least partially within the obstructed viewing region of the first image, wherein the obstructed viewing region is at least partially based on the articulation angle ϕ when the at least a portion of the second image is overlaid; and

displaying the combined image on a display within the vehicle;

wherein the determining an articulation angle ϕ step further comprises determining the articulation angle ϕ by capturing an image of a trailer frame component with the vehicle camera, using image processing techniques on the image to determine a first reference angle α 1 that corresponds to a relative alignment between the vehicle and the trailer frame component at that moment, comparing the first reference angle α 1 to a predetermined reference angle α 0 that corresponds to a relative alignment between the vehicle and the trailer frame component when the vehicle and the trailer are aligned in the same direction, and subtracting one of the reference angles from the other of the reference angles to determine the articulation angle.

2. The method of claim 1 , wherein the determining an articulation angle ϕ step further comprises determining the articulation angle ϕ by sensing a steering wheel angle and using the steering wheel angle to determine the articulation angle.

3. A method of generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method comprising:

capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle;

capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer;

determining an articulation angle ϕ that corresponds to a relative alignment between the vehicle and the trailer, when the vehicle and the trailer are aligned in the same direction the articulation angle ϕ is zero, when the vehicle and the trailer are not aligned in the same direction the articulation angle ϕ is nonzero;

determining an obstructed viewing region within the first image based at least in part on the articulation angle ϕ, the obstructed viewing region being a region of the first image in which the trailer resides;

overlaying at least a portion of the second image at least partially within the obstructed viewing region of the first image, wherein the obstructed viewing region is at least partially based on the articulation angle ϕ when the at least a portion of the second image is overlaid; and

displaying the combined image on a display within the vehicle;

wherein the determining an obstructed viewing region step further comprises determining the obstructed viewing region by retrieving from memory a predefined obstructed viewing region definition that corresponds to an area of the first image that is obstructed by the trailer when the articulation angle ϕ is zero, and modifying the predefined obstructed viewing region definition based at least in part on the determined articulation angle ϕ so that the modified obstructed viewing region obstructs different regions of the first image than the predefined obstructed viewing region definition.

4. The method of claim 3 , wherein the obstructed viewing region is defined by four coordinates that represent vertices of a quadrangle and are located within the first image.

5. The method of claim 1 , wherein the determining an obstructed viewing region step further comprises determining the obstructed viewing region based at least in part on a length of the trailer.

6. The method of claim 1 , wherein the method further comprises:

determining an image overlay area that corresponds to an area on the first image in which the second image will be overlaid and at least partially overlaps with the obstructed viewing region, and

the overlaying step further comprises overlaying the second image within the image overlay area of the first image.

7. A method of generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method comprising:

capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle;

capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer;

determining an articulation angle ϕ that corresponds to a relative alignment between the vehicle and the trailer, when the vehicle and the trailer are aligned in the same direction the articulation angle ϕ is zero, when the vehicle and the trailer are not aligned in the same direction the articulation angle ϕ is nonzero;

determining an obstructed viewing region within the first image based at least in part on the articulation angle ϕ, the obstructed viewing region being a region of the first image in which the trailer resides;

overlaying at least a portion of the second image at least partially within the obstructed viewing region of the first image, wherein the obstructed viewing region is at least partially based on the articulation angle ϕ when the at least a portion of the second image is overlaid; and

displaying the combined image on a display within the vehicle;

wherein the method further comprises:

determining an image overlay area that corresponds to an area on the first image in which the second image will be overlaid and at least partially overlaps with the obstructed viewing region, and

the overlaying step further comprises overlaying the second image within the image overlay area of the first image;

wherein the method further comprises:

identifying a portion of the first image that corresponds to a patch region that is within the obstructed viewing region and outside the image overlay area; and

rendering graphics in the patch region within the obstructed viewing region and outside the image overlay area, wherein the graphics are configured using at least one of scene memorization and image stitching to fill the patch region.

8. The method of claim 3 , wherein the determining an articulation angle ϕ step further comprises determining the articulation angle ϕ by sensing a steering wheel angle and using the steering wheel angle to determine the articulation angle.

9. The method of claim 3 , wherein the determining an obstructed viewing region step further comprises determining the obstructed viewing region based at least in part on a length of the trailer.

10. The method of claim 3 , wherein the method further comprises:

determining an image overlay area that corresponds to an area on the first image in which the second image will be overlaid and at least partially overlaps with the obstructed viewing region, and

the overlaying step further comprises overlaying the second image within the image overlay area of the first image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2019
From: MURAD, MOHANNAD; HUNTZICKER, FRED W.; CHUNG, CAROLINE
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 051194/0592 →
Continuity (2)
Continuation 15889470 · Feb 6, 2018
Related Publication 20200108775A1 · Apr 9, 2020
Cited By (2)
US 12,501,168 US 12,664,628