IP Library Granted Patent US 11,676,300
Granted Patent B2
US 11,676,300 · App. 17/116,628 · Granted Jun 13, 2023

Method for real-time tow ball detection

Inventors: Eduardo Jose Ramirez Llanos (Auburn Hills, MI); Xin Yu (Rochester Hills, MI); Matthew Berkemeier (Beverly Hills, MI)
Assignee: Continental Autonomous Mobility US, LLC
G06T7/74G06V20/56G06T2207/20076G06T2207/20081G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,676,300
App. No.
17/116,628
Granted
Jun 13, 2023
Kind
B2
Abstract

A method for determining a position of a tow ball in an image. The method includes obtaining at least one real-time image from a vehicle. The at least one real-time image is processed with a controller on the vehicle to obtain a feature patch (f) describing at least one real-time image. A convolution is performed of the feature patch (f) and each filter (h) from a set of filters (H) with the filter (h) being based on data representative of known tow hitches fixed to the vehicle. A location of a tow ball on the tow hitch is identified in the at least one real-time image is based on the convolution between the feature patch (f) and each filter (h) from the set of filters (H).

Claims (30)

1. A method for determining a position of a tow ball in an image, the method comprising:

obtaining at least one real-time image from a vehicle;

for each real-time image of the at least one real-time image, processing the real-time image with a controller on the vehicle to obtain a feature patch (f) describing the real time image, the feature patch (f) including at least one of a histogram of oriented gradients (“HOG”), Color Names, RGB, or grayscale;

performing a convolution of the feature patch (f) of each real-time image and each filter (h) from a set of filters (H) with each filter (h) being based on data representative of a known tow hitch fixed to the vehicle; and

identifying a location of a tow ball on the tow hitch in each real-time image of the at least one real-time image based on the convolution between the feature patch (f) of the real-time image and each filter (h) from the set of filters (H),

wherein the set of filters (H) is developed based on an algorithm evaluating extracted features from a database of rear-view images of the vehicle, wherein the extracted features include at least one of HOG, Color Names, RGB, or grayscale.

2. The method of claim 1 , wherein the set of filters (H) is stored in memory of the controller on the vehicle.

3. The method of claim 2 , wherein the set of filters (H) includes an optimization of at least one image of multiple known tow hitches fixed to the vehicle.

4. The method of claim 2 , wherein the set of filters (H) is performed by a computing device having a greater computing power than a computing power of the controller on the vehicle.

5. The method of claim 2 , wherein the at least one real time image includes consecutive real-time images and the method further comprises reducing a search area of the consecutive real-time images by comparing the consecutive real-time images to identify areas with objects that did not move between the consecutive images and searching the areas with objects that did not move.

6. The method of claim 5 , wherein the convolution is only performed on the areas with objects that did not move.

7. The method of claim 1 , wherein the at least one real-time image is captured by a camera on a rear of the vehicle.

8. The method of claim 1 , including receiving a location of the tow ball identified on a display by a user.

9. The method of claim 1 , wherein the set of filters (H) is developed based on an algorithm that identifies learned features from a database of rear-view images of the vehicle.

10. The method of claim 1 , wherein identifying the location of the tow ball includes utilizing an algorithm on the controller that identifies a location of highest probability of the location of the tow ball through a voting process.

11. The method of claim 10 , wherein the voting process identifies at least one pixel in the real-time image from the vehicle with the greatest correlation to the location of the tow ball from the set of filters (H).

12. The method of claim 1 , wherein the at least one real time image includes consecutive real-time images and the method further comprises reducing a search area of the consecutive real-time images by comparing the consecutive real-time images to identify areas with objects that did not move between the consecutive images and searching the areas with objects that did not move.

13. The method of claim 1 , wherein the feature patch (f) includes at least one of Color Names, RGB, or grayscale and wherein the extracted features include at least one of Color Names, RGB, or grayscale.

14. A tow ball identification assembly comprising:

a camera for capturing images of a tow ball; and

a controller configured to perform:

obtaining at least one real-time image from a vehicle;

for each real-time image, processing the real-time image with a controller on the vehicle to obtain a feature patch (f) describing the real-time image, the feature patch (f) including at least one of a histogram of oriented gradients (“HOG”), Color Names, RGB, or grayscale;

for each real-time image, performing a convolution of the feature patch (f) and each filter from a set of filters (H) with each filter (h) being based on data representative of a known tow hitch fixed relative to the vehicle; and

identifying a location of a tow ball on the tow hitch in each real-time image of the at least one real-time image based on the convolution between the feature patch (f) of each real-time image and each filter (h) from the set of filters (H),

wherein the set of filters (H) is developed based on an algorithm evaluating extracted features from a database of rear-view images of the vehicle, the extracted features include at least one of HOG, Color Names, RGB, or grayscale.

15. The assembly of claim 14 , wherein the set of filters (H) is stored in memory of the controller on the vehicle and the set of filters (H) includes an optimization of multiple images of known tow hitches fixed to the vehicle and performing the convolution between the feature patch (f) with each filter (h) from the set of filters (H), which produces a probability of the tow ball position in the image.

16. The assembly of claim 15 , wherein the at least one real time image includes consecutive real-time images and the controller is further configured to reduce a search area of the consecutive real-time images by comparing the consecutive real-time images to identify areas with objects that did not move between the consecutive images and searching the areas with objects that did not move.

17. The assembly of claim 14 , wherein the set of filters (H) is developed based on an algorithm that identifies learned features from a database of rear-view images of the vehicle.

18. The assembly of claim 14 , wherein the feature patch (f) includes at least one of Color Names, RGB, or grayscale and wherein the extracted features include at least one of Color Names, RGB, or grayscale.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 6, 2022
From: CONTINENTAL AUTOMOTIVE SYSTEMS, INC.
To: CONTINENTAL AUTONOMOUS MOBILITY US, LLC.
Reel/Frame 061100/0217 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2021
From: RAMIREZ LLANOS, EDUARDO JOSE; YU, XIN; BERKEMEIER, MATTHEW
To: CONTINENTAL AUTOMOTIVE SYSTEMS, INC.
Reel/Frame 056286/0582 →
Continuity (1)
Related Publication 20220180557A1 · Jun 9, 2022
Cited By (1)
US 12,400,448