IP Library Granted Patent US 11,170,227
Granted Patent B2
US 11,170,227 · App. 15/893,206 · Granted Nov 9, 2021

Generating an image of the surroundings of an articulated vehicle

Inventors: Hans M. Molin (MIssion Viejo, CA); Andreas U. Kuehnle (Strangnas, SE); Marton Gabor Gyori (Budapest, HU); Huba Nemeth (Budapest, HU); Cathy L. Boon (Orange, CA)
Assignee: Bendix Commercial Vehicle Systems LLC
G06K9/00791B60R1/00G01B11/002G01B11/022G01S11/12G06K9/20G06K9/52G06T3/4038G06T7/20H04N5/247H04N5/265H04N5/2624H04N7/181G06K2009/2045G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,170,227
App. No.
15/893,206
Granted
Nov 9, 2021
Kind
B2
Abstract

Systems and methods for generating an image of the surroundings of an articulated vehicle are provided. According to an aspect of the invention, a processor determines a relative position between a first vehicle of an articulated vehicle and a second vehicle of the articulated vehicle; receives a first image from a first camera arranged on the first vehicle and a second image from a second camera arranged on the second vehicle; and combines the first image and the second image based on the relative position between the first vehicle and the second vehicle to generate a combined image of surroundings of the articulated vehicle.

Claims (33)

1. A method comprising:

determining, by a processor, an angle between a first vehicle of an articulated vehicle and a second vehicle of the articulated vehicle as the first and second vehicles rotate laterally relative to each other around a point at which the first and second vehicles are connected to each other, based on a relative position between a first camera arranged on the first vehicle and a second camera arranged on the second vehicle, the first and second cameras being located on a same side of the articulated vehicle;

receiving, by the processor, a first image from the first camera arranged on the first vehicle and a second image from the second camera arranged on the second vehicle, the first and second images being obtained from the same side of the articulated vehicle;

combining, by the processor, the first image and the second image based on the relative position between the first camera and the second camera to generate a combined image of surroundings of the articulated vehicle on the same side thereof, wherein the first image and the second image are combined by rotating the first image and the second image with respect to each other, based on the angle between the first vehicle and the second vehicle;

detecting, by an active sensing system arranged on at least one of the first vehicle and the second vehicle, objects in the surroundings of the articulated vehicle; and

superimposing information and representations of the detected objects on the combined image, wherein the combined image is a birds-eye view image of the surroundings of the articulated vehicle and the superimposed information indicates a type of the detected objects and a likelihood of the articulated vehicle colliding with the detected objects.

2. The method according to claim 1 , wherein the angle is measured by an angular sensor arranged on the articulated vehicle.

3. The method according to claim 1 , wherein the angle between the first vehicle and the second vehicle is determined based on a first motion of the first vehicle and a second motion of the second vehicle, the first motion is measured by at least one first sensor arranged on the first vehicle, and the second motion is measured by at least one second sensor arranged on the second vehicle.

4. The method according to claim 3 , wherein each of the first motion and the second motion comprises at least one of a vehicle speed, a wheel speed, a yaw rate, and an acceleration.

5. The method according to claim 1 , wherein the angle between the first vehicle and the second vehicle is determined by detecting at least one feature of the second vehicle that appears in at least two images from the first camera.

6. The method according to claim 5 , wherein a first one of the images from the first camera is acquired when the angle between the first vehicle and the second vehicle is zero, and a second one of the images from the first camera is acquired when the angle between the first vehicle and the second vehicle is non-zero due to the first and second vehicles rotating laterally relative to each other around the point at which the first and second vehicles are connected to each other.

7. The method according to claim 1 , wherein the angle between the first vehicle and the second vehicle is determined by detecting at least one feature of the first vehicle that appears in at least two images from the second camera.

8. The method according to claim 7 , wherein a first one of the images from the first camera is acquired when the angle between the first vehicle and the second vehicle is zero, and a second one of the images from the first camera is acquired when the angle between the first vehicle and the second vehicle is non-zero due to the first and second vehicles rotating laterally relative to each other around the point at which the first and second vehicles are connected to each other.

9. The method according to claim 1 , wherein the relative position between the first vehicle and the second vehicle is determined continuously.

10. The method according to claim 1 , wherein the first image and the second image are transformed into ground plane images before rotating the first image and the second image.

11. The method according to claim 1 , wherein the components comprise at least one of ultrasonic sensors and radar sensors.

12. The method according to claim 1 , further comprising displaying the combined image and a forward-view image, wherein the forward-view image is acquired by a forward-facing camera arranged on the first vehicle.

13. A system comprising:

a first camera arranged on a first vehicle of an articulated vehicle;

a second camera arranged on a second vehicle of the articulated vehicle, wherein the first and second cameras are arranged on a same side of the articulated vehicle;

an active sensing system arranged on at least one of the first vehicle and the second vehicle, the active sensing system arranged to detect objects in surroundings of the articulated vehicle;

a memory; and

a processor coupled to the memory, the processor comprising:

position determining logic that determines an angle between the first vehicle and the second vehicle as the first and second vehicles rotate laterally relative to each other around a point at which the first and second vehicles are connected to each other, based on a relative position between a first camera arranged on the first vehicle and a second camera arranged on the second vehicle, the first and second cameras being located on a same side of the articulated vehicle;

image receiving logic that receives a first image from the first camera and a second image from the second camera, the first and second images being obtained from the same side of the articulated vehicle;

image combination logic that combines the first image and the second image based on the relative position between the first camera and the second camera to generate a combined image of the surroundings of the articulated vehicle on the same side thereof, wherein the first image and the second image are combined by rotating the first image and the second image with respect to each other, based on the angle between the first vehicle and the second vehicle; and

superimposing logic that superimposes information and representations of the detected objects on the combined image, wherein the combined image is a birds-eye view image of the surroundings of the articulated vehicle and the superimposed information indicates a type of the detected objects and a likelihood of the articulated vehicle colliding with the detected objects.

14. A non-transitory computer-readable medium comprising computer instructions executable by a processor to cause the processor to perform a method comprising:

determining an angle between a first vehicle of an articulated vehicle and a second vehicle of the articulated vehicle as the first and second vehicles rotate laterally relative to each other around a point at which the first and second vehicles are connected to each other, based on a relative position between a first camera arranged on the first vehicle and a second camera arranged on the second vehicle, the first and second cameras being located on a same side of the articulated vehicle;

receiving images from a first camera arranged on the first vehicle and a second camera arranged on the second vehicle, the first and second images being obtained from the same side of the articulated vehicle;

combining the first image and the second image based on the relative position between the first camera and the second camera to generate a combined image of surroundings of the articulated vehicle on the same side thereof, wherein the first image and the second image are combined by rotating the first image and the second image with respect to each other, based on the angle between the first vehicle and the second vehicle;

detecting, by an active sensing system arranged on at least one of the first vehicle and the second vehicle, objects in the surroundings of the articulated vehicle; and

superimposing information and representations of the detected objects on the combined image, wherein the combined image is a birds-eye view image of the surroundings of the articulated vehicle and the superimposed information indicates a type of the detected objects and a likelihood of the articulated vehicle colliding with the detected objects.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2024
From: BENDIX COMMERCIAL VEHICLE SYSTEMS LLC
To: RM ACQUISITION, LLC
Reel/Frame 067824/0004 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 4, 2023
From: MOLIN, HANS M.; KUEHNLE, ANDREAS U.; GYORI, MARTON GABOR; NEMETH, HUBA; BOON, CATHY L.
To: BENDIX COMMERCIAL VEHICLE SYSTEMS LLC
Reel/Frame 065753/0728 →
Continuity (2)
Continuation 14247710 · Apr 8, 2014
Related Publication 20180165524A1 · Jun 14, 2018