Granted Patent B2

US 12,198,382 · App. 17/753,964 · Granted Jan 14, 2025

Inventors: James Hockridge Critchley (Lake Orion, MI); Xin Yu (Rochester Hills, MI); Eduardo Jose Ramirez Llanos (Rochester, MI)

Assignee: Continental Autonomous Mobility US, LLC

G06T7/80B60R1/26B60R11/04G06T7/70G06T2207/10028G06T2207/30244G06T2207/30252

Abstract

Claims (56)

1. A method for calibrating extrinsic parameters of a trailer camera, the trailer camera supported by a trailer attached to a tow vehicle, the method comprising:

determining, at data processing hardware, a three-dimensional feature map from one or more vehicle images received from a camera supported by the tow vehicle;

identifying, at the data processing hardware, reference points within the three-dimensional feature map;

detecting, at the data processing hardware, the identified reference points within one or more trailer images received from the trailer camera after the vehicle and the trailer moved a predefined distance in the forward direction;

determining, at the data processing hardware, a trailer camera location of the trailer camera relative to the detected reference points in the three-dimensional feature map;

determining, at the data processing hardware, a trailer reference point based on the trailer camera location; and

determining, at the data processing hardware, extrinsic parameters of the trailer camera relative to the trailer reference point.

2. The method of claim 1 , wherein the trailer reference point is at a predefined distance from the trailer camera.

3. The method of claim 1 , wherein the trailer reference point overlaps with the trailer camera location.

4. The method of claim 1 , wherein the extrinsic parameters define the position of a center of the trailer camera and a heading of the trailer camera.

5. The method of claim 1 , further comprising:

associating an identifier with each one of the reference points identified in the one or more vehicle images;

wherein detecting the reference points within one or more trailer images received from the trailer camera includes determining the identifier associated with each one of the reference points.

6. The method of claim 1 , further comprising:

transmitting the extrinsic parameters of the trailer camera to one or more vehicle systems causing the vehicle system to execute a behavior using the extrinsic parameters.

7. The method of claim 1 , wherein determining a three-dimensional feature map from one or more vehicle images received from a camera supported by the tow vehicle includes executing at least one of: a Visual Odometry (VO) algorithm, a Simultaneous Localization and Mapping (SLAM) algorithm, or a Structure from Motion (SfM) algorithm.

8. The method of claim 1 , further comprising, before detecting, at the data processing hardware, the reference points within one or more trailer images received from the trailer camera;

transmitting instructions to a user interface, the instruction causing a user display to prompt a driver of the vehicle to drive the vehicle in forward direction;

receiving images from the trailer camera after the vehicle has traveled a predefined distance.

9. A method for calibrating extrinsic parameters of a trailer camera, the trailer camera supported by a trailer attached to a tow vehicle, the method comprising:

determining, at data processing hardware, a three-dimensional feature map from one or more vehicle images received from a camera supported by the tow vehicle;

identifying, at the data processing hardware, reference points within the three-dimensional feature map;

determining, at the data processing hardware, a vehicle pose relative to a first origin point within the three-dimensional feature map;

detecting, at the data processing hardware, the identified reference points within one or more trailer images received from the trailer camera;

determining, at the data processing hardware, a trailer camera location of the trailer camera relative to the detected reference points in the three-dimensional feature map;

determining, at the data processing hardware, a first trailer camera pose of the trailer camera relative to the vehicle pose within the three-dimensional feature map;

determining, at the data processing hardware, a trailer reference point based on the trailer camera location;

determining, at the data processing hardware, a trailer reference pose of the trailer reference point relative to the trailer camera location;

determining, at the data processing hardware, a second trailer camera pose of the trailer camera relative to the trailer reference pose; and

determining, at the data processing hardware, extrinsic parameters of the trailer camera relative to the trailer reference pose, and

transmitting the extrinsic parameters of the trailer camera to one or more vehicle systems of the tow vehicle causing the one or more vehicle systems to execute a behavior using the extrinsic parameters.

10. The method of claim 9 , wherein the trailer reference point is at a predefined distance from the trailer camera.

11. The method of claim 9 , wherein the trailer reference point overlaps with the trailer camera location.

12. The method of claim 9 , wherein the extrinsic parameters define the position of a center of the trailer camera and a heading of the trailer camera.

13. The method of claim 9 , further comprising:

associating an identifier with each one of the reference points identified in the one or more vehicle images;

wherein detecting the reference points within one or more trailer images received from the trailer camera includes determining the identifier associated with each one of the reference points.

14. The method of claim 9 , wherein determining a three-dimensional feature map from one or more vehicle images received from a camera supported by the tow vehicle includes executing at least one of: a Visual Odometry algorithm, a Simultaneous Localization and Mapping algorithm, or a Structure from Motion algorithm.

15. The method of claim 9 , further comprising, before detecting, at the data processing hardware, the reference points within one or more trailer images received from the trailer camera:

transmitting instructions to a user interface, the instruction causing a user display to prompt a driver of the vehicle to drive the vehicle in forward direction;

receiving images from the trailer camera after the vehicle has traveled a predefined distance.

16. A computer software product for calibrating extrinsic parameters of a trailer camera, the trailer camera supported by a trailer attached to a tow vehicle, the computer software product including instructions maintained in non-transitory memory which, when executed by data processing hardware of the tow vehicle, causes the data processing hardware to perform

determining, at data processing hardware, a three-dimensional feature map from one or more vehicle images received from a camera supported by the tow vehicle;

identifying, at the data processing hardware, reference points within the three-dimensional feature map;

detecting, at the data processing hardware, the identified reference points within one or more trailer images received from the trailer camera after the vehicle and the trailer moved a predefined distance in the forward direction;

determining, at the data processing hardware, a trailer camera location of the trailer camera relative to the detected reference points in the three-dimensional feature map;

determining, at the data processing hardware, a trailer reference point based on the trailer camera location;

determining, at the data processing hardware, extrinsic parameters of the trailer camera relative to the trailer reference point; and

transmitting, at the data processing hardware, the extrinsic parameters of the trailer camera to one or more vehicle systems of the tow vehicle causing the one or more vehicle systems to execute a behavior using the extrinsic parameters.

17. The computer software product of claim 16 , wherein the extrinsic parameters define the position of a center of the trailer camera and a heading of the trailer camera.

18. The computer software product of claim 16 , wherein the instructions, when executed by the data processing hardware, further cause the data processing hardware to associate an identifier with each one of the reference points identified in the one or more vehicle images, wherein detecting the reference points within one or more trailer images received from the trailer camera includes determining the identifier associated with each one of the reference points.

19. The computer software product of claim 16 , wherein the instructions, when executed by the data processing hardware, further cause the data processing hardware to,

before detecting the reference points within one or more trailer images received from the trailer camera;

transmit instructions to a user interface, the instruction causing a user display to prompt a driver of the vehicle to drive the vehicle in forward direction; and

receive images from the trailer camera after the vehicle has traveled the predefined distance.

20. The computer software product of claim 16 , determining the three-dimensional feature map from one or more vehicle images received from a camera supported by the tow vehicle includes executing at least one of: a Visual Odometry algorithm, a Simultaneous Localization and Mapping algorithm, or a Structure from Motion algorithm.

Assignments (2)

Continuity (2)

References Cited (27)

Cited By (1)

US 12,326,493