IP Library Granted Patent US 11,157,766
Granted Patent B2
US 11,157,766 · App. 16/708,241 · Granted Oct 26, 2021

Method, apparatus, device and medium for calibrating pose relationship between vehicle sensor and vehicle

Inventors: Chengzhi Han (Beijing, CN); Yizhi Tang (Beijing, CN); Fei Yan (Beijing, CN)
Assignee: Beijing Baidu Netcom Science and Technology Co., Ltd.
G06K9/6212G06K9/00791
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,157,766
App. No.
16/708,241
Granted
Oct 26, 2021
Kind
B2
Abstract

A method, apparatus, device, and medium for calibrating a pose relationship between a vehicle sensor and a vehicle are provided. The method includes: acquiring, by a field-end sensor mounted in a side edge area of the calibration environment, a first image of the calibration environment, and acquiring a second image of the calibration environment by a vehicle sensor, different identifier units being laid on a spatial plane of the calibration environment; determining a first pose transformation relationship between a vehicle coordinate system and a world coordinate system based on the first image; determining a second pose transformation relationship between a vehicle sensor coordinate system and the world coordinate system based on the second image; and calibrating the pose relationship between the vehicle sensor and the vehicle using the above determined pose transformation relationships.

Claims (55)

1. A method for calibrating a pose relationship between a vehicle sensor and a vehicle, the method comprising:

acquiring, by a field-end sensor pre-deployed in a calibration environment, a first image of the calibration environment, and acquiring a second image of the calibration environment by the vehicle sensor, the field-end sensor being mounted in a side edge area of the calibration environment, different identifier boards arranged in a preset mode being laid on a spatial plane of the calibration environment, and each identifier board corresponding to a feature corner point;

determining a first pose transformation relationship between a vehicle coordinate system and a world coordinate system using a first feature corner point identified from the first image and a contact point between the vehicle and ground;

determining a second pose transformation relationship between a vehicle sensor coordinate system and the world coordinate system using a second feature corner point identified from the second image; and

calibrating the pose relationship between the vehicle sensor and the vehicle using the first pose transformation relationship and the second pose transformation relationship.

2. The method according to claim 1 , wherein the different identifier boards on the spatial plane are arranged based on a regular array form, and a different subidentifier is provided within a subarea of any color within each identifier board.

3. The method according to claim 1 , wherein a process of identifying the feature corner point comprises:

identifying the different identifier boards on a target image, and determining pixel coordinates of each identifier board, wherein the target image comprises the first image or the second image; and

determining the feature corner point on the target image based on the pixel coordinates of each identifier board using a feature corner detection technology.

4. The method according to claim 1 , wherein the determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system using the first feature corner point identified from the first image and the contact point between the vehicle and ground comprises:

determining a pose transformation relationship between a pixel coordinate system corresponding to the field-end sensor and the world coordinate system based on pixel coordinates and world coordinates corresponding to the first feature corner point identified from the first image;

determining world coordinates corresponding to the contact point using the pose transformation relationship between the pixel coordinate system corresponding to the field-end sensor and the world coordinate system, and pixel coordinates corresponding to the contact point; and

determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system based on the world coordinates corresponding to the contact point.

5. The method according to claim 4 , wherein the determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system based on the world coordinates corresponding to the contact point comprises:

determining a pose deviation of the contact point with respect to an origin of the vehicle coordinate system; and

determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system based on the pose deviation and the world coordinates corresponding to the contact point.

6. The method according to claim 1 , wherein the determining the second pose transformation relationship between the vehicle sensor coordinate system and the world coordinate system using the second feature corner point identified from the second image comprises:

determining a pose transformation relationship between a pixel coordinate system corresponding to the vehicle sensor and the world coordinate system based on pixel coordinates and world coordinates corresponding to the second feature corner point identified from the second image; and

determining the second pose transformation relationship between the vehicle sensor coordinate system and the world coordinate system using the pose transformation relationship between the pixel coordinate system corresponding to the vehicle sensor and the world coordinate system, and an internal parameter matrix of the vehicle sensor.

7. The method according to claim 4 , wherein a process of determining the world coordinates corresponding to each feature corner point comprises:

determining, based on an identifier board corresponding to each feature corner point, a unique identifier of each feature corner point;

determining the world coordinates corresponding to each feature corner point based on the unique identifier,

wherein information of each feature corner point and a corresponding identifier board, and a corresponding relationship between the unique identifier and the world coordinates of each feature corner point are predetermined in a process of deploying the calibration environment.

8. The method according to claim 1 , wherein the vehicle sensor comprises a front camera, a rear camera, and a circular camera.

9. An apparatus for calibrating a pose relationship between a vehicle sensor and a vehicle, the apparatus comprising:

at least one processor; and

a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring, by a field-end sensor pre-deployed in a calibration environment, a first image of the calibration environment, and acquire a second image of the calibration environment by the vehicle sensor, the field-end sensor being mounted in a side edge area of the calibration environment, different identifier boards arranged in a preset mode being laid on a spatial plane of the calibration environment, and each identifier board corresponding to a feature corner point;

determining a first pose transformation relationship between a vehicle coordinate system and a world coordinate system using a first feature corner point identified from the first image and a contact point between the vehicle and ground;

determining a second pose transformation relationship between a vehicle sensor coordinate system and the world coordinate system using a second feature corner point identified from the second image; and

calibrating the pose relationship between the vehicle sensor and the vehicle using the first pose transformation relationship and the second pose transformation relationship.

10. The apparatus according to claim 9 , wherein the different identifier boards on the spatial plane are arranged based on a regular array form, and a different subidentifier is provided within a subarea of any color within each identifier board.

11. The apparatus according to claim 9 , wherein a process of identifying the feature corner point comprises:

identifying the different identifier boards on a target image, and determine pixel coordinates of each identifier board, wherein the target image comprises the first image or the second image; and

determining the feature corner point on the target image based on the pixel coordinates of each identifier board using a feature corner detection technology.

12. The apparatus according to claim 9 , wherein the determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system using the first feature corner point identified from the first image and the contact point between the vehicle and ground comprises:

determining a pose transformation relationship between a pixel coordinate system corresponding to the field-end sensor and the world coordinate system based on pixel coordinates and world coordinates corresponding to the first feature corner point identified from the first image;

determining the world coordinates corresponding to the contact point using the pose transformation relationship between the pixel coordinate system corresponding to the field-end sensor and the world coordinate system, and pixel coordinates corresponding to the contact point; and

determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system based on the world coordinates corresponding to the contact point.

13. The apparatus according to claim 12 , wherein the determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system based on the world coordinates corresponding to the contact point comprises:

determining a pose deviation of the contact point with respect to an origin of the vehicle coordinate system; and

determining the first pose transformation relationship between the vehicle coordinate system and the world coordinate system based on the pose deviation and the world coordinates corresponding to the contact point.

14. The apparatus according to claim 9 , wherein the determining the second pose transformation relationship between the vehicle sensor coordinate system and the world coordinate system using the second feature corner point identified from the second image comprises:

determining a pose transformation relationship between a pixel coordinate system corresponding to the vehicle sensor and the world coordinate system based on pixel coordinates and world coordinates corresponding to the second feature corner point identified from the second image; and

determining the second pose transformation relationship between the vehicle sensor coordinate system and the world coordinate system using the pose transformation relationship between the pixel coordinate system corresponding to the vehicle sensor and the world coordinate system, and an internal parameter matrix of the vehicle sensor.

15. The apparatus according to claim 12 , wherein a process of determining the world coordinates corresponding to each feature corner point comprises:

determining, based on an identifier board corresponding to each feature corner point, a unique identifier of each feature corner point;

determining the world coordinates corresponding to each feature corner point based on the unique identifier,

wherein information of each feature corner point and a corresponding identifier board, and a corresponding relationship between the unique identifier and the world coordinates of each feature corner point are predetermined in a process of deploying the calibration environment.

16. The apparatus according to claim 9 , wherein the vehicle sensor comprises a front camera, a rear camera, and a circular camera.

17. A non-transitory computer readable storage medium, storing a computer program thereon, wherein the computer program, when executed by a processor, causes the processor to perform operations for calibrating a pose relationship between a vehicle sensor and a vehicle, the operations comprising:

acquiring, by a field-end sensor pre-deployed in a calibration environment, a first image of the calibration environment, and acquiring a second image of the calibration environment by the vehicle sensor, the field-end sensor being mounted in a side edge area of the calibration environment, different identifier boards arranged in a preset mode being laid on a spatial plane of the calibration environment, and each identifier board corresponding to a feature corner point;

determining a first pose transformation relationship between a vehicle coordinate system and a world coordinate system using a first feature corner point identified from the first image and a contact point between the vehicle and ground;

determining a second pose transformation relationship between a vehicle sensor coordinate system and the world coordinate system using a second feature corner point identified from the second image; and

calibrating a pose relationship between the vehicle sensor and the vehicle using the first pose transformation relationship and the second pose transformation relationship.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058241/0248 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2020
From: HAN, CHENGZHI; TANG, YIZHI; YANG, FEI
To: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
Reel/Frame 051604/0105 →
Priority Claims (1)
CN 201910515862.0 · Jun 14, 2019 · national
Continuity (1)
Related Publication 20200394445A1 · Dec 17, 2020