Granted Patent B2

US 12,499,575 · App. 18/116,941 · Granted Dec 16, 2025

Inventors: Pengfei Yang (Fujian, CN); Qingyang Su (Fujian, CN); Tudui Hong (Fujian, CN)

Assignee: Faurecia Clarion Electronics (Xiamen) Co., Ltd.

G06T7/74G01C21/3602G01C21/3632G06T3/606G06T2207/20068G06T2207/20212G06T2207/30244G06T2207/30252

Abstract

Claims (98)

1 . A vehicle navigation image synthesis method applied to a vehicle navigation image synthesis system, comprising:

acquiring M sampling images, wherein the M sampling images include a preset sampling image for establishing a reference coordinate system, the preset sampling image is a sampling image in which a stop line or a zebra crossing is acquired for a first time, and M is a positive integer;

storing N sampling images, wherein N is a positive integer less than or equal to M, the N sampling images are incomplete images, a value of N is determined according to a speed of a target vehicle, a position of the target vehicle, and a following distance of the target vehicle;

determining a plurality of sampling points of each sampling image of the N images;

determining first coordinates of each sampling point of the plurality of sampling points of each sampling image, wherein the first coordinates include a first Y-coordinate and a first X-coordinate; the first Y-coordinate is determined according to parameters of an image acquisition device and a position of each sampling point in a sampling image where each sampling point is located; the first X-coordinate is determined according to projections, on an intermediate coordinate system, of border lines of the sampling image where each sampling point is located; and the intermediate coordinate system is a coordinate system established according to a current position of the target vehicle and a current driving direction of the target vehicle;

mapping the first coordinates of each sampling point to the reference coordinate system to determine second coordinates of each sampling point; and

synthesizing sampling points of the N sampling images according to second coordinates of the sampling points of the N sampling images to determine a target image, wherein the target image is a complete image; and

determining, by the vehicle navigation image synthesis system, image information of an entire junction according to the target image.

2 . The method according to claim 1 , wherein determining the first Y-coordinate of each sampling point includes:

determining a target sampling point and a target sampling image where the target sampling point is located, wherein the target sampling point is a sampling point of a plurality of sampling points of the target sampling image, and the target sampling image is a sampling image of the N sampling images;

establishing an image coordinate system of the target sampling image, wherein an origin of the image coordinate system is a center point of the target sampling image, an X-axis of the image coordinate system is perpendicular to a left border line of the target sampling image or a right border line of the target sampling image, and a Y-axis of the image coordinate system is perpendicular to an upper border line of the target sampling image or a lower border line of the target sampling image;

determining third coordinates of the target sampling point in the image coordinate system, wherein the third coordinates of the target sampling point in the image coordinate system include a third Y-coordinate and a third X-coordinate; the third X-coordinate is an X-coordinate of the target sampling point of the image coordinate system; and the third Y-coordinate is a Y-coordinate of the target sampling point of the image coordinate system; and

determining a first Y-coordinate of the target sampling point according to the parameters of the image acquisition device and the third Y-coordinate, wherein the parameters of the image acquisition device include a vertical distance between the image acquisition device and a ground, a sampling angle of the image acquisition device, and a focal length of the image acquisition device; and the sampling angle of the image acquisition device is an included angle between an acquisition direction of the image acquisition device and a direction perpendicular to the ground.

3 . The method according to claim 2 , wherein determining the first X-coordinate of each sampling point includes:

establishing an intermediate coordinate system corresponding to the target sampling image, wherein an origin of the intermediate coordinate system corresponding to the target sampling image is a current position of the target vehicle corresponding to the target sampling image, a positive direction of a Y-axis of the intermediate coordinate system corresponding to the target sampling image is a current driving direction of the target vehicle corresponding to the target sampling image, and an X-axis of the intermediate coordinate system corresponding to the target sampling image is parallel to the ground;

determining a slope of a first straight line, a slope of a second straight line, and a target quantity, wherein the first straight line is a straight line obtained by projecting the left border line of the target sampling image on the intermediate coordinate system corresponding to the target sampling image; the second straight line is a straight line obtained by projecting the right border line of the target sampling image on the intermediate coordinate system corresponding to the target sampling image; and the target quantity is a half of a number of sampling points of the target sampling image on the X-axis of the image coordinate system; and

determining a first X-coordinate of the target sampling point according to the slope of the first straight line, the slope of the second straight line, the target quantity, the first Y-coordinate of the target sampling point, and the third X-coordinate.

4 . The method according to claim 3 , wherein the first X-coordinate satisfies a formula:

X 1 =X 3 ×((( Y 1 −f )/ k )/ J );

wherein X 1 is the first X-coordinate; X 3 is the third X-coordinate; Y 1 is the first Y-coordinate; f is the slope of the first straight line; k is the slope of the second straight line; and J is the target quantity.

5 . The method according to claim 2 , wherein mapping the first coordinates of each sampling point to the reference coordinate system to determine the second coordinates of each sampling point includes:

mapping the first coordinates of each sampling point to an angle-restoration coordinate system to determine fourth coordinates of each sampling point, wherein the angle-restoration coordinate system is a coordinate system obtained by rotating the intermediate coordinate system by a preset angle; and

mapping the fourth coordinates of each sampling point to the reference coordinate system to determine the second coordinates of each sampling point.

6 . The method according to claim 5 , wherein mapping the first coordinates of each sampling point to the angle-restoration coordinate system to determine the fourth coordinates of each sampling point includes:

acquiring a deviation angle of the target vehicle, wherein the deviation angle of the target vehicle is an included angle between the current driving direction of the target vehicle and a straight-line driving direction of the target vehicle;

determining a preset angle of the target sampling image, wherein the preset angle of the target sampling image is determined according to the deviation angle of the target vehicle within a sampling cycle of the target sampling image;

rotating the target sampling image and the intermediate coordinate system corresponding to the target sampling image clockwise by the preset angle of the target sampling image to determine an angle-restored target sampling image and an angle-restoration coordinate system corresponding to the target sampling image; and

determining coordinates of the target sampling point of the angle-restored target sampling image in the angle-restoration coordinate system corresponding to the target sampling image as fourth coordinates of the target sampling point.

7 . The method according to claim 6 , wherein mapping the fourth coordinates of each sampling point to the reference coordinate system to determine the second coordinates of each sampling point includes:

establishing the reference coordinate system, wherein an origin of the reference coordinate system is a position of the target vehicle within a sampling cycle of the preset sampling image, a positive direction of a Y-axis of the reference coordinate system is a driving direction of the target vehicle within the sampling cycle of the preset sampling image, and an X-axis of the reference coordinate system is parallel to the ground;

determining a first distance difference and a second distance difference, wherein the first distance difference is a distance difference between an origin of the angle-restoration coordinate system and the origin of the reference coordinate system in a direction of an X-axis of the angle-restoration coordinate system, and the second distance difference is a distance difference between the origin of the angle-restoration coordinate system and the origin of the reference coordinate system in a direction of a Y-axis of the angle-restoration coordinate system; and

determining a second X-coordinate and a second Y-coordinate included in the second coordinates, wherein the second X-coordinate is a difference between a fourth X-coordinate of the fourth coordinates and the first distance difference, and the second Y-coordinate is a difference between a fourth Y-coordinate of the fourth coordinate and the second distance difference.

8 . The method according to claim 5 , wherein after synthesizing the sampling points of the N sampling images according to the second coordinates of the sampling points of the N sampling images to determine the target image, the method further comprises:

dividing a target area of the target image into P sampling points;

determining fifth coordinates of each of the P sampling points in the reference coordinate system, wherein the target area is an area whose pixels per inch (PPI) is lower than a preset PPI threshold, and P is a positive integer;

determining a sampling point satisfying a preset condition from the sampling points of the N sampling images according to the fifth coordinates of the first sampling point, wherein the first sampling point is a sampling point of the P sampling points;

determining image information of the first sampling point according to image information of the sampling point satisfying the preset condition; and

adding the image information of the first sampling point to image information of the target image according to the fifth coordinates of the first sampling point.

9 . The method according to claim 2 , wherein the first Y-coordinate satisfies a formula:

Y 1 =tan(arctan( Y 3 /Ob )+β)× h;

wherein Y 1 is the first Y-coordinate; Y 3 is the third Y-coordinate; Ob is the focal length of the image acquisition device; β is the sampling angle of the image acquisition device; and h is the vertical distance between the image acquisition device and the ground.

10 . The method according to claim 1 , wherein storing the N sampling images includes:

acquiring following distances within M sampling cycles, wherein a single sampling cycle corresponds to a single sampling image of the M sampling images;

determining L sampling images according to the following distances within the M sampling cycles, wherein L is a positive integer less than or equal to M; and

determining the N sampling images from the L sampling images; and

storing the N sampling images, wherein positions of the target vehicle corresponding to the N sampling images are different;

in a case that following distances within an a-th sampling cycle to a b-th sampling cycle are all greater than a preset distance threshold, the L sampling images include sampling images within a 1st sampling cycle to the b-th sampling cycle; wherein a is a positive integer, b is a positive integer greater than a, and a difference between a and b is greater than or equal to a first preset quantity threshold; or

in a case that a number of following distances greater than the preset distance threshold of following distances within the 1st sampling cycle to a c-th sampling cycle is greater than a second preset quantity threshold, the L sampling images include sampling images within the 1st sampling cycle to the c-th sampling cycle; wherein a value of c is determined according to the speed of the target vehicle or the current position of the target vehicle, and c is a positive integer greater than 1; or

in a case that a forced synthesis switch is in a turned-on state, the L sampling images include the sampling images within the 1st sampling cycle to the c-th sampling cycle or sampling images within the 1st sampling cycle to a d-th sampling cycle; wherein a value of d is determined according to the speed of the target vehicle or the current position of the target vehicle, and d is a positive integer greater than c.

11 . The method according to claim 1 , wherein after synthesizing the sampling points of the N sampling images according to the second coordinates of the sampling points of the N sampling images to determine the target image, the method further comprises:

determining a driving assistance line according to the target image and driving information of the target vehicle; and

adding the driving assistance line to the target image.

12 . A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the computer implements the vehicle navigation image synthesis method according to claim 1 .

13 . A vehicle navigation image synthesis device applied to a vehicle navigation image synthesis system, comprising:

a communication interface;

a processor coupled to the communication interface and configured to execute a computer program or instructions to implement:

acquiring M sampling images, wherein the M sampling images include a preset sampling image for establishing a reference coordinate system, the preset sampling image is a sampling image in which a stop line or a zebra crossing is acquired for a first time, and M is a positive integer;

storing N sampling images, wherein N is a positive integer less than or equal to M, the N sampling images are incomplete images, a value of N is determined according to a speed of a target vehicle, a position of the target vehicle, and a following distance of the target vehicle;

determining a plurality of sampling points of each sampling image of the N sampling images;

determining first coordinates of each sampling point of the plurality of sampling points of each sampling image, wherein the first coordinates include a first Y-coordinate and a first X-coordinate; the first Y-coordinate is determined according to parameters of an image acquisition device and a position of each sampling point in a sampling image where each sampling point is located; the first X-coordinate is determined according to projections, on an intermediate coordinate system, of border lines of the sampling image where each sampling point is located; and the intermediate coordinate system is a coordinate system established according to a current position of the target vehicle and a current driving direction of the target vehicle;

mapping the first coordinates of each sampling point to the reference coordinate system to determine second coordinates of each sampling point; and

synthesizing sampling points of the N sampling images according to second coordinates of the sampling points of the N sampling images to determine a target image, wherein the target image is a complete image; and

determining, by the vehicle navigation image synthesis system, image information of an entire junction according to the target image.

14 . The device according to claim 13 , wherein the processor is configured to implement:

determining a target sampling point and a target sampling image where the target sampling point is located, wherein the target sampling point is a sampling point of a plurality of sampling points of the target sampling image, and the target sampling image is a sampling image of the N sampling images;

establishing an image coordinate system of the target sampling image, wherein an origin of the image coordinate system is a center point of the target sampling image, an X-axis of the image coordinate system is perpendicular to a left border line of the target sampling image or a right border line of the target sampling image, and a Y-axis of the image coordinate system is perpendicular to an upper border line of the target sampling image or a lower border line of the target sampling image;

determining third coordinates of the target sampling point in the image coordinate system, wherein the third coordinates of the target sampling point in the image coordinate system include a third Y-coordinate and a third X-coordinate; the third X-coordinate is an X-coordinate of the target sampling point of the image coordinate system; and the third Y-coordinate is a Y-coordinate of the target sampling point of the image coordinate system; and

determining a first Y-coordinate of the target sampling point according to the parameters of the image acquisition device and the third Y-coordinate, wherein the parameters of the image acquisition device include a vertical distance between the image acquisition device and a ground, a sampling angle of the image acquisition device, and a focal length of the image acquisition device; and the sampling angle of the image acquisition device is an included angle between an acquisition direction of the image acquisition device and a direction perpendicular to the ground.

15 . The device according to claim 14 , wherein the processor is configured to implement:

establishing an intermediate coordinate system corresponding to the target sampling image, wherein an origin of the intermediate coordinate system corresponding to the target sampling image is a current position of the target vehicle corresponding to the target sampling image, a positive direction of a Y-axis of the intermediate coordinate system corresponding to the target sampling image is a current driving direction of the target vehicle corresponding to the target sampling image, and an X-axis of the intermediate coordinate system corresponding to the target sampling image is parallel to the ground;

determining a slope of a first straight line, a slope of a second straight line, and a target quantity, wherein the first straight line is a straight line obtained by projecting the left border line of the target sampling image on the intermediate coordinate system corresponding to the target sampling image; the second straight line is a straight line obtained by projecting the right border line of the target sampling image on the intermediate coordinate system corresponding to the target sampling image; and the target quantity is a half of a number of sampling points of the target sampling image on the X-axis of the image coordinate system; and

determining a first X-coordinate of the target sampling point according to the slope of the first straight line, the slope of the second straight line, the target quantity, the first Y-coordinate of the target sampling point, and the third X-coordinate.

16 . The device according to claim 14 , wherein the processor is configured to implement:

mapping the first coordinates of each sampling point to an angle-restoration coordinate system to determine fourth coordinates of each sampling point, wherein the angle-restoration coordinate system is a coordinate system obtained by rotating the intermediate coordinate system by a preset angle; and

mapping the fourth coordinates of each sampling point to the reference coordinate system to determine the second coordinates of each sampling point.

17 . The device according to claim 16 , wherein the processor is configured to implement:

acquiring a deviation angle of the target vehicle, wherein the deviation angle of the target vehicle is an included angle between the current driving direction of the target vehicle and a straight-line driving direction of the target vehicle;

determining a preset angle of the target sampling image, wherein the preset angle of the target sampling image is determined according to the deviation angle of the target vehicle within a sampling cycle of the target sampling image;

rotating the target sampling image and the intermediate coordinate system corresponding to the target sampling image clockwise by the preset angle of the target sampling image to determine an angle-restored target sampling image and an angle-restoration coordinate system corresponding to the target sampling image; and

determining coordinates of the target sampling point of the angle-restored target sampling image in the angle-restoration coordinate system corresponding to the target sampling image as fourth coordinates of the target sampling point.

18 . The device according to claim 17 , wherein the processor is configured to implement:

establishing the reference coordinate system, wherein an origin of the reference coordinate system is a position of the target vehicle within a sampling cycle of the preset sampling image, a positive direction of a Y-axis of the reference coordinate system is a driving direction of the target vehicle within the sampling cycle of the preset sampling image, and an X-axis of the reference coordinate system is parallel to the ground;

determining a first distance difference and a second distance difference, wherein the first distance difference is a distance difference between an origin of the angle-restoration coordinate system and the origin of the reference coordinate system in a direction of an X-axis of the angle-restoration coordinate system, and the second distance difference is a distance difference between the origin of the angle-restoration coordinate system and the origin of the reference coordinate system in a direction of a Y-axis of the angle-restoration coordinate system; and

determining a second X-coordinate and a second Y-coordinate included in the second coordinates, wherein the second X-coordinate is a difference between a fourth X-coordinate of the fourth coordinates and the first distance difference, and the second Y-coordinate is a difference between a fourth Y-coordinate of the fourth coordinate and the second distance difference.

19 . The device according to claim 16 , wherein the processor is configured to implement:

dividing a target area of the target image into P sampling points;

determining fifth coordinates of each of the P sampling points in the reference coordinate system, wherein the target area is an area whose PPI is lower than a preset PPI threshold, and P is a positive integer;

determining a sampling point satisfying a preset condition from the sampling points of the N sampling images according to the fifth coordinates of the first sampling point, wherein the first sampling point is a sampling point of the P sampling points;

determining image information of the first sampling point according to image information of the sampling point satisfying the preset condition; and

adding the image information of the first sampling point to image information of the target image according to the fifth coordinates of the first sampling point.

20 . The device according to claim 13 , wherein the processor is configured to implement:

acquiring following distances within M sampling cycles, wherein a single sampling cycle corresponds to a single sampling image of the M sampling images;

determining L sampling images according to the following distances within the M sampling cycles, wherein L is a positive integer less than or equal to M; and

determining the N sampling images from the L sampling images; and

storing the N sampling images, wherein positions of the target vehicle corresponding to the N sampling images are different;

in a case that following distances within an a-th sampling cycle to a b-th sampling cycle are all greater than a preset distance threshold, the L sampling images include sampling images within a 1st sampling cycle to the b-th sampling cycle; wherein a is a positive integer, and b is a positive integer greater than a, and a difference between a and b is greater than or equal to a first preset quantity threshold; or

in a case that a number of following distances greater than the preset distance threshold of following distances within the 1st sampling cycle to a c-th sampling cycle is greater than a second preset quantity threshold, the L sampling images include sampling images within the 1st sampling cycle to the c-th sampling cycle; wherein a value of c is determined according to the speed of the target vehicle or the current position of the target vehicle; and c is a positive integer greater than 1; or

in a case that a forced synthesis switch is in a turned-on state, the L sampling images include the sampling images within the 1st sampling cycle to the c-th sampling cycle or sampling images within the 1st sampling cycle to a d-th sampling cycle; wherein a value of d is determined according to the speed of the target vehicle or the current position of the target vehicle, and d is a positive integer greater than c.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (11)