Granted Patent B2

US 12,254,763 · App. 17/729,834 · Granted Mar 18, 2025

Inventors: Zeshu Shen (Shenzhen, CN); Nengwu Xiang (Shenzhen, CN)

Assignee: Shenzhen Yinwang Intelligent Technologies Co., Ltd.

G08G1/0112G01C21/3667G01C21/387G06V10/25G06V10/74G06V10/80G06V10/95G06V20/56G08G1/0967G06V2201/07

Abstract

Claims (72)

1. An information fusion method, wherein the method is applied to an intelligent vehicle and comprises:

obtaining a plurality of sensing information sets, wherein each sensing information set of the plurality of sensing information sets comprises information about at least one target and different sensing information sets come from sensing systems of different device types, wherein the different device types generating the plurality of different sensing information sets include a vehicle communicating with the intelligent vehicle, a roadside device communicating with the intelligent vehicle, and the intelligent vehicle; and

based on at least two sensing information sets in the plurality of sensing information sets comprising information about a first target, fusing the information about the first target in the at least two sensing information sets to obtain fused information about the first target,

wherein the fusing the information about the first target in the at least two sensing information sets to obtain fused information about the first target comprises:

obtaining confidence of the information about the first target in each sensing information set of the plurality of sensing information sets; and

fusing the information about the first target in the at least two sensing information sets based on the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets to obtain the fused information about the first target;

wherein the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets is confidence of a corresponding sensing information set, and the method further comprises:

calculating a matching degree between information about a target in each sensing information set of the plurality of sensing information sets at a previous moment and fused information about a target corresponding to a sensing information set at the previous moment, wherein calculating a matching degree includes one of obtaining an intersection set between information about the target in a sensing information set of each device type and the information about the target in the fused sensing information set obtained at the previous moment, or calculating an average value of comprehensive distances between information about the same target in a sensing information set of each device type and the fused sensing information set obtained at the previous moment; and

determining confidence of each sensing information set of the plurality of sensing information sets at a current moment based on the matching degree.

2. The method according to claim 1 , wherein the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets is confidence of a corresponding sensing information set, and confidence of each sensing information set of the plurality of sensing information sets is determined by an attribute of a device corresponding to each sensing information set of the plurality of sensing information sets.

3. The method according to claim 1 , wherein before the fusing of the information about the first target in the at least two sensing information sets, the method further comprises:

determining a region of interest (ROI) of the intelligent vehicle based on driving information about the intelligent vehicle; and

filtering out, based on the ROI, information about a target having a position that is not in the ROI in each sensing information set of the plurality of sensing information sets.

4. The method according to claim 1 , wherein the method further comprises:

based on only one sensing information set in the plurality of sensing information sets comprising the information about the first target, searching for a tracking list;

based on no historical information about the first target being recorded in the tracking list, constructing a prediction model based on the information about the first target to obtain prediction information about the first target at a next moment;

setting confidence of the prediction model; and

recording, into the tracking list, the information about the first target, the confidence, and the prediction information about the first target at the next moment.

5. The method according to claim 1 , wherein the method further comprises:

based on only one sensing information set in the plurality of sensing information sets comprising the information about the first target, searching for a tracking list;

based on historical information about the first target being recorded in the tracking list:

comparing the information about the first target with prediction information that is of the first target at a current moment and that is recorded in the tracking list;

obtaining updated confidence based on a comparison result and confidence of a prediction model that is recorded in the tracking list;

adjusting a parameter of the prediction model based on the comparison result, to obtain an adjusted prediction model;

obtaining prediction information about the first target at a next moment based on the information about the first target and the adjusted prediction model; and

recording, into the tracking list, the information about the first target, the updated confidence, and the prediction information about the first target at the next moment.

6. The method according to claim 1 , wherein the method further comprises:

obtaining a map and status information about the intelligent vehicle;

constructing an environment model based on the fused information about the first target, the map, and the status information about the intelligent vehicle; and

displaying the environment model.

7. The method according to claim 1 , wherein each sensing information set of the plurality of sensing information sets is an information set in a standardized format.

8. An apparatus, comprising a processor and a memory, wherein the memory stores program code, and the processor executes the program code so that the apparatus performs the following:

obtaining a plurality of sensing information sets, wherein each sensing information set of the plurality of sensing information sets comprises information about at least one target and different sensing information sets come from sensing systems of different device types, wherein the different device types generating the plurality of different sensing information sets include a vehicle communicating with the apparatus, a roadside device communicating with the apparatus, and the apparatus; and

based on at least two sensing information sets in the plurality of sensing information sets comprising information about a first target, fusing the information about the first target in the at least two sensing information sets to obtain fused information about the first target,

wherein the fusing the information about the first target in the at least two sensing information sets to obtain fused information about the first target comprises:

obtaining confidence of the information about the first target in each sensing information set of the plurality of sensing information sets; and

fusing the information about the first target in the at least two sensing information sets based on the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets to obtain the fused information about the first target;

wherein the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets is confidence of a corresponding sensing information set, and the method further comprises:

calculating a matching degree between information about a target in each sensing information set of the plurality of sensing information sets at a previous moment and fused information about a target corresponding to a sensing information set at the previous moment, wherein calculating a matching degree includes one of obtaining an intersection set between information about the target in a sensing information set of each device type and the information about the target in the fused sensing information set obtained at the previous moment, or calculating an average value of comprehensive distances between information about the same target in a sensing information set of each device type and the fused sensing information set obtained at the previous moment; and

determining confidence of each sensing information set of the plurality of sensing information sets at a current moment based on the matching degree.

9. The apparatus according to claim 8 , wherein the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets is confidence of a corresponding sensing information set, and confidence of each sensing information set of the plurality of sensing information sets is determined by an attribute of a device corresponding to each sensing information set of the plurality of sensing information sets.

10. The apparatus according to claim 8 , wherein before the fusing of the information about the first target in the at least two sensing information sets, the processor is further configured to:

determine a region of interest (ROI) of the apparatus based on driving information about the apparatus; and

filter out, based on the ROI, information about a target having a position that is not in the ROI in each sensing information set.

11. The apparatus according to claim 8 , wherein the processor is further configured to:

based on only one sensing information set in the plurality of sensing information sets comprising the information about the first target, search for a tracking list;

based on no historical information about the first target being recorded in the tracking list, construct a prediction model based on the information about the first target, to obtain prediction information about the first target at a next moment;

set confidence of the prediction model; and

record, into the tracking list, the information about the first target, the confidence, and the prediction information about the first target at the next moment.

12. The apparatus according to claim 8 , wherein:

based on only one sensing information set in the plurality of sensing information sets comprising the information about the first target, searching for a tracking list;

based on historical information about the first target being recorded in the tracking list:

comparing the information about the first target with prediction information that is of the first target at a current moment and that is recorded in the tracking list;

obtaining updated confidence based on a comparison result and confidence of a prediction model that is recorded in the tracking list;

adjusting a parameter of the prediction model based on the comparison result, to obtain an adjusted prediction model;

obtaining prediction information about the first target at a next moment based on the information about the first target and the adjusted prediction model; and

recording, into the tracking list, the information about the first target, the updated confidence, and the prediction information about the first target at the next moment.

13. The apparatus according to claim 8 , wherein the processor is further configured to:

obtain a map and status information about the apparatus;

construct an environment model based on the fused information about the first target, the map, and the status information about the apparatus; and

display the environment model.

14. The apparatus according to claim 8 , wherein each sensing information set of the plurality of sensing information sets is an information set in a standardized format.

15. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer program code, and when the computer program code is executed by a computing device, the computing device performs the method according to claim 1 .

16. A vehicle, wherein the vehicle comprises a memory and a processor, the memory of the vehicle is configured to store a group of computer instructions and a data set, the processor executes the computer instructions stored in the memory, and the processor reads the data set stored in the memory, so that the vehicle is caused to perform the method comprising:

obtaining a plurality of sensing information sets, wherein each sensing information set of the plurality of sensing information sets comprises information about at least one target and different sensing information sets come from sensing systems of different device types, wherein the different device types generating the plurality of different sensing information sets include a second vehicle communicating with the vehicle, a roadside device communicating with the vehicle, and the vehicle; and

based on at least two sensing information sets in the plurality of sensing information sets comprising information about a first target, fusing the information about the first target in the at least two sensing information sets to obtain fused information about the first target,

wherein the fusing the information about the first target in the at least two sensing information sets to obtain fused information about the first target comprises:

obtaining confidence of the information about the first target in each sensing information set of the plurality of sensing information sets; and

fusing the information about the first target in the at least two sensing information sets based on the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets to obtain the fused information about the first target;

wherein the confidence of the information about the first target in each sensing information set of the plurality of sensing information sets is confidence of a corresponding sensing information set, and the method further comprises:

calculating a matching degree between information about a target in each sensing information set of the plurality of sensing information sets at a previous moment and fused information about a target corresponding to a sensing information set at the previous moment, wherein calculating a matching degree includes one of obtaining an intersection set between information about the target in a sensing information set of each device type and the information about the target in the fused sensing information set obtained at the previous moment, or calculating an average value of comprehensive distances between information about the same target in a sensing information set of each device type and the fused sensing information set obtained at the previous moment; and

determining confidence of each sensing information set of the plurality of sensing information sets at a current moment based on the matching degree.

Assignments (3)

Priority Claims (2)

Continuity (2)

References Cited (26)