Granted Patent B2

US 12,548,346 · App. 17/877,054 · Granted Feb 10, 2026

Inventors: Houlong Zhao (Beijing, CN); Kuiyuan Yang (Beijing, CN)

Assignee: Xiaomi EV Technology Co., Ltd.

G06V20/58G06T7/73G06V10/806G06V20/588G06T2207/20081G06T2207/30256G06T2207/30261

Abstract

Claims (92)

1 . A method for target object detection, the method comprising:

obtaining images collected by cameras installed on a target vehicle;

determining a high-dimensional parameter feature in a high-dimensional space corresponding to parameter information of each camera among the cameras;

fusing features of the images via a trained target object detection model according to the high-dimensional parameter features; and

determining position information of a target object based on the fused features, and

wherein an order of the cameras corresponding to the images is the same as an order of the cameras corresponding to the high-dimensional parameter features;

wherein the trained target object detection model is trained by:

obtaining a training sample set, each training sample in the training sample set comprising an input image sequence, training parameter information of a camera corresponding to each image in the input image sequence, and target position information corresponding to the input image sequence,

obtaining high-dimensional parameter features corresponding to the training parameter information by inputting the training parameter information into a parameter mapping model,

inputting the input image sequence and the high-dimensional parameter features corresponding to the training parameter information into a preset model, and obtaining predicted position information output by the preset model,

determining, according to the target position information and the predicted position information, a target loss,

and

training the parameter mapping model and the preset model according to the target loss to obtain a trained parameter mapping model and a trained preset model, and determining the trained preset model as the target object detection model;

wherein determining the high-dimensional parameter feature in the high-dimensional space corresponding to the parameter information of each camera among the cameras comprises:

obtaining, according to the parameter information and a high-dimensional parameter mapping relationship table, the high-dimensional parameter features corresponding to the parameter information;

wherein the high-dimensional parameter mapping relationship table being is generated by:

obtaining the high-dimensional parameter features output by the trained parameter mapping model by inputting preset parameter information of the cameras into the trained parameter mapping model; and

obtaining the high-dimensional parameter mapping relationship table by generating a corresponding relationship between the parameter information input into the trained parameter mapping model and the high-dimensional parameter features output by the trained parameter mapping model.

2 . The method according to claim 1 , wherein the trained target object detection model comprises: a feature extraction sub-model, a feature conversion sub-model, a feature encoding sub-model, and a feature decoding sub-model, wherein,

the feature extraction sub-model is configured to obtain two-dimensional features corresponding to the images by performing feature extraction on the images;

the feature conversion sub-model is configured to obtain three-dimensional features by performing feature conversion according to the two-dimensional features and the high-dimensional parameter features;

the feature encoding sub-model is configured to obtain the fused features by encoding the three-dimensional features; and

the feature decoding sub-model is configured to obtain the position information by decoding the fused features.

3 . The method according to claim 2 , wherein the feature encoding sub-model is an encoder of a Transformer model, and the feature encoding sub-model obtains the fused features by:

obtaining more than one sub-feature of equal size by performing feature division on the three-dimensional features; and

obtaining the fused features by performing, according to a feature vector corresponding to each sub-feature and a position vector corresponding to each sub-feature, the feature fusion on the feature vectors.

4 . The method according to claim 2 , wherein the feature conversion sub-model is a multilayer perceptron, and the feature conversion sub-model obtains the three-dimensional features by performing the feature conversion according to the two-dimensional features and the high-dimensional parameter features by:

assigning values to parameters in the feature conversion sub-model according to the high-dimensional parameter features; and

obtaining the three-dimensional features by performing the feature conversion on the two-dimensional features according to the assigned feature conversion sub-model.

5 . The method according to claim 1 , wherein the target object comprises at least one of a three-dimensional obstacle, a three-dimensional lane line, or a three-dimensional driving area.

6 . An apparatus for target object detection, the apparatus comprising:

a memory, configured to store processor-executable instructions; and

one or more processors communicatively coupled to the memory,

wherein the one or more processors are collectively configured to:

obtain images collected by cameras installed on a target vehicle,

determine a high-dimensional parameter feature in a high-dimensional space corresponding to parameter information of each camera among the cameras,

fuse features of the images via a trained target object detection model according to the high-dimensional parameter features, and

determine position information of a target object based on the fused features, and

wherein an order of the cameras corresponding to the images being the same as an order of the cameras corresponding to the high-dimensional parameter features;

wherein the trained target object detection model is trained by:

obtaining a training sample set, each training sample in the training sample set comprising an input image sequence, training parameter information of a camera corresponding to each image in the input image sequence, and target position information corresponding to the input image sequence,

obtaining high-dimensional parameter features corresponding to the training parameter information by inputting the training parameter information into a parameter mapping model,

inputting the input image sequence and the high-dimensional parameter features corresponding to the training parameter information into a preset model, and obtaining predicted position information output by the preset model,

determining, according to the target position information and the predicted position information, a target loss,

and

training the parameter mapping model and the preset model according to the target loss to obtain a trained parameter mapping model and a trained preset model, and determining the trained preset model as the target object detection model;

wherein determining the high-dimensional parameter feature in the high-dimensional space corresponding to the parameter information of each camera among the cameras comprises:

obtaining, according to the parameter information and a high-dimensional parameter mapping relationship table, the high-dimensional parameter features corresponding to the parameter information,

wherein the high-dimensional parameter mapping relationship table is generated by:

obtaining the high-dimensional parameter features output by the trained parameter mapping model by inputting preset parameter information of the cameras into the trained parameter mapping model; and

obtaining the high-dimensional parameter mapping relationship table by generating a corresponding relationship between the parameter information input into the trained parameter mapping model and the high-dimensional parameter features output by the trained parameter mapping model.

7 . The apparatus according to claim 6 , wherein the trained target object detection model comprises: a feature extraction sub-model, a feature conversion sub-model, a feature encoding sub-model, and a feature decoding sub-model, wherein,

the feature extraction sub-model is configured to obtain two-dimensional features corresponding to the images by performing feature extraction on the images;

the feature conversion sub-model is configured to obtain three-dimensional features by performing feature conversion according to the two-dimensional features and the high-dimensional parameter features;

the feature encoding sub-model is configured to obtain the fused features by encoding the three-dimensional features; and

the feature decoding sub-model is configured to obtain the position information by decoding the fused features.

8 . The apparatus according to claim 7 , wherein the feature encoding sub-model is an encoder of a Transformer model, and the feature encoding sub-model obtains the fused features by:

obtaining more than one sub-feature of equal size by performing feature division on the three-dimensional features; and

obtaining the fused features by performing, according to a feature vector corresponding to each sub-feature and a position vector corresponding to each sub-feature, the feature fusion on the feature vectors.

9 . The apparatus according to claim 7 , wherein the feature conversion sub-model is a multilayer perceptron, and the feature conversion sub-model obtains the three-dimensional features by performing the feature conversion according to the two-dimensional features and the high-dimensional parameter features by:

assigning values to parameters in the feature conversion sub-model according to the high-dimensional parameter features; and

obtaining the three-dimensional features by performing the feature conversion on the two-dimensional features according to the assigned feature conversion sub-model.

10 . The apparatus according to claim 6 , wherein the target object comprises at least one of a three-dimensional obstacle, a three-dimensional lane line, or a three-dimensional driving area.

11 . A non-transitory computer-readable storage medium, storing computer program instructions thereon, wherein the computer program instructions, when executed by a processor, cause the processor to execute a method comprising:

obtaining images collected by cameras installed on a target vehicle;

determining a high-dimensional parameter feature in a high-dimensional space corresponding to parameter information of each camera among the cameras;

fusing features of the images via a trained target object detection model according to the high-dimensional parameter features; and

determining position information of a target object based on the fused features, and

wherein an order of the cameras corresponding to the images is the same as an order of the cameras corresponding to the high-dimensional parameter features;

wherein the trained target object detection model is trained by:

obtaining a training sample set, each training sample in the training sample set comprising an input image sequence, training parameter information of a camera corresponding to each image in the input image sequence, and target position information corresponding to the input image sequence,

obtaining high-dimensional parameter features corresponding to the training parameter information by inputting the training parameter information into a parameter mapping model,

inputting the input image sequence and the high-dimensional parameter features corresponding to the training parameter information into a preset model, and obtaining predicted position information output by the preset model,

determining, according to the target position information and the predicted position information, a target loss,

and

training the parameter mapping model and the preset model according to the target loss to obtain a trained parameter mapping model and a trained preset model, and determining the trained preset model as the target object detection model;

wherein determining the high-dimensional parameter feature in the high-dimensional space corresponding to the parameter information of each camera among the cameras comprises:

obtaining, according to the parameter information and a high-dimensional parameter mapping relationship table, the high-dimensional parameter features corresponding to the parameter information;

wherein the high-dimensional parameter mapping relationship table is generated by:

obtaining the high-dimensional parameter features output by the trained parameter mapping model by inputting preset parameter information of the cameras into the trained parameter mapping model; and

obtaining the high-dimensional parameter mapping relationship table by generating a corresponding relationship between the parameter information input into the trained parameter mapping model and the high-dimensional parameter features output by the trained parameter mapping model.

12 . The non-transitory computer-readable storage medium according to claim 11 , wherein the trained target object detection model comprises: a feature extraction sub-model, a feature conversion sub-model, a feature encoding sub-model, and a feature decoding sub-model, wherein,

the feature extraction sub-model is configured to obtain two-dimensional features corresponding to the images by performing feature extraction on the images;

the feature conversion sub-model is configured to obtain three-dimensional features by performing feature conversion according to the two-dimensional features and the high-dimensional parameter features;

the feature encoding sub-model is configured to obtain the fused features by encoding the three-dimensional features; and

the feature decoding sub-model is configured to obtain the position information by decoding the fused features.

13 . The non-transitory computer-readable storage medium according to claim 12 , wherein the feature encoding sub-model is an encoder of a Transformer model, and the feature encoding sub-model obtains the fused features by:

obtaining more than one sub-feature of equal size by performing feature division on the three-dimensional features; and

obtaining the fused features by performing, according to a feature vector corresponding to each sub-feature and a position vector corresponding to each sub-feature, the feature fusion on the feature vectors.

14 . The non-transitory computer-readable storage medium according to claim 12 , wherein the feature conversion sub-model is a multilayer perceptron, and the feature conversion sub-model obtains the three-dimensional features by performing the feature conversion according to the two-dimensional features and the high-dimensional parameter features by:

assigning values to parameters in the feature conversion sub-model according to the high-dimensional parameter features; and

obtaining the three-dimensional features by performing the feature conversion on the two-dimensional features according to the assigned feature conversion sub-model.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (14)