IP Library Granted Patent US 11,605,211
Granted Patent B2
US 11,605,211 · App. 17/036,903 · Granted Mar 14, 2023

Object detection model training method and apparatus, and device

Inventors: Changzheng Zhang (Shenzhen, CN); Xin Jin (Beijing, CN); Dandan Tu (Shenzhen, CN)
Assignee: HUAWEI CLOUD COMPUTING TECHNOLOGIES CO., LTD.
G06V10/255G06K9/6215G06K9/6218G06K9/6256G06N3/0454G06T7/11G06V10/454G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,605,211
App. No.
17/036,903
Granted
Mar 14, 2023
Kind
B2
Abstract

An object detection model training method performed by a computing device, includes obtaining a system parameter including at least one of a receptive field of a backbone network, a size of a training image, a size of a to-be-detected object in the training image, a training computing capability, or a complexity of the to-be-detected object, determining a configuration parameter based on the system parameter, establishing a variable convolution network based on the configuration parameter and a feature map of the backbone network, recognizing the to-be-detected object based on a feature of the variable convolution network, and training the backbone network and the variable convolution network, where a convolution core used by any variable convolution layer may be offset in any direction in a process of performing convolution.

Claims (78)

1. A method implemented by a computing device and comprising:

obtaining a system parameter comprising at least one of a receptive field of a backbone network of an object detection model, a first size of a training image, a second size of a to-be-detected object in the training image, a training computing capability, or a complexity of the to-be-detected object;

determining a configuration parameter of i variable convolution networks based on the system parameter, wherein the configuration parameter comprises at least one of a quantity of the i variable convolution networks, a quantity L i of variable convolution layers comprised in an i th variable convolution network, a sliding span of a first convolution core of the i th variable convolution network, a maximum offset distance, or a third size of the first convolution core, and wherein both i and L i are integers greater than zero;

obtaining the training image;

establishing the backbone network based on the training image;

establishing the i variable convolution networks based on a feature map of the backbone network and the configuration parameter, wherein an offset distance does not exceed the maximum offset distance when a second convolution core of each of the variable convolution networks performs a convolution;

inputting features of the i variable convolution networks into a classifier of the object detection model to obtain the to-be-detected object in the training image;

comparing the to-be-detected object from the classifier and a prior result of the training image to obtain a comparison result; and

activating, based on the comparison result, at least one of a model parameter of a third convolution core of the backbone network, model parameters of second convolution cores of the i variable convolution networks, or the maximum offset distance.

2. The method of claim 1 , wherein establishing the i variable convolution networks comprises:

offsetting, in any direction, a first convolution sheet of a fourth convolution core of a first variable convolution layer in the i variable convolution networks by a quantity of features to obtain a second convolution sheet, wherein the quantity of features does not exceed the maximum offset distance; and

performing the convolution on a model parameter of the first convolution sheet and a feature of the first variable convolution layer corresponding to the second convolution sheet to obtain a feature of a second variable convolution layer of the i variable convolution networks.

3. The method of claim 1 , wherein establishing the i variable convolution networks comprises:

offsetting, in any direction, a first convolution point in a fifth convolution core of a third variable convolution layer in the i variable convolution networks by a quantity of features to obtain a second convolution point, wherein the quantity of features does not exceed the maximum offset distance; and

performing the convolution on a model parameter of the first convolution point and a feature of the third variable convolution layer corresponding to the second convolution point to obtain a feature in a fourth variable convolution layer of the i variable convolution networks.

4. The method of claim 1 , wherein establishing the i variable convolution networks comprises dividing the feature map into i parts, and wherein the i parts respectively serve as an initial variable convolution layer of the i variable convolution networks.

5. The method of claim 1 , wherein the training computing capability indicates a computing capability of the computing device.

6. The method of claim 1 , wherein the complexity of the to-be-detected object indicates a detection complexity of the to-be-detected object.

7. An apparatus comprising:

an initialization system configured to:

obtain a system parameter comprising at least one of a receptive field of a backbone network, a first size of a training image, a second size of a to-be-detected object in the training image, a training computing capability, or a complexity of the to-be-detected object; and

determine a configuration parameter of i variable convolution networks based on the system parameter, wherein the configuration parameter comprises at least one of a quantity of the i variable convolution networks, a quantity L i of variable convolution layers comprised in an i th variable convolution network, a sliding span of a first convolution core of the i th variable convolution network, a maximum offset distance, or a third size of the first convolution core, and wherein both i and L i are integers greater than zero;

an object detection model system coupled to the initialization system and configured to:

obtain the training image;

establish the backbone network based on the training image;

establish the i variable convolution networks based on a feature map of the backbone network and the configuration parameter, wherein an offset distance does not exceed the maximum offset distance when a second convolution core of each of the variable convolution networks performs a convolution; and

input features of the i variable convolution networks into a classifier of the object detection model system to obtain the to-be-detected object in the training image; and

an activation system coupled to the initialization system and the object detection model system and configured to:

compare the to-be-detected object of the classifier and a prior result of the training image to obtain a comparison result; and

activate, based on the comparison result, at least one of a model parameter of a third convolution core of the backbone network, model parameters of second convolution cores of the i variable convolution networks, or the maximum offset distance.

8. The apparatus of claim 7 , wherein the object detection model system is further configured to:

offset, in any direction, a first convolution sheet of a fourth convolution core of a first variable convolution layer in the i variable convolution networks by a quantity of features to obtain a second convolution sheet, wherein the quantity of features does not exceed the maximum offset distance; and

perform the convolution on a model parameter of the first convolution sheet and a feature of the first variable convolution layer corresponding to the second convolution sheet to obtain a feature of a second variable convolution layer of the i variable convolution networks.

9. The apparatus of claim 7 , wherein the object detection model system is further configured to:

offset, in any direction, a first convolution point in a fifth convolution core of a third variable convolution layer in the i variable convolution networks by a quantity of features to obtain a second convolution point, wherein the quantity of features does not exceed the maximum offset distance; and

perform the convolution on a model parameter of the first convolution point and a feature of the third variable convolution layer corresponding to the second convolution point to obtain a feature in a fourth variable convolution layer of the i variable convolution networks.

10. The apparatus of claim 7 , wherein the object detection model system is further configured to divide the feature map into i parts, and wherein the i parts respectively serve as an initial variable convolution layer of the i variable convolution networks.

11. The apparatus of claim 7 , wherein the training computing capability indicates a computing capability of the apparatus.

12. The apparatus of claim 7 , wherein the complexity of the to-be-detected object indicates a detection complexity of the to-be-detected object.

13. A computing device system comprising:

a plurality of computing devices, wherein each of the computing devices comprises:

a memory configured to store instructions; and

a processor coupled to the memory, wherein the instructions cause the processor to be configured to:

obtain a system parameter comprising at least one of a receptive field of a backbone network of an object detection model, a first size of a training image, a second size of a to-be-detected object in the training image, a training computing capability, or a complexity of the to-be-detected object;

determine a configuration parameter of i variable convolution networks based on the system parameter, wherein the configuration parameter comprises at least one of a quantity of the i variable convolution networks, a quantity L i of variable convolution layers comprised in an i th variable convolution network, a sliding span of a first convolution core of the i th variable convolution network, a maximum offset distance, or a third size of the first convolution core, and wherein both i and L i are integers greater than zero;

obtain the training image;

establish the backbone network based on the training image;

establish the i variable convolution networks based on a feature map of the backbone network and the configuration parameter, wherein an offset distance does not exceed the maximum offset distance when a second convolution core of each of the i variable convolution networks performs a convolution;

input features of the i variable convolution networks into a classifier of the object detection model to obtain the to-be-detected object in the training image; and

compare the to-be-detected object of the classifier and a prior result of the training image to obtain a comparison result; and

activate, based on the comparison result, at least one of a model parameter of a third convolution core of the backbone network, model parameters of second convolution cores of the i variable convolution networks, or the maximum offset distance.

14. The computing device system of claim 13 , wherein the instructions further cause the processor to be configured to:

offset, in any direction, a first convolution sheet of a fourth convolution core of a first variable convolution layer in the i variable convolution networks by a quantity of features to obtain a second convolution sheet, wherein the quantity of features does not exceed the maximum offset distance; and

perform the convolution on a model parameter of the first convolution sheet and a feature of the first variable convolution layer corresponding to the second convolution sheet to obtain a feature of a second variable convolution layer of the i variable convolution networks.

15. The computing device system of claim 13 , wherein the instructions further cause the processor to be configured to:

offset, in any direction, a first convolution point in a fifth convolution core of a third variable convolution layer in the i variable convolution networks by a quantity of features to obtain a second convolution point, wherein the quantity of features does not exceed the maximum offset distance; and

perform the convolution on a model parameter of the first convolution point and a feature of the third variable convolution layer corresponding to the second convolution point to obtain a feature in a fourth variable convolution layer of the i variable convolution networks.

16. The computing device system of claim 13 , wherein the instructions further cause the processor to be configured to divide the feature map into i parts, and wherein the i parts respectively serve as an initial variable convolution layer of the i variable convolution networks.

17. The computing device system of claim 13 , wherein the training computing capability indicates a computing capability of a corresponding computing device.

18. The computing device system of claim 13 , wherein the complexity of the to-be-detected object indicates a detection complexity of the to-be-detected object.

19. A non-transitory computer readable storage medium configured to store instructions that, when executed by a computing device in a computing device system, cause the computing device to:

obtain a system parameter comprising at least one of a receptive field of a backbone network of an object detection model, a first size of a training image, a second size of a to-be-detected object in the training image, a training computing capability, or a complexity of the to-be-detected object;

determine a configuration parameter of i variable convolution networks based on the system parameter, wherein the configuration parameter comprises at least one of a quantity of the i variable convolution networks, a quantity L i of variable convolution layers comprised in an i th variable convolution network, a sliding span of a first convolution core of the i th variable convolution network, a maximum offset distance, or a third size of the first convolution core, and wherein both i and L i are integers greater than zero;

obtain the training image;

establish the backbone network based on the training image;

establish the i variable convolution networks based on a feature map of the backbone network and the configuration parameter, wherein an offset distance does not exceed the maximum offset distance when a second convolution core of each of the I variable convolution networks performs a convolution;

input features of the i variable convolution networks into a classifier of the object detection model to obtain the to-be-detected object in the training image;

compare the to-be-detected object of the classifier and a prior result of the training image to obtain a comparison result; and

activate, based on the comparison result, at least one of a model parameter of a third convolution core of the backbone network, model parameters of second convolution cores of the i variable convolution networks, or the maximum offset distance.

20. A non-transitory, computer-readable medium comprising a computer program product that, when executed by a processor, causes a computing device in a computing device system to:

obtain a system parameter comprising at least one of a receptive field of a backbone network of an object detection model, a first size of a training image, a second size of a to-be-detected object in the training image, a training computing capability, or a complexity of the to-be-detected object;

determine a configuration parameter of i variable convolution networks based on the system parameter, wherein the configuration parameter comprises at least one of a quantity of the i variable convolution networks, a quantity L i of variable convolution layers comprised in an i th variable convolution network, a sliding span of a first convolution core of the i th variable convolution network, a maximum offset distance, or a third size of the first convolution core, and wherein both i and L i are integers greater than zero;

obtain the training image;

establish the backbone network based on the training image;

establish the i variable convolution networks based on a feature map of the backbone network and the configuration parameter, wherein an offset distance does not exceed the maximum offset distance when a second convolution core of each of the i variable convolution networks performs a convolution;

input features of the i variable convolution networks into a classifier of the object detection model to obtain the to-be-detected object in the training image;

compare the to-be-detected object of the classifier and a prior result of the training image to obtain a comparison result; and

activate, based on the comparison result, at least one of a model parameter of a third convolution core of the backbone network, model parameters of second convolution cores of the i variable convolution networks, or the maximum offset distance.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: HUAWEI TECHNOLOGIES CO., LTD.
To: HUAWEI CLOUD COMPUTING TECHNOLOGIES CO., LTD.
Reel/Frame 059267/0088 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2020
From: ZHANG, CHANGZHENG; JIN, XIN; TU, DANDAN
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 054274/0567 →