IP Library Granted Patent US 12,243,284
Granted Patent B2
US 12,243,284 · App. 17/587,689 · Granted Mar 4, 2025

Image classification method and apparatus

Inventors: Kai Han (Beijing, CN); Yunhe Wang (Beijing, CN); Han Shu (Beijing, CN); Chunjing Xu (Shenzhen, CN)
Assignee: Huawei Technologies Co., Ltd.
G06V10/454G06V10/764G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,284
App. No.
17/587,689
Granted
Mar 4, 2025
Kind
B2
Abstract

This application relates to an image recognition technology in the field of computer vision in the field of artificial intelligence, and provides an image classification method and apparatus. The method includes: obtaining an input feature map of a to-be-processed image; performing convolution processing on the input feature map based on M convolution kernels of a neural network, to obtain a candidate output feature map of M channels, where M is a positive integer; performing matrix transformation on the M channels of the candidate output feature map based on N matrices, to obtain an output feature map of N channels, where a quantity of channels of each of the N matrices is less than M, N is greater than M, and N is a positive integer; and classify the to-be-processed image based on the output feature map, to obtain a classification result of the to-be-processed image.

Claims (43)

1. An image classification method, comprising:

obtaining an input feature map of a to-be-processed image;

performing convolution processing on the input feature map based on M convolution kernels of a neural network, to obtain an intermediate feature map comprising M channels, wherein M is a positive integer;

performing convolution processing on the M channels of the intermediate feature map based on N matrices, to obtain an output feature map comprising N channels, wherein the N matrices represent N convolution kernels, the N matrices comprise M groups of convolution kernels, and the M groups of convolution kernels respectively correspond to the M channels of the intermediate feature map, and wherein a quantity of channels of each of the N matrices is less than M, N is greater than M, and N is a positive integer; and

classifying the to-be-processed image based on the output feature map, to obtain a classification result of the to-be-processed image.

2. The image classification method according to claim 1 , wherein

performing the convolution processing on the M channels of the intermediate feature map based on the N matrices, to obtain the output feature map comprising the N channels, comprises:

performing convolution on a corresponding channel of the M channels of the intermediate feature map based on each of the M groups of convolution kernels, to obtain the output feature map comprising the N channels.

3. The image classification method according to claim 2 , wherein performing the convolution on a corresponding channel of the M channels of the intermediate feature map based on each of the M groups of convolution kernels, to obtain the output feature map comprising the N channels, comprises:

performing, based on each of the M groups of convolution kernels, depthwise convolution on the corresponding channel of the M channels of the intermediate feature map, to obtain the output feature map comprising the N channels.

4. The image classification method according to claim 2 , wherein a convolution kernel in each of the M groups of convolution kernels is the same as a convolution kernel in another group of the M groups of convolution kernels.

5. The image classification method according to claim 1 , wherein classifying the to-be-processed image based on the output feature map, to obtain a classification result of the to-be-processed image, comprises:

combining the intermediate feature map and the output feature map to obtain a feature splicing feature map, wherein a quantity of channels of the feature splicing feature map is M+N; and

classifying the to-be-processed image based on the feature splicing feature map, to obtain the classification result of the to-be-processed image.

6. An image classification apparatus, comprising:

one or more processors; and

one or more memories coupled to the one or more processors and store programming instructions for execution by the one or more processors to:

obtain an input feature map of a to-be-processed image;

perform convolution processing on the input feature map based on M convolution kernels of a neural network, to obtain an intermediate feature map comprising M channels, wherein M is a positive integer;

perform convolution processing on the M channels of the intermediate feature map based on N matrices, to obtain an output feature map comprising N channels, wherein the N matrices represent N convolution kernels, the N matrices comprise M groups of convolution kernels, and the M groups of convolution kernels respectively correspond to the M channels of the intermediate feature map, and wherein a quantity of channels of each of the N matrices is less than M, N is greater than M, and N is a positive integer; and

classify the to-be-processed image based on the output feature map, to obtain a classification result of the to-be-processed image.

7. The image classification apparatus according to claim 6 , wherein

the one or more memories store programming instructions for execution by the one or more processors to perform convolution on a corresponding channel of the M channels of the intermediate feature map based on each of the M groups of convolution kernels, to obtain the output feature map comprising the N channels.

8. The image classification apparatus according to claim 7 , wherein the one or more memories store programming instructions for execution by the one or more processors to:

perform, based on each of the M groups of convolution kernels, depthwise convolution on the corresponding channel of the M channels of the intermediate feature map, to obtain the output feature map comprising the N channels.

9. The image classification apparatus according to claim 7 , wherein a convolution kernel in each of the M groups of convolution kernels is the same as a convolution kernel in another group of the M groups of convolution kernels.

10. The image classification apparatus according to claim 6 , wherein the one or more memories store programming instructions for execution by the one or more processors to:

combining the intermediate feature map and the output feature map to obtain a feature splicing feature map, wherein a quantity of channels of the feature splicing feature map is M+N; and

classify the to-be-processed image based on the feature splicing feature map, to obtain the classification result of the to-be-processed image.

11. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores one or more instructions that when executed by a device, cause the device to perform operations comprising:

obtaining an input feature map of a to-be-processed image;

performing convolution processing on the input feature map based on M convolution kernels of a neural network, to obtain an intermediate feature map comprising M channels, wherein M is a positive integer;

performing convolution processing on the M channels of the intermediate feature map based on N matrices, to obtain an output feature map comprising N channels, wherein the N matrices represent N convolution kernels, the N matrices comprise M groups of convolution kernels, and the M groups of convolution kernels respectively correspond to the M channels of the intermediate feature map, and wherein a quantity of channels of each of the N matrices is less than M, N is greater than M, and N is a positive integer; and

classifying the to-be-processed image based on the output feature map, to obtain a classification result of the to-be-processed image.

12. The non-transitory computer-readable storage medium according to claim 11 , wherein

performing the convolution processing on the M channels of the intermediate feature map based on the N matrices, to obtain the output feature map comprising the N channels, comprises:

performing convolution on a corresponding channel of the M channels of the intermediate feature map based on each of the M groups of convolution kernels, to obtain the output feature map comprising the N channels.

13. The non-transitory computer-readable storage medium according to claim 12 , wherein performing the convolution on a corresponding channel of the M channels of the intermediate feature map based on each of the M groups of convolution kernels, to obtain the output feature map comprising the N channels, comprises:

performing, based on each of the M groups of convolution kernels, depthwise convolution on the corresponding channel of the M channels of the intermediate feature map, to obtain the output feature map comprising the N channels.

14. The non-transitory computer-readable storage medium according to claim 12 , wherein a convolution kernel in each of the M groups of convolution kernels is the same as a convolution kernel in another group of the M groups of convolution kernels.

15. The non-transitory computer-readable storage medium according to claim 11 , wherein classifying the to-be-processed image based on the output feature map, to obtain a classification result of the to-be-processed image, comprises:

combining the intermediate feature map and the output feature map to obtain a feature splicing feature map, wherein a quantity of channels of the feature splicing feature map is M+N; and

classifying the to-be-processed image based on the feature splicing feature map, to obtain the classification result of the to-be-processed image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2023
From: HAN, KAI; WANG, YUNHE; SHU, HAN; XU, CHUNJING
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 062575/0064 →
Priority Claims (1)
CN 201910697287.0 · Jul 30, 2019 · national
Continuity (2)
Continuation PCTCN2020105830 · Jul 30, 2020
Related Publication 20220157041A1 · May 19, 2022
References Cited (20)
US 20180137406A1 · Howard et al. · 2018 [cited by applicant]
US 20180365580A1 · Musuvathi et al. · 2018 [cited by applicant]
US 20190188537A1 · Dutta · 2019 [cited by examiner]
US 20190220709A1 · Freeman · 2019 [cited by examiner]
CN 107451075A · 2017 [cited by applicant]
CN 109670506A · 2019 [cited by applicant]
CN 110020682A · 2019 [cited by applicant]
CN 112686317A · 2021 [cited by applicant]
CN 114492754A · 2022 [cited by applicant]
CN 115879508A · 2023 [cited by applicant]
Lecun et al., Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, vol. 86, No. 11, pp. 2278-2324, Nov. 1998. [cited by examiner]
AlexeyAB, “GhostNet: More Features from Cheap Operatins—75.7% top-1 (better than than MobileNetV3),” retrieved on Apr. 14, 2022, retrieved from URL: <https://github.com/AlexeyAB/darknet/issues/4418>, Nov. 2019, 9 pages. [cited by applicant]
Huawei, “Ghostnet: Efficient Networks by Generating More Features from Cheap Operations,” retrieved on Apr. 14, 2022, retrieved from URL: <https://pytorch.org/hub/pytorch_vision_ghostnet>, 2019, 3 pages. [cited by applicant]
Intel, “Intel® Distribution of OpenVINO™ Toolkit,” Intel Developer Tools, retrieved on Apr. 14, 2022, retrieved from URL: <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/overview.html>, 2019, 9… [cited by applicant]
PCT International Search Report and Written Opinion issued in International Application No. PCT/CN2020/105830 on Nov. 4, 2020, 15 pages (with English translation). [cited by applicant]
Extended European Search Report issued in European Application No. 20848554.0 on Sep. 2, 2022, 6 pages. [cited by applicant]
He et al., “Identity Mappings in Deep Residual Networks,” European Conference on Computer Vision, Sep. 17, 2016, 16 pages. [cited by applicant]
Lecun et al., “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, vol. 86, No. 11, Nov. 1998, 46 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/CN2023/120541, mailed on Dec. 26, 2023, 10 pages (with partial English translation). [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/CN2023/123636, mailed on Dec. 26, 2023, 13 pages (with partial English translation). [cited by applicant]
Cited By (1)
US 12,634,595