Granted Patent B2

US 12,548,293 · App. 18/378,405 · Granted Feb 10, 2026

Inventors: Jinxi Xiang (Guangdong, CN); Sen Yang (Guangdong, CN); Jun Zhang (Guangdong, CN); Dongxian Jiang (Guangdong, CN); Yingyong Hou (Guangdong, CN); Xiao Han (Guangdong, CN)

Assignee: Tencent Technology (Shenzhen) Compay Limited

G06V10/762G06V10/267G06V10/40G06V10/764G06V10/7715G06V10/776G06V10/806

Abstract

Claims (122)

1 . An image detection method, performed by a computer device, the method comprising:

obtaining an image;

performing feature extraction processing on the image to obtain a feature representation subset of the image, the image comprising at least two sub-images, the feature representation subset comprising at least two sub-image features, and the at least two sub-image features corresponding to the at least two sub-images respectively;

generating attention weights corresponding to the at least two sub-image features;

performing weighting aggregation processing on the at least two sub-image features according to the attention weights to obtain a first feature vector;

performing clustering sampling processing on the at least two sub-image features to obtain at least two classification clusters comprising sampled sub-image features;

determining a block sparse self-attention for each of the sampled sub-image features according to the at least two classification clusters and a block sparse matrix;

determining a second feature vector according to at least two block sparse self-attentions respectively corresponding to the at least two classification clusters, a block sparse self-attention for a sampled sub-image feature being determined based on sampled sub-image features in a classification cluster to which the sampled sub-image feature belongs; and

determining a classification result of the image according to the first feature vector and the second feature vector.

2 . The method according to claim 1 , wherein the performing feature extraction processing on the image to obtain the feature representation subset of the image comprises:

recognizing a background region and a foreground region in the image;

performing image segmentation on the image according to the background region and the foreground region to obtain a foreground image;

scaling the foreground image according to a scaling magnification to obtain a scaled foreground image;

cropping the scaled foreground image according to a preset sub-image length and a preset sub-image width to obtain the at least two sub-images, the preset sub-image length being less than a length of the scaled foreground image, and the preset sub-image width being less than a width of the scaled foreground image; and

performing image feature extraction processing on the at least two sub-images to obtain sub-image features corresponding to the at least two sub-images; and

determining a feature representation subset of the image according to the sub-image features corresponding to the at least two sub-images.

3 . The method according to claim 1 , wherein the generating the attention weights corresponding to the at least two sub-image features, and performing weighting aggregation processing on the at least two sub-image features according to the attention weights to obtain the first feature vector comprises:

inputting the at least two sub-image features into a first attention sub-network in an image recognition model, the first attention sub-network comprising a weight learning network layer and a weighting aggregation network layer;

performing weight fitting processing on the at least two sub-image features using the weight learning network layer to obtain attention weights corresponding to the at least two sub-image features; and

weighting each of the at least two sub-image features according to the attention weight using the weighting aggregation network layer to obtain a weighted sub-image feature corresponding to each of the at least two sub-image features, and aggregating the weighted sub-image features corresponding to the at least two sub-image features to obtain the first feature vector.

4 . The method according to claim 1 , wherein the performing clustering sampling processing on the at least two sub-image features to obtain the at least two classification clusters, determining the block sparse self-attention according to the at least two classification clusters and the block sparse matrix, and determining the second feature vector according to the at least two block sparse self-attentions comprises:

inputting the at least two sub-image features into a second attention sub-network in an image recognition model, the second attention sub-network comprising a clustering sampling network layer, a global self-attention network layer, a self-attention network layer, and a mean pooling network layer;

performing clustering sampling processing on the at least two sub-image features using the clustering sampling network layer to obtain the at least two classification clusters, a classification cluster comprising the sampled sub-image features, a sum number of sampled sub-image features comprised in the at least two classification clusters being N, and N being a positive integer less than a number of the at least two sub-image features;

determining block sparse global self-attention weight matrices of N sampled sub-image features based on the block sparse matrix using the global self-attention network layer;

determining the block sparse self-attention for each of the sampled sub-image features according to the at least two classification clusters and the block sparse global self-attention weight matrices using the self-attention network layer; and

performing mean pooling processing on the at least two block sparse self-attentions using the mean pooling network layer to obtain the second feature vector.

5 . The method according to claim 4 , wherein the performing clustering sampling processing on the at least two sub-image features using the clustering sampling network layer to obtain the at least two classification clusters comprises:

clustering the at least two sub-image features using the clustering sampling network layer to obtain the at least two classification clusters;

obtaining a k th classification cluster from the at least two classification clusters, k being a positive integer, and the k th classification cluster comprising at least one clustered sub-image feature;

obtaining a vector distance between the at least one clustered sub-image feature and a cluster center of the k th classification cluster as a reference distance; and

sequentially obtaining h clustered sub-image features from the at least one clustered sub-image feature according to the reference distance, and taking the h clustered sub-image features as the sampled sub-image features comprised in the k th classification cluster, h being a positive integer, and h being less than or equal to a number of the at least one clustered sub-image feature.

6 . The method according to claim 4 , wherein the second attention sub-network comprises a query weight matrix and a key weight matrix, and the determining the block sparse global self-attention weight matrices of N sampled sub-image features based on the block sparse matrix using the global self-attention network layer comprises:

constructing a sampled sub-image feature matrix according to the sampled sub-image features comprised in the at least two classification clusters using the global self-attention network layer;

multiplying the sampled sub-image feature matrix and the query weight matrix to obtain a query matrix, and multiplying the sampled sub-image feature matrix and the key weight matrix to obtain a key matrix;

determining a block sparse global correlation matrix according to the query matrix, a transposition matrix corresponding to the key matrix, and the block sparse matrix; and

normalizing the block sparse global correlation matrix to obtain a block sparse global self-attention weight matrix.

7 . The method according to claim 6 , wherein the second attention sub-network comprises a value weight matrix, the N sampled sub-image features comprise a sampled sub-image feature N j , j being a positive integer less than or equal to N, and the determining the block sparse self-attention according to the at least two classification clusters and the block sparse global self-attention weight matrices using the self-attention network layer comprises:

multiplying the sampled sub-image feature matrix and the value weight matrix using the self-attention network layer to obtain a value matrix;

taking the sampled sub-image feature in a classification cluster to which the sampled sub-image feature N j belongs as a target sampled sub-image feature;

obtaining a block sparse global self-attention weight between the sampled sub-image feature N j and the target sampled sub-image feature from the block sparse global self-attention weight matrix as a target block sparse global self-attention weight;

obtaining, from the value matrix, a value vector corresponding to the target sampled sub-image feature as a target value vector; and

determining a block sparse self-attention for the sampled sub-image feature N j according to the target value vector and the target block sparse global self-attention weight.

8 . The method according to claim 1 , wherein the determining the classification result of the image according to the first feature vector and the second feature vector comprises:

inputting the first feature vector and the second feature vector into a classification sub-network of an image recognition model, the classification sub-network comprising a feature fusion network layer and a classification network layer;

performing feature fusion processing on the first feature vector and the second feature vector using the feature fusion network layer to obtain a fusion feature vector; and

classifying the fusion feature vector using the classification network layer to obtain the classification result of the image.

9 . An image detection method, performed by a computer device, the method comprising:

obtaining a sample image;

performing feature extraction processing on the sample image to obtain a sample feature representation subset of the sample image, the sample image comprising at least two sample sub-images, the sample feature representation subset comprising at least two sample sub-image features, and the at least two sample sub-image features corresponding to the at least two sample sub-images respectively;

inputting the at least two sample sub-images into an initial image recognition model, generating sample attention weights corresponding to the at least two sample sub-image features using the initial image recognition model;

performing weighting aggregation processing on the at least two sample sub-image features according to the sample attention weights corresponding to the at least two sample sub-image features to obtain a first sample feature vector;

performing clustering sampling processing on the at least two sample sub-image features using the initial image recognition model to obtain at least two sample classification clusters comprising sample sampled sub-image features;

determining a sample block sparse self-attention for each of the sample sampled sub-image features according to the at least two sample classification clusters and a block sparse matrix;

determining a second sample feature vector according to at least two sample block sparse self-attentions respectively corresponding to the at least two sample classification clusters, a sample block sparse self-attention for a sample sampled sub-image feature being determined based on sample sampled sub-image features in a sample classification cluster to which the sample sampled sub-image feature belongs;

determining a sample classification result of the sample image according to the first sample feature vector and the second sample feature vector using the initial image recognition model; and

adjusting model parameters of the initial image recognition model according to the at least two sample classification clusters, the attention weights corresponding to the at least two sample sub-image features, the sample classification result, and a classification label corresponding to the sample image to obtain an image recognition model for recognizing a classification result of an image.

10 . The method according to claim 9 , wherein the adjusting the model parameters of the initial image recognition model comprises:

determining a divergence loss value according to the at least two sample classification clusters and the sample attention weights corresponding to the at least two sample sub-image features;

determining a classification loss value according to the sample classification result and the classification label corresponding to the sample image;

performing weighting summation on the divergence loss value and the classification loss value to obtain a total model loss value; and

adjusting the model parameters of the initial image recognition model according to the total model loss value to obtain the image recognition model.

11 . The method according to claim 10 , wherein the determining the divergence loss value according to the at least two sample classification clusters and the sample attention weights corresponding to the at least two sample sub-image features comprises:

obtaining an i th sample classification cluster from the at least two sample classification clusters, i being a positive integer, and i being less than or equal to a number of the at least two sample classification clusters;

taking sample sub-image features comprised in the i th sample classification cluster as target sample sub-image features;

determining a category divergence loss value corresponding to the i th sample classification cluster according to sample attention weights corresponding to the target sample sub-image features and a number of the target sample sub-image features; and

accumulating category divergence loss values corresponding to the sample classification clusters to obtain the divergence loss value.

12 . The method according to claim 11 , wherein the determining the category divergence loss value corresponding to the i th sample classification cluster according to sample attention weights corresponding to the target sample sub-image features and the number of the target sample sub-image features comprises:

obtaining a fitting attention distribution comprising sample attention weights corresponding to the target sample sub-image features;

normalizing the fitting attention weight distribution to obtain a normalized fitting attention distribution;

taking a uniform attention distribution corresponding to the number of the target sample sub-image features as an attention distribution label; and

determining the category divergence loss value corresponding to the i th sample classification cluster according to the normalized fitting attention distribution and the attention distribution label.

13 . An image detection apparatus, comprising:

a memory operable to store computer-readable instructions; and

a processor circuitry operable to read the computer-readable instructions, the processor circuitry when executing the computer-readable instructions is configured to:

obtain an image;

perform feature extraction processing on the image to obtain a feature representation subset of the image, the image comprising at least two sub-images, the feature representation subset comprising at least two sub-image features, and the at least two sub-image features corresponding to the at least two sub-images respectively;

generate attention weights corresponding to the at least two sub-image features;

perform weighting aggregation processing on the at least two sub-image features according to the attention weights to obtain a first feature vector;

perform clustering sampling processing on the at least two sub-image features to obtain at least two classification clusters comprising sampled sub-image features;

determine a block sparse self-attention for each of the sampled sub-image features according to the at least two classification clusters and a block sparse matrix;

determine a second feature vector according to at least two block sparse self-attentions respectively corresponding to the at least two classification clusters, a block sparse self-attention for a sampled sub-image feature being determined based on sampled sub-image features in a classification cluster to which the sampled sub-image feature belongs; and

determine a classification result of the image according to the first feature vector and the second feature vector.

14 . The apparatus according to claim 13 , wherein the processor circuitry is configured to:

recognize a background region and a foreground region in the image;

perform image segmentation on the image according to the background region and the foreground region to obtain a foreground image;

scale the foreground image according to a scaling magnification to obtain a scaled foreground image;

crop the scaled foreground image according to a preset sub-image length and a preset sub-image width to obtain the at least two sub-images, the preset sub-image length being less than a length of the scaled foreground image, and the preset sub-image width being less than a width of the scaled foreground image; and

perform image feature extraction processing on the at least two sub-images to obtain sub-image features corresponding to the at least two sub-images; and

determine a feature representation subset of the image according to the sub-image features corresponding to the at least two sub-images.

15 . The apparatus according to claim 13 , wherein the processor circuitry is configured to:

input the at least two sub-image features into a first attention sub-network in an image recognition model, the first attention sub-network comprising a weight learning network layer and a weighting aggregation network layer;

perform weight fitting processing on the at least two sub-image features using the weight learning network layer to obtain attention weights corresponding to the at least two sub-image features; and

weight each of the at least two sub-image features according to the attention weight using the weighting aggregation network layer to obtain a weighted sub-image feature corresponding to each of the at least two sub-image features, and aggregate the weighted sub-image features corresponding to the at least two sub-image features to obtain the first feature vector.

16 . The apparatus according to claim 13 , wherein the processor circuitry is configured to:

input the at least two sub-image features into a second attention sub-network in an image recognition model, the second attention sub-network comprising a clustering sampling network layer, a global self-attention network layer, a self-attention network layer, and a mean pooling network layer;

perform clustering sampling processing on the at least two sub-image features using the clustering sampling network layer to obtain the at least two classification clusters, a classification cluster comprising the sampled sub-image features, a sum number of sampled sub-image features comprised in the at least two classification clusters being N, and N being a positive integer less than a number of the at least two sub-image features;

determine block sparse global self-attention weight matrices of N sampled sub-image features based on the block sparse matrix using the global self-attention network layer;

determine the block sparse self-attention for each of the sampled sub-image features according to the at least two classification clusters and the block sparse global self-attention weight matrices using the self-attention network layer; and

perform mean pooling processing on the at least two block sparse self-attentions using the mean pooling network layer to obtain the second feature vector.

17 . The apparatus according to claim 16 , wherein the processor circuitry is configured to:

cluster the at least two sub-image features using the clustering sampling network layer to obtain the at least two classification clusters;

obtain a k th classification cluster from the at least two classification clusters, k being a positive integer, and the k th classification cluster comprising at least one clustered sub-image feature;

obtain a vector distance between the at least one clustered sub-image feature and a cluster center of the k th classification cluster as a reference distance; and

sequentially obtain h clustered sub-image features from the at least one clustered sub-image feature according to the reference distance, and take the h clustered sub-image features as the sampled sub-image features comprised in the k th classification cluster, h being a positive integer, and h being less than or equal to a number of the at least one clustered sub-image feature.

18 . The apparatus according to claim 16 , wherein the second attention sub-network comprises a query weight matrix and a key weight matrix, and the processor circuitry is configured to:

construct a sampled sub-image feature matrix according to the sampled sub-image features comprised in the at least two classification clusters using the global self-attention network layer;

multiply the sampled sub-image feature matrix and the query weight matrix to obtain a query matrix, and multiply the sampled sub-image feature matrix and the key weight matrix to obtain a key matrix;

determine a block sparse global correlation matrix according to the query matrix, a transposition matrix corresponding to the key matrix, and the block sparse matrix; and

normalize the block sparse global correlation matrix to obtain a block sparse global self-attention weight matrix.

19 . The apparatus according to claim 13 , wherein the processor circuitry is configured to:

input the first feature vector and the second feature vector into a classification sub-network of an image recognition model, the classification sub-network comprising a feature fusion network layer and a classification network layer;

perform feature fusion processing on the first feature vector and the second feature vector using the feature fusion network layer to obtain a fusion feature vector; and

classify the fusion feature vector using the classification network layer to obtain the classification result of the image.

20 . A non-transitory machine-readable media, having instructions stored on the machine-readable media, the instructions configured to, when executed, cause a machine to:

obtain an image;

perform feature extraction processing on the image to obtain a feature representation subset of the image, the image comprising at least two sub-images, the feature representation subset comprising at least two sub-image features, and the at least two sub-image features corresponding to the at least two sub-images respectively;

generate attention weights corresponding to the at least two sub-image features;

perform weighting aggregation processing on the at least two sub-image features according to the attention weights to obtain a first feature vector;

perform clustering sampling processing on the at least two sub-image features to obtain at least two classification clusters comprising sampled sub-image features;

determine a block sparse self-attention for each of the sampled sub-image features according to the at least two classification clusters and a block sparse matrix;

determine a second feature vector according to at least two block sparse self-attentions respectively corresponding to the at least two classification clusters, a block sparse self-attention for a sampled sub-image feature being determined based on sampled sub-image features in a classification cluster to which the sampled sub-image feature belongs; and

determine a classification result of the image according to the first feature vector and the second feature vector.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (19)