IP Library Granted Patent US 11,281,925
Granted Patent B2
US 11,281,925 · App. 16/902,235 · Granted Mar 22, 2022

Method and terminal for recognizing object node in image, and computer-readable storage medium

Inventors: Feng Li (Shenzhen, CN); Xiaoxiang Zuo (Shenzhen, CN); Haoyuan Li (Shenzhen, CN); Jun Cheng (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06K9/2054G06K9/629G06K9/6298
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,281,925
App. No.
16/902,235
Granted
Mar 22, 2022
Kind
B2
Abstract

A method for recognizing object nodes in an image includes: performing image detection on an original image, to determine a target image region that comprises a target image object, the original image being a two-dimensional image; performing image cropping on the original image according to the target image region, to obtain a subimage comprising the target image object; and performing node recognition on the subimage based on a preset node recognition model, to obtain image position information of each node of the target image object in the original image.

Claims (86)

1. A method for recognizing object nodes in an image, executed by a computer device, comprising:

performing image detection on an original image, to determine a target image region that comprises a target image object, the original image being a two-dimensional image, by:

invoking a preset second region detection model to perform convolution calculation on the original image, to obtain a third image feature;

performing R times of downsampling processing and S sets of convolution calculation processing on the third image feature to obtain a fourth image feature, the R times of downsampling processing comprising downsampling the third image feature and downsampling an image feature obtained after each set of convolution calculation processing, and R and S each being a positive integer;

performing convolution calculation processing on the fourth image feature to obtain an intermediate image feature;

performing dimension conversion on the fourth image feature to obtain a first feature vector, and performing dimension conversion on the intermediate image feature to obtain a second feature vector;

performing feature fusion on the first feature vector and the second feature vector to obtain quadruple data; and

determining a region position in the original image according to a first offset and a second offset in the quadruple data, determining a region size according to a width value and a height value in the quadruple data, and obtaining the target image region according to the region position and the region size;

performing image cropping on the original image according to the target image region, to obtain a subimage comprising the target image object; and

performing node recognition on the subimage based on a preset node recognition model, to obtain image position information of each node of the target image object in the original image.

2. The method according to claim 1 , wherein the performing image detection on an original image, to determine a target image region comprises:

invoking a preset first region detection model to perform image feature recognition on the original image, to obtain a region probability matrix;

and

determining the target image region further according to the region probability matrix.

3. The method according to claim 2 , wherein the invoking a preset first region detection model to perform image feature recognition on the original image, to obtain a region probability matrix comprises:

invoking the preset first region detection model to perform convolution calculation on the original image, to obtain a first image feature;

performing downsampling processing on the first image feature, to obtain a second image feature;

performing upsampling processing on the second image feature, to obtain a target image feature; and

performing deconvolution calculation on the target image feature, to obtain the region probability matrix.

4. The method according to claim 3 , wherein:

performing the downsampling processing comprises N times of downsampling processing and M sets of convolution calculation processing, N and M both being positive integers; and

performing the upsampling processing comprises N times of upsampling processing and M sets of deconvolution calculation processing.

5. The method according to claim 1 , wherein the processing the fourth image feature further comprises:

performing downsampling processing on the fourth image feature, and performing convolution calculation processing on an image feature obtained after the downsampling processing.

6. The method according to claim 1 , wherein the performing node recognition on the subimage based on a preset node recognition model comprises:

invoking the preset node recognition model to perform node recognition on the subimage, to obtain node probability matrices of P channels; and

obtaining the image position information of each node of the target image object according to the node probability matrices of the P channels.

7. The method according to claim 6 , wherein the performing node recognition on the subimage, to obtain node probability matrices of P channels comprises:

performing convolution calculation on the subimage, to obtain a convolutional image feature;

performing node recognition processing on the convolutional image feature, to obtain a first node recognition feature; and

obtaining the node probability matrices of the P channels according to the first node recognition feature.

8. The method according to claim 7 , wherein the obtaining the node probability matrices of the P channels according to the first node recognition feature comprises:

obtaining a second node recognition feature according to the first node recognition feature; and

performing node information adjustment processing on the first node recognition feature and the second node recognition feature, to obtain the node probability matrices of the P channels.

9. The method according to claim 1 , wherein the performing image cropping on the original image according to the target image region, to obtain a subimage comprising the target image object comprises:

performing image cropping on the original image according to the target image region, to obtain a cropped image, the cropped image comprising the target image object; and

adjusting a resolution of the cropped image to obtain the subimage.

10. The method according to claim 1 , further comprising:

obtaining a node connection image of the target image object according to the image position information of each node of the target image object; and

superposing the node connection image onto the original image, to obtain a superposed image.

11. The method according to claim 1 , wherein the performing dimension conversion on the fourth image feature, to obtain a first feature vector comprises:

performing scaling processing on the fourth image feature, to obtain a scaled fourth image feature, a resolution of the scaled fourth image feature being the same as a resolution of the intermediate image feature; and

performing dimension conversion on the scaled fourth image feature, to obtain the first feature vector.

12. An intelligent terminal, comprising a processor and a memory coupled to the processor, the processor being configured to:

perform image detection on an original image, to determine a target image region that comprises a target image object, the original image being a two-dimensional image, by:

invoking a preset second region detection model to perform convolution calculation on the original image, to obtain a third image feature;

performing R times of downsampling processing and S sets of convolution calculation processing on the third image feature to obtain a fourth image feature, the R times of downsampling processing comprising downsampling the third image feature and downsampling an image feature obtained after each set of convolution calculation processing, and R and S each being a positive integer;

performing convolution calculation processing on the fourth image feature to obtain an intermediate image feature;

performing dimension conversion on the fourth image feature to obtain a first feature vector, and performing dimension conversion on the intermediate image feature to obtain a second feature vector;

performing feature fusion on the first feature vector and the second feature vector to obtain quadruple data; and

determining a region position in the original image according to a first offset and a second offset in the quadruple data, determining a region size according to a width value and a height value in the quadruple data, and obtaining the target image region according to the region position and the region size;

perform image cropping on the original image according to the target image region, to obtain a subimage comprising the target image object; and

perform node recognition on the subimage based on a preset node recognition model, to obtain image position information of each node of the target image object in the original image.

13. The intelligent terminal according to claim 12 , wherein the processor is further configured to:

invoke a preset first region detection model to perform image feature recognition on the original image, to obtain a region probability matrix;

and

determine the target image region further according to the region probability matrix.

14. The intelligent terminal according to claim 13 , wherein the processor is further configured to:

invoke the preset first region detection model to perform convolution calculation on the original image, to obtain a first image feature;

perform downsampling processing on the first image feature, to obtain a second image feature;

perform upsampling processing on the second image feature, to obtain a target image feature; and

perform deconvolution calculation on the target image feature to obtain a region probability matrix.

15. The intelligent terminal according to claim 14 , wherein the processor is further configured to:

perform the downsampling processing comprises N times of downsampling processing and M sets of convolution calculation processing; and

perform the upsampling and fusion processing comprises N times of upsampling processing.

16. The intelligent terminal according to claim 12 , wherein the processor is further configured to:

obtain a node connection image of the target image object according to the image position information of each node of the target image object; and

superpose the node connection image onto the original image, to obtain a superposed image.

17. The intelligent terminal according to claim 12 , wherein the processing the fourth image feature further comprises:

performing downsampling processing on the fourth image feature, and performing convolution calculation processing on an image feature obtained after the downsampling processing.

18. The intelligent terminal according to claim 12 , wherein the performing node recognition on the subimage based on a preset node recognition model comprises:

invoking the preset node recognition model to perform node recognition on the subimage, to obtain node probability matrices of P channels; and

obtaining the image position information of each node of the target image object according to the node probability matrices of the P channels.

19. The intelligent terminal according to claim 12 , wherein the performing image cropping on the original image according to the target image region, to obtain a subimage comprising the target image object comprises:

performing image cropping on the original image according to the target image region, to obtain a cropped image, the cropped image comprising the target image object; and

adjusting a resolution of the cropped image to obtain the subimage.

20. A non-transitory computer-readable storage medium, storing a computer-readable instruction, the computer-readable instruction, when executed by one or more processors, causing the one or more processors to perform:

performing image detection on an original image, to determine a target image region that comprises a target image object, the original image being a two-dimensional image, by:

invoking a preset second region detection model to perform convolution calculation on the original image, to obtain a third image feature;

performing R times of downsampling processing and S sets of convolution calculation processing on the third image feature to obtain a fourth image feature, the R times of downsampling processing comprising downsampling the third image feature and downsampling an image feature obtained after each set of convolution calculation processing, and R and S each being a positive integer;

performing convolution calculation processing on the fourth image feature to obtain an intermediate image feature;

performing dimension conversion on the fourth image feature to obtain a first feature vector, and performing dimension conversion on the intermediate image feature to obtain a second feature vector;

performing feature fusion on the first feature vector and the second feature vector to obtain quadruple data; and

determining a region position in the original image according to a first offset and a second offset in the quadruple data, determining a region size according to a width value and a height value in the quadruple data, and obtaining the target image region according to the region position and the region size;

performing image cropping on the original image according to the target image region, to obtain a subimage comprising the target image object; and

performing node recognition on the subimage based on a preset node recognition model, to obtain image position information of each node of the target image object in the original image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 15, 2020
From: LI, FENG; ZUO, XIAOXIANG; LI, HAOYUAN; CHENG, JUN
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 052944/0777 →
Priority Claims (1)
CN 201810346029.3 · Apr 16, 2018 · national
Continuity (2)
Continuation PCTCN2019077843 · Mar 12, 2019
Related Publication 20200311453A1 · Oct 1, 2020