IP Library › Granted Patent US 11,631,275
Granted Patent B2
US 11,631,275 · App. 17/006,071 · Granted Apr 18, 2023

Image processing method and apparatus, terminal, and computer-readable storage medium

Inventors: Wei Xiong (Shenzhen, CN); Fei Huang (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06V40/161G06K9/6256G06K9/6262G06T3/40G06V40/172G06V40/174
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,631,275
App. No.
17/006,071
Granted
Apr 18, 2023
Kind
B2
Abstract

Embodiments of the present disclosure disclose an image processing method and apparatus, a terminal, and a computer-readable storage medium, that are generally related to the field of computer technologies. The method can include obtaining a trained pixel classification model. The pixel classification model can be used for determining a classification identifier of each pixel in any image with the classification identifier including a head classification identifier. The head classification identifier can be used for indicating that a corresponding pixel is located in a head region. The method can further include classifying each pixel in the target image based on the pixel classification model to obtain a classification identifier of the pixel in the target image, and determining a head region in the target image according to pixels whose classification identifiers are a head classification identifier and editing the head region.

Claims (59)

1. An image processing method that is performed by a terminal, the method comprising:

obtaining a trained pixel classification model that determines a classification identifier of each pixel in an image, the classification identifier indicating whether the respective pixel is (i) located in a head region or (ii) not located in the head region;

performing face detection on a target image to detect a face region in the target image;

performing recognition on the detected face region, based on a trained expression recognition model, to determine an expression class of the face region;

determining whether the determined expression class of the detected face region is a predetermined target expression class;

determining whether to perform head detection on the target image based on whether the determined expression class of the detected face region is the predetermined target expression class, such that the head detection is performed on the target image only in response to a determination that the determined expression class of the detected face region is the predetermined target expression class, and the head detection is not performed on the target image in response to a determination that the determined expression class is not the predetermined target expression class;

in response to a determination that the head detection is to be performed on the target image, applying the trained pixel classification model to classify each pixel in the target image as (i) located in the head region or (ii) not located in the head region; and

determining the head region in the target image, the head region being defined by first pixels in the target image, each of the first pixels being classified as located in the head region, and editing the determined head region.

2. The method according to claim 1 , wherein before the obtaining the trained pixel classification model, the method further comprises:

obtaining a plurality of sample images and a sample classification identifier of each pixel in the plurality of sample images; and

performing training according to the plurality of sample images and the sample classification identifier of each pixel in the plurality of sample images, until a classification accuracy of the trained pixel classification model reaches a first preset threshold.

3. The method according to claim 1 , wherein the editing the head region further comprises:

determining a target processing mode, according to a preset correspondence between the target expression class and the target processing mode; and

editing the head region by using the determined target processing mode.

4. The method according to claim 1 , further comprising obtaining the trained expression recognition model by

obtaining a plurality of sample face images and a sample expression class of each sample face image; and

performing training according to the plurality of sample face images and the sample expression class of each sample face image until a recognition accuracy of the trained expression recognition model reaches a second preset threshold.

5. The method according to claim 1 , wherein before classifying each pixel in the target image using the pixel classification model, the method further comprises:

obtaining a target video, the target video including a plurality of images arranged in sequence; and

using each of the plurality of images as the target image.

6. The method according to claim 1 , wherein the editing the head region further comprises at least one of scaling up the head region, scaling down the head region, adding a light effect to the head region, and displaying a dynamic effect of shaking the head region.

7. An image processing apparatus, comprising:

processing circuitry configured to

obtain a trained pixel classification model that determines a classification identifier of each pixel in an image, the classification identifier indicating whether the respective pixel is (i) located in a head region or (ii) not located in the head region;

perform face detection on a target image to detect a face region in the target image;

perform recognition on the detected face region, based on a trained expression recognition model, to determine an expression class of the face region;

determine whether the determined expression class of the detected face region is a predetermined target expression class;

determining whether to perform head detection on the target image based on whether the determined expression class of the detected face region is the predetermined target expression class, such that the head detection is performed on the target image only in response to a determination that the determined expression class of the detected face region is the predetermined target expression class, and the head detection is not performed on the target image in response to a determination that the determined expression class is not the predetermined target expression class;

in response to a determination that the head detection is to be performed on the target image, apply the trained pixel classification model to classify each pixel in the target image as (i) located in the head region or (ii) not located in the head region; and

determine the head region in the target image, the head region being defined by first pixels in the target image, each of the first pixels being classified as located in the head region, and edit the determined head region.

8. The apparatus according to claim 7 , wherein the processing circuitry is further configured to:

obtain a plurality of sample images and a sample classification identifier of each pixel in the plurality of sample images; and

perform training according to the plurality of sample images and the sample classification identifier of each pixel in the plurality of sample images until a classification accuracy of the trained pixel classification model reaches a first preset threshold.

9. The apparatus according to claim 7 , wherein the processing circuitry is further configured to:

determine a target processing mode, according to a preset correspondence between the target expression class and the target processing mode; and

edit the head region by using the determined target processing mode.

10. The apparatus according to claim 7 , wherein the processing circuitry is further configured to:

obtain a plurality of sample face images and a sample expression class of each sample face image; and

perform training according to the plurality of sample face images and the sample expression class of each sample face image, until a recognition accuracy of the trained expression recognition model reaches a second preset threshold.

11. The apparatus according to claim 7 , wherein the processing circuitry is further configured to:

obtain a target video including a plurality of images arranged in sequence, each of the plurality of images being used as the target image.

12. An image processing terminal comprising processing circuitry configured to cause the image processing terminal to implement the image processing method according to claim 1 .

13. The terminal according to claim 12 , wherein the processing circuitry further performs:

obtaining a plurality of sample images and a sample classification identifier of each pixel in the plurality of sample images; and

performing training according to the plurality of sample images and the sample classification identifier of each pixel in the plurality of sample images, until a classification accuracy of the trained pixel classification model reaches a first preset threshold.

14. The terminal according to claim 12 , wherein the processing circuitry further performs:

determining a target processing mode, according to a preset correspondence between the target expression class and the target processing mode; and

editing the head region by using the determined target processing mode.

15. A non-transitory computer-readable storage medium that stores at least one instruction that, when executed by processing circuitry, causes the processing circuitry to perform:

obtaining a trained pixel classification model that determines a classification identifier of each pixel in any image, the classification identifier indicating whether the respective pixel is (i) located in a head region or (ii) not located in the head region;

performing face detection on a target image to detect a face region in the target image;

performing recognition on the detected face region, based on a trained expression recognition model to determine an expression class of the face region;

determining whether the determined expression class of the detected face region is a predetermined target expression class;

determining whether to perform head detection on the target image based on whether the determined expression class of the detected face region is the predetermined target expression class, such that the head detection is performed on the target image only in response to a determination that the determined expression class of the detected face region is the predetermined target expression class, and the head detection is not performed on the target image in response to a determination that the determined expression class is not the predetermined target expression class;

in response to a determination that the head detection is to be performed on the target image, applying the trained pixel classification model to classify each pixel in the target image as (i) located in the head region or (ii) not located in the head region; and

determining the head region in the target image, the head region being defined by first pixels in the target image, each of the first pixels being classified as located in the head region, and editing the determined head region.

16. The non-transitory computer-readable storage medium according to claim 15 , wherein the processing circuitry further performs:

obtaining a plurality of sample images and a sample classification identifier of each pixel in the plurality of sample images; and

performing training according to the plurality of sample images and the sample classification identifier of each pixel in the plurality of sample images until a classification accuracy of the trained pixel classification model reaches a first preset threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2020
From: XIONG, WEI; HUANG, FEI
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 053630/0417 →
Priority Claims (1)
CN 201810812675.4 · Jul 23, 2018 · national
Continuity (2)
Continuation PCTCN2019089825 · Jun 3, 2019
Related Publication 20200394388A1 · Dec 17, 2020
Cited By (1)
US 12,608,975