IP Library › Granted Patent US 11,568,590
Granted Patent B2
US 11,568,590 · App. 17/373,420 · Granted Jan 31, 2023

Cartoonlization processing method for image, electronic device, and storage medium

Inventors: Tianwei Lin (Beijing, CN); Fu Li (Beijing, CN); Xin Li (Beijing, CN); Henan Zhang (Beijing, CN); Hao Sun (Beijing, CN)
Assignee: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
G06T15/02G06T5/50G06T7/11G06T7/90G06T11/001G06V40/171G06T2207/20221G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,568,590
App. No.
17/373,420
Granted
Jan 31, 2023
Kind
B2
Abstract

The disclosure discloses a cartoonlization processing method for an image, and relates to a field of computational vision, image processing, face recognition, deep learning technologies. The method includes: performing skin color recognition on a facial image to be processed to determine a target skin color of a face in the facial image; processing the facial image by utilizing any cartoonizing model in a cartoonizing model set to obtain a reference cartoonized image corresponding to the facial image in a case that the cartoonizing model set does not contain a cartoonizing model corresponding to the target skin color; determining a pixel adjustment parameter based on the target skin color and a reference skin color corresponding to the any cartoonizing model; and adjusting a pixel value of each pixel point in the reference cartoonized image based on the pixel adjustment parameter, to obtain a target cartoonized image corresponding to the facial image.

Claims (67)

1. A cartoonlization processing method for an image, comprising:

performing skin color recognition on a facial age to be processed to determine a target skin color of a face in the facial image;

in a case that a cartoonizinq model set contains a cartoonizinq model corresponding to the target skin color, processing the facial image by using the cartoonizinq model corresponding to the target skin color to obtain a cartoonized image corresponding to the target skin color,

and in a case that the cartoonizing model set does not contain the cartoonizinq model corresponding to the target skin color:

processing the facial image by utilizing any of the cartoonizing models in the cartoonizing model set to obtain a reference cartoonized image corresponding to the facial image;

determining a pixel adjustment parameter based on the target skin color and a reference skin color corresponding to the any cartoonizing model utilized for processing the facial image; and

adjusting a pixel value of each pixel point of the plurality of the pixel points in the reference cartoonized image based on the pixel adjustment parameter, to obtain a target cartoonized image corresponding to the facial image;

wherein adjusting the pixel value of each pixel point of the plurality of the pixel points in the reference cartoonized image based on the pixel adjustment parameter comprises:

determining a first variable corresponding to each pixel point in the reference cartoonized image, the first variable being a ratio of a current pixel value of the pixel point in the reference cartoonized image to a set coefficient; and

determining an adiusted pixel value of each pixel point based on a power function of each first variable and the pixel adjustment parameter.

2. The method of claim 1 , before determining the pixel adjustment parameter based on the target skin color and the reference skin color corresponding to the cartoonizing model utilized for processing the facial image, further comprising:

performing feature recognition on the facial image to determine a feature of the facial image; and

updating the pixel adjustment parameter based on the feature of the facial image.

3. The method of claim 1 , wherein determining the pixel adjustment parameter based on the target skin color and the reference skin color corresponding to the cartoonizing model utilized for processing the facial image comprising:

determining the pixel adjustment parameter based on a difference degree between the target skin color and the reference skin color.

4. The method of claim 1 , after obtaining the target cartoonized image corresponding to the facial image, further comprising:

obtaining a facial image set and a cartoonized image set corresponding to the target skin color in a case that a number of target cartoonized images matching the target skin color is greater than a first threshold; and

performing training on an initial cartoonizing model based on the facial image set and the cartoonized image set to obtain the target cartoonizing model corresponding to the target skin color.

5. The method of claim 1 , before performing skin color recognition on the facial image to be processed, further comprising:

performing face segmentation on an image to be processed by utilizing a face segmentation model to obtain the facial image to be processed.

6. The method of claim 5 , further comprising:

performing background cartoonlization processing on the image to be processed to obtain a background-cartoonized image; and

merging the target cartoonized image and the background-cartoonized image to obtain a cartoonized image corresponding to the image to be processed.

7. An electronic device, comprising:

at least one processor; and

a memory, communicatively coupled to the at least one processor,

wherein the memory is configured to store instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is caused to execute a cartoonlization processing method for an image, the method comprising:

performing skin color recognition on a facial image to be processed to determine a target skin color of a face in the facial image;

in a case that a cartoonizing model set contains a cartoonizing model corresponding to the target skin color, processing the facial imaqe by using the cartoonizinq model corresponding to the target skin color to obtain a cartoonized image corresponding to the target skin color,

and in a case that the cartoonizing model set does not contain the cartoonizinq model corresponding to the target skin color:

processing the facial image by utilizing any of the cartoonizing models in the cartoonizing model set to obtain a reference cartoonized image corresponding to the facial image;

determining a pixel adjustment parameter based on the target skin color and a reference skin color corresponding to the cartoonizing model utilized for processing the facial image; and

adjusting a pixel value of each pixel point of the plurality of the pixel points in the reference cartoonized image based on the pixel adjustment parameter, to obtain a target cartoonized image corresponding to the facial image;

wherein adjusting the pixel value of each pixel point of the plurality of the pixel points in the reference cartoonized image based on the pixel adjustment parameter comprises:

determining a first variable corresponding to each pixel point in the reference cartoonized image, the first variable being a ratio of a current pixel value of the pixel point in the reference cartoonized image to a set coefficient; and

determining an adjusted pixel value of each pixel point based on a power function of each first variable and the pixel adjustment parameter.

8. The electronic device of claim 7 , wherein, before determining the pixel adjustment parameter based on the target skin color and the reference skin color corresponding to the cartoonizing model utilized for processing the facial image, the method further comprises: performing feature recognition on the facial image to determine a feature of the facial image: and updating the pixel adjustment parameter based on the feature of the facial image.

9. The electronic device of claim 7 , wherein determining the pixel adjustment parameter based on the target skin color and the reference skin color corresponding to the cartoonizing model utilized for processing the facial image comprising:

determining the pixel adjustment parameter based on a difference degree between the target skin color and the reference skin color.

10. The electronic device of claim 7 , wherein, after obtaining the target cartoonized image corresponding to the facial image, the further comprises:

obtaining a facial image set and a cartoonized image set corresponding to the target skin color in a case that a number of target cartoonized images matching the target skin color is greater than a first threshold; and

performing training on an initial cartoonizing model based on the facial image set and the cartoonized image set to obtain the target cartoonizing model corresponding to the target skin color.

11. The electronic device of claim 7 , wherein, before performing skin color recognition on the facial image to be processed, the method further comprises:

performing face segmentation on an image to be processed by utilizing a face segmentation model to obtain the facial image to be processed.

12. The electronic device of claim 11 , wherein the further comprises:

performing background cartoonlization processing on the image to be processed to obtain a background-cartoonized image; and

merging the target cartoonized image and the background-cartoonized image to obtain a cartoonized image corresponding to the image to be processed.

13. A non-transitory computer readable storage medium having computer instructions stored thereon, wherein the computer instructions are configured to cause a computer to execute the cartoonlization processing method for an image, the method comprising:

performing skin color recognition on a facial image to be processed to determine a target skin color of a face in the facial image;

in a case that a cartoonizinq model set contains a cartoonizinq model corresponding to the target skin color, processing the facial image by using the cartoonizing model corresponding to the target skin color to obtain a cartoonized image corresponding to the target skin color,

and in a case that the cartoonizinq model set does not contain the cartoonizinq model corresponding to the target skin color:

processing the facial image by utilizing any of the cartoonizing models in the cartoonizing model set to obtain a reference cartoonized image corresponding to the facial image in a case that the cartoonizing model set does not contain a cartoonizing model corresponding to the target skin color;

determining a pixel adjustment parameter based on the target skin color and a reference skin color corresponding to the cartoonizing model utilized for processing the facial image; and

adjusting a pixel value of each pixel point of the plurality of the pixel points in the reference cartoonized image based on the pixel adjustment parameter, to obtain a target cartoonized image corresponding to the facial image;

wherein adjusting the pixel value of each pixel point of the plurality of the pixel points in the reference cartoonized image based on the pixel adjustment parameter comprises:

determining a first variable corresponding to each pixel point in the reference cartoonized image, the first variable being a ratio of a current pixel value of the pixel point in the reference cartoonized image to a set coefficient; and

determining an adjusted pixel value of each pixel point based on a power function of each first variable and the pixel adjustment parameter.

14. The storage medium of claim 13 , wherein, before determining the pixel adjustment parameter based on the target skin color and the reference skin color corresponding to the ilany llcartoonizing model utilized for processing the facial image, the method further comprises:

performing feature recognition on the facial image to determine a feature of the facial image; and

updating the pixel adjustment parameter based on the feature of the facial image.

15. The storage medium of claim 13 , wherein determining the pixel adjustment parameter based on the target skin color and the reference skin color corresponding to the cartoonizing model utilized for processing the facial image comprising:

determining the pixel adjustment parameter based on a difference degree between the target skin color and the reference skin color.

16. The storage medium of claim 13 , wherein, after obtaining the target cartoonized image corresponding to the facial image, the further comprises:

obtaining a facial image set and a cartoonized image set corresponding to the target skin color in a case that a number of target cartoonized images matching the target skin color is greater than a first threshold; and

performing training on an initial cartoonizing model based on the facial image set and the cartoonized image set to obtain the target cartoonizing model corresponding to the target skin color.

17. The storage medium of claim 13 , wherein, before performing skin color recognition on the facial image to be processed, the method further comprises:

performing face segmentation on an image to be processed by utilizing a face segmentation model to obtain the facial image to be processed.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2021
From: LIN, TIANWEI; LI, FU; LI, XIN; ZHANG, HENAN; SUN, HAO
To: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
Reel/Frame 056828/0311 →
Priority Claims (1)
CN 202010844614.3 · Aug 20, 2020 · national
Continuity (1)
Related Publication 20210343065A1 · Nov 4, 2021