IP Library Granted Patent US 11,069,151
Granted Patent B2
US 11,069,151 · App. 16/542,025 · Granted Jul 20, 2021

Methods and devices for replacing expression, and computer readable storage media

Inventor: Chengpeng Du (Guangdong, CN)
Assignee: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.
G06T19/20G06F3/0482G06F3/04845G06K9/00302G06K9/00281G06T13/40G06T2219/2021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,151
App. No.
16/542,025
Granted
Jul 20, 2021
Kind
B2
Abstract

The present disclosure provides a method and a device for replacing an expression. The method includes: acquiring a current expression represented by a currently-reconstructed 3D face model; acquiring a target expression from a user; acquiring, based on the current expression and the target expression, values for adjusting coordinates of a first set of key points on the currently-reconstructed 3D face model; and adjusting the coordinates of the first set of key points on the currently-reconstructed 3D face model based on the values, to generate a 3D face model representing the target expression.

Claims (60)

1. A method for replacing an expression, comprising:

acquiring a current expression represented by a currently-reconstructed three-dimensional (3D) face model, wherein the current expression is acquired by acts of:

scanning the currently-reconstructed 3D face model to acquire a plurality of key portions and key points of the plurality of key portions;

extracting a feature vector of the plurality of key portions based on coordinates of the key points of the plurality of key portions and distances among the plurality of key portions; and

analyzing the feature vector by a pre-trained neural network model to determine the current expression;

acquiring a target expression from a user, comprising:

capturing an expression of the user by a camera; matching the expression captured by the camera with a preset list of expressions, and

in response to the expression captured by the camera matching one expression in the preset list, using the expression captured by the camera as the target expression;

acquiring, based on the current expression and the target expression, values for adjusting coordinates of a first set of key points on the currently-reconstructed 3D face model, comprising:

querying, based on the current expression and the target expression, a preset database to acquire the values for adjusting the coordinates of the first set of key points, the preset database comprises a plurality of expressions, and values for adjusting coordinates of a corresponding set of key points from one of the plurality of expressions to another of the plurality of expressions; and

adjusting the coordinates of the first set of key points on the currently-reconstructed 3D face model based on the values, to generate a 3D face model representing the target expression.

2. The method of claim 1 , further comprising:

displaying one or more adjustable widgets, each of the one or more adjustable widgets being configured to adjust a corresponding key portion on the 3D face model representing the target expression within a preset range;

acquiring an operation on one of the one or more adjustable widgets;

acquiring an adjustment angle based on the operation; and

adjusting the corresponding key portion based on the adjustment angle.

3. The method of claim 1 , further comprising:

acquiring a preset state feature of a key portion corresponding to the target expression; and

adjusting a state of the key portion in the 3D face model representing the target expression based on the preset state feature.

4. An electronic device, comprising:

a computer program;

a memory, configured to store the computer program; and

a processor, configured to execute the computer program to carry out:

acquiring a current expression represented by a currently-reconstructed 3D face model, wherein the current expression is acquired by acts of:

scanning the currently-reconstructed 3D face model to acquire a plurality of key portions and key points of the plurality of key portions;

extracting a feature vector of the plurality of key portions based on coordinates of the key points of the plurality of key portions and distances among the plurality of key portions; and

analyzing the feature vector by a pre-trained neural network model to determine the current expression;

acquiring a target expression from a user, comprising:

capturing an expression of the user by a camera; matching the expression captured by the camera with a preset list of expressions; and

in response to the expression captured by the camera matching one expression in the preset list, using the expression captured by the camera as the target expression;

acquiring, based on the current expression and the target expression, values for adjusting coordinates of a first set of key points on the currently-reconstructed 3D face model, comprising:

querying, based on the current expression and the target expression, a preset database to acquire the values for adjusting the coordinates of the first set of key points, the preset database comprises a plurality of expressions, and values for adjusting coordinates of a corresponding set of key points from one of the plurality of expressions to another of the plurality of expressions; and

adjusting the coordinates of the first set of key points on the currently-reconstructed 3D face model based on the values, to generate a 3D face model representing the target expression.

5. The electronic device of claim 4 , wherein the processor is further configured to execute the computer program to carry out:

displaying one or more adjustable widgets, each of the one or more adjustable widgets being configured to adjust a corresponding key portion on the 3D face model representing the target expression within a preset range;

acquiring an operation on one of the one or more adjustable widgets from the user;

acquiring an adjustment angle based on the operation; and

adjusting the corresponding key portion based on the adjustment angle.

6. The electronic device of claim 4 , wherein the processor is further configured to execute the computer program to carry out:

acquiring a preset state feature of a key portion corresponding to the target expression; and

adjusting a state of the key portion in the 3D face model representing the target expression based on the preset state feature.

7. A non-transient computer readable storage medium having a computer program stored thereon, wherein the computer program causes an electronic device to carry out:

acquiring a current expression represented by a currently-reconstructed 3D face model, wherein the current expression is acquired by acts of:

scanning the currently-reconstructed 3D face model to acquire a plurality of key portions and key points of the plurality of key portions;

extracting a feature vector of the plurality of key portions based on coordinates of the key points of the plurality of key portions and distances among the plurality of key portions; and

analyzing the feature vector by a pre-trained neural network model to determine the current expression;

acquiring a target expression from a user, comprising:

capturing an expression of the user by a camera; matching the expression captured by the camera with a preset list of expressions; and

in response to the expression captured by the camera matching one expression in the pre list, using the expression captured by the camera as the target expression;

acquiring, based on the current expression and the target expression, values for adjusting coinages of a first set of key points on the currently-reconstructed 3D face model, comprising:

querying, based on the current expression and the target expression, a preset database to acquire the values for adjusting the coordinates of the first set of key points, the preset database comprises a plurality of expressions, and values for adjusting coordinate; of a corresponding set of key points from one of the plurality of expressions to another of the plurality of expression; and

adjusting the coordinates of the first set of key points on the currently-reconstructed 3D face model based on the values, to generate a 3D face model representing the target expression.

8. The non-transient computer readable storage medium of claim 7 , wherein the computer program causes an electronic device to further carry out:

displaying one or more adjustable widgets, each of the one or more adjustable widgets being configured to adjust a corresponding key portion on the 3D face model representing the target expression within a preset range;

acquiring an operation on one of the one or more adjustable widgets from the user;

acquiring an adjustment angle based on the operation; and

adjusting the corresponding key portion based on the adjustment angle,

or

acquiring a preset state feature of a key portion corresponding to the target expression; and

adjusting a state of the key portion in the 3D face model representing the target expression based on the preset state feature.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2019
From: DU, CHENGPENG
To: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.
Reel/Frame 050093/0311 →
Priority Claims (1)
CN 201810934577.8 · Aug 16, 2018 · national
Continuity (1)
Related Publication 20200058171A1 · Feb 20, 2020