IP Library Granted Patent US 11,069,115
Granted Patent B2
US 11,069,115 · App. 16/795,794 · Granted Jul 20, 2021

Method of controlling display of avatar and electronic device therefor

Inventors: Donghwan Seo (Gyeonggi-do, KR); Hyejin Kang (Gyeonggi-do, KR); Junho An (Gyeonggi-do, KR); Minsheok Choi (Gyeonggi-do, KR); Yonggyoo Kim (Gyeonggi-do, KR); Junyoung Park (Gyeonggi-do, KR); Chanhee Yoon (Gyeonggi-do, KR); Wooyong Lee (Gyeonggi-do, KR); Jonghoon Won (Gyeonggi-do, KR)
Assignee: Samsung Electronics Co., Ltd.
G06T13/40G06K9/00268G06K9/00302G06T7/73H04N5/23229G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,115
App. No.
16/795,794
Granted
Jul 20, 2021
Kind
B2
Abstract

An electronic device and method are disclosed. The electronic device includes a camera, a display, and a processor. The processor implements the method, including capturing an image using a camera of the electronic device for animation of an avatar, the image including at least a part of a face of a user, analyzing, by a processor, a portion of the image including the at least the part of the face to determine whether an entirety of the face is captured within the image, and selecting a primary image or an alternative image for display of the avatar based on the determination, including: displaying the avatar on a display of the electronic device using the alternative image when less than the entirety of the face is captured within the image.

Claims (51)

1. A method of an electronic device, the method comprising:

capturing an image using a camera of the electronic device for animation of an avatar, the image including at least a part of a face of a user;

analyzing, by a processor, a portion of the image including the at least the part of the face to determine whether an entirety of the face is captured within the image; and

selecting a primary image or an alternative image for display of the avatar based on the determination, including: displaying the avatar on a display of the electronic device using the alternative image when less than the entirety of the face is captured within the image,

wherein analyzing the portion of the image further comprises detecting whether the face is partially obscured by an obstacle based on detecting an overlap between the face and an object within the captured image, and the avatar is displayed using the alternative image when the face is partially obscured by the obstacle.

2. The method of claim 1 , further comprising:

identifying, by a processor, a plurality of feature points in the captured image corresponding to major features of the face and calculating reliability scores for each of the plurality of feature points,

wherein detecting the overlap between the face and the object is based partly on one or more of the reliability scores being less than or equal to a threshold score.

3. The method of claim 2 , further comprising: identifying, by a processor, a plurality of feature points in the captured image corresponding to major features of the face,

wherein the displaying the avatar using the alternative image comprises:

substituting movement information of a first feature point with movement information of a second feature point, and animating the avatar using the substituted movement information.

4. The method of claim 1 , wherein the alternative image is generated by replacing at least a part of the avatar using another at least a portion of another image of the avatar that includes a predetermined shape.

5. The method of claim 4 , wherein the alternative image comprises an image of the avatar captured during a previous animation frame of the avatar.

6. The method of claim 1 , wherein determining whether the entirety of the face is captured further includes:

detecting a size of the face in the captured image; and

determining whether the detected size of the face is smaller than a threshold size,

wherein the captured image is determined to lack the entirety of the face when the detected size is smaller than the threshold size.

7. The method of claim 1 , further comprising:

storing in a memory an identification of a peripheral area of the captured image, indicating a region of the captured image distorted by a lens of the camera,

wherein the avatar is displayed using the alternative image when the face of the user is located in the peripheral area.

8. The method of claim 1 , further comprising:

capturing a second image using the camera of the electronic device for the animation of the avatar; and

in response to detecting that the second image lacks the face of the user entirely, displaying the avatar on the display using the alternative image.

9. An electronic device, comprising:

a camera;

a display;

a memory storing instructions; and

a processor, operatively coupled to the camera, the display and the memory,

wherein the instructions are executable by the processor to cause the electronic device to:

capture an image using a camera of the electronic device for animation of an avatar, the image including at least a part of a face of a user,

analyze, by a processor, a portion of the image including the at least the part of the face to determine whether an entirety of the face is captured within the image, and

select a primary image or an alternative image for display of the avatar based on the determination, including: displaying the avatar on a display of the electronic device using the alternative image when less than the entirety of the face is captured within the image,

wherein analyzing the portion of the image further comprises detecting whether the face is partially obscured by an obstacle based on detecting an overlap between the face and an object within the captured image, and the avatar is displayed using the alternative image when the face is partially obscured by the obstacle.

10. The electronic device of claim 9 , wherein the instructions are further executable by the processor to cause the electronic device to:

identify a plurality of feature points in the captured image corresponding to major features of the face and calculating reliability scores for each of the plurality of feature points,

wherein detecting the overlap between the face and the object is based partly on one or more of the reliability scores being less than or equal to a threshold score.

11. The electronic device of claim 10 , the instructions further executable by the processor to cause the electronic device to:

identify, by a processor, a plurality of feature points in the captured image corresponding to major features of the face, wherein displaying the avatar using the alternative image comprises:

substitute movement information of a first feature point with movement information of a second feature point, and animating the avatar using the substituted movement information.

12. The electronic device of claim 9 , wherein the alternative image is generated by replacing at least a part of the avatar using another at least a portion of another image of the avatar that includes a predetermined shape.

13. The electronic device of claim 12 , wherein the alternative image comprises an image of the avatar captured during a previous animation frame of the avatar.

14. The electronic device of claim 9 , wherein determining whether the entirety of the face is captured further includes:

detecting a size of the face in the captured image; and

determining whether the detected size of the face is smaller than a threshold size,

wherein the captured image is determined to lack the entirety of the face when the detected size is smaller than the threshold size.

15. The electronic device of claim 9 , further comprising:

storing in a memory an identification of a peripheral area of the captured image, indicating a region of the captured image distorted by a lens of the camera,

wherein the avatar is displayed using the alternative image when the face of the user is located in the peripheral area.

16. The electronic device of claim 9 , the instructions further executable by the processor to cause the electronic device to:

capture a second image using the camera of the electronic device for the animation of the avatar, and

in response to detecting that the second image lacks the face of the user entirely, display the avatar on the display using the alternative image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2020
From: SEO, DONGHWAN; KANG, HYEJIN; AN, JUNHO; CHOI, MINSHEOK; KIM, YONGGYOO; PARK, JUNYOUNG; YOON, CHANHEE; LEE, WOOYONG; WON, JONGHOON
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 051870/0920 →
Priority Claims (1)
KR 10-2019-0019718 · Feb 20, 2019 · national
Continuity (1)
Related Publication 20200265627A1 · Aug 20, 2020