IP Library › Granted Patent US 10,379,734
Granted Patent B2
US 10,379,734 · App. 14/764,149 · Granted Aug 13, 2019

Systems and methods for interactive image caricaturing by an electronic device

Inventors: Xuan Zou (Shanghai, CN); Fan Ling (Shanghai, CN); Ning Bi (San Diego, CA); Lei Zhang (Shanghai, CN)
Assignee: QUALCOMM Incorporated
G06F3/04883G06F3/04845G06T11/00G06T11/60G06T17/20G06F2203/04808
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,379,734
App. No.
14/764,149
Granted
Aug 13, 2019
Kind
B2
Abstract

A method for interactive image caricaturing by an electronic device is described. The method includes detecting at least one feature location of an image. The method further includes generating, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line. The method additionally includes obtaining a gesture input. The method also includes determining at least one caricature action based on the at least one gesture input. The method further includes generating a caricature image based on the image mesh, the at least one caricature action and the image.

Claims (66)

1. A method for interactive image caricaturing by an electronic device, comprising:

detecting at least one feature location of an image;

generating, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component in the image;

obtaining at least one gesture input;

determining at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

generating a caricature image based on the image mesh, the at least one caricature action and the image.

2. The method of claim 1 , wherein the image is a face image.

3. The method of claim 1 , wherein the at least one gesture input comprises a touch input.

4. The method of claim 1 , further comprising displaying at least one of the image and the caricature image.

5. The method of claim 1 , wherein generating a caricature image comprises applying the at least one caricature action to the image.

6. The method of claim 5 , wherein the at least one caricature action provides shifting information for the at least one vertex of the image mesh and provides the pixel map that maps the image vertex pixel to the corresponding caricature image vertex pixel based on the shifting information, and wherein generating the caricature image comprises interpolating a location of other pixels in the caricature image based on the pixel map.

7. The method of claim 1 , further comprising validating the at least one gesture input.

8. The method of claim 7 , wherein validating the at least one gesture input comprises determining whether a motion of the gesture input is greater than a motion threshold.

9. The method of claim 1 , further comprising maintaining a caricature action list.

10. The method of claim 9 , further comprising adding the at least one caricature action to the caricature action list.

11. The method of claim 1 , wherein the at least one gesture input comprises multiple gesture inputs.

12. The method of claim 11 , further comprising determining a motion of the gesture inputs relative to one another.

13. The method of claim 11 , further comprising determining that the gesture inputs are horizontally aligned.

14. The method of claim 11 , further comprising determining that the gesture inputs are vertically aligned.

15. An electronic device for interactive image caricaturing, comprising:

feature location detection circuitry configured to detect at least one feature location of an image;

image mesh generation circuitry coupled to the feature location detection circuitry, wherein the image mesh generation circuitry is configured to generate, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component in the image;

sensing circuitry configured to obtain at least one gesture input;

caricature action determination circuitry coupled to the sensing circuitry, wherein the caricature action determination circuitry is configured to determine at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

caricature image generation circuitry coupled to the image mesh generation circuitry and the caricature action determination circuitry, wherein the caricature image generation circuitry is configured to generate a caricature image based on the image mesh, the at least one caricature action and the image.

16. The electronic device of claim 15 , wherein the image is a face image.

17. The electronic device of claim 15 , wherein the at least one gesture input comprises a touch input.

18. The electronic device of claim 15 , further comprising a display that is coupled to the caricature image generation circuitry and configured to display at least one of the image and the caricature image.

19. The electronic device of claim 15 , wherein generating a caricature image comprises applying the at least one caricature action to the image.

20. The electronic device of claim 19 , wherein the at least one caricature action provides shifting information for the at least one vertex of the image mesh and provides the pixel map that maps the image vertex pixel to the corresponding caricature image vertex pixel based on the shifting information, and wherein generating the caricature image comprises interpolating a location of other pixels in the caricature image based on the pixel map.

21. The electronic device of claim 15 , further comprising gesture input validation circuitry coupled to the sensing circuitry, wherein the gesture input validation circuitry is configured to validate the at least one gesture input.

22. The electronic device of claim 21 , wherein validating the at least one gesture input comprises determining whether a motion of the gesture input is greater than a motion threshold value.

23. The electronic device of claim 15 , wherein the electronic device is configured to maintain a caricature action list.

24. The electronic device of claim 23 , wherein the electronic device is configured to add the at least one caricature action to the caricature action list.

25. The electronic device of claim 15 , wherein the at least one gesture input comprises multiple gesture inputs.

26. The electronic device of claim 25 , further comprising gesture input validation circuitry configured to determine a motion of the gesture inputs relative to one another.

27. The electronic device of claim 25 , further comprising gesture input validation circuitry configured to determine that the gesture inputs are horizontally aligned.

28. The electronic device of claim 25 , further comprising gesture input validation circuitry configured to determine that the gesture inputs are vertically aligned.

29. A non-transitory tangible computer-readable medium having instructions thereon, the instructions comprising:

code for causing an electronic device to detect at least one feature location of an image;

code for causing the electronic device to generate, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component in the image;

code for causing the electronic device to obtain at least one gesture input;

code for causing the electronic device to determine at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

code for causing the electronic device to generate a caricature image based on the image mesh, the at least one caricature action and the image.

30. The computer-readable medium of claim 29 , wherein the image is a face image.

31. The computer-readable medium of claim 29 , wherein the at least one gesture input comprises a touch input.

32. The computer-readable medium of claim 29 , wherein the instructions further comprise code for causing the electronic device to display at least one of the image and the caricature image.

33. The computer-readable medium of claim 29 , wherein the code for causing the electronic device to generate a caricature image comprises code for causing the electronic device to apply the at least one caricature action to the image.

34. The computer-readable medium of claim 29 , wherein the instructions further comprise code for causing the electronic device to validate the at least one gesture input.

35. The computer-readable medium of claim 29 , wherein the at least one gesture input comprises multiple gesture inputs.

36. The computer-readable medium of claim 29 , wherein the instructions further comprise code for causing the electronic device to maintain a caricature action list.

37. The computer-readable medium of claim 36 , wherein the instructions further comprise code for causing the electronic device to add the at least one caricature action to the caricature action list.

38. An apparatus for interactive image caricaturing, comprising:

means for detecting at least one feature location of an image;

means for generating, based on the at least one feature location, an image mesh that comprises a grid of at least one horizontal line and at least one vertical line, wherein at least one vertex of the image mesh corresponds to a feature location of a facial component of the image;

means for obtaining at least one gesture input;

means for determining at least one caricature action based on the at least one gesture input, wherein the caricature action provides a pixel map that maps an image vertex pixel of the at least one vertex to a corresponding caricature image vertex pixel; and

means for generating a caricature image based on the image mesh, the at least one caricature action and the image.

39. The apparatus of claim 38 , wherein the image is a face image.

40. The apparatus of claim 38 , wherein the at least one gesture input comprises a touch input.

41. The apparatus of claim 38 , further comprising means for displaying at least one of the image and the caricature image.

42. The apparatus of claim 38 , wherein the means for generating a caricature image comprises means for applying the at least one caricature action to the image.

43. The apparatus of claim 38 , further comprising means for validating the at least one gesture input.

44. The apparatus of claim 38 , wherein the at least one gesture input comprises multiple gesture inputs.

45. The apparatus of claim 38 , further comprising means for maintaining a caricature action list.

46. The apparatus of claim 45 , further comprising means for adding the at least one caricature action to the caricature action list.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2015
From: ZOU, XUAN; LING, FAN; BI, NING; ZHANG, LEI
To: QUALCOMM INCORPORATED
Reel/Frame 036514/0927 →
Continuity (1)
Related Publication 20160011769A1 · Jan 14, 2016