IP Library › Granted Patent US 12,182,911
Granted Patent B2
US 12,182,911 · App. 17/447,081 · Granted Dec 31, 2024

Image generation method, image generation apparatus, and image generation system

Inventors: Yanghua Jin (Tokyo, JP); Huachun Zhu (Tokyo, JP); Yingtao Tian (Tokyo, JP)
Assignee: Preferred Networks, Inc.
G06T11/60G06F18/25G06N3/049G06T13/40G06N3/045G06N3/047G06N3/065G06N3/126
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,182,911
App. No.
17/447,081
Granted
Dec 31, 2024
Kind
B2
Abstract

An image generation apparatus includes at least one memory, and at least one processor configured to acquire first latent information of a first image and second latent information of a second image, generate fusion latent information by using the first latent information and the second latent information, and generate a fusion image by inputting the fusion latent information into a trained generative model.

Claims (61)

1. An image generation apparatus comprising:

at least one memory; and

at least one processor configured to:

acquire first latent information of a first image and second latent information of a second image, the first image including data related to a visual representation of a first object, and the second image including data related to a visual representation of a second object,

generate fusion latent information by using the first latent information and the second latent information, and

generate a fusion image by inputting the fusion latent information into a trained generative model, the fusion image including data related to a visual representation of a third object,

wherein the at least one processor uses randomly determined information to generate the fusion latent information,

wherein the first object, the second object, and the third object are respectively a first character, a second character, and a third character, the first character, the second character, and the third character being different from each other,

wherein the visual representation of the first object indicates an appearance of the first character, the visual representation of the second object indicates an appearance of the second character, and the visual representation of the third object indicates an appearance of the third character, and

wherein the first latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the first character, the second latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the second character, and the fusion latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the third character.

2. The image generation apparatus as claimed in claim 1 , wherein the at least one processor is configured to perform a genetic operation on the first latent information and the second latent information to generate the fusion latent information.

3. The image generation apparatus as claimed in claim 2 , wherein the genetic operation includes at least one of crossover, mutation, or selection.

4. The image generation apparatus as claimed in claim 1 , wherein the at least one processor is configured to fuse elements of the first latent information and elements of the second latent information to generate the fusion latent information.

5. The image generation apparatus as claimed in claim 1 , wherein the at least one processor is configured to select elements of the first latent information and elements of the second latent information to generate the fusion latent information.

6. The image generation apparatus as claimed in claim 1 , wherein the at least one processor is configured to perform at least one of an arithmetic operation or a logical operation of the first latent information and the second latent information to generate the fusion latent information.

7. The image generation apparatus as claimed in claim 1 , wherein the first latent information includes main and sub information, and the second latent information includes main and sub information.

8. The image generation apparatus as claimed in claim 7 , wherein the at least one processor is configured to:

select one of the main and the sub information of the first latent information,

select one of the main and the sub information of the second latent information, and

generate the fusion latent information by using the selected information of the first latent information and the selected information of the second latent information.

9. The image generation apparatus as claimed in claim 1 , wherein the at least one processor is configured to generate the fusion latent information that includes main and sub information.

10. The image generation apparatus as claimed in claim 1 , wherein the first latent information and the second latent information are represented in blockchain compliant codes.

11. The image generation apparatus as claimed in claim 1 , wherein the trained generative model is a generator trained in accordance with a generative adversarial network.

12. The image generation apparatus as claimed in claim 1 , wherein the at least one processor displays, on a display device, the first image, the second image, the fusion image, and the information of the at least one of the hair style, the hair color, the eye color, the skin color, the expression, or the attachment of the third character.

13. The image generation apparatus as claimed in claim 1 , wherein the at least one processor displays, on a display device, the first image, the second image, and the fusion image in a form of a family tree.

14. The image generation apparatus as claimed in claim 1 , wherein the using of the randomly determined information includes applying fluctuation by using a random noise.

15. The image generation apparatus as claimed in claim 1 , wherein the using of the randomly determined information includes randomly determining an element of the fusion latent information.

16. The image generation apparatus as claimed in claim 1 , wherein the use of the randomly determined information to generate the fusion latent information includes an application of fluctuation.

17. An image generation method comprising:

acquiring, by at least one processor, first latent information of a first image and second latent information of a second image, the first image including data related to a visual representation of a first object, and the second image including data related to a visual representation of a second object,

generating, by the at least one processor, fusion latent information by using the first latent information and the second latent information, and

generating, by the at least one processor, a fusion image by inputting the fusion latent information into a trained generative model, the fusion image including data related to a visual representation of a third object,

wherein the generating of the fusion latent information includes generating, by the at least one processor, the fusion latent information by using randomly determined information,

wherein the first object, the second object, and the third object are respectively a first character, a second character, and a third character, the first character, the second character, and the third character being different from each other,

wherein the visual representation of the first object indicates an appearance of the first character, the visual representation of the second object indicates an appearance of the second character, and the visual representation of the third object indicates an appearance of the third character, and

wherein the first latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the first character, the second latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the second character, and the fusion latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the third character.

18. The image generation method as claimed in claim 17 , wherein the generating of the fusion image includes performing a genetic operation on the first latent information and the second latent information.

19. The image generation method as claimed in claim 18 , wherein the genetic operation includes at least one of crossover, mutation, or selection.

20. An image generation apparatus comprising:

at least one memory; and

at least one processor configured to:

acquire first latent information of a first image and second latent information of a second image, the first image including data related to a visual representation of a first object, and the second image including data related to a visual representation of a second object,

generate fusion latent information by using the first latent information and the second latent information, and

generate a fusion image by inputting the fusion latent information into a trained generative model, the fusion image including data related to a visual representation of a third object,

wherein the at least one processor displays, on a display device, the first image, the second image, and the fusion image in a form of a family tree,

wherein the first object, the second object, and the third object are respectively a first character, a second character, and a third character, the first character, the second character, and the third character being different from each other,

wherein the visual representation of the first object indicates an appearance of the first character, the visual representation of the second object indicates an appearance of the second character, and the visual representation of the third object indicates an appearance of the third character, and

wherein the first latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the first character, the second latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the second character, and the fusion latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the third character.

21. The image generation apparatus as claimed in claim 20 , wherein the at least one processor uses randomly determined information to generate the fusion latent information.

22. The image generation apparatus as claimed in claim 20 , wherein the at least one processor displays, on the display device, the information of the at least one of the hair style, the hair color, the eye color, the skin color, the expression, or the attachment of the third character.

23. An image generation method comprising:

acquiring, by at least one processor, first latent information of a first image and second latent information of a second image, the first image including data related to a visual representation of a first object, and the second image including data related to a visual representation of a second object,

generating, by the at least one processor, fusion latent information by using the first latent information and the second latent information,

generating, by the at least one processor, a fusion image by inputting the fusion latent information into a trained generative model, the fusion image including data related to a visual representation of a third object, and

displaying, by the at least one processor, on a display device, the first image, the second image, and the fusion image in a form of a family tree,

wherein the first object, the second object, and the third object are respectively a first character, a second character, and a third character, the first character, the second character, and the third character being different from each other,

wherein the visual representation of the first object indicates an appearance of the first character, the visual representation of the second object indicates an appearance of the second character, and the visual representation of the third object indicates an appearance of the third character, and

wherein the first latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the first character, the second latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the second character, and the fusion latent information includes information of at least one of a hair style, a hair color, an eye color, a skin color, an expression, or an attachment of the third character.

24. The image generation method as claimed in claim 23 , wherein the generating of the fusion latent information includes generating the fusion latent information by using randomly determined information.

25. The image generation method as claimed in claim 23 , further comprising:

displaying, by the at least one processor, on the display device, the information of the at least one of the hair style, the hair color, the eye color, the skin color, the expression, or the attachment of the third character.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 8, 2021
From: JIN, YANGHUA; ZHU, HUACHUN; TIAN, YINGTAO
To: PREFERRED NETWORKS, INC.
Reel/Frame 057408/0704 →
Continuity (3)
Continuation PCTUS2019022688 · Mar 18, 2019
Provisional Application 62816315 · Mar 11, 2019
Related Publication 20210398336A1 · Dec 23, 2021