IP Library Granted Patent US 10,916,044
Granted Patent B2
US 10,916,044 · App. 15/577,084 · Granted Feb 9, 2021

Information processing apparatus, information processing method, and program

Inventors: Maki Imoto (Tokyo, JP); Tomoya Narita (Kanagawa, JP)
Assignee: SONY CORPORATION
G06T11/60G06F3/0486G06K9/00281G06T11/00H04N1/387H04N7/147H04N7/18G06K9/00302
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,916,044
App. No.
15/577,084
Granted
Feb 9, 2021
Kind
B2
Abstract

There is provided an information processing apparatus, an information processing method, and a program capable of allowing a user to more intuitively perform operations for synthesis, the information processing apparatus including: a control unit configured to generate a synthesized object model by synthesizing a first object model and a second object model, and cause the generated synthesized object model to be displayed in association with an image corresponding to the first object model or the second object model. The control unit causes the first object model or the second object model to be moved, for the synthesis, in response to an input based on a user's motion with respect to a screen on which an object model or an image corresponding to the object model is displayed.

Claims (66)

1. An information processing apparatus comprising:

a control unit configured to

generate a synthesized object model by synthesizing a first object model and a second object model based on a user operation, and

cause a synthesized image corresponding to the generated synthesized object model to be displayed in association with an image corresponding to the first object model or an image corresponding to the second object model,

wherein the control unit causes the image corresponding to one of the first object model or the second object model to be a synthesis source and to be moved, for the synthesis, in response to the user operation based on a user's motion with respect to a screen on which the synthesis source and a synthesis destination comprising the image corresponding to another of the first object model or the second object model are each displayed,

wherein the control unit generates the synthesized object model by synthesizing the first object model and the second object model using synthesis processing including generation of mesh information and generation of texture information,

wherein the synthesis processing is initiated when the synthesis source is moved toward the synthesis destination in response to the user operation,

wherein the mesh information is generated by detecting feature points of at least one of the first object model or the second object model using image recognition, and

wherein the control unit is implemented via at least one processor.

2. The information processing apparatus according to claim 1 ,

wherein the first object model includes a first face model,

the second object model includes a second face model, and

the synthesized object model includes a synthesized face model.

3. The information processing apparatus according to claim 2 ,

wherein, after movement from one of the first face model and the second face model to the other is completed, the control unit causes the synthesized face model to be displayed in association with the image corresponding to the other face model.

4. The information processing apparatus according to claim 2 ,

wherein, when the user operation based on the user's motion is interrupted, the control unit does not perform the synthesis.

5. The information processing apparatus according to claim 2 ,

wherein the user operation based on the user's motion includes an operation of assigning both a start point and an end point or an operation of assigning only the start point.

6. The information processing apparatus according to claim 2 ,

wherein the user operation based on the user's motion includes a gesture operation.

7. The information processing apparatus according to claim 2 ,

wherein the control unit causes the first face model or the second face model corresponding to the image associated with the synthesized face model to be displayed to be adjacent to the synthesized face model when the synthesized face model is displayed.

8. The information processing apparatus according to claim 2 ,

wherein the first face model or the second face model to be moved includes a face model that is lined up in a face list displayed on the screen or a face model that is generated from an image lined up in the face list.

9. The information processing apparatus according to claim 8 ,

wherein the control unit selects a face model or an image lined up in the face list on a basis of an attribute of a face that is expressed by the first face model or the second face model.

10. The information processing apparatus according to claim 8 ,

wherein the control unit adds the first face model or the second face model moved toward the face list to the face list.

11. The information processing apparatus according to claim 8 ,

wherein the control unit performs prior display of a second synthesized face model related to the synthesized face model.

12. The information processing apparatus according to claim 11 ,

wherein the prior display of the second synthesized face model includes causing the first face model or the second face model, which is being moved, to be changed into the second synthesized face model.

13. The information processing apparatus according to claim 12 ,

wherein the second synthesized face model includes a synthesized face mask in which face feature information of the synthesized face model is changed in accordance with information related to the movement of the first face model or the second face model, which is being moved.

14. The information processing apparatus according to claim 11 ,

wherein the prior display of the second synthesized face model includes causing a face model that is lined up in the face list or a face model that is generated from an image lined up in the face list, to be changed into the second synthesized face model.

15. The information processing apparatus according to claim 14 ,

wherein the control unit causes a face model selected from or all the face models from among face models lined up in the face list or face models that are generated from images lined up in the face list, to be changed into the second synthesized face model.

16. The information processing apparatus according to claim 2 ,

wherein, in a case in which a plurality of face models or images corresponding to the face models are displayed on the screen, the control unit generates the respective synthesized face models on a basis of a moved face model and the plurality of face models or the images corresponding to the face models.

17. The information processing apparatus according to claim 16 ,

wherein, when a face model is moved to one of the plurality of face models or one of the images corresponding to the face models, the control unit performs the synthesis.

18. The information processing apparatus according to claim 2 ,

wherein at least one of the first face model or the second face model includes a face model of the user.

19. The information processing apparatus according to claim 1 ,

wherein at least one of the image corresponding to the first object model or the image corresponding to the second object model includes a through image that is temporarily recorded by image capturing.

20. The information processing apparatus according to claim 1 ,

wherein the user's motion on which the user operation is based comprises a drag and drop motion from an initial location of the synthesis source to a location of the synthesis destination, and

wherein the control unit causes the synthesized image to be displayed when the drop motion is performed at the location of the synthesis destination.

21. The information processing apparatus according to claim 1 ,

wherein the synthesized object model is further generated according to a synthesis mode that determines an amount of the generated mesh information for each object model and an amount of the generated texture information for each object model to be used in the synthesis processing based on which object model corresponds to the synthesis source and which object model corresponds to the synthesis destination.

22. An information processing method, executed by at least one processor, the method comprising:

generating a synthesized object model by synthesizing a first object model and a second object model based on a user operation;

causing a synthesized image corresponding to the generated synthesized object model to be displayed in association with an image corresponding to the first object model or an image corresponding to the second object model; and

causing the image corresponding to one of the first object model or the second object model to be a synthesis source and to be moved, for the synthesis, in response to the user operation based on a user's motion with respect to a screen on which the synthesis source and a synthesis destination comprising the image corresponding to another of the first object model or the second object model are each displayed,

wherein the synthesized object model is generated by synthesizing the first object model and the second object model using synthesis processing including generation of mesh information and generation of texture information,

wherein the synthesis processing is initiated when the synthesis source is moved toward the synthesis destination in response to the user operation, and

wherein the mesh information is generated by detecting feature points of at least one of the first object model or the second object model using image recognition.

23. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:

generating a synthesized object model by synthesizing a first object model and a second object model based on a user operation;

causing a synthesized image corresponding to the generated synthesized object model to be displayed in association with an image corresponding to the first object model or an image corresponding to the second object model; and

causing the image corresponding to one of the first object model or the second object model to be a synthesis source and to be moved, for the synthesis, in response to the user operation based on a user's motion with respect to a screen on which the synthesis source and a synthesis destination comprising the image corresponding to another of the first object model or the second object model are each displayed,

wherein the synthesized object model is generated by synthesizing the first object model and the second object model using synthesis processing including generation of mesh information and generation of texture information,

wherein the synthesis processing is initiated when the synthesis source is moved toward the synthesis destination in response to the user operation, and

wherein the mesh information is generated by detecting feature points of at least one of the first object model or the second object model using image recognition.

Assignments (2)
CHANGE OF NAME Recorded Dec 17, 2025
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 073986/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2017
From: IMOTO, MAKI; NARITA, TOMOYA
To: SONY CORPORATION
Reel/Frame 044511/0767 →
Priority Claims (1)
JP 2015-143720 · Jul 21, 2015 · national
Continuity (1)
Related Publication 20180182144A1 · Jun 28, 2018
Cited By (1)
US 12,646,274