Image modification techniques

1. A method of processing one or more images, comprising:

obtaining a two-dimensional (2D) input image from a camera;

obtaining a three-dimensional (3D) representation of the 2D input image;

determining a first portion of the 3D representation of the 2D input image corresponding to a target object in the 2D input image;

obtaining an object mask associated with one or more additional objects, different from the target object, in the 2D input image, and wherein the determining the first portion of the 3D representation is based on the object mask;

determining, based on the object mask, a second portion of the 3D representation corresponding to at least one of the one or more additional objects; and

adjusting a pose of the first portion of the 3D representation and fixing one or more vertices of the second portion of the 3D representation to generate an adjusted 3D mesh;

generating a 2D output image having a modified version of the target object based on the adjusted pose of the first portion of the 3D representation of the 2D input image corresponding to the target object to be output on a display.

2. The method of claim 1 , wherein at least a sub-portion of the modified version of the target object has an enlarged size in the 2D output image with respect to a corresponding sub-portion of the target object in the 2D input image.

3. The method of claim 1 , wherein adjusting the pose of the first portion of the 3D representation of the 2D input image includes one or more of adjusting depth, vertical shifting, horizontal shifting, pitch rotation, yaw rotation, and roll rotation.

4. The method of claim 1 , wherein adjusting the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object comprises:

decreasing depth values of first one or more vertices of the first portion of the 3D representation of the 2D input image corresponding to the target object.

5. The method of claim 4 , wherein adjusting the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object further comprises increasing depth values of second one or more vertices of the first portion of the 3D representation of the 2D input image.

6. The method of claim 1 , further comprising:

obtaining a user input identifying the target object, wherein the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object is adjusted based on the user input.

7. The method of claim 6 , further comprising:

obtaining an indication of amount of adjustment of the target object; and

adjusting the pose of the 3D representation of the 2D input image by an amount corresponding to the indication of amount of adjustment of the target object.

8. The method of claim 6 , further comprising:

determining an amount of adjustment of the target object, based on one or more of an adjustment range, an apparent size change, and a maximum amount of distortion; and

adjusting the pose of the 3D representation of the 2D input image by an amount corresponding to the amount of adjustment of the target object.

9. The method of claim 1 , further comprising:

obtaining an object mask associated with the target object in the 2D input image;

determining, based on the object mask, the first portion of the 3D representation of the 2D input image corresponding to the target object;

determining, based on the object mask, a second portion of the 3D representation corresponding to one or more additional objects; and

combining the first portion of the 3D representation having the adjusted pose and the second portion of the 3D representation to generate a combined 3D mesh.

10. The method of claim 9 , wherein generating the 2D output image includes converting the combined 3D mesh to the 2D output image.

11. The method of claim 1 , wherein adjusting the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object comprises:

decreasing depth values of vertices of the first portion of the 3D representation of the 2D input image corresponding to the target object such that the vertices have a pre-determined numerical correspondence with respect to the fixed one or more vertices of the second portion of the 3D representation.

12. The method of claim 1 , wherein generating the 2D output image includes converting the adjusted 3D mesh to the 2D output image.

13. The method of claim 1 , wherein determining the first portion of the 3D representation of the 2D input image comprises at least one of feature detection and object detection.

14. The method of claim 1 , wherein determining the first portion of the 3D representation of the 2D input image is based on depth information corresponding to the first portion of the 3D representation of the 2D input image.

15. An apparatus for processing one or more images, comprising:

a memory; and

one or more processors coupled to the memory and configured to:

obtain a two-dimensional (2D) input image from a camera;

obtain a three-dimensional (3D) representation of the 2D input image;

determine a first portion of the 3D representation of the 2D input image corresponding to a target object in the 2D input image;

obtain an object mask associated with one or more additional objects, different from the target object, in the 2D input image, wherein the first portion of the 3D representation is determined based on the object mask;

determine, based on the object mask, a second portion of the 3D representation corresponding to at least one of the one or more additional objects;

adjust a pose of the first portion of the 3D representation and fix one or more vertices of the second portion of the 3D representation to generate an adjusted 3D mesh; and

generate a 2D output image having a modified version of the target object based on the adjusted pose of the first portion of the 3D representation of the 2D input image corresponding to the target object to be output on a display.

16. The apparatus of claim 15 , wherein at least a sub-portion of the modified version of the target object has an enlarged size in the 2D output image with respect to a corresponding sub-portion of the target object in the 2D input image.

17. The apparatus of claim 15 , wherein adjusting the pose of the first portion of the 3D representation of the 2D input image includes one or more of adjusting depth, vertical shifting, horizontal shifting, pitch rotation, yaw rotation, and roll rotation.

18. The apparatus of claim 15 , wherein the one or more processors are configured to:

decrease depth values of first one or more vertices of the first portion of the 3D representation of the 2D input image corresponding to the target object.

19. The apparatus of claim 18 , wherein adjusting the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object further comprises increasing depth values of second one or more vertices of the first portion of the 3D representation of the 2D input image.

20. The apparatus of claim 15 , wherein the one or more processors are configured to obtain a user input identifying the target object, wherein the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object is adjusted based on the user input.

21. The apparatus of claim 20 , wherein the one or more processors are configured to:

obtain an indication of amount of adjustment of the target object; and

adjust the pose of the 3D representation of the 2D input image by an amount corresponding to the indication of amount of adjustment of the target object.

22. The apparatus of claim 20 , wherein the one or more processors are configured to:

determine an amount of adjustment of the target object, based on one or more of an adjustment range, an apparent size change, and a maximum amount of distortion; and

adjust the pose of the 3D representation of the 2D input image by an amount corresponding to the amount of adjustment of the target object.

23. The apparatus of claim 15 , wherein the one or more processors are configured to:

obtain an object mask associated with the target object in the 2D input image;

determine, based on the object mask, the first portion of the 3D representation of the 2D input image corresponding to the target object;

determine, based on the object mask, a second portion of the 3D representation corresponding to one or more additional objects; and

combine the first portion of the 3D representation having the adjusted pose and the second portion of the 3D representation to generate a combined 3D mesh.

24. The apparatus of claim 23 , generating the 2D output image includes converting the combined 3D mesh to the 2D output image.

25. The apparatus of claim 16 , wherein, to adjust the pose of the first portion of the 3D representation of the 2D input image corresponding to the target object, the one or more processors are configured to:

decrease depth values of vertices of the first portion of the 3D representation of the 2D input image corresponding to the 2D input image such that the vertices have a pre-determined numerical correspondence with respect to the fixed one or more vertices of the second portion of the 3D representation.

26. The apparatus of claim 16 , wherein generating the 2D output image includes converting the adjusted 3D mesh to the 2D output image.

27. The apparatus of claim 15 , wherein determining the first portion of the 3D representation of the 2D input image comprises at least one of feature detection and object detection.

28. The apparatus of claim 15 , wherein determining the first portion of the 3D representation of the 2D input image is based on depth information corresponding to the first portion of the 3D representation of the 2D input image.

29. A method of processing one or more images, comprising:

obtaining a two-dimensional (2D) input image from an image capture device;

obtaining a three-dimensional (3D) representation of the 2D input image, the 3D representation of the 2D input image including a first portion corresponding to a target object in the 2D input image, and a second portion corresponding to at least one or more additional objects in the 2D input image;

obtaining an object mask associated with one or more additional objects and determining the first portion and the second portion of the 3D representation based on the object mask;

adjusting a pose of the first portion of the 3D representation and fixing one or more vertices of the second portion of the 3D representation to generate an adjusted 3D mesh; and

generating a 2D output image having a modified version of the target object based on the adjusted pose of the first portion of the 3D representation of the 2D input image to be output on a display.

30. A method of processing one or more images, comprising:

generating a three-dimensional (3D) representation of a 2D input image, the 3D representation of the 2D input image including a first portion corresponding to a target object in the 2D input image, and a second portion corresponding to at least one or more additional objects in the 2D input image;

determining the first portion and the second portion of the 3D representation based on an object mask associated with one or more additional objects;

adjusting a pose of the first portion of the 3D representation and fixing one or more vertices of the second portion of the 3D representation to generate an adjusted 3D mesh; and

generating a 2D output image having a modified version of the target object based on the adjusted pose of the first portion of the 3D representation of the 2D input image to be output on a display.