3D-consistent 2D manipulation of images
View Patent ↗One embodiment involves receiving selection of a first quadrilateral and a second quadrilateral in an image being edited in an image editing application. An edge of the first quadrilateral may be shared with an edge of the second quadrilateral. In this embodiment, one or more manipulations associated with the first quadrilateral and/or the second quadrilateral may be received. In response to the received manipulation(s), an updated view of the image is generated or otherwise displayed according to one embodiment. The updated view may be based at least in part on the image, the received manipulation(s), and/or one or more constraints. For example, the updated view may be based at least in part on a three-dimensional scene constraint. In embodiments, the updated view is consistent with a three-dimensional scene in the image. In some embodiments, the updated view is generated in real-time or substantially real-time.
1. A computer-implemented method for manipulation of two-dimensional images that maintains three-dimensional consistency, the method comprising:
receiving, in a computing device, selection of a first quadrilateral in an image being edited in an image editing application;
receiving, in the computing device, selection of a second quadrilateral in the image, the second quadrilateral sharing an edge with the first quadrilateral;
receiving, in the computing device, a manipulation associated with a vertex of the first quadrilateral or the second quadrilateral; and
in response to the manipulation:
determining, in the computing device, new positions for remaining vertices of the first and second quadrilaterals that satisfy a three-dimensional scene constraint, wherein the three-dimensional scene constraint specifies a function based at least in part upon maintaining a consistent ratio of two vertices, in three-dimensional homogeneous coordinates, from two different homographs for the shared edge between the first and second quadrilaterals; and
generating, in the computing device, an updated view of the image that corresponds to the new positions of the vertices.
2. The method of claim 1 , wherein the updated view of the image is generated in real-time or substantially real-time.
3. The method of claim 1 , wherein receiving the manipulation comprises:
receiving selection of a corner of the first quadrilateral or the second quadrilateral; and
receiving input indicating that the corner should be moved to a new location.
4. The method of claim 1 , wherein receiving the manipulation comprises:
receiving selection of a vanishing point corresponding to the first quadrilateral or the second quadrilateral; and
receiving input indicating that that the vanishing point should be moved to a new location.
5. The method of claim 1 , wherein the three-dimensional scene constraint further comprises specifying that a line of the first quadrilateral or the second quadrilateral should be horizontal.
6. The method of claim 1 , wherein the three-dimensional scene constraint further comprises specifying that a line of the first quadrilateral or the second quadrilateral should be vertical.
7. The method of claim 1 , wherein the three-dimensional scene constraint further comprises specifying that movement of one or more of the remaining vertices of the first quadrilateral or the second quadrilateral should be minimized in the updated view.
8. The method of claim 1 , further comprising:
identifying, in the computing device, a line in a three-dimensional scene in the image, wherein the three-dimensional scene constraint further comprises specifying that the line should be straight.
9. The method of claim 1 , wherein there is no tearing along the shared edge between the first quadrilateral and the second quadrilateral in the updated view.
10. The method of claim 1 , further comprising:
after generating the updated view of the image:
receiving an additional input, the additional input comprising at least one of an additional constraint, an additional manipulation, or selection of another quadrilateral; and
generating another view of the image, the other view being (i) based at least in part on the received additional input and (ii) consistent with a three-dimensional scene in the image.
11. The method of claim 1 , further comprising storing the updated view.
12. The method of claim 1 , wherein the updated view comprises a straightened line for at least one of the first quadrilateral or the second quadrilateral.
13. The method of claim 1 , wherein the updated view comprises moving at least one vanishing point associated with the first quadrilateral or the second quadrilateral.
14. The method of claim 1 , wherein the updated view is generated without reconstructing a three-dimensional model associated with the image.
15. The method of claim 1 , wherein the manipulation comprises selection and movement of the vertex of the first quadrilateral or the second quadrilateral to a new location in the image.
16. A non-transitory computer-readable medium comprising a program for manipulation of two-dimensional images that maintains three-dimensional consistency, the program comprising:
program code for receiving selection of a first quadrilateral in an image being edited in an image editing application;
program code for receiving selection of a second quadrilateral in the image, the second quadrilateral sharing an edge with the first quadrilateral;
program code for receiving a manipulation associated with a vertex of the first quadrilateral or the second quadrilateral; and
program code that in response to the manipulation:
determines new positions for remaining vertices of the first and second quadrilaterals that satisfy a three-dimensional scene constraint, wherein the three-dimensional scene constraint specifies a function based at least in part upon maintaining a consistent ratio of two vertices, in three-dimensional homogeneous coordinates, from two different homographs for the shared edge between the first and second quadrilaterals; and
generates an updated view of the image that corresponds to the new positions of the vertices.
17. The non-transitory computer-readable medium of claim 16 , wherein the updated view of the image is generated in real-time or substantially real-time.
18. The non-transitory computer-readable medium of claim 16 , wherein the three-dimensional scene constraint further comprises specifying that a line of the first quadrilateral or the second quadrilateral should be vertical.
19. A system for manipulation of two-dimensional images that maintains three-dimensional consistency, the system comprising:
an input device;
a memory; and
a processor in communication with the input device and the memory, the processor configured for:
receiving selection of a first quadrilateral in an image being edited in an image editing application;
receiving selection of a second quadrilateral in the image, the second quadrilateral sharing an edge with the first quadrilateral;
receiving a manipulation associated with a vertex of the first quadrilateral or the second quadrilateral; and
in response to the manipulation:
determining new positions for remaining vertices of the first and second quadrilaterals that satisfy a three-dimensional scene constraint, wherein the three-dimensional scene constraint specifies a function based at least in part upon maintaining a consistent ratio of two vertices, in three-dimensional homogeneous coordinates, from two different homographs for the shared edge between the first and second quadrilaterals; and
generating an updated view of the image that corresponds to the new positions of the vertices.
20. The system of claim 19 , wherein after generating the updated view of the image:
receiving an additional input, the additional input comprising at least one of an additional constraint, an additional manipulation, or selection of another quadrilateral; and
generating another view of the image, the other view being (i) based at least in part on the received additional input and (ii) consistent with a three-dimensional scene in the image.