3D-consistent 2D manipulation of images
One embodiment involves receiving selection of a first quadrilateral and a second quadrilateral in an image being edited in an image editing application. An edge of the first quadrilateral may be shared with an edge of the second quadrilateral. In this embodiment, one or more manipulations associated with the first quadrilateral and/or the second quadrilateral may be received. In response to the received manipulation(s), an updated view of the image is generated or otherwise displayed according to one embodiment. The updated view may be based at least in part on the image, the received manipulation(s), and/or one or more constraints. For example, the updated view may be based at least in part on a three-dimensional scene constraint. In embodiments, the updated view is consistent with a three-dimensional scene in the image. In some embodiments, the updated view is generated in real-time or substantially real-time.
1. A method comprising:
receiving, by an image editing application, selection of a first quadrilateral in an image being edited in the image editing application;
receiving, by the image editing application, selection of a second quadrilateral in the image, the second quadrilateral sharing an edge with the first quadrilateral;
receiving, by the image editing application, a manipulation associated with at least one of the first quadrilateral or the second quadrilateral;
calculating, by the image editing application, a consistency error associated with the manipulation, the consistency error being based on the shared edge of the quadrilaterals; and
displaying, by the image editing application, an updated view of the image based at least in part on a three-dimensional scene constraint, the three-dimensional scene constraint being based at least in part on the consistency error and the manipulation.
2. The method of claim 1 , wherein the updated view of the image is generated in real-time or substantially real-time.
3. The method of claim 1 ,
wherein receiving the manipulation comprises:
receiving selection of a corner of the first quadrilateral or the second quadrilateral; and
receiving input indicating that the corner should be moved to a new location; and
wherein displaying the updated view comprises displaying an updated portion of the image corresponding to the corner at the new location.
4. The method of claim 1 ,
wherein receiving the manipulation comprises:
receiving selection of a vanishing point corresponding to the first quadrilateral or the second quadrilateral; and
receiving input indicating that that the vanishing point should be moved to a new location; and
wherein displaying the updated view comprises displaying the image such that vanishing point corresponds with the new location.
5. The method of claim 1 ,
wherein receiving the manipulation comprises:
receiving selection of a pin on a line of the first quadrilateral or the second quadrilateral, a location on the pin not being a corner; and
receiving input indicating that the pin should be moved to a new location; and
wherein displaying the updated view comprises displaying the image to correspond with the pin having the new location.
6. The method of claim 1 , wherein the three-dimensional scene constraint further comprises specifying that a line of the first quadrilateral or the second quadrilateral should be horizontal.
7. The method of claim 1 , wherein the three-dimensional scene constraint further comprises specifying that a line of the first quadrilateral or the second quadrilateral should be vertical.
8. The method of claim 1 , wherein the three-dimensional scene constraint further comprises specifying that a vertex of the first quadrilateral or the second quadrilateral should not be moved in the updated view.
9. The method of claim 1 , wherein the three-dimensional scene constraint further comprises an equation associated with the first quadrilateral and the second quadrilateral.
10. The method of claim 1 , further comprising:
identifying a line in a three-dimensional scene in the image, wherein the three-dimensional scene constraint further comprises specifying that the line should be straight.
11. The method of claim 1 , wherein there is no tearing along the shared edge between the first quadrilateral and the second quadrilateral in the updated view.
12. The method of claim 1 , further comprising:
after displaying the updated view of the image:
receiving an additional input, the additional input comprising at least one of an additional constraint, an additional manipulation, or selection of another quadrilateral; and
displaying another view of the image, the another view being based at least in part on the received additional input, the another view being consistent with the three-dimensional scene in the image.
13. The method of claim 1 , further comprising storing the updated view.
14. The method of claim 1 , wherein the updated view comprises a straightened line for at least one of the first quadrilateral or the second quadrilateral.
15. The method of claim 1 , wherein the updated view comprises moving at least one vanishing point associated with the first quadrilateral or the second quadrilateral.
16. The method of claim 1 , wherein the updated view is displayed without reconstructing a three-dimensional model associated with the image.
17. The method of claim 1 , wherein the manipulation comprises selection and movement of a vertex corresponding to at least one of the first quadrilateral or the second quadrilateral to a new location in the image; and
wherein the method further comprises:
calculating at least one new vertex for at least one of the first quadrilateral or the second quadrilateral, wherein the calculated at least one new vertex is based at least in part on the movement of the vertex to the new location and the three-dimensional scene constraint; and
wherein the updated view is based at least in part on the calculated at least one new vertex.
18. The method of claim 1 ,
wherein the manipulation comprises:
selecting a vertex corresponding to at least one of the first quadrilateral or the second quadrilateral; and
moving the vertex to a new location in the image; and
wherein the updated view minimizes changes in locations of vertices, other than the selected vertex, corresponding to the first quadrilateral and the second quadrilateral.
19. A non-transitory computer-readable medium comprising:
program code for receiving selection of a first quadrilateral in an image;
program code for receiving selection of a second quadrilateral in the image, the second quadrilateral sharing an edge with the first quadrilateral;
program code for receiving a manipulation associated with at least one of the first quadrilateral or the second quadrilateral;
program code for calculating a consistency error associated with the manipulation, the consistency error being based on the shared edge of the quadrilaterals; and
program code for generating an updated view of the image based at least in part on a three-dimensional scene constraint, the three-dimensional scene constraint being based at least in part on the consistency error and the manipulation.
20. A system comprising:
an input device;
a display;
a memory; and
a processor in communication with the input device, the display, and the memory, the processor configured for:
receiving, via the input device, selection of a first quadrilateral in an image, the image being a two-dimensional image, the image comprising a three-dimensional scene in the two-dimensional image;
receiving, via the input device, selection of a second quadrilateral in the image, the second quadrilateral sharing an edge with the first quadrilateral;
receiving, via the input device, a manipulation associated with at least one of the first quadrilateral or the second quadrilateral;
calculating a consistency error associated with the manipulation, the consistency error being based on the shared edge of the quadrilaterals; and
displaying, on the display, an updated view of the image, the updated view being based at least in part on a three-dimensional scene constraint, the three-dimensional scene constraint being based at least in part on the consistency error and the manipulation.