Granted Patent B2

US 12,505,596 · App. 18/320,664 · Granted Dec 23, 2025

Inventors: Zhihong Ding (Fremont, CA); Scott Cohen (Sunnyvale, CA); Matthew Joss (Seattle, WA); Jianming Zhang (Campbell, CA); Darshan Prasad (San Jose, CA); Celso Gomes (Bellevue, WA); Jonathan Brandt (Santa Cruz, CA)

Assignee: Adobe Inc.

G06T11/60G06F3/04842G06T3/40G06T5/77G06T7/50G06V10/761G06T2207/20084

Abstract

Claims (71)

1 . A computer-implemented method comprising:

determining a vanishing point associated with a digital image portraying an object;

extracting, from the digital image, a reference line that extends from the vanishing point to the object;

detecting user input selecting a selectable option for performing a perspective-aware object move operation within the digital image;

determining, in response to the user input selecting the selectable option, to restrict movement of the object to one or more movements along the reference line;

detecting one or more user interactions for moving the object within the digital image along the reference line; and

performing a perspective-based resizing of the object within the digital image based on moving the object with respect to the vanishing point along the reference line.

2 . The computer-implemented method of claim 1 , wherein:

moving the object within the digital image with respect to the vanishing point along the reference line comprises moving the object along the reference line towards the vanishing point; and

performing the perspective-based resizing of the object based on moving the object with respect to the vanishing point along the reference line comprises diminishing a size of the object based on moving the object along the reference line towards the vanishing point.

3 . The computer-implemented method of claim 1 , wherein:

moving the object within the digital image with respect to the vanishing point along the reference line comprises moving the object along the reference line away from the vanishing point; and

performing the perspective-based resizing of the object based on moving the object with respect to the vanishing point along the reference line comprises increasing a size of the object based on moving the object along the reference line away from the vanishing point.

4 . The computer-implemented method of claim 1 ,

further comprising generating a perspective scaling ratio for the object based on moving the object within the digital image along the reference line,

wherein performing the perspective-based resizing of the object within the digital image comprises performing the perspective-based resizing of the object using the perspective scaling ratio for the object.

5 . The computer-implemented method of claim 4 , wherein generating the perspective scaling ratio for the object based on moving the object within the digital image along the reference line comprises:

determining a first distance of the object from a horizon line associated with the digital image before the object is moved along the reference line;

determining a second distance of the object from the horizon line after the object is moved along the reference line; and

generating the perspective scaling ratio using the first distance and the second distance.

6 . The computer-implemented method of claim 1 ,

wherein moving the object within the digital image with respect to the vanishing point along the reference line comprises moving the object to a position where a portion of the object overlaps with a portion of an additional object portrayed within the digital image; and

further comprising modifying the digital image to occlude the portion of the object or the portion of the additional object based on comparing an object depth of the object to an object depth of the additional object.

7 . The computer-implemented method of claim 6 ,

further comprising determining that the object depth of the object is greater than the object depth of the additional object based on comparing the object depth of the object to the object depth of the additional object,

wherein modifying the digital image to occlude the portion of the object or the portion of the additional object comprises modifying the digital image to occlude the portion of the object using the portion of the additional object based on determining that the object depth of the object is greater than the object depth of the additional object.

8 . The computer-implemented method of claim 1 , further comprising providing, for display within the digital image, one or more perspective-based size previews along the reference line upon selection of the object for movement within the digital image.

9 . The computer-implemented method of claim 1 , further comprising:

generating a content fill for the object within the digital image before detecting the one or more user interactions for moving the object; and

exposing the content fill within the digital image in response to moving the object with respect to the vanishing point.

10 . A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

determining a vanishing point associated with a digital image portraying an object at a first position within the digital image;

moving the object with respect to the vanishing point to a second position within the digital image in response to one or more user interactions with the object;

determining a perspective scaling ratio for the object based on the first position and the second position by:

detecting a horizon line associated with the digital image;

determining a first distance from the first position to the horizon line and a second distance from the second position to the horizon line; and

determining a ratio using between the first distance and the second distance; and

performing a perspective-based resizing of the object within the digital image using the perspective scaling ratio.

11 . The non-transitory computer-readable medium of claim 10 , wherein:

determining the first distance from the first position to the horizon line comprises determining the first distance between a first average pixel coordinate pair for the object and the horizon line; and

determining the second distance from the second position to the horizon line comprises determining the second distance between a second average pixel coordinate pair for the object and the horizon line.

12 . The non-transitory computer-readable medium of claim 11 , wherein detecting the horizon line for the digital image comprises detecting the horizon line using a neural network.

13 . The non-transitory computer-readable medium of claim 10 , wherein determining the vanishing point associated the digital image comprises determining a vanishing point located outside of the digital image.

14 . The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:

generating a plurality of perspective-based size previews for the object that indicate sizes for the object at positions within the digital image other than the first position;

detecting a selection of the object for movement within the digital image; and

providing, for display within the digital image in response to the selection, the plurality of perspective-based size previews along a line from the object to the vanishing point.

15 . The non-transitory computer-readable medium of claim 14 , wherein generating the plurality of perspective-based size previews for the object that indicate the sizes for the object at the positions within the digital image other than the first position comprises:

generating a first perspective-based size preview indicating a larger size than a size of the object at the first position, the larger size corresponding to a position that is farther from the vanishing point than the first position; and

generating a second perspective-based size preview indicating a smaller size than the size of the object as the first position, the smaller size corresponding to a position that is closer to the vanishing point than the first position.

16 . The non-transitory computer-readable medium of claim 10 , wherein:

the operations further comprise moving the vanishing point associated with the digital image to a modified location in response to at least one user interaction with the vanishing point; and

moving the object with respect to the vanishing point to the second position within the digital image comprises moving the object with respect to the vanishing point at the modified location to the second position within the digital image.

17 . A system comprising:

at least one memory device comprising; and

at least one processor configured to cause the system to:

detect user input selecting a selectable option for performing a perspective-aware object move operation within a digital image;

determine, in response to the user input selecting the selectable option, to restrict movement of a first object depicted within the digital image to one or more movements along a reference line that extends from a vanishing point of the digital image to the first object;

detect one or more user interactions for moving a first the first object; and

modify the digital image in response to the one or more user interactions with the first object by:

moving the first object with respect to the vanishing point along the reference line from a first position to a second position within the digital image, a portion of the first object at the second position overlapping with a portion of a second object portrayed within the digital image; and

based on moving the first object with respect to the vanishing point of the digital image from the first position to the second position:

performing a perspective-based resizing of the first object within the digital image; and

occluding the portion of the first object with the portion of the second object based on the second position of the first object and an object depth of the second object.

18 . The system of claim 17 , further comprising:

determining, using a depth prediction neural network, an object depth for the first object at the second position; and

occluding the portion of the first object with the portion of the second object based on the second position of the first object and the object depth of the second object comprises occluding the portion of the first object with the portion of the second object based on the object depth of the first object and the object depth of the second object.

19 . The system of claim 18 , wherein occluding the portion of the first object with the portion of the second object based on the object depth of the first object and the object depth of the second object comprises occluding the portion of the first object with the portion of the second object based on determining that the object depth of the first object is greater than the object depth of the second object.

20 . The system of claim 17 , wherein the at least one processor is further configured to cause the system to:

move the first object with respect to the vanishing point from the second position to a third position within the digital image, an additional portion of the first object at the third position overlapping with an additional portion of the second object; and

occlude the additional portion of the second object with the additional portion of the first object based on an updated object depth of the first object at the third position and the object depth of the second object.

Assignments (1)

Continuity (9)

References Cited (250)