IP Library › Granted Patent US 11,138,776
Granted Patent B2
US 11,138,776 · App. 16/416,123 · Granted Oct 5, 2021

Adaptive image armatures with interactive composition guidance

Inventors: Radomir Mech (Mountain View, CA); Jose Ignacio Echevarria Vallespi (San Jose, CA); Jingwan Lu (Santa Clara, CA); Jianming Zhang (Campbell, CA); Jane Little E (Redwood City, CA)
Assignee: ADOBE INC.
G06T11/60G06T7/0002G06T7/32G06T7/344G06T7/70G06T7/97G06T2200/24G06T2207/20092G06T2207/30168H04N5/23222H04N5/232945
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,138,776
App. No.
16/416,123
Granted
Oct 5, 2021
Kind
B2
Abstract

Various methods and systems are provided for image-management operations that includes generating adaptive image armatures based on an alignment between composition lines of a reference armature and a position of an object in an image. In operation, a reference armature for an image is accessed. The reference armature includes a plurality of composition lines that define a frame of reference for image composition. An alignment map is determined using the reference armature. The alignment map includes alignment information that indicates alignment between the composition lines of the reference armature and the position of the object in the image. Based on the alignment map, an adaptive image armature is determined. The adaptive image armature includes a subset of the composition lines of the reference armature. The adaptive image armature is displayed.

Claims (63)

1. A computer-implemented method for generating adaptive image armatures, the method comprising:

accessing a reference armature for an image, wherein the reference armature comprises a plurality of composition lines that are a frame of reference for image composition:

determining an alignment map using the reference armature, wherein the alignment map comprises alignment information indicating an alignment between the plurality of composition lines of the reference armature and a position of an object in the image;

generating alignment scores for each composition line of the plurality of composition lines corresponding to the reference armature, wherein each alignment score for each composition line represents an extent to which the position of the object in the image aligns with the composition line in the reference armature overlaid on the alignment map;

determining an adaptive image armature comprising a subset of composition lines selected based on the alignment map and the alignment scores, wherein determining the adaptive image armature comprises automatically selecting the subset of composition lines from the plurality of composition lines of the reference armature, the alignment map and alignment scores support ranking the plurality of composition lines associated with automatically selecting the subset of composition lines; and

causing display of the adaptive image armature on the image.

2. The method of claim 1 , further comprising iteratively accessing a plurality of images having the object in a plurality of subsequent positions, wherein the reference armature is dynamically referenced to generate corresponding adaptive image armatures for the plurality of subsequent positions of the object.

3. The method of claim 1 , wherein determining the alignment map comprises:

generating a saliency map for the image, wherein the saliency map is a value-based representation of the position of the object in the image;

overlaying the reference armature comprising the plurality of composition lines on the saliency map; and

generating the alignment map comprising the alignment information indicating the alignment between the plurality of composition lines overlaid on the saliency map having the value-based representation of the position of the object.

4. The method of claim 1 , wherein determining the adaptive image armature using the alignment map comprises:

generating a ranking for each composition line of the plurality of composition lines, the ranking based on the generated alignment scores associated with the alignment map; and

based on the ranking for each composition line, automatically selecting the subset of composition lines from the plurality of composition lines for the adaptive image armature.

5. The method of claim 1 , wherein the adaptive image armature is caused to be displayed with visual guidance for fitting the object to the adaptive image armature based on an adjustment to the position of the object in the image relative to the displayed adaptive image armature.

6. The method of claim 5 , wherein the visual guidance is iteratively updated based on identifying changes in the position of the object to subsequent positions of the object.

7. The method of claim 1 , further comprising:

identifying a position change of the position of the object to a subsequent position;

based on the position change in the position, dynamically generating a subsequent iteration of the alignment map, wherein the subsequent iteration of the alignment map comprises updated alignment information indicating an alignment between the position of composition lines and the subsequent position;

updating the adaptive image armature, wherein the updated adaptive image armature comprises an updated subset of composition lines from the plurality of composition lines of reference armature; and

causing display of the updated adaptive image armature on the image.

8. One or more computer storage media storing computer-useable instructions that, when executed by one or more computing devices, causes the one or more computing devices to perform operations, the operations comprising:

determining an alignment map for an image, wherein the alignment map comprises alignment information indicating an alignment between a plurality of composition lines of a reference armature and an identified position of an object in an image, the reference armature comprises the plurality of composition lines that are a frame of reference for image composition;

generating alignment scores for each composition line of the plurality of composition lines corresponding to the reference armature, wherein each alignment score for each composition line represents an extent to which the position of the object in the image aligns with the composition line in the reference armature overlaid on the alignment map;

determining an adaptive image armature comprising a subset of composition lines selected based on the alignment map for the image and the alignment scores, wherein determining the adaptive image armature comprises automatically selecting the subset of composition lines from the plurality of composition lines of the reference armature, the alignment map and alignment scores support ranking the plurality of composition lines associated with automatically selecting the subset of composition lines; and

determining that the adaptive image armature of the image correlates with a model armature template, wherein the model armature template comprises a model subset of composition lines from the plurality of composition lines of the reference armature; and

based on the adaptive image armature correlating with the model armature template, determining that the image corresponds with the model armature template.

9. The one or more computer storage media of claim 8 , wherein determining the alignment map for the image comprises:

generating a saliency map for the image, wherein the saliency map is a value-based representation of the identified position of the object in the image;

overlaying the reference armature comprising the plurality of composition lines of the saliency map; and

generating the alignment map for the image comprising the alignment information indicating the alignment between the plurality of composition lines overlaid on the saliency map having the value-based representation of the identified position of the object.

10. The one or more computer storage media of claim 8 , wherein determining the adaptive image armature using the alignment map comprises:

generating a ranking for each composition line of the plurality of composition lines, the ranking based on the generated alignment scores associated with the alignment map; and

based on the ranking for each composition line, automatically selecting the subset of composition lines from the plurality of composition lines for adaptive image armature.

11. The one or more computer storage media of claim 8 , wherein the adaptive image armature is caused to be displayed with visual guidance for fitting the object to the adaptive image armature based on an adjustment to the identified position of the object in the image relative to the adaptive image armature.

12. The one or more computer storage media of claim 11 , wherein the visual guidance is iteratively updated based on identifying changes in the identified position of the object to subsequent positions of the object.

13. The one or more computer storage media of claim 8 , further comprising:

in response to receiving a selection of a composition-style, the composition-style including at least two model armature templates, identifying at least one image in a plurality of images whose composition matches one of the at least two model armature templates; and

causing the display of the matching adaptive image armature on the at least one image whose composition matches either of the at least two model armature templates.

14. A system for utilizing adaptive image armatures, the system comprising:

one or more processors; and

one or more computer storage media storing computer-useable instructions that, when used by one or more processors, cause the one or more processors to execute:

means for accessing a reference armature for an image, wherein the reference armature comprises a plurality of composition lines that are a frame of reference of image composition;

means for determining an alignment map using the reference armature, wherein the alignment map comprises alignment information indicating an alignment between the plurality of composition lines of the reference armature and a position of the object in the image;

means for generating alignment scores for each composition line of the plurality of composition lines corresponding to the reference armature, wherein each alignment score for each composition line represents an extent to which the position of the object in the image aligns with the composition line in the reference armature overlaid on the alignment map;

means for determining an adaptive image armature comprising a subset of composition lines selected based on the alignment map and the alignment scores, wherein determining the adaptive image armature comprises automatically selecting the subset of composition lines from the plurality of composition lines of the reference armature, the alignment map and alignment scores that support ranking the plurality of composition lines associated with automatically selecting the subset of composition lines; and

means for causing display of the adaptive image armature on the image.

15. The system of claim 14 , further comprising:

means for determining an alignment map for a stored image, wherein the alignment map for the stored image comprises alignment information indicating an alignment between the plurality of composition lines of the reference armature and an identified position of an identified object in the stored image;

means for determining an adaptive armature for the stored image based on the alignment map for the stored image, wherein the adaptive image armature for the stored image comprises a second subset of composition lines from the plurality of composition lines of the reference armature;

means for determining that the adaptive image armature for the stored image correlates with a model armature template, wherein the model armature template comprises a model subset of composition lines from the plurality of composition lines of the reference armature; and

means on the adaptive image armature for the stored image correlating with the model armature template, determining that the stored image corresponds with the model armature template.

16. The system of claim 15 , wherein, in response to receiving a selection of a composition-style, the composition-style including at least two model armature templates, identifying at least one image in a plurality of images whose composition matches one of the at least two model armature templates; and

causing display of the matching adaptive image armature on the at least one image whose composition matches either of the at least two model armature templates.

17. The system of claim 14 , wherein determining the alignment map comprises:

generating a saliency map for the image, wherein the saliency map is a value-based on representation of the position of the object in the image;

overlaying the reference armature comprising the plurality of composition lines on the saliency map; and

generating the alignment map comprising the alignment information indicating alignment between the plurality of composition lines overlaid on the saliency map having the value-based representation of the position of the object.

18. The system of claim 14 , wherein determining the adaptive image armature using the alignment map comprises:

generating a ranking for each composition line of the plurality of composition lines, the ranking based on the generated alignment scores associated with the alignment map; and

based on the ranking for each composition lines, automatically selecting the subset of composition lines from the plurality of composition lines for the adaptive image armature.

19. The system of claim 14 , wherein the adaptive image armature is caused to be displayed with visual guidance for fitting the object to the adaptive image armature based on an adjustment to the position of the object in the image relative to the displayed adaptive image armature.

20. The system of claim 19 , wherein the visual guidance is iteratively updated based on the identifying changes in the position of the object to subsequent positions of the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2019
From: MECH, RADOMIR; ECHEVARRIA VALLESPI, JOSE IGNACIO; LU, JINGWAN; ZHANG, JIANMING; E, JANE LITTLE
To: ADOBE INC.
Reel/Frame 050905/0809 →
Continuity (1)
Related Publication 20200364914A1 · Nov 19, 2020