Self-portrait enhancement techniques
View Patent ↗Systems and approaches are provided for optimizing self-portraiture. The background of the self-portrait can be enhanced by image registration or stitching techniques of images captured using one or more conventional cameras. Multiple standard resolution images can be stitched together to generate a panoramic or a composite image of a higher resolution. Foreground elements, such as one or more representations of users, can also be enhanced in various ways. The representations of the users can be composited to exclude undesirable elements, such as image data of one of the users extending her arm to capture the self-portrait. An ideal pose of the users can automatically be selected and other image enhancements, such as histogram optimization, brightness and contrast optimization, color-cast correction, or reduction or removal of noise, can automatically be performed to minimize user effort in capturing self-portraits.
1. A computer-implemented method, comprising:
obtaining a plurality of images;
determining one or more background elements from each of the plurality of images;
determining a plurality of foreground elements from each of the plurality of images;
generating a background image comprising a first composite of at least a portion of the one or more background elements;
determining a score of each of the plurality of foreground elements according to one or more visual characteristics;
obtaining a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and
generating a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements.
2. The computer-implemented method of claim 1 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.
3. The computer-implemented method of claim 1 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.
4. The computer-implemented method of claim 1 , further comprising:
obtaining one or more selections relating to display of the background image,
wherein the self-portrait image is rendered based at least in part upon the one or more selections.
5. The computer-implemented method of claim 4 , wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image.
6. A computing device, comprising:
one or more processors;
one or more cameras;
memory including instructions that, when executed by the one or more processors, cause the computing device to:
obtain a plurality of images;
determine one or more background elements from each of the plurality of images;
determine a plurality of foreground elements from each of the plurality of images;
generate a background image comprising a first composite of at least a portion of the one or more background elements;
determine a score of each of the plurality of foreground elements according to one or more visual characteristics;
obtaining a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and
generate a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements.
7. The computing device of claim 6 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.
8. The computing device of claim 6 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.
9. The computing device of claim 6 , wherein the instructions, when executed to generate the background image, further enable the computing device to:
generate a preliminary background image by stitching together at least the portion of the one or more background elements; and
fill in at least a portion of the preliminary background image using an inpainting process.
10. The computing device of claim 6 , wherein the instructions, when executed by the one or more processors, further enable the computing device to:
obtain one or more selections relating to display of the background image,
wherein the self-portrait image is rendered based at least in part upon the one or more selections.
11. The computing device of claim 10 , wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image.
12. A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to:
obtain a plurality of images;
determine one or more background elements from each of the plurality of images;
determine a plurality of foreground elements from each of the plurality of images;
generate a background image comprising a first composite of at least a portion of the one or more background elements;
determine a score of each of the plurality of foreground elements according to one or more visual characteristics;
obtain a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and
generate a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements.
13. The non-transitory computer-readable storage medium of claim 12 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.
14. The non-transitory computer-readable storage medium of claim 12 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.
15. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions, when executed to generate the background image, further cause the processor to:
generate a preliminary background image by stitching together at least the portion of the one or more background elements; and
fill in at least a portion of the preliminary background image using an inpainting process.
16. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions, when executed by the processor, further cause the processor to:
obtain one or more selections relating to display of the background image, wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image; and
wherein the self-portrait image is rendered based at least in part upon the one or more selections.