IP Library Granted Patent US 10,282,815
Granted Patent B2
US 10,282,815 · App. 15/377,875 · Granted May 7, 2019

Environmental map generation from a digital image

Inventors: Xue Bai (Seattle, WA); Elya Shechtman (Seattle, WA); Sylvain Philippe Paris (Jamaica Plain, MA)
Assignee: Adobe Inc.
G06T3/20G06T3/40G06T5/005G06T7/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,282,815
App. No.
15/377,875
Granted
May 7, 2019
Kind
B2
Abstract

Environmental map generation techniques and systems are described. A digital image is scaled to achieve a target aspect ratio using a content aware scaling technique. A canvas is generated that is dimensionally larger than the scaled digital image and the scaled digital image is inserted within the canvas thereby resulting in an unfilled portion of the canvas. An initially filled canvas is then generated by filling the unfilled portion using a content aware fill technique based on the inserted digital image. A plurality of polar coordinate canvases is formed by transforming original coordinates of the canvas into polar coordinates. The unfilled portions of the polar coordinate canvases are filled using a content-aware fill technique that is initialized based on the initially filled canvas. An environmental map of the digital image is generated by combining a plurality of original coordinate canvas portions formed from the polar coordinate canvases.

Claims (43)

1. In a digital medium environment to generate an environmental map of a digital image, a method implemented by a computing device, the method comprising:

receiving, by the computing device, a canvas having a digital image inserted therein and an unfilled portion;

generating, by the computing device, an initially filled canvas by filling the unfilled portion using a content aware fill technique based on the inserted digital image;

forming, by the computing device, a plurality of polar coordinate canvases by transforming original coordinates of the canvas into polar coordinates;

filling, by the computing device, the unfilled portion of respective ones of the plurality of polar coordinate canvases using a content-aware fill technique that is initialized based on the initially filled canvas;

forming, by the computing device, a plurality of original coordinate canvas portions by transforming the polar coordinates of the filled plurality of polar coordinate canvases back into original coordinates; and

generating, by the computing device, an environmental map of the digital image by combining the plurality of original coordinate canvas portions.

2. The method as described in claim 1 , further comprising scaling, by the computing device, the digital image to achieve a target aspect ratio, the scaling based on a content aware scaling technique and wherein the scaled digital image is inserted into the canvas.

3. The method as described in claim 2 , wherein the content aware scaling technique is configured such that scaling of portion of the image that is identified as salient is minimized with respect to another portion of the image that is not identified as important.

4. The method as described in claim 2 , wherein the content aware scaling technique is performed using seam filling.

5. The method as described in claim 2 , further comprising inserting, by the computing device, the scaled digital image into the canvas, the canvas is dimensionally larger than the scaled image thereby resulting in the unfilled portion of the canvas.

6. The method as described in claim 5 , wherein the inserting includes inserting the digital image at a center of the canvas that is twice as large, dimensionally, as the digital image.

7. The method as described in claim 1 , wherein the filling further comprises reflecting a portion of the inserted digital image in the canvas.

8. The method as described in claim 7 , wherein the reflecting fills horizontal unfilled portions of the canvas and the filling using the content aware fill technique fills vertical unfilled portions of the canvas.

9. The method as described in claim 1 , wherein the target aspect ratio is two to one, width to height.

10. The method as described in claim 1 , further comprising inserting an object within the digital image as having an appearance that is based at least partially on the environmental map and outputting the digital image as having the object.

11. The method as described in claim 1 , wherein the original coordinates are defined based on a vertical axis and a horizontal axis.

12. In a digital medium environment to generate an environmental map of a digital image, an environmental mapping system comprising:

a content aware scaling module implemented at least partially in hardware of a computing device to scale the digital image to achieve a target aspect ratio using a content aware scaling technique;

a canvas generation module implemented at least partially in hardware of a computing device to generate a canvas that is dimensionally larger than the scaled digital image and insert the scaled digital image within the canvas thereby resulting in an unfilled portion of the canvas; and

an environmental mapping module implemented at least partially in hardware of a computing device to generate the environmental map of the digital image based on the canvas, the environmental map encoding light information of the digital image.

13. The system as described in claim 12 , wherein the content aware scaling technique is configured such that:

scaling of portion of the image that is identified as important is minimized with respect to another portion of the image that is not identified as important; or

using seam filling.

14. The system as described in claim 12 , wherein the environmental map encodes the light information of the digital image as a 360-degree panoramic image.

15. The system as described in claim 12 , wherein the environmental mapping module includes:

an initial content aware fill module implemented at least partially in hardware of a computing device to generate an initially filled canvas by filling the unfilled portion using a content aware fill technique based on the inserted digital image;

a polar coordinate canvas formation module implemented at least partially in hardware of a computing device to form a plurality of polar coordinate canvases by transforming original coordinates of the canvas into polar coordinates;

a polar coordinate content aware fill module implemented at least partially in hardware of a computing device to fill the unfilled portion of respective ones of the plurality of polar coordinate canvases using a content-aware fill technique that is initialized based on the initially filled canvas;

an original coordinate transformation module implemented at least partially in hardware to form a plurality of original coordinate canvas portions by transforming the polar coordinates of the filled plurality of polar coordinate canvases back into original coordinates; and

a map generation module implemented at least partially in hardware of a computing device to generate an environmental map of the digital image by combining the plurality of original coordinate canvas portions.

16. In a digital medium environment to generate an environmental map of a digital image, a system comprising:

means for scaling the digital image to achieve a target aspect ratio, the scaling based on a content aware scaling;

means for generating a canvas that is dimensionally larger than the scaled digital image that includes the scaled digital image and an unfilled portion of the canvas;

means for generating an initially filled canvas by filling the unfilled portion using a content aware fill technique based on the inserted digital image;

means for forming a plurality of polar coordinate canvases by transforming original coordinates of the canvas into polar coordinates;

means for filling the unfilled portion of respective ones of the plurality of polar coordinate canvases using a content-aware fill technique that is initialized based on the initially filled canvas;

means for forming a plurality of original coordinate canvas portions by transforming the polar coordinates of the filled plurality of polar coordinate canvases back into original coordinates; and

means for generating an environmental map of the digital image by combining the plurality of original coordinate canvas portions.

17. The system as described in claim 16 , wherein the target aspect ratio is two to one, width to height.

18. The system as described in claim 16 , further comprising means for reflecting portion of the digital image in the canvas.

19. The system as described in claim 16 , wherein the canvas is twice as large, dimensionally, as the digital image.

20. The system as described in claim 16 , wherein the original coordinates are defined based on a vertical axis and a horizontal axis.

Assignments (2)
CHANGE OF NAME Recorded Jan 21, 2019
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 048103/0226 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2016
From: BAI, XUE; SHECHTMAN, ELYA; PARIS, SYLVAIN PHILIPPE
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 040696/0410 →
Continuity (2)
Provisional Application 62414453 · Oct 28, 2016
Related Publication 20180122044A1 · May 3, 2018