Virtual lens simulation for video and photo cropping

1. A method for simulating a virtual lens when applying a crop or zoom effect to an input image, the method comprising:

receiving, by a processor, the input image, the input image having a first field of view of a scene, the input image depicting the scene with an input lens distortion centered in the first field of view, the input lens distortion caused by lens characteristics of a lens used to capture the input image, the lens centered at the first field of view during capture of the input image;

identifying, by the processor, the input lens distortion within the input image;

obtaining, by the processor, a selection of a sub-frame representing a portion of the input image, the sub-frame having a second field of view of the scene smaller than the first field of view;

determining, by the processor, an input lens distortion effect present in the sub-frame based on the input lens distortion centered in the first field of view, a location of the sub-frame within the first field of view, and a size of the second field of view; and

generating, by the processor, an output image based on the input lens distortion and the input lens distortion effect present in the sub-frame, the output image including the sub-frame remapped from the input lens distortion centered in the first field of view to the input lens distortion centered in the second field of view such that a portion of the scene depicted in the sub-frame appears to have been captured using the lens centered at the second field of view.

2. The method of claim 1 , wherein the input lens distortion caused by the lens characteristics causes straight lines in the scene to appear as curved lines in at least a portion of the input image.

3. The method of claim 1 , wherein the input lens distortion comprises a distortion produced by a conventional lens, a wide angle lens, a fisheye lens, a hemispherical lens, a zoom lens, or a flat lens.

4. The method of claim 1 , wherein obtaining the selection of the sub-frame representing the portion of the input image comprises:

automatically identifying the sub-frame based on metadata associated with the input image or image characteristics of the input image;

automatically obtaining a sub-frame center location, a sub-frame size, and a scaling factor for transforming the input image; and

applying the crop or zoom effect applied to the input image based on the sub-frame center location, the sub-frame size, and the scaling factor to generate the sub-frame.

5. The method of claim 1 , wherein obtaining the selection of the sub-frame representing the portion of the input image further comprises:

receiving a manual selection of the sub-frame from post-processing tools.

6. The method of claim 1 , wherein generating the output image comprises:

obtaining metadata associated with the input image, the metadata specifying the lens characteristics of the lens used to capture the input image; and

processing the sub-frame using the lens characteristics specified in the metadata.

7. The method of claim 1 , wherein generating the output image comprises:

applying an inverse input lens distortion function to the sub-frame to remove the input lens distortion effect present in the sub-frame to generate a rectilinear image, wherein straight lines in the scene within the sub-frame appear as straight lines in the rectilinear image; and

applying an input lens distortion function to apply the input lens distortion to the rectilinear image.

8. The method of claim 1 , wherein the sub-frame is remapped from the input lens distortion centered in the first field of view to the input lens distortion centered in the second field of view by applying a direct function to remap the sub-frame without an intermediate step of removing the input lens distortion effect present in the sub-frame.

9. A non-transitory computer-readable storage medium storing instructions for simulating a virtual lens when applying a crop or zoom effect to an input image, the instructions when executed by one or more processors causing the one or more processors to perform steps including:

receiving the input image, the input image having a first field of view of a scene, the input image depicting the scene with an input lens distortion centered in the first field of view, the input lens distortion caused by lens characteristics of a lens used to capture the input image, the lens centered at the first field of view during capture of the input image;

identifying the input lens distortion within the input image;

obtaining a selection of a sub-frame representing a portion of the input image, the sub-frame having a second field of view of the scene smaller than the first field of view;

determining an input lens distortion effect present in the sub-frame based on the input lens distortion centered in the first field of view, a location of the sub-frame within the first field of view, and a size of the second field of view; and

generating an output image based on the input lens distortion and the input lens distortion effect present in the sub-frame, the output image including the sub-frame remapped from the input lens distortion centered in the first field of view to the input lens distortion centered in the second field of view such that a portion of the scene depicted in the sub-frame appears to have been captured using the lens centered at the second field of view.

10. The non-transitory computer-readable storage medium of claim 9 , wherein the input lens distortion caused by the lens characteristics causes straight lines in the scene to appear as curved lines in at least a portion of the input image.

11. The non-transitory computer-readable storage medium of claim 9 , wherein the input lens distortion comprises a distortion produced by a conventional lens, a wide angle lens, a fisheye lens, a hemispherical lens, a zoom lens, or a flat lens.

12. The non-transitory computer-readable storage medium of claim 9 , wherein obtaining the selection of the sub-frame representing the portion of the input image comprises:

automatically identifying the sub-frame based on metadata associated with the input image or image characteristics of the input image;

automatically obtaining a sub-frame center location, a sub-frame size, and a scaling factor for transforming the input image; and

applying the crop or zoom effect applied to the input image based on the sub-frame center location, the sub-frame size, and the scaling factor to generate the sub-frame.

13. The non-transitory computer-readable storage medium of claim 9 , wherein obtaining the selection of the sub-frame representing the portion of the input image further comprises:

receiving a manual selection of the sub-frame from post-processing tools.

14. The non-transitory computer-readable storage medium of claim 9 , wherein generating the output image comprises:

obtaining metadata associated with the input image, the metadata specifying the lens characteristics of the lens used to capture the input image; and

processing the sub-frame using the lens characteristics specified in the metadata.

15. The non-transitory computer-readable storage medium of claim 9 , wherein generating the output image comprises:

applying an inverse input lens distortion function to the sub-frame to remove the input lens distortion effect present in the sub-frame to generate a rectilinear image, wherein straight lines in the scene within the sub-frame appear as straight lines in the rectilinear image; and

applying an input lens distortion function to apply the input lens distortion to the rectilinear image.

16. The non-transitory computer-readable storage medium of claim 9 , wherein the sub-frame is remapped from the input lens distortion centered in the first field of view to the input lens distortion centered in the second field of view by applying a direct function to remap the sub-frame without an intermediate step of removing the input lens distortion effect present in the sub-frame.

17. A system for simulating a virtual lens when applying a crop or zoom effect to an input image, the system comprising:

one or more processors; and

a non-transitory computer-readable storage medium storing instructions for simulating the virtual lens when applying a crop or zoom effect to the input image, the instructions when executed by one or more processors causing the one or more processors to perform steps including:

receiving the input image, the input image having a first field of view of a scene, the input image depicting the scene with an input lens distortion centered in the first field of view, the input lens distortion caused by lens characteristics of a lens used to capture the input image, the lens centered at the first field of view during capture of the input image;

identifying the input lens distortion within the input image;

obtaining a selection of a sub-frame representing a portion of the input image, the sub-frame having a second field of view of the scene smaller than the first field of view;

determining an input lens distortion effect present in the sub-frame based on the input lens distortion centered in the first field of view, a location of the sub-frame within the first field of view, and a size of the second field of view; and

generating an output image based on the input lens distortion and the input lens distortion effect present in the sub-frame, the output image including the sub-frame remapped from the input lens distortion centered in the first field of view to the input lens distortion centered in the second field of view such that a portion of the scene depicted in the sub-frame appears to have been captured using the lens centered at the second field of view.

18. The system of claim 17 , wherein the input lens distortion caused by the lens characteristics causes straight lines in the scene to appear as curved lines in at least a portion of the input image.

19. The system of claim 17 , wherein the input lens distortion comprises a distortion produced by a conventional lens, a wide angle lens, a fisheye lens, a hemispherical lens, a zoom lens, or a flat lens.

20. The system of claim 17 , wherein obtaining the selection of the sub-frame representing the portion of the input image comprises:

automatically identifying the sub-frame based on metadata associated with the input image or image characteristics of the input image;

automatically obtaining a sub-frame center location, a sub-frame size, and a scaling factor for transforming the input image; and

applying the crop or zoom effect applied to the input image based on the sub-frame center location, the sub-frame size, and the scaling factor to generate the sub-frame.