Scaled perspective zoom on resource constrained devices
A dolly zoom effect can be applied to one or more images captured via a resource-constrained device (e.g., a mobile smartphone) by manipulating the size of a target feature while the background in the one or more images changes due to physical movement of the resource-constrained device. The target feature can be detected using facial recognition or shape detection techniques. The target feature can be resized before the size is manipulated as the background changes (e.g., changes perspective).
1. A method comprising:
generating an image using an image sensor of a user device;
receiving, at the user device, a user selection indicating a depiction of a physical object in the image;
generating a zoom video sequence by maintaining a scaling of only the depiction of the physical object in the zoom video sequence as the user device moves closer to or away from the physical object;
receiving, through a touchscreen of the user device, an instruction to change the depiction of the physical object from an initial size to a size that is manipulated by increased scaling in the zoom video sequence; and
storing the zoom video sequence on the user device.
2. The method of claim 1 , wherein the image comprises a first region including the depiction of the physical object, and a second region including a remaining area of the image excluding the first region.
3. The method of claim 2 , wherein generating the zoom video sequence comprises:
maintaining the scaling of the first region as the user device moves closer to or away from the physical object; and
adjusting the scaling of the second region as the user device moves closes to or away from the physical object.
4. The method of claim 2 , further comprising:
resizing the first region;
maintaining the scaling of the resized first region as the user device moves closer to or away from the physical object; and
adjusting the scaling of the second region as the user device moves closes to or away from the physical object.
5. The method of claim 1 , further comprising:
generating, using a convolutional neural network, image feature areas for different image features in the image;
identifying, on the user device, a target image feature from one of the image feature areas, the target image feature corresponding to the physical object depicted in the image; and
resizing the target image feature,
wherein generating the zoom video sequence comprises:
maintaining a scaling of the resized target image feature in the zoom video sequence without maintaining the scaling of image features areas that are not the target image feature such that the resized target image feature covers the depiction of the physical object in the zoom video sequence as the user device moves closer to or away from the physical object.
6. The method of claim 1 , wherein the physical object is one of a human face, a segment of a human body, or a shape in the image.
7. The method of claim 1 , further comprising:
stabilizing the depiction of the physical object in the zoom video sequence such that the depiction of the physical object remains in an initial area in the zoom video sequence.
8. The method of claim 1 , wherein the image is from a live video feed generated by a front-facing camera of the user device, the user device having the front-facing camera and a rear-facing camera.
9. The method of claim 1 , further comprising:
generating an ephemeral message comprising the zoom video sequence; and
transmitting the ephemeral message to a network server.
10. A user device comprising:
one or more processors;
an image sensor; and
a memory storing instructions that, when executed by the one or more processors, cause the user device to perform operations comprising:
generating an image using the image sensor;
receiving, at the user device, a user selection indicating a depiction of a physical object in the image;
generating a zoom video sequence by maintaining a scaling of only the depiction of the physical object in the zoom video sequence as the user device moves closer to or away from the physical object;
receiving, through a touchscreen of the user device, an instruction to change the depiction of the physical object from an initial size to a size that is manipulated by increased scaling in the zoom video sequence; and
storing the zoom video sequence on the user device.
11. The user device of claim 10 , wherein the image comprises a first region including the depiction of the physical object, and a second region including a remaining area of the image excluding the first region.
12. The user device of claim 11 , wherein generating the zoom video sequence comprises:
maintaining the scaling of the first region as the user device moves closer to or away from the physical object; and
adjusting the scaling of the second region as the user device moves closes to or away from the physical object.
13. The user device of claim 11 , wherein the operations further comprise:
resizing the first region;
maintaining the scaling of the resized first region as the user device moves closer to or away from the physical object; and
adjusting the scaling of the second region as the user device moves closes to or away from the physical object.
14. The user device of claim 10 , wherein the operations further comprise:
generating, using a convolutional neural network, image feature areas for different image features in the image;
identifying, on the user device, a target image feature from one of the image feature areas, the target image feature corresponding to the physical object depicted in the image; and
resizing the target image feature,
wherein generating the zoom video sequence comprises:
maintaining a scaling of the resized target image feature in the zoom video sequence without maintaining the scaling of image features areas that are not the target image feature such that the resized target image feature covers the depiction of the physical object in the zoom video sequence as the user device moves closer to or away from the physical object.
15. The user device of claim 10 , wherein the operations further comprise:
receiving, through a touchscreen of the user device, an instruction to change the depiction of the physical object from an initial size to a size that is manipulated by increased scaling in the zoom video sequence.
16. The user device of claim 10 , wherein the physical object is one of a human face, a segment of a human body, or a shape in the image.
17. The user device of claim 10 , further comprising:
stabilizing the depiction of the physical object in the zoom video sequence such that the depiction of the physical object remains in an initial area in the zoom video sequence.
18. The user device of claim 10 , wherein the image is from a live video feed generated by the image sensor, the image sensor comprising a front-facing camera of the user device,
wherein the operations further comprise:
generating an ephemeral message comprising the zoom video sequence; and
transmitting the ephemeral message to a network server.
19. A non-transitory machine-readable storage device embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
generating an image using an image sensor of a user device;
receiving, at the user device, a user selection indicating a depiction of a physical object in the image;
generating a zoom video sequence by maintaining a scaling of only the depiction of the physical object in the zoom video sequence as the user device moves closer to or away from the physical object;
receiving, through a touchscreen of the user device, an instruction to change the depiction of the physical object from an initial size to a size that is manipulated by increased scaling in the zoom video sequence; and
storing the zoom video sequence on the user device.