Edge-based electronic image stabilization
View Patent ↗An image stabilization system applies a “pinned-edge” or “soft pinned edge” image stabilization technique to digital video to compensate for unwanted camera motion in a captured video. In these stabilization techniques, a warping function is applied to an image frame to achieve a non-uniform shifting of depicted points in the image frame such that a reference point is stabilized with respect to a reference frame. In pinned-edge image stabilization, the final stabilized output video has the same dimensions as the pre-stabilized input video captured by the image sensor. In soft pinned-edge image stabilization, the pre-stabilized input video has slightly larger dimensions than the stabilized output video but these larger dimensions are still reduced compared to traditional electronic image stabilization.
1. A method for stabilizing a digital video, the method comprising:
receiving a reference video frame depicting a scene, the reference video frame depicting a reference point represented by one or more pixels at a reference pixel location;
receiving a second frame depicting at least a portion of the scene, wherein movement of the reference point between the first and second frame results in the reference point being depicted at a different pixel location in the second frame;
generating a stabilized second frame by non-uniformly shifting at least some points depicted in the second frame by an amount based on distances of the points from an edge of the second frame such that a greater shift is applied to non-edge points than edge points; and
generating a stabilized video comprising a sequence of frames including the reference frame and the stabilized second frame, wherein depicted motion of the reference point between the first frame and the stabilized second frame is reduced relative to depicted motion of the reference point between the first frame and the second frame.
2. The method of claim 1 , wherein non-uniformly shifting the at least some points comprises:
applying zero shift to edge points depicted in the second frame.
3. The method of claim 1 , wherein non-uniformly shifting the at least some points comprises:
applying a non-zero shift to edge points depicted in the second frame.
4. The method of claim 1 , wherein non-uniformly shifting the at least some points comprises applying a shift as a linear function of the distance.
5. The method of claim 1 , wherein non-uniformly shifting the at least some points comprises applying a shift as a non-linear function of the distance.
6. The method of claim 1 , wherein a field of view of the stabilized second frame has equivalent dimensions a field of view of the second frame as originally captured.
7. The method of claim 1 , wherein a field of view of the stabilized second frame has smaller dimensions than a field of view of the second frame as originally captured.
8. A non-transitory computer-readable storage medium storing computer-executable instructions for stabilizing a digital video, the computer-executable instructions when executed by one more processors cause the one or more processors to perform steps including:
receiving a reference video frame depicting a scene, the reference video frame depicting a reference point represented by one or more pixels at a reference pixel location;
receiving a second frame depicting at least a portion of the scene, wherein movement of the reference point between the first and second frame results in the reference point being depicted at a different pixel location in the second frame;
generating a stabilized second frame by non-uniformly shifting at least some points depicted in the second frame by an amount based on distances of the points from an edge of the second frame such that a greater shift is applied to non-edge points than edge points; and
generating a stabilized video comprising a sequence of frames including the reference frame and the stabilized second frame, wherein depicted motion of the reference point between the first frame and the stabilized second frame is reduced relative to depicted motion of the reference point between the first frame and the second frame.
9. The non-transitory computer-readable storage medium of claim 8 , wherein non-uniformly shifting the at least some points comprises:
applying zero shift to edge points depicted in the second frame.
10. The non-transitory computer-readable storage medium of claim 8 , wherein non-uniformly shifting the at least some points comprises:
applying a non-zero shift to edge points depicted in the second frame.
11. The non-transitory computer-readable storage medium of claim 8 , wherein non-uniformly shifting the at least some points comprises applying a shift as a linear function of the distance.
12. The non-transitory computer-readable storage medium of claim 8 , wherein non-uniformly shifting the at least some points comprises applying a shift as a non-linear function of the distance.
13. The non-transitory computer-readable storage medium of claim 8 , wherein a field of view of the stabilized second frame has equivalent dimensions a field of view of the second frame as originally captured.
14. The non-transitory computer-readable storage medium of claim 8 , wherein a field of view of the stabilized second frame has smaller dimensions than a field of view of the second frame as originally captured.
15. A camera system for capturing and stabilizing digital video, the camera system comprising:
a camera system for capturing digital video;
one or more processors; and
a computer-readable storage medium storing computer-executable instructions for stabilizing the digital video, the computer-executable instructions when executed by the one more processors cause the one or more processors to performs steps including:
receiving a reference video frame depicting a scene, the reference video frame depicting a reference point represented by one or more pixels at a reference pixel location;
receiving a second frame depicting at least a portion of the scene, wherein movement of the reference point between the first and second frame results in the reference point being depicted at a different pixel location in the second frame;
generating a stabilized second frame by non-uniformly shifting at least some points depicted in the second frame by an amount based on distances of the points from an edge of the second frame such that a greater shift is applied to non-edge points than edge points; and
generating a stabilized video comprising a sequence of frames including the reference frame and the stabilized second frame, wherein depicted motion of the reference point between the first frame and the stabilized second frame is reduced relative to depicted motion of the reference point between the first frame and the second frame.
16. The camera system of claim 15 , wherein non-uniformly shifting the at least some points comprises:
applying zero shift to edge points depicted in the second frame.
17. The camera system of claim 15 , wherein non-uniformly shifting the at least some points comprises:
applying a non-zero shift to edge points depicted in the second frame.
18. The camera system of claim 15 , wherein non-uniformly shifting the at least some points comprises applying a shift as a linear function of the distance.
19. The camera system of claim 15 , wherein non-uniformly shifting the at least some points comprises applying a shift as a non-linear function of the distance.
20. The camera system of claim 15 , wherein a field of view of the stabilized second frame has equivalent dimensions a field of view of the second frame as originally captured.
21. The camera system of claim 15 , wherein a field of view of the stabilized second frame has smaller dimensions than a field of view of the second frame as originally captured.