Granted Patent B2

US 12,538,029 · App. 18/523,003 · Granted Jan 27, 2026

Inventor: Stepan Moskovchenko (Belmont, CA)

Assignee: Skydio, Inc.

H04N23/69G06T3/40G06V20/17H04N23/683

Abstract

Claims (56)

1 . A method of operating an unmanned aerial vehicle (UAV), the method comprising:

capturing a raw video stream input using one or more camera sensors of the UAV;

determining a zoom level for the raw video stream input; and

responsive to determining the zoom level being less than or equal to 3×,

zooming the raw video stream input to a field of view at a first zoom level of less than or equal to 3× to generate a zoomed video stream; and

applying electronic image stabilization to the zoomed video stream to generate a zoomed and stabilized video stream; and

responsive to determining the zoom level being greater than 3×,

first zooming the raw video stream input to a first field of view at 3× to generate a 3× zoomed video stream;

second zooming the 3× zoomed video stream to a second field of view at a second zoom level to generate a fully zoomed video stream at greater than 3×; and

applying electronic image stabilization to the fully zoomed video stream to generate a zoomed and stabilized video stream.

2 . The method of claim 1 further comprising establishing a communications link between the UAV and a ground station.

3 . The method of claim 2 further comprising receiving, by the UAV via the communications link, a signal encoding data representative of the zoom level from the ground station.

4 . The method of claim 3 , wherein determining the zoom level for the raw video stream input comprises determining the zoom level according to the data representative of the zoom level.

5 . The method of claim 1 further comprising encoding the zoomed and stabilized video stream to generate a zoomed, stabilized and encoded video stream.

6 . The method of claim 5 further comprising scaling the zoomed and stabilized video stream to generate an unencoded, zoomed, scaled and stabilized video stream.

7 . The method of claim 6 , wherein encoding the zoomed and stabilized video stream comprises encoding the unencoded, zoomed, scaled and stabilized video stream.

8 . The method of claim 5 further comprising transmitting the zoomed, stabilized and encoded video stream to a ground station communicably coupled to the UAV via a communications link.

9 . The method of claim 1 further comprising:

detecting an acceleration of the one or more camera sensors; and

computing a vector representative of the acceleration of the one or more camera sensors.

10 . The method of claim 9 , wherein applying electronic image stabilization to the zoomed video stream comprises:

producing a stabilization viewport inside a perimeter of the field of view, wherein the stabilization viewport and the perimeter of the field of view are separated by a margin;

computing an opposing vector counteracting the vector; and

moving the stabilization viewport within the margin according to the opposing vector to generate the zoomed and stabilized video stream.

11 . The method of claim 10 , wherein producing the stabilization viewport comprises cropping the field of view by a predetermined percentage defining the margin.

12 . The method of claim 10 further comprising generating mesh data defining coordinates of the stabilization viewport within the field of view according to the vector and at least one property of a lens of the one or more camera sensors, wherein moving the stabilization viewport comprises moving the stabilization viewport further according to the mesh data.

13 . The method of claim 12 , wherein producing the stabilization viewport comprises generating a texture map for the stabilization viewport according to the mesh data, and wherein moving the stabilization viewport comprises moving the stabilization viewport further according to the texture map.

14 . The method of claim 10 further comprising transmitting the zoomed and stabilized video stream as the stabilization viewport to a ground station communicably coupled to the UAV via a communications link.

15 . The method of claim 9 , wherein applying electronic image stabilization to the fully zoomed video stream comprises:

producing a stabilization viewport inside a perimeter of the second field of view, wherein the stabilization viewport and the perimeter of the second field of view are separated by a margin;

computing an opposing vector counteracting the vector; and

moving the stabilization viewport within the margin according to the opposing vector to generate the zoomed and stabilized video stream.

16 . The method of claim 15 , wherein producing the stabilization viewport comprises cropping the second field of view by a predetermined percentage defining the margin.

17 . The method of claim 15 further comprising generating mesh data defining coordinates of the stabilization viewport within the second field of view according to the vector and at least one property of a lens of the one or more camera sensors, wherein moving the stabilization viewport comprises moving the stabilization viewport further according to the mesh data.

18 . The method of claim 17 , wherein producing the stabilization viewport comprises generating a texture map for the stabilization viewport according to the mesh data, and wherein moving the stabilization viewport comprises moving the stabilization viewport further according to the texture map.

19 . The method of claim 15 further comprising transmitting the zoomed and stabilized video stream as the stabilization viewport to a ground station communicably coupled to the UAV via a communications link.

20 . A system for operating an unmanned aerial vehicle (UAV), the system comprising:

one or more camera sensors for capturing a raw video stream input; and

at least one computing device disposed onboard the UAV and operably coupled to the one or more camera sensors, the at least one computing device configured to:

determine a zoom level for the raw video stream input;

responsive to determining the zoom level being less than or equal to 3×,

zoom the raw video stream input to a field of view at a first zoom level of less than or equal to 3× to generate a zoomed video stream; and

apply electronic image stabilization to the zoomed video stream to generate a zoomed and stabilized video stream; and

responsive to determining the zoom level being greater than 3×,

first zoom the raw video stream input to a first field of view at 3× to generate a 3× zoomed video stream;

second zoom the 3× zoomed video stream to a second field of view at a second zoom level to generate a fully zoomed video stream at greater than 3×; and

apply electronic image stabilization to the fully zoomed video stream to generate a zoomed and stabilized video stream.

21 . One or more non-transitory computer readable media having stored thereon program instructions which, when executed by at least one processor, cause an unmanned aerial vehicle (UAV) to:

determine a zoom level for a raw video stream input captured by one or more camera sensors of the UAV;

responsive to determining the zoom level being less than or equal to 3×,

zoom the raw video stream input to a field of view at a first zoom level of less than or equal to 3× to generate a zoomed video stream; and

apply electronic image stabilization to the zoomed video stream to generate a zoomed and stabilized video stream; and

responsive to determining the zoom level being greater than 3×,

first zoom the raw video stream input to a first field of view at 3× to generate a 3× zoomed video stream;

second zoom the 3× zoomed video stream to a second field of view at a second zoom level to generate a fully zoomed video stream at greater than 3×; and

apply electronic image stabilization to the fully zoomed video stream to generate a zoomed and stabilized video stream.

Assignments (1)

Continuity (2)

References Cited (10)