Structure from Motion for drone videos
View Patent ↗Aspects of the subject disclosure may include, for example, a method comprising obtaining, by a processing system including a processor, first and second models for a structure of an object, based respectively on ground-level and aerial observations of the object. Model parameters are determined for a three-dimensional (3D) third model of the object based on the first and second models; the determining comprises a transfer learning procedure. Data representing observations of the object is captured at an airborne unmanned aircraft system (UAS) operating at an altitude between that of the ground-level observations and the aerial observations. The method also comprises dynamically adjusting the third model in accordance with the operating altitude of the UAS; updating the adjusted third model in accordance with the data; and determining a 3D representation of the structure of the object, based on the updated adjusted third model. Other embodiments are disclosed.
1. A device comprising:
a processing system including a processor; and
a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations comprising:
obtaining data representing a plurality of two-dimensional (2D) images of an object, the images captured at a plurality of operating altitudes in an operating altitude range;
constructing a three-dimensional (3D) model of the object in accordance with the data, wherein the constructing comprises a transfer learning procedure using a first plurality of observations of the object obtained at a first altitude range lower than the operating altitude range and a second plurality of observations of the object obtained at a second altitude range higher than the operating altitude range, wherein the 3D model is based on parameters comprising structure-from-motion (SfM) model parameters for the operating altitude range; and
adjusting the 3D model in accordance with updated data representing a new image captured at a new operating altitude.
2. The device of claim 1 , wherein the first plurality of observations comprise ground level or near-ground-level observations of the object and the second plurality of observations comprise aerial observations of the object.
3. The device of claim 1 , wherein the images comprise video images captured using an airborne unmanned aircraft system (UAS).
4. The device of claim 3 , wherein the operations further comprise:
predicting a future operating altitude of the UAS; and
determining a predicted 3D model of the object in accordance with the predicted future operating altitude.
5. The device of claim 3 , wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on a manifold.
6. The device of claim 5 , wherein the manifold is a Grassmannian manifold.
7. The device of claim 5 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on the first plurality of observations and a second endpoint corresponding to a second model of the object based on the second plurality of observations.
8. The device of claim 1 , wherein the operations further comprise:
determining correspondence parameters representing a correspondence between the data and the 3D model; and
updating the correspondence parameters in accordance with the data.
9. The device of claim 8 , wherein the updating of the correspondence parameters comprises adjusting the correspondence parameters to reduce an error between the 3D model and the data.
10. The device of claim 9 , wherein the 3D model is adjusted in real time, and wherein the adjusting comprises comparing the 3D model with live video images of the object.
11. A method comprising:
obtaining, by a processing system including a processor, data representing a plurality of two-dimensional (2D) images of an object, the images captured at a plurality of operating altitudes in an operating altitude range;
constructing, by the processing system, a three-dimensional (3D) model of the object in accordance with the data, wherein the constructing comprises a transfer learning procedure using a first plurality of observations of the object obtained at a first altitude range lower than the operating altitude range and a second plurality of observations of the object obtained at a second altitude range higher than the operating altitude range, wherein the 3D model is based on parameters comprising structure-from-motion (SfM) model parameters for the operating altitude range; and
adjusting, by the processing system, the 3D model in real time, in accordance with updated data representing a new image captured at a new operating altitude.
12. The method of claim 11 , wherein the first plurality of observations comprise ground level or near-ground-level observations of the object and the second plurality of observations comprise aerial observations of the object.
13. The method of claim 11 , wherein the images comprise video images captured using an airborne unmanned aircraft system (UAS).
14. The method of claim 13 , further comprising:
predicting, by the processing system, a future operating altitude of the UAS; and
determining, by the processing system, a predicted 3D model of the object in accordance with the predicted future operating altitude,
wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on a manifold.
15. The method of claim 14 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on the first plurality of observations and a second endpoint corresponding to a second model of the object based on the second plurality of observations.
16. The method of claim 11 , further comprising:
determining, by the processing system, correspondence parameters representing a correspondence between the data and the 3D model; and
updating, by the processing system, the correspondence parameters in accordance with the data by adjusting the correspondence parameters to reduce an error between the 3D model and the data.
17. A non-transitory machine-readable medium comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations comprising:
obtaining data representing a plurality of two-dimensional (2D) images of an object, the images captured at a plurality of operating altitudes in an operating altitude range;
constructing a three-dimensional (3D) model of the object in accordance with the data, wherein the constructing comprises a transfer learning procedure using a first plurality of observations of the object obtained at a first altitude range lower than the operating altitude range and a second plurality of observations of the object obtained at a second altitude range higher than the operating altitude range, wherein the 3D model is based on parameters comprising structure-from-motion (SfM) model parameters for the operating altitude range; and
adjusting the 3D model in accordance with updated data representing a live video image captured at a new operating altitude.
18. The non-transitory machine-readable medium of claim 17 , wherein the first plurality of observations comprise ground level or near-ground-level observations of the object and the second plurality of observations comprise aerial observations of the object.
19. The non-transitory machine-readable medium of claim 17 , wherein the images comprise video images captured using an airborne unmanned aircraft system (UAS).
20. The non-transitory machine-readable medium of claim 19 , wherein the operations further comprise:
predicting a future operating altitude of the UAS; and
determining a predicted 3D model of the object in accordance with the predicted future operating altitude,
wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on a manifold.