Structure from motion for drone videos
Aspects of the subject disclosure may include, for example, a method comprising obtaining, by a processing system including a processor, first and second models for a structure of an object, based respectively on ground-level and aerial observations of the object. Model parameters are determined for a three-dimensional (3D) third model of the object based on the first and second models; the determining comprises a transfer learning procedure. Data representing observations of the object is captured at an airborne unmanned aircraft system (UAS) operating at an altitude between that of the ground-level observations and the aerial observations. The method also comprises dynamically adjusting the third model in accordance with the operating altitude of the UAS; updating the adjusted third model in accordance with the data; and determining a 3D representation of the structure of the object, based on the updated adjusted third model. Other embodiments are disclosed.
1. A device comprising:
a processing system including a processor; and
a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations comprising:
obtaining data representing a plurality of two-dimensional (2D) video images of an object, the video images captured using an airborne unmanned aircraft system (UAS) operating in a first altitude range;
constructing a three-dimensional (3D) model of the object in accordance with the data, the constructing using ground level or near-ground-level observations of the object obtained in a second altitude range and aerial observations of the object obtained in a third altitude range, wherein the first altitude range comprises operating altitudes of the UAS above the second altitude range and below the third altitude range; and
dynamically adjusting the 3D model in accordance with updated data representing a new video image captured at a new operating altitude of the UAS.
2. The device of claim 1 , wherein the constructing comprises a transfer learning procedure.
3. The device of claim 2 , wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein the transfer learning procedure is used on a manifold to determine the parameters.
4. The device of claim 3 , wherein the manifold is a Grassmannian manifold.
5. The device of claim 3 , wherein the parameters comprise structure-from-motion (SfM) model parameters for the first altitude range.
6. The device of claim 5 , wherein the parameters are determined using a machine-learning algorithm.
7. The device of claim 3 , wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on the manifold.
8. The device of claim 7 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on a first plurality of observations comprising the ground level or near-ground-level observations and a second endpoint corresponding to a second model of the object based on a second plurality of observations comprising the aerial observations.
9. The device of claim 1 , wherein the operations further comprise:
predicting, by the processing system, a future operating altitude of the UAS; and
determining a predicted 3D model of the object in accordance with the predicted future operating altitude.
10. The device of claim 1 , wherein the first altitude range is from about 100 feet to about 2000 feet.
11. A method comprising:
obtaining, by a processing system including a processor, data representing a plurality of two-dimensional (2D) video images of an object, the video images captured using an airborne unmanned aircraft system (UAS) operating in a first altitude range;
constructing, by the processing system, a three-dimensional (3D) model of the object in accordance with the data, the constructing using a first plurality of observations of the object obtained in a second altitude range and a second plurality of observations of the object obtained in a third altitude range, wherein the first altitude range comprises operating altitudes of the UAS above the second altitude range and below the third altitude range; and
dynamically adjusting the 3D model in accordance with updated data representing a new video image captured at a new operating altitude of the UAS.
12. The method of claim 11 , wherein the constructing comprises a transfer learning procedure.
13. The method of claim 12 , wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein the transfer learning procedure is used on a manifold to determine the parameters.
14. The method of claim 13 , wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on the manifold.
15. The method of claim 14 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on the first plurality of observations and a second endpoint corresponding to a second model of the object based on the second plurality of observations.
16. The method of claim 15 , wherein the first plurality of observations comprise ground level or near-ground-level observations of the object and the second plurality of observations comprise aerial observations of the object.
17. A non-transitory machine-readable medium comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations comprising:
obtaining data representing a plurality of two-dimensional (2D) video images of an object, the video images captured using an airborne unmanned aircraft system (UAS) operating in a first altitude range;
constructing a three-dimensional (3D) model of the object in accordance with the data, the constructing using observations of the object obtained in an altitude range outside the first altitude range; and
dynamically adjusting the 3D model in accordance with updated data representing a new video image captured at a new operating altitude of the UAS.
18. The non-transitory machine-readable medium of claim 17 , wherein the constructing the 3D model of the object comprises a transfer learning procedure used on a manifold to determine parameters varying non-linearly with an operating altitude of the UAS.
19. The non-transitory machine-readable medium of claim 18 , wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on the manifold, wherein the non-linear path has a first endpoint corresponding to a first model of the object based on ground level or near-ground-level observations of the object and a second endpoint corresponding to a second model of the object based on aerial observations of the object.
20. The non-transitory machine-readable medium of claim 18 , wherein the manifold is a Grassmannian manifold.