Structure from motion for drone videos
Aspects of the subject disclosure may include, for example, a method comprising obtaining, by a processing system including a processor, first and second models for a structure of an object, based respectively on ground-level and aerial observations of the object. Model parameters are determined for a three-dimensional (3D) third model of the object based on the first and second models; the determining comprises a transfer learning procedure. Data representing observations of the object is captured at an airborne unmanned aircraft system (UAS) operating at an altitude between that of the ground-level observations and the aerial observations. The method also comprises dynamically adjusting the third model in accordance with the operating altitude of the UAS; updating the adjusted third model in accordance with the data; and determining a 3D representation of the structure of the object, based on the updated adjusted third model. Other embodiments are disclosed.
1. A method comprising:
obtaining, by a processing system including a processor, data representing a plurality of two-dimensional (2D) video images of an object, the video images captured at a plurality of operating altitudes using an airborne unmanned aircraft system (UAS);
constructing, by the processing system, a three-dimensional (3D) model of the object in accordance with the data, wherein the constructing comprises a transfer learning procedure using a first plurality of ground level or near-ground-level observations of the object and a second plurality of aerial observations of the object, wherein the plurality of operating altitudes are in an altitude range greater than that of the first plurality of observations and less than that of the second plurality of observations; and
dynamically adjusting, by the processing system, the 3D model in accordance with updated data representing a new video image captured at a new operating altitude of the UAS.
2. The method of claim 1 , wherein the altitude range is from about 100 feet to about 2000 feet.
3. The method of claim 1 , further comprising:
predicting, by the processing system, a future operating altitude of the UAS; and
determining, by the processing system, a predicted 3D model of the object in accordance with the predicted future operating altitude.
4. The method of claim 1 , wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on a manifold.
5. The method of claim 4 , wherein the parameters comprise structure-from-motion (SfM) model parameters for the altitude range.
6. The method of claim 5 , further comprising obtaining, by the processing system, the parameters using a machine-learning algorithm.
7. The method of claim 4 , wherein the manifold is a Gras smannian manifold.
8. The method of claim 4 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on the first plurality of observations and a second endpoint corresponding to a second model of the object based on the second plurality of observations.
9. The method of claim 1 , further comprising:
determining, by the processing system, correspondence parameters representing a correspondence between the data and the 3D model; and
updating, by the processing system, the correspondence parameters in accordance with the data.
10. The method of claim 9 , wherein updating the correspondence parameters comprises adjusting, by the processing system, the correspondence parameters to reduce an error between the 3D model and the data.
11. The method of claim 1 , wherein the 3D model is adjusted in real time, and wherein the adjusting comprises comparing the 3D model with live video images of the object.
12. A device comprising:
a processing system including a processor; and
a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations comprising:
obtaining data representing a plurality of two-dimensional (2D) video images of an object, the video images captured at a plurality of operating altitudes using an airborne unmanned aircraft system (UAS);
constructing a three-dimensional (3D) model of the object in accordance with the data, wherein the constructing comprises a transfer learning procedure using a first plurality of ground level or near-ground-level observations of the object and a second plurality of aerial observations of the object; and
dynamically adjusting, by the processing system, the 3D model in accordance with updated data representing a new video image captured at a new operating altitude of the UAS,
wherein the 3D model is adjusted in real time, and wherein the adjusting comprises comparing the 3D model with live video images of the object.
13. The device of claim 12 , wherein the plurality of operating altitudes are in an altitude range greater than that of the first plurality of observations and less than that of the second plurality of observations.
14. The device of claim 13 , wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on a manifold.
15. The device of claim 14 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on the first plurality of observations and a second endpoint corresponding to a second model of the object based on the second plurality of observations.
16. The device of claim 14 , wherein the parameters comprise structure-from-motion (SfM) model parameters for the altitude range, and wherein the parameters are obtained using a machine-learning algorithm.
17. A non-transitory machine-readable medium comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations comprising:
obtaining data representing a plurality of two-dimensional (2D) video images of an object, the video images captured at a plurality of operating altitudes using an airborne unmanned aircraft system (UAS);
constructing a three-dimensional (3D) model of the object in accordance with the data, wherein the constructing comprises a transfer learning procedure using a first plurality of ground level or near-ground-level observations of the object and a second plurality of aerial observations of the object, wherein the plurality of operating altitudes are in an altitude range greater than that of the first plurality of observations and less than that of the second plurality of observations, wherein the 3D model is based on parameters varying non-linearly with an operating altitude of the UAS, and wherein variation of the 3D model with the operating altitude corresponds to a non-linear path on a manifold; and
dynamically adjusting the 3D model in accordance with updated data representing a new video image captured at a new operating altitude of the UAS.
18. The non-transitory machine-readable medium of claim 17 , wherein the manifold is a Grassmannian manifold.
19. The non-transitory machine-readable medium of claim 17 , wherein the non-linear path has a first endpoint corresponding to a first model of the object based on the first plurality of observations and a second endpoint corresponding to a second model of the object based on the second plurality of observations.
20. The non-transitory machine-readable medium of claim 17 , wherein the parameters comprise structure-from-motion (SfM) model parameters for the altitude range, and wherein the parameters are obtained using a machine-learning algorithm.