Employing three-dimensional (3D) data predicted from two-dimensional (2D) images using neural networks for 3D modeling applications and other applications
The disclosed subject matter is directed to employing machine learning models configured to predict 3D data from 2D images using deep learning techniques to derive 3D data for the 2D images. In some embodiments, a method is provided that comprises receiving, by a system operatively coupled to a processor, a two-dimensional image, and determining, by the system, auxiliary data for the two-dimensional image, wherein the auxiliary data comprises orientation information regarding a capture orientation of the two-dimensional image. The method further comprises, deriving, by the system, three-dimensional information for the two-dimensional image using one or more neural network models configured to infer the three-dimensional information based on the two-dimensional image and the auxiliary data.
1. A system, comprising:
a memory that stores computer executable components; and
a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
a reception component that receives a two-dimensional image of an environment;
a pre-processing component that alters one or more characteristics of the two-dimensional image to transform the image into a pre-processed image in accordance with a standard representation format, the one or more characteristics of the two-dimensional image including at least one of an orientation, a shape, and a level of magnification of the two-dimensional image;
a depth derivation component that derives three-dimensional information for the pre-processed two-dimensional image using one or more neural network models configured to infer the three-dimensional information based on the pre-processed two-dimensional image; and
a rendering component that renders a three-dimensional model of the environment based on the derived three-dimensional information and providing the three-dimensional model of the environment.
2. The system of claim 1 , wherein the pre-processing component alters the one or more characteristics based one or more image capture parameters associated with capture of the two-dimensional image.
3. The system of claim 2 , wherein the pre-processing component extracts the one or more image capture parameters from metadata associated with the two-dimensional image.
4. The system of claim 2 , wherein the one or more image capture parameters comprise one or more camera settings of a camera used to capture the two-dimensional image.
5. The system of claim 2 , wherein the one or more image capture parameters are selected from a group consisting of, camera lens parameters, lighting parameters, and color parameters.
6. The system of claim 1 , wherein the pre-processing component alters the one or more characteristics based on variances between the one or more characteristics and one or more defined image characteristics of the standard representation format.
7. The system of claim 6 , wherein the one or more characteristics comprise one or more visual characteristics of the two-dimensional image.