Camera calibration method and apparatus based on deep learning
Disclosed herein is a camera calibration method based on deep learning including acquiring an image captured by a camera and predicting an intrinsic parameter of the camera by applying, to the acquired image, a neural network module trained to predict the intrinsic parameter.
1. A camera calibration method based on deep learning, comprising:
acquiring an image captured by a camera; and
predicting an intrinsic parameter of the camera by applying, to the acquired image, a neural network module trained to predict the intrinsic parameter;
wherein the neural network module includes a first convolutional neural network (CNN) and a second CNN trained to predict different intrinsic parameters, and
wherein the predicting of the intrinsic parameter includes:
predicting a first intrinsic parameter of the camera by applying the first CNN to the acquired image;
concatenating one or more feature maps composed of the first intrinsic parameter with any one of fully connected layers of the second CNN; and
predicting a second intrinsic parameter of the camera by applying the second CNN to the acquired image.
2. The camera calibration method of claim 1 , wherein the neural network module is trained using a plurality of learning images derived from a single panoramic planar image.
3. The camera calibration method of claim 2 , wherein the plurality of learning images is generated by projecting the single panoramic planar image onto three-dimensional (3D) geometry to convert the single panoramic planar image into a 360-degree 3D image and changing an intrinsic parameter of a virtual camera set in the 360-degree 3D image.
4. The camera calibration method of claim 2 , wherein the predicting of the intrinsic parameter includes predicting the intrinsic parameter of the camera by calculating a probability of a similarity between a ground truth parameter set in the neural network module and output data output from an output layer of the neural network module using a classification model.
5. A camera calibration apparatus based on deep learning, comprising:
an acquisition unit configured to acquire an image captured by a camera; and
a prediction unit configured to predict an intrinsic parameter of the camera by applying, to the acquired image, a neural network module trained to predict the intrinsic parameter;
wherein the neural network module includes a first convolutional neural network (CNN) and a second CNN trained to predict different intrinsic parameters, and
wherein the prediction unit includes:
a first prediction unit configured to predict a first intrinsic parameter of the camera by applying the first CNN to the acquired image; and
a second prediction unit configured to concatenate one or more feature maps composed of the first intrinsic parameter with any one of fully connected layers of the second CNN and to predict a second intrinsic parameter of the camera by applying the second CNN to the acquired image.
6. The camera calibration apparatus of claim 5 , wherein the neural network module is trained using a plurality of learning images derived from a single panoramic planar image.
7. The camera calibration apparatus of claim 6 , wherein the plurality of learning images is generated by projecting the single panoramic planar image onto three-dimensional (3D) geometry to convert the single panoramic planar image into a 360-degree 3D image and changing an intrinsic parameter of a virtual camera set in the 360-degree 3D image.
8. The camera calibration apparatus of claim 6 , wherein the prediction unit predicts the intrinsic parameter of the camera by calculating a probability of a similarity between a ground truth parameter set in the neural network module and output data output from an output layer of the neural network module using a classification model.