Scene labeling of RGB-D data with interactive option
A method for generating labels for an image of low quality is described. The method includes mapping image data and depth information of said image to a 3D point cloud; segmenting the 3-D point cloud and the image into super voxels and image patches, fusing features obtained from the super voxels and image patches, by using a fusion model, applying classifiers to fused features, wherein the fusion model and the classifiers are generated from a dataset including image data of selected quality and quantity, corresponding point cloud and image labels, and generating scene labels based on applied classifiers.
1. A method for generating labels for an image, comprising:
mapping image data and depth information of said image to a three-dimensional (3D) point cloud;
segmenting the 3D point cloud and the image into super voxels and image patches, respectively;
fusing features obtained from the super voxels and image patches by using a fusion model;
applying classifiers to the fused features, wherein the fusion model and the classifiers are generated from a stored dataset including image data having a selected quality and quantity along with corresponding point clouds and image labels; and
generating scene labels based on the applied classifiers.
2. The method according to claim 1 wherein the image data and depth information comprises red, green and blue color data and depth information.
3. The method according to claim 1 wherein the features are obtained by using one of an unsupervised or a supervised feature learning framework on the image and the 3D point cloud separately.
4. The method according to claim 1 wherein the classifiers are obtained using an exemplar Support Vector Machine (SVM) method.
5. The method according to claim 1 further including interactively correcting the generated scene labels based on user input.
6. The method according to claim 1 wherein the classifiers are updated based on user input.
7. An Apparatus for scene labeling an image, comprising:
at least one processor configured to
(a) map image data and depth information of said image to a three-dimensional-(3D) point cloud;
(b) segment the 3D point cloud and the image into super voxels and image patches, respectively;
(c) fuse features that are obtained from the super voxels and image patches by using a fusion model;
(d) apply classifiers to the fused features; and
(e) generate scene labels based on the applied pre-trained classifiers, wherein the fusion model and the classifiers are generated from a stored dataset including image data having a selected quality and quantity along with corresponding point clouds and image labels.
8. The apparatus according to claim 7 wherein the image data and the depth information comprises red, green and blue color data and depth information.
9. The apparatus according to claim 7 wherein the fusion model comprises a bimodal Deep Boltzman Machine model.
10. The apparatus according to claim 7 wherein the features are obtained using one of an unsupervised or a supervised feature learning framework on the image and the 3D point cloud separately.
11. The apparatus according to claim 7 wherein the classifiers are obtained using an exemplar Support Vector Machine (SVM) apparatus.
12. The apparatus according to claim 7 wherein the processor interactively corrects the generated scene labels based on user input.
13. The apparatus according to claim 7 wherein the processor updates the classifiers based on user input.