System and method for generating a multi-dimensional image

1. A method of generating a multi-dimensional image of an object in a scene, the method comprising:

capturing a three-dimensional (3D) low-resolution depth image of the scene into a low-spatial resolution depth map in a memory and a two-dimensional (2D) high-resolution light intensity image of the scene into the memory;

converting the 2D high-resolution light intensity image from the memory to a spectral-augmented high-resolution image of a selected channel, wherein the channel is a selected spectral band, wherein the converting includes receiving a selection of an object of interest and selecting a channel based upon stored parameters for the object of interest, wherein the converting includes adding spectral reflectance information for the selected channel;

segmenting the object from the spectral-augmented high-resolution image, resulting in a segmented version of the spectral-augmented high-resolution image; and

generating a high-resolution depth image of the object based at least in part on the 3D low-resolution depth image and the segmented version of the spectral-augmented high-resolution image.

2. The method of claim 1 , further comprising generating a 3D high-resolution spectral image of the object based on the high-resolution depth image and the spectral-augmented high-resolution image, the 3D spectral image comprising volume and spectral information.

3. The method of claim 2 , further comprising augmenting a user's direct vision of the scene by superimposing the 3D high-resolution spectral image on the direct vision.

4. The method of claim 1 , wherein the object is a human body part under a medical operation or examination by a physician, and wherein the high-resolution depth image is used to provide an augmented vision for the physician.

5. The method of claim 1 , wherein the object comprises non-rigid human or animal organ tissues.

6. The method of claim 1 , wherein the 2D light intensity image comprises two, three, or four channels.

7. The method of claim 1 , wherein the 2D light intensity image comprises electronic bandpass wavelength selection.

8. The method of claim 1 , wherein the channel of the spectral-augmented high-resolution image is variably selected to make differences between the object and its surrounding region more noticeable in the spectral-augmented high-resolution image than in the 2D light intensity image.

9. The method of claim 1 , wherein the channel is selected based on a user input depending at least on spectral reflectance properties of the object.

10. The method of claim 1 , wherein converting the 2D high-resolution light intensity image to a spectral-augmented high-resolution image of a selected channel comprises:

reconstructing a spectral image having at least three channels from the 2D high-resolution light intensity image; and

filtering the spectral image to generate an image of the selected channel.

11. The method of claim 1 , wherein generating a high-resolution depth image of the object comprises:

generating a high-resolution depth image of the scene by interpolating depth values from the low-resolution depth image; and

segmenting the object from the high-resolution depth image.

12. The method of claim 1 , wherein each of the 3D low-resolution depth image and the 2D high-resolution light intensity image comprises a plurality of images in a video, and the method is performed in real time to track the object in the scene across the images.

13. A system for generating a multi-dimensional image of an object in a scene, the system comprising:

means for capturing a three-dimensional (3D) low-resolution depth image of the scene into a low-spatial resolution depth map in a memory and a two-dimensional (2D) high-resolution light intensity image of the scene into the memory;

means for converting the 2D high-resolution light intensity image from the memory to a spectral-augmented high-resolution image of a selected channel, wherein the channel is a selected spectral band, wherein the converting includes receiving a selection of an object of interest and selecting a channel based upon stored parameters for the object of interest, wherein the converting includes adding spectral reflectance information for the selected channel; and

means for generating a high-resolution depth image of the object based at least in part on the 3D low-resolution depth image and the segmented version of the spectral-augmented high-resolution image.

14. The system of claim 13 , further comprising means for segmenting the object from the spectral-augmented high-resolution image into a segmented version of the spectral-augmented high-resolution image.