Depth from time of flight camera
View Patent ↗Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.
1. A method comprising:
receiving at least one originally captured image depicting one or more objects in an environment of a time of flight camera for a single frame by the time of flight camera;
training a region detector as a trained region detector using training data comprising other originally captured time of flight images, the other originally captured time of flight images having labeled regions of interest;
inputting the at least one originally captured image to the trained region detector and receiving in response one or more regions of interest in the at least one originally captured image, a region of interest comprising image elements which are predicted, based at least in part on regions of interest in earlier frames, to depict at least part of the one or more objects; and
computing depth from the one or more regions of interest of the at least one originally captured image.
2. The method as claimed in claim 1 the trained region detector having been trained to learn associations between the image elements of other originally captured time of flight images and the regions of interest.
3. The method as claimed in claim 1 comprising computing depth only from the one or more regions of interest of the originally captured image.
4. The method as claimed in claim 1 wherein the at least one originally captured image from the time of flight camera is a plurality of infra-red intensity values.
5. The method as claimed in claim 1 comprising:
receiving a plurality of originally captured images for the single frame by the time of flight camera;
aggregating the plurality of originally captured images to produce an aggregated original image; and
inputting the aggregated original image to the trained region detector.
6. The method as claimed in claim 1 comprising:
receiving a plurality of originally captured images for the single frame by the time of flight camera; and
inputting each of the originally captured images to the trained region detector.
7. The method as claimed in claim 1 comprising using the trained region detector to classify image elements of the originally captured image as being in a region of interest or not.
8. The method as claimed in claim 7 comprising clustering the classified image elements.
9. The method as claimed in claim 7 comprising applying a sliding window detector to the classified image elements.
10. The method as claimed in claim 1 , wherein the originally captured time of flight images of the training data depict examples of the one or more objects.
11. The method as claimed in claim 1 wherein the trained region detector is a random decision forest.
12. A computing system comprising:
an input/output controller arranged to receive at least one originally captured image depicting one or more objects in an environment of a time of flight camera for a single frame by the time of flight camera;
a region detector trained using training data comprising other originally captured time of flight images, the other originally captured time of flight images having labeled regions of interest, the region detector arranged to compute from the at least one originally captured image one or more regions of interest in the at least one originally captured image, a region of interest comprising image elements which are predicted, based at least in part on regions of interest in earlier frames, to depict at least part of the one or more objects; and
a depth map computation logic arranged to compute depth from the one or more regions of interest of the at least one originally captured image.
13. The computing system of claim 12 the region detector comprising a classifier having been trained to learn associations between the image elements of other originally captured time of flight images and the regions of interest.
14. The computing system of claim 13 the region detector comprising a clustering component arranged to cluster labeled image elements output by the classifier.
15. The computing system of claim 13 the region detector comprising a sliding window component arranged to compute one or more regions from labeled image elements output by the classifier.
16. The computing system of claim 13 integrated in a time of flight camera.
17. A computer-storage medium storing instructions which when executed by a computing device control the device to:
receive at least one originally captured image depicting one or more objects in an environment of a time of flight camera captured for a single frame by the time of flight camera;
send the at least one originally captured image to a trained region detector, having been trained, using training data comprising other originally captured time of flight images, the other originally captured time of flight images having labeled regions of interest, to learn associations between image elements of the other originally captured time of flight images and regions of interest, and receive in response one or more regions of interest in the at least one originally captured image, a region of interest comprising image elements which are predicted, based at least in part on regions of interest in earlier frames, to depict at least part of the one or more objects; and
compute depth from the one or more regions of interest of the at least originally captured image.
18. The computer-storage medium of claim 17 storing instructions which when executed by a computing device control the device to:
receive a plurality of originally captured images for the single frame by the time of flight camera;
aggregate the plurality of originally captured images to produce an aggregated original image; and
input the aggregated original image to the trained region detector.
19. The computer-storage medium of claim 17 storing instructions which when executed by a computing device control the device to:
receive a plurality of originally captured images for the single frame by the time of flight camera; and
input each of the originally captured images to the trained region detector.
20. The computer-storage medium of claim 17 storing instructions which when executed by a computing device control the device to compute depth only from the one or more regions of interest.