Distributed depth data processing
Examples are provided that relate to processing depth camera data over a distributed computing system, where phase unwrapping is performed prior to denoising. One example provides a time-of-flight camera comprising a time-of-flight depth image sensor, a logic machine, a communication subsystem, and a storage machine holding instructions executable by the logic machine to process time-of-flight image data acquired by the time-of-flight depth image sensor by, prior to denoising, performing phase unwrapping pixel-wise on the time-of-flight image data to obtain coarse depth image data comprising depth values; and send the coarse depth image data and active brightness image data to a remote computing system via the communication subsystem for denoising.
1. A time-of-flight camera, comprising:
a time-of-flight depth image sensor;
a logic machine;
a communication subsystem; and
a storage machine holding instructions executable by the logic machine to:
process time-of-flight image data acquired by the time-of-flight depth image sensor by, prior to denoising, performing phase unwrapping pixel-wise on the time-of-flight image data to obtain coarse depth image data comprising depth values; and
send the coarse depth image data and the active brightness image data to a remote computing system via the communication subsystem for denoising.
2. The time-of-flight camera of claim 1 , wherein the instructions are executable to compress the coarse depth image data and the active brightness image data before sending the coarse depth image data and the active brightness image data to the remote computing system.
3. The time-of-flight camera of claim 1 , wherein the instructions are further executable to, prior to denoising, perform a pixel-wise signal calibration correction.
4. The time-of-flight camera of claim 1 , further comprising a device incorporating the time-of-flight camera, and wherein the remote computing system is remote from the device incorporating the time-of-flight camera.
5. The time-of-flight camera of claim 1 , wherein the instructions are executable to segment a coarse depth image including the coarse depth image data, and send the coarse depth image data to the remote computing system after segmenting.
6. The time-of-flight camera of claim 1 , wherein the instructions are further executable to process the time-of-flight image data by, prior to denoising, performing active brightness averaging pixel-wise on the time-of-flight image data to obtain active brightness image data.
7. A computing system comprising:
a logic machine;
a communication subsystem; and
a storage machine holding instructions executable by the logic machine to:
receive coarse depth image data from a remote device comprising a time-of-flight camera,
reconstruct time-of-flight (ToF) phase data from the coarse depth image data,
apply a spatial denoising filter comprising a convolution kernel to the ToF phase data to form denoised depth image data, and
output the denoised depth image data.
8. The computing system of claim 7 , wherein the instructions are further executable to receive active brightness image data from the remote device, and denoise the coarse depth image data based at least on the active brightness image data.
9. The computing system of claim 7 , wherein the instructions are further executable to segment the coarse depth image data based upon a threshold signal to noise ratio.
10. The computing system of claim 9 , wherein the instructions are executable to perform denoising on coarse depth image data with lower signal to noise ratios using a relatively larger convolution kernel, and perform denoising on coarse depth image data with higher signal to noise ratios using a relatively smaller convolution kernel.
11. The computing system of claim 7 , wherein the convolution kernel comprises a size in a range of 3×3 to 19×19.
12. The computing system of claim 7 , wherein the denoised depth image data is output to the remote device.
13. The computing system of claim 7 , wherein the instructions are further executable to denoise the coarse depth image data using temporal filtering based on prior stored coarse depth image data.
14. The computing system of claim 13 , wherein the temporal filtering is performed based upon 3-7 previously-received coarse depth images.
15. Enacted on a time-of-flight camera, a method comprising:
receiving time-of-flight image data from a time-of-flight depth image sensor of the time-of-flight camera;
prior to denoising, performing active brightness averaging pixel-wise on the time-of-flight image data to obtain active brightness image data,
prior to denoising, performing phase unwrapping pixel-wise on the time-of-flight image data to obtain coarse depth image data, the coarse depth image data comprising phase errors, and
sending the coarse depth image data and the active brightness image data to a remote computing system via a communication subsystem for correction of at least some of the phase errors.
16. The method of claim 15 , further comprising compressing the coarse depth image data and the active brightness image data before sending the coarse depth image data and the active brightness image data to the remote system.
17. The method of claim 15 , further comprising, prior to denoising, performing a signal calibration correction pixel-wise on the time-of-flight image data.
18. The method of claim 15 , wherein the remote computing system is remote from a device incorporating the time-of-flight camera.
19. The method of claim 15 , further comprising segmenting the coarse depth image data.
20. The method of claim 15 , further comprising receiving, at a device incorporating the time-of-flight camera, denoised depth image data and denoised active brightness image data from the remote system.