Multi-dimensional data capture of an environment using plural devices

1. A method of delivering three-dimensional (3D) data to one or more users, the method comprising:

generating, by a computer processing system, a series of time-indexed 3D frames that combines 3D frames of image data of a scene in motion captured by one or more data capture devices over a first period of time, wherein the series of time-indexed 3D frames are generated by:

receiving 3D frames of image data, from each data capture device of one or more data capture devices, each data capture device including a depth sensor that captures a frame of depth data having an array of depth measurements corresponding to an array of pixels of image data captured by the one or more data capture devices, the 3D frames of image data from each of the one or more data capture devices including an array of pixels of image data of the scene in motion and time-indexed depth data frames indicating an array of distance measurements corresponding to the array of pixels of image data to points on surfaces of the scene in motion sampled over the first period of time from one or more points of view,

receiving location data for the first period of time, the location data including data for a location of each of the one or more data capture devices,

determining 3D coordinates in a common coordinate system for each of the points of the depth data and corresponding pixels of image data using the location data and the array of distance measurements captured by each of the one or more data capture devices at each sampled time of the first period of time, and

generating the series of time-indexed 3D frames of the scene in motion, wherein each of the time-indexed 3D frames includes a 3D point cloud with the 3D coordinates of the points of the depth data for each of the one or more data capture devices in the common coordinate system at a given sampled time of the first period of time; and

transmitting, by the computer processing system, the series of time-indexed 3D frames to one or more users for viewing, wherein a first point of view of a first user for the series of time-indexed 3D frames may be selected from any of a plurality of different points of view within the 3D point cloud of each of the 3D frames.

2. The method of claim 1 , wherein the first user can change from viewing the series of time-indexed 3D frames from the first point of view to viewing the series of time-indexed 3D frames from a second point of view of the plurality of different points of view within the 3D point cloud of each of the 3D frames.

3. The method of claim 1 , wherein transmitting includes transmitting via a local area network (LAN), an intranet, an extranet, or the Internet.

4. The method of claim 1 , wherein generating the series of time-indexed 3D frames further includes:

receiving color data of the surfaces of the scene in motion from each of the one or more data capture devices; and

associating the color data from each of the one or more data capture devices with the time-indexed depth data and with the coordinates in the common coordinate system.

5. The method of claim 1 , wherein the time-indexed depth data for each of the one or more data capture devices includes distance measurements to surfaces within 4π steradian captured by a plurality of depth sensors.

6. The method of claim 1 , wherein the location data includes averaged motion data from one or more accelerometers.

7. The method of claim 1 , wherein the location data includes geospatial data from a GPS module of each of the one or more data capture devices indicating a geospatial location of the respective data capture device.

8. The method of claim 1 , wherein the location data further includes relative location data between multiple data capture devices.

9. The method of claim 8 , wherein the relative location data is based on recognition by a data capture device of a shape of another data capture device.

10. The method of claim 1 , wherein generating the series of time-indexed 3D frames further includes:

receiving relative orientation data indicating how each data capture device of multiple data capture devices is oriented with to relation each other of the multiple data capture devices during the first period of time; and

associating the time-indexed depth data for each of the multiple data capture devices with the coordinates in the common coordinate system based on the relative orientation data.

11. The method of claim 1 , wherein generating the series of time-indexed 3D frames further includes:

recognizing a spatial overlap in a portion of a first time-indexed depth data and a second time-indexed depth data;

discarding the overlapping portion from one of the first time-indexed depth data and the second time-indexed depth data; and

retaining the overlapping portion from the other of the first time-indexed depth data and the second time-indexed depth data with the associated coordinates.

12. A system for delivering three-dimensional (3D) data to one or more users, the system comprising:

an apparatus to generate a series of time-indexed 3D frames that combine 3D frames of image data of a scene in motion captured by one or more data capture devices over a first period of time, the apparatus including:

one or more data capture devices, each data capture device including a depth sensor that captures a frame of depth data having an array of depth measurements corresponding to an array of pixels of image data captured by the one or more data capture devices, wherein the depth sensor to detect time-indexed depth data indicating an array of distance measurements to points on surfaces of the scene in motion sampled over the first period of time from one or more points of view, an image camera to record image data of the scene in motion including an array of pixels of image data corresponding the array of depth measurements, and a sensor to sense a location of the data capture device, and

a data processor to receive 3D frames of image data including the time-indexed depth data, the image data, and location data from each of the one or more data capture devices for the first period of time, the data processor to:

determine 3D coordinates in a common coordinate system for each of the points of the depth data and corresponding pixels of image data using the location data and the array of distance measurements captured by each of the one or more data capture devices at each sampled time of the first period of time; and

generate the series of time-indexed 3D frames of the scene in motion, wherein each of the time-indexed 3D frames includes a 3D point cloud with the 3D coordinates of the points of the depth data for each of the one or more data capture devices in the common coordinate system at a given sampled time of the first period of time; and

an apparatus to transmit the series of time-indexed 3D frames to one or more users for viewing, wherein a first point of view of a first user for the series of time-indexed 3D frames may be selected from any of a plurality of different points of view within the 3D point cloud of each of the 3D frames.

13. The system of claim 12 , wherein the first user can change from viewing the series of time-indexed 3D frames from the first point of view to viewing the series of time-indexed 3D frames from a second point of view of the plurality of different points of view within the 3D point cloud of each of the 3D frames.

14. The system of claim 12 , wherein transmitting includes transmitting via a local area network (LAN), an intranet, an extranet, or the Internet.

15. The system of claim 12 , wherein the data processor is further to:

receive color data of the scene in motion from each of the one or more data capture devices; and

associate the color data from each of the one or more data capture devices with the time-indexed depth data and with the coordinates in the common coordinate system.

16. The system of claim 12 , wherein the time-indexed depth data for each of the one or more data capture devices includes distance measurements to surfaces within 4π steradian captured by a plurality of depth sensors.

17. The system of claim 12 , wherein the location data includes averaged motion data from one or more accelerometers.

18. The system of claim 12 , wherein the location data includes geospatial data from a GPS module of each of the one ore more data capture device indicating a geospatial location of the respective data capture device.

19. The system of claim 12 , wherein the location data further includes relative location data between multiple data capture devices.

20. The system of claim 19 , wherein the relative location data is based on recognition by a data capture device of a shape of another data capture device.

21. The system of claim 12 , wherein the data processor is further to:

receive relative orientation data indicating how each data capture device of multiple data capture devices is oriented with relation to each other of the multiple data capture devices during the first period of time; and

associate the time-indexed depth data for each of the multiple data capture devices with coordinates in the common coordinate system based on the relative orientation data.

22. The system of claim 12 , wherein the data processor is further to:

recognize a spatial overlap in a portion of a first time-indexed depth data and a second time-indexed depth data;

discard the overlapping portion from one of the first time-indexed depth data and the second time-indexed depth data; and

retain the overlapping portion from the other of the first and second time-indexed depth data with the associated coordinates.

23. A non-transitory computer-readable storage medium having stored thereon data representing sequences of instructions that, when executed by a processor, cause the processor to perform operations comprising:

generating a series of time-indexed three-dimensional (3D) frames that combines 3D frames of image data of a scene in motion captured by one or more data capture devices over a first period of time, wherein the series of time-indexed 3D frames are generated by:

receiving 3D frames of image data, from each data capture device of one or more data capture devices, each data capture device including a depth sensor that captures a frame of depth data having an array of depth measurements corresponding to an array of pixels of image data captured by the one or more data capture devices, the 3D frames of image data from each of the one or more data capture device including an array of pixels of image data of the scene in motion and time-indexed depth data frames indicating an array of distance measurements corresponding to the array of pixels of image data to points on surfaces of the scene in motion sampled over the first period of time from one or more points of view,

receiving location data for the first period of time, the location data including data for a location of each of the one or more data capture devices,

determining 3D coordinates in a common coordinate system for each of the points of the depth data and corresponding pixels of image data using the location data and the array of distance measurements captured by each of the one or more data capture devices at each sampled time of the first period of time, and

generating the series of time-indexed 3D frames of the scene in motion, wherein each of the time-indexed 3D frames includes a 3D point cloud with the 3D coordinates of the points of the depth data for each of the one or more data capture devices in the common coordinate system at a given sampled time of the first period of time; and

transmitting the series of time-indexed 3D frames to one or more users for viewing, wherein a first point of view of a first user for the series of time-indexed 3D frames may be selected from any of a plurality of different points of view within the 3D point cloud of each of the 3D frames.

24. The medium of claim 23 , wherein the first user can change from viewing the series of time-indexed 3D frames from the first point of view to viewing the series of time-indexed 3D frames from a second point of view of the plurality of different points of view within the 3D point cloud of each of the 3D frames.

25. The medium of claim 23 , wherein transmitting includes transmitting via a local area network (LAN), an intranet, an extranet, or the Internet.