Digital camera device for 3D imaging
Described are digital camera devices for 3D Imaging. Embodiments of the invention capture and process sensory data including image data to produce 3D content and generate depth information. Digital camera devices improve upon existing camera devices by implementing dynamic calibration processes that improve the quality of image capture and image projection. In some embodiments the digital camera devices include an imaging system for performing computer vision tasks including scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, object tracking, facial recognition, object counting, 3D imaging, image enhancement and image restoration.
1. A digital camera device for capturing 3D content, the digital camera apparatus having variable camera settings and comprising:
a plurality of camera modules configured to capture one or more channels of stereoscopic video, the plurality of camera modules comprising a master camera module and a plurality of slave camera modules, the master camera module comprising:
a processor;
a memory;
a sensor;
a lens;
a master streaming service; and
a master camera control unit having a dual camera controller, a master mono camera controller and a master proxy camera controller, the master mono camera controller controlling operation of the master camera module, the master proxy camera controller interfacing with one or more slave proxy camera controllers to distribute control commands and dynamic calibration metadata to the plurality of slave camera modules,
an imaging system communicatively coupled to the master camera control unit and interfacing with the plurality of camera modules through a master streaming service to receive two or more channels of stereoscopic video, the imaging system comprising:
a camera calibration module instructing one or more processors to generate dynamic calibration metadata comprising one or more sets of calibration parameters for one or more camera modules, the dynamic calibration metadata including an identifier associating each set of calibration parameters with a particular camera module, the camera calibration module distributing the dynamic calibration metadata to the master camera control unit; and
the master camera control unit calibrating the plurality of camera modules by transmitting dynamic calibration metadata having the master camera module id to the master mono camera controller and dynamic calibration metadata having a slave camera module id to a slave camera within the plurality of slave camera modules identified by the slave camera module id.
2. The digital camera device of claim 1 , wherein the dynamic calibration metadata further comprises real time calibration metadata describing calibration parameters for non-integer camera settings.
3. The digital camera device of claim 1 , wherein the dynamic calibration metadata further comprises real time calibration metadata describing calibration parameters for uncalibrated camera settings.
4. The digital camera device of claim 1 , wherein the dynamic calibration metadata further comprises re-calibration data including pixel position coordinates correcting calibration errors when applied to image pixel data during the process of mapping image pixels to display pixels.
5. The digital camera device of claim 1 , wherein the dynamic calibration metadata further comprises re-calibration data including re-calibration parameters describing a stereoscopic calibration position without from calibration errors.
6. The digital camera device of claim 1 , further comprising a vision processing module included in the imaging system, the vision processing module receiving one or more stereoscopic image frames from the plurality of camera modules and calculating depth information from the one or more stereoscopic image frames.
7. The digital camera device of claim 6 , further comprising a 3D rendering engine included in the imaging system, the 3D rendering engine combining RGB color data with depth information generated by the vision processing module to create 3D content.
8. The digital camera device of claim 7 , wherein the 3D content includes one or more 3D models of objects appearing in image data captured by the plurality of camera devices.
9. The digital camera device of claim 1 , further comprising a remote streaming service included in the imaging system, the remote streaming service including a remote streaming client and a remote streaming server, the remote streaming client generating 3D image and video streams from 3D content created by the 3D rendering engine, the remote streaming server distributing 3D image and video streams to a plurality of client devices over a content steaming network.
10. The digital camera device of claim 1 , further comprising a special sensor selected from the group comprising inertial measurement units, gyroscopes, accelerometers, other motion sensors, altimeters, and magnetometers.
11. The digital camera device of claim 1 , further comprising an acoustic sensor selected from the group comprising microphones and transducers.
12. The digital camera device of claim 1 , wherein the sensor is an optical sensor selected from the group comprising structured light capture cameras, stereoscopic capture cameras, photometric capture cameras, modulated light capture cameras, visible light sensors (e.g., RGB cameras), ultraviolet (UV) cameras, near infrared cameras, infrared cameras, ambient light sensors, time of flight (TOF) sensors, and optical emitters.
13. The digital camera device of claim 1 , further comprising a touch sensor selected from the group comprising force sensors, capacitive touch sensors, resistive touch sensors, and thermal touch sensors.
14. The digital camera device of claim 1 , further comprising a location sensor selected from the group comprising a GPS system, beacon, trilateration system, and triangulation system.
15. The digital camera device of claim 1 , further comprising an emissions based sensor selected from the group comprising a lidar system, dot field projector, vertical cavity surface-emitting laser sensor and CMOS laser sensor.
16. The digital camera device of claim 1 , wherein the plurality of camera modules comprises eight standard angle lens camera modules having a field of view between 60° and 110°, the eight standard angle lens camera modules mounted to a 360° circular rig.
17. The digital camera device of claim 1 , wherein the plurality of camera modules comprises ten camera modules with five camera modules having a standard field of view between 60° and 110° and five camera modules having a wide field of view greater than 110°.
18. The digital camera device of claim 17 , wherein the ten camera modules are mounted to a 360° decagon rig.
19. A digital camera apparatus for capturing 3D content, the digital camera apparatus having variable camera settings and comprising:
a plurality of camera modules, the plurality of camera modules comprising a master camera module and a plurality of slave camera modules, the master camera module comprising:
a processor;
a memory;
a sensor;
a lens; and
a master camera control unit communicatively coupled to a mono camera controller included in each camera module, the plurality of camera modules configured to capture one or more channels of stereoscopic video,
an imaging system communicatively coupled to the master camera control unit and interfacing with the plurality of camera modules to receive two or more channels of stereoscopic video, the imaging system comprising one or more processors configured to re-calibrate the plurality of camera modules using re-calibration data produced from one or more pairs of stereoscopic image frames extracted from the two or more channels of stereoscopic video, the re-calibration data distributed to the master camera control unit and including an identifier associating each set of re-calibration data generated by the master imaging system with every camera module capturing a stereoscopic image frame used to generate the set of recalibration data,
the master camera control unit using the identifier to distribute re-calibration data to the camera module capturing a stereo image frame used to generate the re-calibration data,
a mono camera controller included a camera module associated with the identifier receiving the re-calibration data and modifying one or more calibration parameters included in a calibration file using the re-calibration data to correct a calibration error.
20. A digital camera apparatus for capturing 3D content, the digital camera apparatus having variable camera settings and comprising:
a plurality of camera modules configured to capture one or more channels of stereoscopic video, the plurality of camera modules comprising a master camera module and a plurality of slave camera modules, the master camera module comprising:
a processor;
a memory;
a sensor;
a lens; and
a master camera control unit interfacing with a master mono camera controller and a proxy camera controller to distribute instructions for controlling each camera module;
an imaging system communicatively coupled a system control unit interfacing with the master camera control unit, the imaging system comprising a user interface and a camera calibration system,
the user interface receiving one or more commands for modifying one or more camera settings of the plurality of camera modules,
the camera calibration system instructing a processor to generate one or more modified camera settings from the one or more commands received by the user interface, for modified camera settings not associated with calibration metadata included in a static calibration file, the camera calibration system further instructing a processor to extract one or more sets of calibration metadata associated with camera settings proximate to the modified camera settings and generate one or more sets of real time calibration metadata describing how to calibrate the plurality of camera modules at the modified camera settings, the camera calibration system providing the optimal calibration metadata and the modified camera settings to the master camera control unit, and
the master camera control unit instructing system control unit to adjust the position of each camera module in the plurality of camera modules to configure the digital camera apparatus to capture video channels with the modified camera settings, the master camera control unit further instructing the mono camera controller for every camera module to adjust the calibration of the camera, module as described by the real time calibration metadata.