High-speed optical tracking with compression and/or CMOS windowing
A high-speed optical tracking with compression and CMOS windowing for tracking applications that require high-speed and/or low latency when delivering pose (orientation and position) data. The high-speed optical tracking with compression and CMOS windowing generally includes an optical tracking system ( 10 ) that sense fiducial ( 22 ) and by extension markers ( 20 ). The markers can either be integrated on tools ( 20 ) handled by an operator (e.g. surgeon) ( 50 ) or fixed on the subject to track (e.g. patient) ( 40 ). The low-level processing ( 101, 107, 140 ) is realized within the tracking system to reduce the overall data to be transferred and considerably reduce the processing efforts. Pose data ( 127 ) are finally computed and transferred to a PC/monitor ( 30,31 ) or to tablet ( 60 ) to be used by an end-user application ( 130 ).
1. A method for high speed tracking of optical fiducials and markers, comprising:
detecting, by at least one optical sensor, at least one fiducial attached to a marker placed on at least one of an object and a person, the at least one optical sensor providing sensor data;
determining at least one of the position and orientation of the marker by processing the sensor data, wherein processing the sensor data comprises:
transferring the sensor data to a compression module configured to compress the sensor data to form a compressed image, and
determining the at least one of the position and orientation of the marker from the compressed sensor data; and
transmitting the at least one of the determined position and orientation of the marker through a communication channel as processed data to an application.
2. The method as defined in claim 1 , wherein the sensor data comprises partial sensor data comprising sufficient information for determining the at least one of the position and orientation of the marker.
3. The method as defined in claim 1 , wherein processing the sensor data further comprises segmenting the compressed sensor data depending on an intensity of a plurality of pixels to form a segmented image.
4. The method as defined in claim 3 , wherein processing the sensor data further comprises:
detecting one or more blobs in the segmented image; and
determining the position of the one or more blobs.
5. The method as defined in claim 1 , further comprising making a geometry matching between a cloud of 3D reconstructed points and a geometry of the object.
6. The method as defined in claim 1 , wherein the application comprises a navigation application configured to track the marker.
7. The method as defined in claim 1 , wherein at least one of the sensor data and the processed data is used to control a robotic structure.
8. The method as defined in claim 7 , wherein the robotic structure is used in a surgical application.
9. The method as defined in claim 7 , wherein the robotic structure is held by an operator.
10. The method as defined in claim 1 , wherein the at least one optical sensor is a plenoptic camera.
11. The method as defined in claim 10 , wherein the plenoptic camera comprises an array of micro-lenses placed in front of an array of sensors, wherein each micro-lens and one or more corresponding pixels of the sensor array form a micro-camera.
12. The method as defined in claim 11 , wherein the compression module is configured to receive information from a subset of the micro-camera pixels.
13. The method according to claim 11 , wherein information from at least one micro-camera is used to determine the position of the at least one fiducial in the compressed image.
14. A device for the high-speed tracking of optical fiducials and markers, comprising:
at least one fiducial attached to a marker placed on at least one of an object and a person;
an optical tracking system configured to detect the at least one fiducial and provide sensor data;
a processing unit configured to:
compress the sensor data to form a compressed image, and
determine at least one of the position and orientation of the marker from the compressed sensor data; and
a communication unit configured to transfer the at least one of the determined position and orientation of the marker.
15. The device as defined in claim 14 , wherein the sensor data comprises partial sensor data detected within a window of time.
16. The device as defined in claim 14 , further comprising an electronic device configured to receive the at least one of the determined position and orientation.
17. The device as defined in claim 16 , wherein the electronic device is a tablet.
18. The device as defined in claim 14 , wherein the communication unit is configured to transfer the at least one of the determined position and orientation of the marker via a wireless communication channel.
19. The device as defined in claim 14 , wherein the optical tracking system comprises at least one camera.
20. The device as defined in claim 14 , wherein the device is configured to control a robotic structure.
21. The device as defined in claim 20 , wherein the robotic structure is used in a surgical application.
22. The device as defined in claim 20 , wherein the robotic structure is held by an operator.
23. The device as defined in claim 14 , wherein the optical tracking system comprises a plenoptic camera.
24. The device as defined in claim 23 , wherein the plenoptic camera comprises an array of micro-lenses placed in front of an array of sensors, wherein each micro-lens and one or more corresponding pixels of the sensor array form a micro-camera.
25. The device as defined in claim 24 , wherein the processing unit is configured to receive information from a portion of the micro-camera pixels.
26. The device as defined in claim 24 , wherein information from at least one micro-camera is used to define the position of the at least one fiducial in the compressed image.