Tracking device for hand tracking in surgical environment
A system for tracking a hand within a surgical environment is disclosed. The system comprises a hand covering including an opening to receive the hand and a plurality of trackable regions that each include at least one tracking feature. The system further comprises a tracking unit including one or more sensors configured to detect a location of each tracking feature. The system further comprises a processor configured to cause the system to receive the location of each tracking feature from the tracking unit, determine a position and an orientation of each trackable region of the hand covering, and calculate an overall position, orientation, and/or pose of the hand. Various actions or settings may be controlled within the surgical workspace by utilizing particular gestures or movements that are identified by the system via the processor.
1. A system for tracking a hand within a surgical environment, the system comprising:
a hand covering including:
an opening configured to receive at least a portion of the hand therein,
a plurality of trackable regions, wherein each trackable region substantially corresponds to an articulating segment of the hand, and wherein the plurality of trackable regions include a collection region, and
a plurality of tracking features, wherein each tracking feature is disposed on one of the plurality of trackable regions;
a tracking unit including one or more sensors configured to detect a location of each tracking feature;
a processor; and
a non-transitory, computer-readable medium storing instructions that, when executed, cause the system to:
receive initiation input to initiate a collection mode,
continuously track a position and an orientation of the collection region during the collection mode, thereby collecting a plurality of points defining a three-dimensional surface by:
receiving the location of each tracking feature from the tracking unit,
determining, based on the location, the position and the orientation of each trackable region of the hand covering, and
calculating a pose of the hand based on the position and the orientation of each trackable region of the hand covering, and
receive termination input to terminate the collection mode, wherein at least one of the initiation input and the termination input comprises at least one of a pushbutton input, a voice command input, a force input, and a gesture input.
2. The system of claim 1 , wherein the instructions, when executed, further cause the system to calculate one or more of a position of the hand and an orientation of the hand based on the position and the orientation of each trackable region of the hand covering.
3. The system of claim 2 , further comprising a surgical instrument,
wherein the instructions, when executed, further cause the system to:
determine a proximity of the hand to the surgical instrument based on the position of the hand, and
selectively control an operation of the surgical instrument based on the proximity of the hand to the surgical instrument.
4. The system of claim 3 , wherein the surgical instrument comprises one of a robotic arm, an end effector, and a handpiece.
5. The system of claim 3 , wherein the operation of the surgical instrument comprises locking of one or more of a position, an orientation, and a pose of the surgical instrument.
6. The system of claim 1 , wherein the instructions, when executed, further cause the system to:
identify, based on the pose of the hand, a hand gesture from a set of predetermined hand gestures; and
execute one or more actions associated with the hand gesture.
7. The system of claim 6 , wherein the one or more actions comprise advancing a computer-aided surgical system to a subsequent stage of a surgical plan.
8. The system of claim 1 , further comprising an augmented reality display,
wherein the instructions, when executed, further cause the system to display a representation of the hand on the augmented reality display in real time based on one or more of the pose of the hand, a position of the hand, and an orientation of the hand.
9. The system of claim 1 , wherein each tracking feature comprises a distinct colored surface of the trackable region, and
wherein the distinct colored surface is optically detectable by the one or more sensors.
10. The system of claim 1 , wherein each tracking feature comprises one or more of an infrared marker, an electromagnetic sensor, and a Fiber Bragg grating sensor.
11. A system for tracking a finger within a surgical environment, the system comprising:
a tracking device configured to be disposed on a finger of a hand, wherein the tracking device includes at least one tracking feature affixed thereto;
a tracking unit including one or more sensors configured to detect a location of each tracking marker;
a processor; and
a non-transitory, computer-readable medium storing instructions that, when executed, cause the system to:
receive initiation input to initiate a collection mode,
continuously track a position and an orientation of the finger during the collection mode, thereby collecting a plurality of points defining a three-dimensional surface by:
receiving the location of each tracking feature from the tracking unit, and
determining, based on the location of each tracking marker, the position and the orientation of the finger, and
receive termination input to terminate the collection mode, wherein at least one of the initiation input and the termination input comprises at least one of a pushbutton input, a voice command input, a force input, and a gesture input.
12. The system of claim 11 , wherein the instructions, when executed, further cause the system to:
track one or more motions of the finger based on the position and the orientation of the finger;
identify a gesture from a set of predetermined gestures based on the one or more motions; and
execute one or more actions associated with the gesture.
13. The system of claim 12 , wherein the one or more actions comprise advancing a computer-aided surgical system to a subsequent stage of a surgical plan.
14. The system of claim 11 , further comprising a surgical instrument,
wherein the instructions, when executed, further cause the system to:
determine a proximity of the finger to the surgical instrument based on the position of the finger, and
selectively control an operation of the surgical instrument based on the proximity of the finger to the surgical instrument.
15. The system of claim 14 , wherein the surgical instrument comprises one of a robotic arm, an end effector, and a handpiece.
16. The system of claim 14 , wherein the operation of the surgical instrument comprises locking of one or more of a position, an orientation, and a pose of the surgical instrument.
17. The system of claim 11 , wherein each tracking feature comprises one or more of an infrared marker, an electromagnetic sensor, and a Fiber Bragg grating sensor.