Image rotation using software for endoscopic applications
The disclosure extends to endoscopic devices and systems for image correction of a rotating sensor. The disclosure allows for the distal image sensor to rotate as the user rotates the lumen with respect to a fixed handpiece. The system includes an angle sensor located at the junction of the rotating lumen and the fixed handpiece. Periodic measurements of angle are used in the system's image processing chain in order to effect suitable software image rotations, thereby providing a final displayed image or video stream with the desired orientation.
1. An endoscopic device comprising:
a hand-piece;
a proximal portion and a distal portion comprising a tip;
a lumen;
an image sensor disposed within the lumen for providing visualization of an area, wherein the image sensor is at the distal portion near the tip of the endoscope;
an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen;
wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece; and
an image signal processing pipeline for performing rotation transformations upon images that are captured by the image sensor based on the angle of rotation detected by the angle sensor, wherein the image signal processing pipeline is configured to rotate the images counter to the angle of rotation detected by the angle sensor to maintain a constant image horizon for a user on a the display, and wherein an orientation of rotated images for display on the display is rotationally different than the orientation of the lumen.
2. The endoscopic device of claim 1 , wherein the angle sensor is a rotation-detecting Hall-effect sensor.
3. The endoscopic device of claim 2 , wherein the rotation-detecting Hall-effect sensor is located in the hand-piece.
4. The endoscopic device of claim 2 , wherein the device further comprises a diametrically-polarized magnetic annulus and wherein the rotation-detecting Hall-effect sensor produces a voltage that is used to detect an angle of the diametrically-polarized, magnetic annulus.
5. The endoscopic device of claim 1 , wherein the angle sensor produces a voltage that is used to detect the angle of rotation of the hand-piece relative to the lumen.
6. The endoscopic device of claim 1 , wherein the angle sensor is a potentiometer.
7. The endoscopic device of claim 6 , wherein the potentiometer comprises a carbon filament, wherein the carbon filament of the potentiometer is disposed on said lumen.
8. The endoscopic device of claim 1 , where the angle sensor comprises a light source and a photo diode that rotate relative to a gradient disc.
9. The endoscopic device of claim 8 , wherein said photo diode detects the electromagnetic energy from said light source that is reflected by said gradient disc.
10. The endoscopic device of claim 1 , wherein said image sensor incorporates a two-dimensional array of pixels capable of detecting electromagnetic radiation.
11. The endoscopic device of claim 10 , wherein said image rotation transformation comprises taking each initial pixel's integer x,y coordinates and transforming them to final, real number pixel coordinates by applying a rotation kernel.
12. The endoscopic device of claim 11 , wherein said image rotation transformation further comprises truncating the final, real number pixel coordinates to integer values, then assigning values to blank pixels in a final image using the values of nearby, filled pixels.
13. The endoscopic device of claim 12 , wherein said assignment is performed using bilinear interpolation.
14. The endoscopic device of claim 12 , wherein said assignment is performed using bicubic interpolation.
15. The endoscopic device of claim 12 , wherein said assignment is performed using nearest neighbor substitution.
16. The endoscopic device of claim 10 , wherein said image rotation transformation comprises taking each final pixel's integer x,y coordinates and transforming them to initial real number x,y coordinates by applying an inverse rotation kernel.
17. The endoscopic device of claim 16 , wherein said image rotation transformation further comprises estimating a pixel value at the initial real number x,y coordinates, using data from one or more closest integer coordinate locations.
18. The endoscopic device of claim 17 , wherein said estimation is performed using nearest neighbor substitution.
19. The endoscopic device of claim 17 , wherein said estimation is performed using bilinear interpolation.
20. The endoscopic device of claim 17 , wherein said estimation is performed using bicubic interpolation.
21. An endoscopic system comprising:
an endoscope unit comprising:
a hand-piece;
a proximal portion and a distal portion comprising a tip;
a lumen;
an image sensor disposed within the lumen for providing visualization of an area, wherein the image sensor is at the distal portion near the tip of the endoscope;
wherein said image sensor incorporates a two dimensional array of pixels capable of detecting electromagnetic radiation;
an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen, wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece; and
an image signal processing pipeline for performing rotation transformations upon images that are captured by the image sensor based on the angle of rotation detected by the angle sensor, wherein the image signal processing pipeline is configured to rotate the images counter to the angle of rotation detected by the angle sensor to maintain a constant image horizon for a user on a display, and wherein an orientation of rotated images for display on the display is rotationally different than the orientation of the lumen.
22. A method of rotating an image in an endoscopic application comprising:
providing an endoscope unit comprising:
a hand-piece;
a proximal portion and a distal portion comprising a tip;
a lumen;
an image sensor disposed within the lumen for providing visualization of an area,
wherein the image sensor is at the distal portion near the tip of the endoscope;
wherein said image sensor incorporates a two dimensional array of pixels capable of detecting electromagnetic radiation;
detecting an angle of rotation of the hand-piece relative to the lumen using an angle sensor, wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece;
performing image rotation transformations upon images captured by the image sensor using an image signal processing pipeline based on the angle of rotation detected by the angle sensor, wherein the image signal processing pipeline is configured to rotate the images counter to the angle of rotation detected by the angle sensor to maintain a constant image horizon for a user on a display, and wherein an orientation of rotated images for display on the display is rotationally different than the orientation of the lumen.
23. The method of claim 22 , wherein the step of performing image rotation transformation comprises taking each initial pixel's integer x,y coordinates and transforming them to final, real number pixel coordinates by applying a rotation kernel.
24. The method of claim 23 , wherein said image rotation transformation further comprises truncating the final, real number pixel coordinates to integer values, then assigning values to blank pixels in a final image using the values of nearby, filled pixels.
25. The method of claim 24 , wherein said assignment is performed using bilinear interpolation.
26. The method of claim 24 , wherein said assignment is performed using bicubic interpolation.
27. The method of claim 24 , wherein said assignment is performed using nearest neighbor substitution.
28. The method of claim 22 , wherein the step of performing image rotation transformation comprises taking each final pixel's integer x,y coordinates and transforming them to initial real number x,y coordinates by applying an inverse rotation kernel.
29. The method of claim 28 , wherein said image rotation transformation further comprises estimating a pixel value at the initial real number x,y coordinates, using the data from one or more closest integer coordinate locations.
30. The method of claim 28 , wherein said estimation is performed using nearest neighbor substitution.
31. The method of claim 28 , wherein said estimation is performed using bilinear interpolation.
32. The method of claim 28 , wherein said estimation is performed using bicubic interpolation.
33. An endoscopic device comprising:
a hand-piece;
a proximal portion and a distal portion comprising a tip;
a lumen;
an image sensor disposed within the lumen for providing visualization of an area, wherein the image sensor is at the distal portion near the tip of the endoscope;
an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen;
wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece; and
an image signal processing pipeline for performing rotation transformations upon images that are captured by the image sensor based on the angle of rotation detected by the angle sensor, wherein the image signal processing pipeline is configured to:
determine a quantized angle of rotation based on the angle of rotation detected by the angle sensor;
determine that the quantized angle of rotation has remained constant for at least a plurality of frames comprising a predetermine number of frames; and
in response do determining that the quantized angle of rotation has remained constant for at least the plurality of frames, rotate the images counter to the angle of rotation detected by the angle sensor to maintain a substantially constant image horizon for a user on the display, and wherein an orientation of rotated images for display on the display is rotationally different than the orientation of the lumen.