IP Library › Granted Patent US 12,236,624
Granted Patent B2
US 12,236,624 · App. 17/483,305 · Granted Feb 25, 2025

Optical caliper for 3-D endoscopic imaging and measurement

Inventors: Carlos Omar Morales (Tewksbury, MA); Xuanye Wang (Reading, MA); Eduardo Tepichin-Rodríguez (Puebla, MX); Andrea-Fernanda Muñoz-Potosi (Santander, CO); Luis-Gabriel Valdivieso-González (Santander, CO); Ponciano Rodriguez-Montero (Puebla, MX)
Assignees: SMITH & NEPHEW, INC.; SMITH & NEPHEW ORTHOPAEDICS AG; SMITH & NEPHEW ASIA PACIFIC PTE. LIMITED
G06T7/521G06T5/50G06T7/10G06T15/06G06T2207/10048G06T2207/10068
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,236,624
App. No.
17/483,305
Filed
Sep 23, 2021
Granted
Feb 25, 2025
Kind
B2
Art Unit
2665
USPC
382/154
Abstract

A system for an endoscope includes a camera head coupled to an optical channel. The system also includes a light port coupled to the endoscope and configured to receive a first light ray, wherein the first light ray travels through a first optical path along an optical axis of the optical channel. The system also includes a depth measurement module coupled to the endoscope, wherein the depth measurement module is configured to transmit a second light ray through a second optical path along the optical axis. The system also includes an image sensor coupled to the camera head and configured to receive a first set of images pertaining to the first light ray and a second set of images pertaining to the second light ray. The system also includes a processing device configured to use the first and second sets of images to generate one or more 3-D images.

Claims (48)

1. A system for an endoscope, comprising:

a camera head coupled to an optical channel;

a light port coupled to the endoscope and configured to receive a first light ray, wherein the first light ray travels through a first optical path along an optical axis of the optical channel;

a depth measurement module coupled to the endoscope, wherein the depth measurement module is configured to transmit a second light ray through a second optical path along the optical axis and includes a light source configured to transmit the second light ray and a microcontroller coupled to a Liquid Crystal Display (LCD) configured to generate a grating through which the second light ray passes to project a grid pattern on a surface;

an objective lens configured to collimate the second light ray;

an image sensor coupled to the camera head and configured to receive a first set of images pertaining to the first light ray and a second set of images pertaining to the second light ray; and

a processing device configured to receive the first and second sets of images from the image sensor and use the first and second sets of images to generate one or more three-dimensional (3-D) images of the surface, wherein the processing device adjusts a density of the grid pattern using the LCD to displace a focal point of the objective lens and varies a distance between self-images of the grating and the focal point of the objective lens to obtain depth information of the surface.

2. The system of claim 1 , wherein the depth measurement module comprises a beam splitter configured to direct at least a portion of the second light ray through the optical channel.

3. The system of claim 2 , wherein the processing device is further configured to generate, using the grid pattern, the one or more 3-D images.

4. The system of claim 2 , wherein the microcontroller is configured to adjust a rotation of the grid pattern using the LCD.

5. The system of claim 1 , wherein the second optical path is disposed at least between a distal end of the endoscope and the camera head.

6. The system of claim 1 , wherein the first set of images comprise two-dimensional (2-D) images.

7. The system of claim 1 , wherein the second set of images comprise the depth information.

8. The system of claim 1 , wherein the first and second light rays comprise different portions of an electromagnetic spectrum.

9. The system of claim 1 , wherein the first light ray is a white light and the second light ray is an ultraviolet (UV) ray or an infrared ray.

10. The system of claim 9 , further comprising a second sensor configured to receive the second light ray.

11. The system of claim 1 , wherein the processing device is configured to use at least one of distortion and phase information obtained from the second set of images to generate the one or more 3-D images.

12. The system of claim 1 , wherein the processing device is configured to generate, via an artificial intelligence engine, a machine learning model trained to adjust at least one of the density of the grid pattern or a rotation of the grid pattern displayed on the LCD.

13. A method of generating a three-dimensional (3-D) image of an object, comprising:

transmitting a first light ray through a first optical path associated with an optical axis of an optical channel;

transmitting a second light ray through a second optical path associated with the optical axis;

passing the second light ray through a grating generated using a microcontroller coupled to a Liquid Crystal Display (LCD) to form a grid pattern;

projecting the grid pattern on a surface;

receiving a first set of images captured by an image sensor, wherein the first set of images pertain to the first light ray;

receiving a second set of images captured from the image sensor, wherein the second set of images pertain to the second light ray and the grid pattern;

adjusting a density of the grid pattern using the LCD to displace a focal point of an objective lens configured to collimate the second light ray;

varying a distance between self-images of the grating and the focal point of the objective lens to obtain depth information of the surface;

generating, using the first and second sets of images, the 3-D image of the surface.

14. The method of claim 13 , further comprising:

rotating the grid pattern about the optical axis; and

generating, using the grid pattern, the 3-D image.

15. The method of claim 13 , further comprising

generating, using the grid pattern, the 3-D image.

16. The method of claim 13 , further comprising:

generating, via an artificial intelligence engine, a machine learning model trained to generate image data based on at least one of the first and second sets of images and the grid pattern.

17. The method of claim 13 , wherein the first optical path and a section of the second optical path are disposed in the optical channel.

18. A system for an endoscope, comprising:

a camera head coupled to an optical channel;

a light port coupled to the endoscope and configured to receive a first light ray, wherein the first light ray travels through a first optical path along an optical axis of the optical channel;

a depth measurement module coupled to the endoscope and including a microcontroller coupled to a Liquid Crystal Display (LCD), wherein the depth measurement module is configured to generate a grating using the microcontroller and the LCD and pass a second light ray through the LCD to create a grid pattern and transmit the grid pattern toward a beam splitter, wherein the beam splitter is configured to reflect at least a portion of the grid pattern through the optical axis and onto a surface;

an objective lens configured to collimate the second light ray;

an image sensor configured to receive a first set of images captured from the camera head, wherein the first set of images pertain to the first light ray;

a second image sensor configured to receive a second set of images captured from the camera head, wherein the second set of images pertain to the second light ray and the grid pattern; and

a processing device configured to receive the first and second sets of images and the grid pattern from the image sensors and use the first and second sets of images and the grid pattern to generate one or more three-dimensional (3-D) images of the surface, wherein the processing device adjusts a density of the grid pattern using the LCD to displace a focal point of the objective lens and varies a distance between self-images of the grating and the focal point of the objective lens to obtain depth information of the surface.

19. The system of claim 18 , wherein the processing device is configured to generate, via an artificial intelligence engine, a machine learning model trained to generate image data based on at least one of the first and second sets of images and the grid pattern.

20. The system of claim 1 , wherein the processing device is configured to:

calculate a displacement of the focal point relative to the self-images of the grating; and

generate the one or more three-dimensional (3-D) images of the surface based on the displacement.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2021
From: MORALES, CARLOS OMAR; WANG, XUANYE; RODRIGUEZ, EDUARDO TEPICHIN; POTOSI, ANDREA FERNANDA MUNOZ; GONZALEZ, LUIS GABRIEL VALDIVIESO; MONTERO, PONCIANO RODRIGUEZ
To: SMITH & NEPHEW, INC.
Reel/Frame 057592/0080 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2021
From: SMITH & NEPHEW, INC.
To: SMITH & NEPHEW, INC.; SMITH & NEPHEW ORTHOPAEDICS AG; SMITH & NEPHEW ASIA PACIFIC PTE. LIMITED
Reel/Frame 057592/0145 →
Continuity (2)
Provisional Application 63082056 · Sep 23, 2020
Related Publication 20220092805A1 · Mar 24, 2022
References Cited (44)
US 5200838A · Nudelman et al. · 1993 [cited by applicant]
US 7061628B2 · Franke · 2006 [cited by examiner]
US 7453581B2 · Greve · 2008 [cited by examiner]
US 8444652B2 · Amis et al. · 2013 [cited by applicant]
US 9689809B2 · Buchwald · 2017 [cited by examiner]
US 9782056B2 · McDowall · 2017 [cited by examiner]
US 10307296B2 · Tan et al. · 2019 [cited by applicant]
US 10499996B2 · de Almeida Barreto · 2019 [cited by applicant]
US 10750151B2 · Babayoff · 2020 [cited by examiner]
US 10973398B2 · McDowall · 2021 [cited by examiner]
US 11321860B2 · Matsumoto · 2022 [cited by examiner]
US 11457981B2 · Govari · 2022 [cited by examiner]
US 11593946B2 · Miyata · 2023 [cited by examiner]
US 20080278809A1 · Redert · 2008 [cited by examiner]
US 20110017827A1 · Hayashi · 2011 [cited by examiner]
US 20110181748A1 · Ohnishi · 2011 [cited by examiner]
US 20130050833A1 · Lewis · 2013 [cited by examiner]
US 20130335531A1 · Lee · 2013 [cited by examiner]
US 20150346475A1 · Moriuchi · 2015 [cited by examiner]
US 20150351967A1 · Lim et al. · 2015 [cited by applicant]
US 20160223754A1 · Yamazaki · 2016 [cited by examiner]
US 20170085789A1 · Tatsuta · 2017 [cited by examiner]
US 20180284429A1 · He · 2018 [cited by examiner]
US 20200218057A1 · Blazejewski · 2020 [cited by examiner]
US 20210051270A1 · Ohta · 2021 [cited by examiner]
US 20210161718A1 · Ng et al. · 2021 [cited by applicant]
US 20210196384A1 · Shelton, IV · 2021 [cited by examiner]
US 20230186476A1 · Ghazvinian Zanjani · 2023 [cited by examiner]
CN 110840386A · 2020 [cited by applicant]
CN 111685711A · 2020 [cited by applicant]
EP 2720013A2 · 2014 [cited by examiner]
JP 5909443B2 · 2016 [cited by examiner]
WO 2021071988A1 · 2021 [cited by applicant]
WO 2021071991A1 · 2021 [cited by applicant]
WO 2021168408A1 · 2021 [cited by applicant]
WO 2021203077A1 · 2021 [cited by applicant]
WO 2021203082A1 · 2021 [cited by applicant]
WO 2021257672A1 · 2021 [cited by applicant]
WO 2022006041A1 · 2022 [cited by applicant]
WO 2022006248A1 · 2022 [cited by applicant]
WO 2022159726A1 · 2022 [cited by applicant]
Tepichin-Rodriguez et al., “Talbot effect based tunable setup for the measurement of stepped surfaces: Plane and spherical wavefront illumination,” Optics Communications 473 (2020) 126044, pp. 1-8. [cited by applicant]
Wen et al., “The talbot effect: recent advances in classical optics, nonlinear optics, and quantum optics,” 2013, Advances in optics and photonics, 5(1), pp. 83-130. [cited by applicant]
International Patent Application No. PCT/US2022/041845 filed Aug. 29, 2022 listing Quist, Brian William as first Inventor, entitled, “Methods and Systems of Ligament Repair,” 108 pages. [cited by applicant]