IP Library Granted Patent US 12,217,413
Granted Patent B2
US 12,217,413 · App. 18/409,027 · Granted Feb 4, 2025

Surgical kit inspection systems and methods for inspecting surgical kits having parts of different types

Inventors: Perry J. Simson (Bradenton, FL); Graeme A. Field (Jacksonville, FL); Patrick M. Caldwell (Jacksonville, FL)
Assignee: Howmedica Osteonics Corp.
G06T7/0008A61B34/70B25J19/0066G06F18/214G06T7/001G06T7/60G06T7/73G06V10/141G06V10/82G06V20/10G06V30/1437G06V30/153G06T2207/30164G06V2201/034
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,217,413
App. No.
18/409,027
Granted
Feb 4, 2025
Kind
B2
Abstract

Surgical kit inspection systems and methods are provided for inspecting surgical kits having parts of different types. The surgical kit inspection system comprises a vision unit including a first camera unit and a second camera unit to capture images of parts of a first type and a second type in each kit and to capture images of loose parts from each kit that are placed on a light surface. A robot supports the vision unit to move the first and second camera units relative to the parts in each surgical kit. One or more controllers obtain unique inspection instructions for each of the surgical kits to control inspection of each of the surgical kits and control movement of the robot and the vision unit accordingly to provide output indicating inspection results for each of the surgical kits.

Claims (31)

1. An inspection system for inspecting a kit of parts, the inspection system comprising:

a light surface to receive parts from the kit;

one or more cameras configured to obtain one or more images of parts disposed on the light surface;

a robotic manipulator supporting the one or more cameras such that the one or more cameras are capable of being moved by the robotic manipulator relative to the parts disposed on the light surface; and

one or more controllers being configured to:

obtain unique inspection instructions for the kit to control inspection of the kit,

position the robotic manipulator relative to the parts of the kit in accordance with the unique inspection instructions,

operate the one or more cameras to capture the one or more images of the parts disposed on the light surface,

detect parts in the one or more images,

compare the detected parts to a predefined list of parts for the kit, and

generate output indicating inspection result for the kit.

2. The inspection system of claim 1 , further comprising a light source to illuminate the parts and the light surface.

3. The inspection system of claim 2 , wherein the light source is arranged beneath the light surface to back-illuminate the parts.

4. The inspection system of claim 2 , wherein the one or more controllers are further configured to operate the light source in accordance with the unique inspection instructions.

5. The inspection system of claim 1 , wherein the one or more cameras includes a first camera having a first imaging lens and a first liquid lens to obtain images of unique geometric features of the parts disposed on the light surface.

6. The inspection system of claim 5 , wherein the one or more controllers are further configured to use a first identification method to detect unique geometric features of parts in the images.

7. The inspection system of claim 5 , wherein the one or more cameras includes a second camera having a second imaging lens, different than the first imaging lens, and a second liquid lens to obtain images of characters on the parts.

8. The inspection system of claim 7 , wherein the robotic manipulator supports the first and second cameras such that the first and second cameras are capable of being moved by the robotic manipulator.

9. The inspection system of claim 7 , wherein the one or more controllers are further configured to use a first identification method to detect unique geometric features of parts in the images, and a second identification method to detect characters on parts in the images.

10. The inspection system of claim 1 , wherein the robotic manipulator includes a collaborative robotic arm having one or more sensors to detect forces or torques applied to the one or more cameras by the operator to control movement of the collaborative robotic arm.

11. The inspection system of claim 1 , wherein the output generated by the one or more controllers indicates one or more of the following: if any parts are missing; or if any parts have a defect.

12. The inspection system of claim 1 , wherein the one or more controllers are configured to employ a deep learning algorithm to detect the parts in the kit.

13. The inspection system of claim 1 , wherein the one or more controllers are further configured to position the robotic manipulator at a plurality of poses in accordance with the unique inspection instructions.

14. The inspection system of claim 13 , wherein the one or more controllers are further configured to operate the one or more cameras at each of the plurality of poses to capture one or more images at each of the plurality of poses.

15. The inspection system of claim 14 , wherein the one or more controllers are further configured to detect parts in the one or more images captured at each of the plurality of poses.

16. The inspection system of claim 1 , wherein the one or more cameras are configured to obtain one or more images of at least one of: unique geometric features of parts, and characters on parts.

17. The inspection system of claim 1 , wherein the one or more cameras includes a first camera having a first imaging lens to obtain images of unique geometric features of parts of a first type.

18. The inspection system of claim 17 , wherein the one or more cameras further includes a second camera having a second imaging lens, different than the first imaging lens, to obtain images of characters on parts of a second type.

19. The inspection system of claim 18 , wherein the first camera further includes a first liquid lens; and

wherein the second camera further includes a second liquid lens.

20. The inspection system of claim 18 , wherein the one or more controllers are further configured to use a first identification method to detect unique geometric features of parts in the images when utilizing the first camera, and a second identification method to detect characters on parts in the images when utilizing the second camera.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2024
From: SIMSON, PERRY J.
To: HOWMEDICA OSTEONICS CORP.
Reel/Frame 066239/0935 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2024
From: FIELD, GRAEME A.; CALDWELL, PATRICK M.
To: TEKNICAL, LLC
Reel/Frame 066240/0090 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2024
From: TEKNICAL, LLC
To: HOWMEDICA OSTEONICS CORP.
Reel/Frame 066240/0105 →
Continuity (4)
Continuation 18084692 · Dec 20, 2022
Continuation 17342664 · Jun 9, 2021
Provisional Application 63036596 · Jun 9, 2020
Related Publication 20240144466A1 · May 2, 2024
References Cited (90)
US 5515599A · Best · 1996 [cited by applicant]
US 7164968B2 · Treat et al. · 2007 [cited by applicant]
US 7180014B2 · Farber et al. · 2007 [cited by applicant]
US 7557710B2 · Sanchez et al. · 2009 [cited by applicant]
US 7644016B2 · Nycz et al. · 2010 [cited by applicant]
US 7997847B2 · Treat et al. · 2011 [cited by applicant]
US 8567880B2 · Treat et al. · 2013 [cited by applicant]
US 9305218B2 · Lewis et al. · 2016 [cited by applicant]
US 10071856B2 · Hance et al. · 2018 [cited by applicant]
US 10154885B1 · Barnett et al. · 2018 [cited by applicant]
US 10169696B2 · Lee · 2019 [cited by applicant]
US 10239701B2 · Wicks et al. · 2019 [cited by applicant]
US 10357325B2 · Sayani et al. · 2019 [cited by applicant]
US 10360531B1 · Stallman et al. · 2019 [cited by applicant]
US 10390902B2 · Singh et al. · 2019 [cited by applicant]
US 10417754B2 · Kimura et al. · 2019 [cited by applicant]
US 10504623B2 · Woods et al. · 2019 [cited by applicant]
US 10517680B2 · Moctezuma et al. · 2019 [cited by applicant]
US 10520926B2 · Scelsi et al. · 2019 [cited by applicant]
US 10956791B2 · Gafni et al. · 2021 [cited by applicant]
US 11439469B2 · Poltaretskyi et al. · 2022 [cited by applicant]
US 11443501B2 · Donhowe et al. · 2022 [cited by applicant]
US 11593931B2 · Simson et al. · 2023 [cited by applicant]
US 11594324B2 · Quintini et al. · 2023 [cited by applicant]
US 11617625B2 · Rosinski · 2023 [cited by applicant]
US 11620813B2 · Gafni et al. · 2023 [cited by applicant]
US 20070268133A1 · Sanchez et al. · 2007 [cited by applicant]
US 20080108873A1 · Gattani et al. · 2008 [cited by applicant]
US 20090317002A1 · Dein · 2009 [cited by applicant]
US 20150224650A1 · Xu et al. · 2015 [cited by applicant]
US 20150324739A1 · Baker et al. · 2015 [cited by applicant]
US 20160042130A1 · Broninx · 2016 [cited by applicant]
US 20160074128A1 · Dein · 2016 [cited by applicant]
US 20160328813A1 · Montgomery et al. · 2016 [cited by applicant]
US 20170032313A1 · McCullough et al. · 2017 [cited by applicant]
US 20170098049A1 · Sweeney · 2017 [cited by applicant]
US 20170243157A1 · Piron et al. · 2017 [cited by applicant]
US 20190080446A1 · Kuzmin et al. · 2019 [cited by applicant]
US 20190142304A1 · Crawford et al. · 2019 [cited by applicant]
US 20190261566A1 · Robertson et al. · 2019 [cited by applicant]
US 20190279354A1 · Inazumi et al. · 2019 [cited by applicant]
US 20190290796A1 · Ma et al. · 2019 [cited by applicant]
US 20190303721A1 · Kurtz et al. · 2019 [cited by applicant]
US 20190321126A1 · Otto et al. · 2019 [cited by applicant]
US 20190389070A1 · Sirkett · 2019 [cited by applicant]
US 20200005069A1 · Wang et al. · 2020 [cited by applicant]
US 20200034591A1 · Bachelder et al. · 2020 [cited by applicant]
US 20200184248A1 · Donhowe et al. · 2020 [cited by applicant]
US 20200193241A1 · Katayama et al. · 2020 [cited by applicant]
US 20200205926A1 · Keibel · 2020 [cited by applicant]
US 20200273581A1 · Wolf et al. · 2020 [cited by applicant]
US 20200397509A1 · Chen et al. · 2020 [cited by applicant]
US 20210193310A1 · Jacobs et al. · 2021 [cited by applicant]
US 20210212784A1 · Ramadorai · 2021 [cited by applicant]
US 20210268659A1 · Olmstead · 2021 [cited by applicant]
US 20210344820A1 · Boren et al. · 2021 [cited by applicant]
US 20210383523A1 · Simson et al. · 2021 [cited by applicant]
US 20220292815A1 · Donnelly et al. · 2022 [cited by applicant]
US 20230117419A1 · Simson et al. · 2023 [cited by applicant]
US 20230140072A1 · Hawkins · 2023 [cited by examiner]
DE 102010021037A1 · 2011 [cited by applicant]
JP 2022083768A · 2022 [cited by applicant]
WO 2005020063A1 · 2005 [cited by applicant]
WO 2008033574A2 · 2008 [cited by applicant]
WO 2018221599A1 · 2018 [cited by applicant]
WO 2018228818A1 · 2018 [cited by applicant]
WO 2019045282A1 · 2019 [cited by applicant]
WO 2020031192A1 · 2020 [cited by applicant]
WO 2021115583A1 · 2021 [cited by applicant]
WO 2021124716A1 · 2021 [cited by applicant]
WO 2021216535A1 · 2021 [cited by applicant]
E. Carpintero, C. Pérez, R. Morales, N. García, A. Candela and J. Azorín, “Development of a robotic scrub nurse for the operating theatre,” 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Bi… [cited by examiner]
Advanced Illumination, “DL 194 General Purpose Diffuse Lights Specification Sheet”, Dec. 14, 2016, Updated May 17, 2017, 4 pages. [cited by applicant]
Advanced Illumination, “DL 2230 Extra Small Dome Light Specification Sheet”, Dec. 14, 2016, Updated May 17, 2020, 4 pages. [cited by applicant]
Cognex, “Deep Learning for Machine Vision—VisionPro Vidi 3.1 Video”, https://www.cognex.com/resources/on-demand-webinars/deep-learning-for-machine-vision-visionpro-vidi-3-1 2021, 4 pages. [cited by applicant]
Cognex, “Deep Learning vs. Machine Vision”, 2019, 12 pages. [cited by applicant]
Cognex, “ViDi Deep Learning for Factory Automation Best Practices Guide”, 2019, 16 pages. [cited by applicant]
Cognex, “VisionPro ViDi Brochure”, 2019, 2 pages. [cited by applicant]
Edmund Optics, “TechSpec Compact Fixed Focal Length Lens #59-870 16 mm FL f/1.6”, 2014, 5 pages. [cited by applicant]
Edmund Optics, “TechSpec Compact Fixed Focal Length Lens #59-872 35 mm FL f/1.65”, 2014, 5 pages. [cited by applicant]
English language abstract and machine-assisted English translation for DE 10 2010 021 037 A1 extracted from espacenet.com database on Jun. 16, 2021, 15 pages. [cited by applicant]
English language abstract for WO 2018/228818 A1 extracted from espacenet.com database on Jun. 16, 2021, 2 pages. [cited by applicant]
O'Mahony, Niall, “Deep Learning vs. Traditional Computer Vision”, 2020, 17 pages. [cited by applicant]
Opto Engineering, “Product Data Sheet—ENMT-M1224-MPW2-MM”, 2020, 2 pages. [cited by applicant]
Opto Engineering, “Product Data Sheet—MC150X”, 2020, 3 pages. [cited by applicant]
Optotune, “Datasheet: EL-10-30-Series”, 2019 16 pages. [cited by applicant]
Tabernik, Domen et al., “Segmentation-Based Deep-Learning Approach for Surface-Defect Detection” Jun. 11, 2019, 17 pages. [cited by applicant]
Universal Robots, “e-Series from Universal Robots Brochure”, Jun. 2018, 12 pages. [cited by applicant]
Wilson, Andy, “Lens Choices Proliferate for Developers of Machine Vision Systems”, Visions System Design, Sep. 12, 2016, 9 pages. [cited by applicant]
Zhang, Chen L. et al., “Optimization of Virtual and Real Registration Technology Based on Augmented Reality in a Surgical Navigation System”, Biomed Eng Online, vol. 19, No. 1, Jan. 8, 2020, 28 pages. [cited by applicant]