IP Library › Granted Patent US 11,931,107
Granted Patent B1
US 11,931,107 · App. 17/062,799 · Granted Mar 19, 2024

Intraoperative three-dimensional bone model generation

Inventors: Sied W. Janna (Memphis, TN); Daniel Farley (Memphis, TN)
Assignees: Smith & Nephew, Inc.; Smith & Nephew Orthopaedics AG; Smith & Nephew Asia Pacific Pte. Limited
A61B34/10A61B34/20A61B90/36A61B90/39G06N3/08A61B2034/102A61B2034/2046A61B2090/363A61B2090/3983
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,931,107
App. No.
17/062,799
Granted
Mar 19, 2024
Kind
B1
Abstract

Methods, non-transitory computer readable media, and surgical computing devices are disclosed herein for creating a three-dimensional (3D) model based on a plurality of received two-dimensional (2D) medical images containing patient anatomy and a tracking fiducial. Once the 2D images are received, a determination is made regarding any potential processing steps required to put the images into a standard view. Once the images are processed, a user may modify or adjust various factors. Using the size and orientation of the tracking fiducial a 3D virtual model is created based on a repository of known patient data, such as a bone atlas. The 3D virtual model can then be output to a display device and optionally used to facilitate a surgical procedure. The 3D virtual model of patient anatomy can advantageously be generated more quickly and using fewer resources with this technology.

Claims (50)

1. A method for patient anatomy modeling to facilitate efficient surgical planning and procedures, the method implemented by one or more surgical computing devices and comprising:

obtaining at least first and second two-dimensional images of anatomy of a patient and at least one tracking fiducial attached to the patient anatomy;

generating a feature vector comprising information regarding a location or proximity of one or more features of the patient anatomy or tracking fiducial identified in each of the first and second two-dimensional images;

applying the feature vector to a stored repository of a plurality of three-dimensional virtual models to identify one of the three-dimensional virtual models that is a closest match to the patient anatomy; and

automatically controlling one or more surgical tools according to a surgical plan generated based on the one of the three-dimensional virtual models.

2. The method of claim 1 , further comprising determining suitability of the second two-dimensional image based on whether a second plane of the second two-dimensional image is substantially orthogonal with respect to a first plane of the first two-dimensional image, wherein the first and second planes are determined based on at least one marker attached to the tracking fiducial or an imaging device.

3. The method of claim 2 , further comprising controlling the imaging device to obtain the second two-dimensional image when a current orientation of the marker is substantially ninety degrees offset from a prior orientation of the marker when the first two-dimensional image was captured.

4. The method of claim 1 , further comprising processing image data associated with the second two-dimensional image to modify one or more characteristics of the second two-dimensional image to match another one or more characteristics of the first two-dimensional image, wherein the image data is processed using a location of the tracking fiducial in the first and second two-dimensional images as a reference.

5. The method of claim 1 , further comprising:

comparing surface data associated with the patient anatomy obtained via a point probe device to the one of the three-dimensional virtual models to generate or output an indication of an accuracy of the one of the three-dimensional virtual models; and

processing the one of the three-dimensional virtual models based on the comparison before outputting the one of the three-dimensional virtual models.

6. The method of claim 1 , further comprising one or more of:

training a machine learning model based on the stored repository;

applying the machine learning model to the feature vector to identify the one of the three-dimensional virtual models; or

updating the machine learning model based on the feature vector and the one of the three-dimensional virtual models.

7. A non-transitory computer readable medium having stored thereon instructions for patient anatomy modeling to facilitate efficient surgical planning and procedures comprising executable code that, when executed by one or more processors, causes the processors to:

obtain a plurality of two-dimensional images of anatomy of a patient and a tracking fiducial attached to the patient anatomy, wherein at least one of the two-dimensional images is substantially orthogonal with respect to at least another of the two-dimensional images;

generate a feature vector comprising location data for a plurality of features identified in at least two or more of the two-dimensional images;

apply the feature vector to a plurality of three-dimensional virtual models to identify one of the three-dimensional virtual models that is a closest match to the patient anatomy; and

automatically generate a surgical plan for a surgical procedure associated with the anatomy of the patient based on the one of the three-dimensional virtual models.

8. The non-transitory computer readable medium of claim 7 , wherein the executable code, when executed by the processors, further causes the processors to determine suitability of the two-dimensional images based on an analysis of a plurality of planes of the two-dimensional images determined based on at least one marker attached to the tracking fiducial or an imaging device.

9. The non-transitory computer readable medium of claim 7 , wherein the executable code, when executed by the processors, further causes the processors to modify one or more characteristics of one or more of the two-dimensional images to match another one or more characteristics of another one or more of the two-dimensional images.

10. The non-transitory computer readable medium of claim 9 , wherein the characteristics of the one or more of the two-dimensional images are modified using a location of the tracking fiducial in the one or more of the two-dimensional images as a reference.

11. The non-transitory computer readable medium of claim 7 , wherein the executable code, when executed by the processors, further causes the processors to:

compare surface data associated with the patient anatomy obtained via a point probe device to the one of the three-dimensional virtual models to generate or output an indication of an accuracy of the one of the three-dimensional virtual models; and

process the one of the three-dimensional virtual models based on the comparison before outputting the one of the three-dimensional virtual models.

12. The non-transitory computer readable medium of claim 7 , wherein the stored repository comprises a bone atlas and the executable code, when executed by the processors, further causes the processors to one or more of:

train a machine learning model based on the bone atlas;

apply the machine learning model to the feature vector to identify the one of the three-dimensional virtual models; or

update the machine learning model based on the feature vector and the one of the three-dimensional virtual models.

13. A surgical computing device, comprising:

a display device;

a non-transitory computer readable medium comprising programmed instructions stored thereon; and

one or more processors coupled to the computer readable medium and configured to execute the stored programmed instructions to:

obtain at least first and second two-dimensional images of anatomy of a patient and at least one tracking fiducial attached to the patient anatomy, wherein the second two-dimensional image is substantially orthogonal with respect to the first two-dimensional image;

generate a feature vector comprising information regarding a location or proximity of one or more features of the patient anatomy or tracking fiducial identified in each of the first and second two-dimensional images;

apply the feature vector to a stored repository of a plurality of three-dimensional virtual models to identify one of the three-dimensional virtual models that is a closest match to the patient anatomy; and

process the one of the three-dimensional virtual models to improve a match of the one of the three-dimensional virtual models with the patient anatomy prior to outputting the one of the three-dimensional virtual models to the display device.

14. The surgical computing device of claim 13 , wherein the processors are further configured to execute the stored programmed instructions to determine suitability of the second two-dimensional image based on whether a second plane of the second two-dimensional image is substantially orthogonal with respect to a first plane of the first two-dimensional image, wherein the first and second planes are determined based on at least one marker attached to the tracking fiducial or an imaging device.

15. The surgical computing device of claim 14 , wherein the processors are further configured to execute the stored programmed instructions to control an imaging device to obtain the second two-dimensional image when a current orientation of the marker is substantially ninety degrees offset from a prior orientation of the marker when the first two-dimensional image was captured.

16. The surgical computing device of claim 13 , wherein the processors are further configured to execute the stored programmed instructions to process image data associated with the second two-dimensional image to modify one or more characteristics of the second two-dimensional image to match another one or more characteristics of the first two-dimensional image, wherein the image data is processed using a location of the tracking fiducial in the first and second two-dimensional images as a reference.

17. The surgical computing device of claim 13 , wherein the one of the three-dimensional virtual models is processed based on a comparison of surface data associated with the patient anatomy obtained via a point probe device to the one of the three-dimensional virtual models.

18. The surgical computing device of claim 13 , wherein the stored repository comprises a bone atlas and the processors are further configured to execute the stored programmed instructions to one or more of:

train a machine learning model based on the bone atlas;

apply the machine learning model to the feature vector to identify the one of the three-dimensional virtual models; or

update the machine learning model based on the feature vector and the one of the three-dimensional virtual models.

19. The surgical computing device of claim 13 , wherein the processors are further configured to execute the stored programmed instructions to output the one of the three-dimensional virtual models to the display device alone with a visual representation of the first and second two dimensional images orientated based on a substantially ninety degree offset angle.

20. The surgical computing device of claim 19 , wherein the processors are further configured to execute the stored programmed instructions to:

lock the visual representation to the one of the three-dimensional virtual models based on a determined relative position of the first and second two-dimensional images and the one of the three-dimensional virtual models; and

move the locked visual representation and one of the three-dimensional virtual models in response to a received user input.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 2, 2021
From: SMITH & NEPHEW, INC.
To: SMITH & NEPHEW, INC.; SMITH & NEPHEW ORTHOPAEDICS AG; SMITH & NEPHEW ASIA PACIFIC PTE. LIMITED
Reel/Frame 057886/0807 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2021
From: JANNA, SIED W.; FARLEY, DANIEL
To: SMITH & NEPHEW, INC.
Reel/Frame 056159/0204 →
Continuity (1)
Provisional Application 62910161 · Oct 3, 2019
Cited By (1)
US 12,369,981