IP Library › Granted Patent US 11,234,665
Granted Patent B2
US 11,234,665 · App. 16/845,431 · Granted Feb 1, 2022

System and method for increasing the accuracy of a medical imaging device

Inventors: Morten Boll (Espergærde, DK); Thomas Sangild Sørensen (Aarhus C, DK); Jens Peter Holme (Copenhagen S, DK); Sami Brandt (Malmö, SE); Henrik Öjelund (Kgs. Lyngby, DK)
Assignee: 3SHAPE A/S
A61B6/5229A61B6/032A61B6/0492A61B6/4085A61B6/501A61B6/5264G06T7/0014G06T7/20G06T7/70G16H50/50H04N5/247G06T2207/10081G06T2207/30204
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,234,665
App. No.
16/845,431
Granted
Feb 1, 2022
Kind
B2
Abstract

A method for improving the accuracy of a digital medical model of a part of a patient, the method includes obtaining a set of at least 2 medical images of the patient, where an element including a predefined geometry and/or predefined information was attached to the patient during the recording of the medical images; obtaining at least 2 tracking images taken with at least one camera having a known positional relationship relative to the medical imaging device, the tracking images depicting at least part of the element; determining any movement of the element between acquisition of the at least 2 tracking images; and generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images.

Claims (66)

1. A method for improving the accuracy of a digital medical model of a part of a patient, the medical model comprising at least two medical images recorded with a medical imaging device, the method comprising:

obtaining a set of at least two medical images of the patient, where an element comprising a predefined geometry or predefined information was in a stationary relationship with the patient during the recording of the medical images;

at a first time, simultaneously obtaining at least three tracking images taken with at least three cameras having known positional relationships relative to the medical imaging device, said tracking images depicting at least part of the element;

at a second time, simultaneously obtaining at least three additional tracking images taken with the at least three cameras, said tracking images depicting at least part of the element;

determining any movement of the element between acquisition of the at least three tracking images and the at least three additional tracking images; and

generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images;

wherein:

fiducial markers in the element are recognized in each image;

a pattern of the fiducial markers is recognized in each tracking image to achieve a best fit to a known predefined pattern of the fiducial markers on the element from each tracking image;

the position and orientation of the element in each of the three tracking images each time t is determined; and

a weighted average of the position and orientation of the element from the tracking images is used to determine the best fit for the position and orientation of the element at each time t.

2. The method according to claim 1 , wherein the predefined information of the element comprises the fiducial markers in a predefined pattern, size, shape or colour.

3. The method according to claim 1 , wherein the predefined information of the element comprises a geometric shape of the element.

4. The method according to claim 1 , wherein compensating for any movement of the patient between the acquisition of the medical images comprises:

associating a time stamp with each of the medical images and each of the tracking images;

determining a position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition;

adjusting a position of each pixel or voxel of the acquired medical image with an amount corresponding to the movement of the element.

5. The method according to claim 1 , wherein a coordinate system used in determining the position and orientation of the element can be fixed arbitrarily.

6. The method according to claim 1 , wherein the tracking images taken and the medical images are time stamped using a same clock.

7. The method according to claim 1 , wherein determining a position and orientation of the element at each time stamp comprises:

recognizing the fiducial markers in each tracking image;

obtaining a digital representation in a database of a known predefined pattern or shape of the fiducial markers;

recognizing a pattern of the fiducial markers in each image to achieve a best fit to the known predefined pattern of the fiducial markers on the element from each tracking image.

8. The method according to claim 1 , wherein determining the position and orientation of the element at each time stamp comprises:

recognizing the fiducial markers in each tracking image;

using a classification of indices of the fiducial markers; and

matching a known pattern of the fiducial markers on the element to a pattern of the fiducial markers on the tracking image using the classification of the indices of the fiducial markers.

9. The method according to claim 8 , wherein recognizing the fiducial markers comprises performing principal component analysis on the tracking images in order to segment the fiducial markers.

10. The method according to claim 8 , wherein the three cameras are placed at points of an equilateral triangle.

11. The method according to claim 10 , wherein:

a camera position and rotation of each of the cameras is calibrated or determined;

intrinsic parameters such as focal length, skew, principal point and lens distortion are calibrated or determined for each of the cameras;

a position of each of the fiducial markers is determined directly in the camera co-ordinate frame;

a position or orientation of the element from the three tracking images is determined using a cost function to minimise a difference in the determined position of the fiducial markers in each of the tracking images.

12. The method according to claim 8 , wherein the digital medical model is generated in real time.

13. The method according to claim 8 , wherein the medical imaging device is a cone beam computed tomography device.

14. A method according to claim 1 , wherein compensating for any movement of the patient between the acquisition of the medical images comprises:

associating a time stamp with each of the medical images, and each of the tracking images;

determining a position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition;

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition.

15. The method according to claim 14 , wherein the predefined information of the element comprises at least one fiducial marker in a predefined pattern, size, shape or colour.

16. The method according to claim 15 , wherein the predefined information of the element comprises a geometric shape of the element.

17. The method according to claim 14 , wherein compensating for any movement of the patient between the acquisition of the medical images comprises:

associating a time stamp with each of the medical images, and each of the tracking images;

determining a position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition;

adjusting a position of each pixel or voxel of the acquired medical image with an amount corresponding to the movement of the element.

18. The method according to claim 17 , wherein determining the position and orientation of the element at each time stamp comprises:

recognizing a plurality of the individual fiducial markers in each tracking image;

obtaining a digital representation in a database of a known predefined pattern and/or shape of the fiducial markers;

recognizing the pattern of the fiducial markers in each image to achieve a best fit to a known predefined pattern of the fiducial markers on the element from each tracking image.

19. The method according to claim 17 , wherein determining the position and orientation of the element at each time stamp comprises:

recognizing a plurality of the fiducial markers in each tracking image;

using a classification of indices of the fiducial markers; and

matching a known pattern of the fiducial markers on the element to a pattern of the fiducial markers on the tracking image using the classification of the indices of the fiducial markers.

20. The method according to claim 17 , wherein recognizing the fiducial markers comprises performing principal component analysis on the tracking images in order to segment the fiducial markers.

21. The method according to claim 17 , wherein the three cameras are placed at points of an equilateral triangle.

22. The method according to claim 21 , wherein: a camera position and rotation of each of the cameras is calibrated or determined;

focal length, skew, principal point and lens distortion are calibrated or determined for each of the cameras; and

a position or orientation of the element from the three images is determined using a cost function to minimise a difference in a determined position of the fiducial markers in each of the tracking images.

23. The method according to claim 14 , wherein compensating for any movement of the patient between the acquisition of the medical images comprises:

associating a time stamp with each of the medical images, and each of the tracking images;

determining a position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition;

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition.

24. The method according to claim 14 , wherein a coordinate system used in determining the position and orientation of the element can be fixed arbitrarily.

25. The method according to claim 14 , wherein the tracking images taken and the medical images are time stamped using a same clock.

26. The method according to claim 14 , wherein the medical imaging device is a cone beam computed tomography device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2022
From: 3SHAPE A/S
To: NEWTON2 APS
Reel/Frame 061673/0078 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 10, 2020
From: BOLL, MORTEN; SØRENSEN, THOMAS SANGILD; HOLME, JENS PETER; BRANDT, SAMI; ÖJELUND, HENRIK
To: 3SHAPE A/S
Reel/Frame 052367/0824 →
Priority Claims (1)
DK PA 2015 70121 · Mar 4, 2015 · national
Continuity (3)
Continuation 16420338 · May 23, 2019
Continuation 15555502
Related Publication 20200305827A1 · Oct 1, 2020