IP Library › Granted Patent US 10,692,214

Granted Patent B2

US 10,692,214 · App. 16/045,551 · Granted Jun 23, 2020

Method and apparatus to generate and track standardized anatomical regions automatically

Inventor: Oleg Bisker (Warren, NJ)

Assignee: Canfield Scientific, Incorporated

G06T7/0016 G06K9/3233 G06K9/6202 G06T7/344 G06T7/74 G06T7/75 G06T17/20 G06K2009/6213 G06T2207/10028 G06T2207/30088 G06T2207/30201

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 10,692,214
App. No.: 16/045,551
Granted: Jun 23, 2020
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Methods and apparatus are disclosed that assist a user in tracking changes that occur in an anatomical region of one or more subjects based on three-dimensional images of the region captured at different times, such as before and after treatment. In exemplary implementations, a three-dimensional anatomical reference model derived from a population of relevance to the subjects is deformed to fit at least a baseline image of each of the subjects. The deformed model of each subject's baseline image is further deformed to fit a follow-up image of the subject's anatomical region. The deformed models thus generated are used to establish sparse correspondences between the images from which surrounding, denser correspondences between the images are additionally found.

Claims (209)

1. An anatomical imaging apparatus comprising:

a storage device containing instructions; and

a processor for executing the instructions to:

obtain a reference model including a reference anatomical region;

obtain first and second images of a subject anatomical region corresponding to the reference anatomical region, the second image having been captured after the first image;

determine a correspondence relationship between the first and second images by using the reference model; and

control a display device to display at least one of the first and second images so as to indicate the correspondence relationship;

wherein the reference model is derived from a population of subjects having at least one characteristic in common with a subject of the subject anatomical region.

2. The apparatus of claim 1 , wherein the reference model and the first and second images are three-dimensional.

3. The apparatus of claim 1 , wherein the storage device contains instructions for execution by the processor to:

obtain a region of interest (ROI) on the reference model including the reference anatomical region, and

transfer the ROI to at least one of the first and second images.

4. The apparatus of claim 3 , wherein the storage device contains instructions for execution by the processor to:

obtain a plurality of further first images of a plurality of subjects' respective anatomical regions corresponding to the reference anatomical region; and

transfer the ROI to the plurality of further first images.

5. The apparatus of claim 1 , wherein determining a correspondence relationship between the first and second images includes:

fitting the reference model to the first image of the subject anatomical region to generate a first deformed model; and

fitting at least one of the reference model and the first deformed model to the second image of the subject anatomical region to generate a second deformed model.

6. The apparatus of claim 5 , wherein determining a correspondence relationship between the first and second images includes:

determining corresponding seed points in the first and second images using the first and second deformed models; and

determining corresponding mesh vertices proximate to the matching seed points in the first and second images.

7. The apparatus of claim 6 , wherein determining corresponding seed points includes:

determining a first point of the first deformed model and a second point of the second deformed model that correspond to the same point on the reference model;

determining a first location in the first image corresponding to the first point of the first deformed model; and

determining a second location in the second image corresponding to the second point of the first deformed model,

wherein the corresponding seed points are at the first and second locations.

8. The apparatus of claim 6 , wherein determining corresponding mesh vertices includes:

selecting a first area surrounding a first mesh vertex of the first image; and

finding a second area of the second image that is correlated to the first area, the second area surrounding a second mesh vertex of the second image,

wherein the first and second mesh vertices are corresponding mesh vertices.

9. The apparatus of claim 1 , wherein indicating the correspondence relationship includes:

determining one or more differences between the first and second images; and

generating a visual representation of the one or more differences for display by the display device.

10. The apparatus of claim 1 , wherein the storage device contains instructions for execution by the processor to:

obtain a third image of the subject anatomical region corresponding to the reference anatomical region, the third image having been captured after the second image;

determine a correspondence relationship between at least one of the first and second images and the third image by using the reference model; and

control the display device to display the third image and at least one of the first and second images so as to indicate the correspondence relationship between the third image and at least one of the first and second images.

11. The apparatus of claim 1 , wherein the storage device contains instructions for execution by the processor to generate at least one measurement indicative of the correspondence relationship between the first and second images.

12. A method performed by an anatomical imaging apparatus comprising:

obtaining a reference model including a reference anatomical region;

obtaining first and second images of a subject anatomical region corresponding to the reference anatomical region, the second image having been captured after the first image;

determining a correspondence relationship between the first and second images by using the reference model; and

controlling a display device to display at least one of the first and second images so as to indicate the correspondence relationship,

wherein the reference model is derived from a population of subjects having at least one characteristic in common with a subject of the subject anatomical region.

13. The method of claim 12 , wherein the reference model and the first and second images are three-dimensional.

14. The method of claim 12 , comprising:

obtaining a region of interest (ROI) on the reference model including the reference anatomical region, and

transferring the ROI to at least one of the first and second images.

15. The method of claim 14 , comprising:

obtaining a plurality of further first images of a plurality of subjects' respective anatomical regions corresponding to the reference anatomical region; and

transferring the ROI to the plurality of further first images.

16. The method of claim 12 , wherein determining a correspondence relationship between the first and second images includes:

fitting the reference model to the first image of the subject anatomical region to generate a first deformed model; and

fitting at least one of the reference model and the first deformed model to the second image of the subject anatomical region to generate a second deformed model.

17. The method of claim 16 , wherein determining a correspondence relationship between the first and second images includes:

determining corresponding seed points in the first and second images using the first and second deformed models; and

determining corresponding mesh vertices proximate to the matching seed points in the first and second images.

18. The method of claim 17 , wherein determining corresponding seed points includes:

determining a first point of the first deformed model and a second point of the second deformed model that correspond to the same point on the reference model;

determining a first location in the first image corresponding to the first point of the first deformed model; and

determining a second location in the second image corresponding to the second point of the first deformed model,

wherein the corresponding seed points are at the first and second locations.

19. The method of claim 17 , wherein determining corresponding mesh vertices includes:

selecting a first area surrounding a first mesh vertex of the first image; and

finding a second area of the second image that is correlated to the first area, the second area surrounding a second mesh vertex of the second image,

wherein the first and second mesh vertices are corresponding mesh vertices.

20. The method of claim 12 , wherein indicating the correspondence relationship includes:

determining one or more differences between the first and second images; and

generating a visual representation of the one or more differences for display by the display device.

21. The method of claim 12 , comprising:

obtaining a third image of the subject anatomical region corresponding to the reference anatomical region, the third image having been captured after the second image;

determining a correspondence relationship between at least one of the first and second images and the third image by using the reference model; and

controlling the display device to display the third image and at least one of the first and second images so as to indicate the correspondence relationship between the third image and at least one of the first and second images.

22. A non-transient computer readable storage medium containing instructions for execution by a processor for carrying out the method of claim 12 .

23. The method of claim 12 comprising generating at least one measurement indicative of the correspondence relationship between the first and second images.

24. An anatomical imaging apparatus comprising:

a storage device containing instructions; and

a processor for executing the instructions to:

obtain a reference model including a reference anatomical region;

obtain first and second images of a subject anatomical region corresponding to the reference anatomical region, the second image having been captured after the first image;

determine a correspondence relationship between the first and second images by using the reference model; and

control a display device to display at least one of the first and second images so as to indicate the correspondence relationship, wherein indicating the correspondence relationship includes:

determining one or more differences between the first and second images; and

generating a visual representation of the one or more differences for display by the display device.

25. The apparatus of claim 24 , wherein the reference model and the first and second images are three-dimensional.

26. The apparatus of claim 24 , wherein the storage device contains instructions for execution by the processor to:

obtain a region of interest (ROI) on the reference model including the reference anatomical region, and

transfer the ROI to at least one of the first and second images.

27. The apparatus of claim 26 , wherein the storage device contains instructions for execution by the processor to:

obtain a plurality of further first images of a plurality of subjects' respective anatomical regions corresponding to the reference anatomical region; and

transfer the ROI to the plurality of further first images.

28. The apparatus of claim 24 , wherein determining a correspondence relationship between the first and second images includes:

fitting the reference model to the first image of the subject anatomical region to generate a first deformed model; and

fitting at least one of the reference model and the first deformed model to the second image of the subject anatomical region to generate a second deformed model.

29. The apparatus of claim 28 , wherein determining a correspondence relationship between the first and second images includes:

determining corresponding seed points in the first and second images using the first and second deformed models; and

determining corresponding mesh vertices proximate to the matching seed points in the first and second images.

30. The apparatus of claim 29 , wherein determining corresponding seed points includes:

determining a first point of the first deformed model and a second point of the second deformed model that correspond to the same point on the reference model;

determining a first location in the first image corresponding to the first point of the first deformed model; and

determining a second location in the second image corresponding to the second point of the first deformed model,

wherein the corresponding seed points are at the first and second locations.

31. The apparatus of claim 29 , wherein determining corresponding mesh vertices includes:

selecting a first area surrounding a first mesh vertex of the first image; and

finding a second area of the second image that is correlated to the first area, the second area surrounding a second mesh vertex of the second image,

wherein the first and second mesh vertices are corresponding mesh vertices.

32. The apparatus of claim 24 , wherein the storage device contains instructions for execution by the processor to:

obtain a third image of the subject anatomical region corresponding to the reference anatomical region, the third image having been captured after the second image;

determine a correspondence relationship between at least one of the first and second images and the third image by using the reference model; and

33. The apparatus of claim 24 , wherein the storage device contains instructions for execution by the processor to generate at least one measurement indicative of the correspondence relationship between the first and second images.

34. A method performed by an anatomical imaging apparatus comprising:

obtaining a reference model including a reference anatomical region;

obtaining first and second images of a subject anatomical region corresponding to the reference anatomical region, the second image having been captured after the first image;

determining a correspondence relationship between the first and second images by using the reference model; and

controlling a display device to display at least one of the first and second images so as to indicate the correspondence relationship, wherein indicating the correspondence relationship includes:

determining one or more differences between the first and second images; and

generating a visual representation of the one or more differences for display by the display device.

35. The method of claim 34 , wherein the reference model and the first and second images are three-dimensional.

36. The method of claim 34 , comprising:

obtaining a region of interest (ROI) on the reference model including the reference anatomical region, and

transferring the ROI to at least one of the first and second images.

37. The method of claim 36 , comprising:

obtaining a plurality of further first images of a plurality of subjects' respective anatomical regions corresponding to the reference anatomical region; and

transferring the ROI to the plurality of further first images.

38. The method of claim 34 , wherein determining a correspondence relationship between the first and second images includes:

fitting the reference model to the first image of the subject anatomical region to generate a first deformed model; and

fitting at least one of the reference model and the first deformed model to the second image of the subject anatomical region to generate a second deformed model.

39. The method of claim 38 , wherein determining a correspondence relationship between the first and second images includes:

determining corresponding seed points in the first and second images using the first and second deformed models; and

determining corresponding mesh vertices proximate to the matching seed points in the first and second images.

40. The method of claim 39 , wherein determining corresponding seed points includes:

determining a first point of the first deformed model and a second point of the second deformed model that correspond to the same point on the reference model;

determining a first location in the first image corresponding to the first point of the first deformed model; and

determining a second location in the second image corresponding to the second point of the first deformed model,

wherein the corresponding seed points are at the first and second locations.

41. The method of claim 39 , wherein determining corresponding mesh vertices includes:

selecting a first area surrounding a first mesh vertex of the first image; and

finding a second area of the second image that is correlated to the first area, the second area surrounding a second mesh vertex of the second image,

wherein the first and second mesh vertices are corresponding mesh vertices.

42. The method of claim 34 , comprising:

obtaining a third image of the subject anatomical region corresponding to the reference anatomical region, the third image having been captured after the second image;

determining a correspondence relationship between at least one of the first and second images and the third image by using the reference model; and

43. The method of claim 34 comprising generating at least one measurement indicative of the correspondence relationship between the first and second images.

44. A non-transient computer readable storage medium containing instructions for execution by a processor for carrying out the method of claim 34 .

45. An anatomical imaging apparatus comprising:

a storage device containing instructions; and

a processor for executing the instructions to:

obtain a reference model including a reference anatomical region;

obtain first and second images of a subject anatomical region corresponding to the reference anatomical region, the second image having been captured after the first image;

determine a correspondence relationship between the first and second images by using the reference model; and

control a display device to display at least one of the first and second images so as to indicate the correspondence relationship,

wherein determining a correspondence relationship between the first and second images includes:

fitting the reference model to the first image of the subject anatomical region to generate a first deformed model;

fitting at least one of the reference model and the first deformed model to the second image of the subject anatomical region to generate a second deformed model;

determining corresponding seed points in the first and second images using the first and second deformed models; and

determining corresponding mesh vertices proximate to the matching seed points in the first and second images, wherein determining corresponding mesh vertices includes:

selecting a first area surrounding a first mesh vertex of the first image; and

finding a second area of the second image that is correlated to the first area, the second area surrounding a second mesh vertex of the second image,

wherein the first and second mesh vertices are corresponding mesh vertices.

46. The apparatus of claim 45 , wherein the reference model and the first and second images are three-dimensional.

47. The apparatus of claim 45 , wherein the storage device contains instructions for execution by the processor to:

obtain a region of interest (ROI) on the reference model including the reference anatomical region, and

transfer the ROI to at least one of the first and second images.

48. The apparatus of claim 47 , wherein the storage device contains instructions for execution by the processor to:

obtain a plurality of further first images of a plurality of subjects' respective anatomical regions corresponding to the reference anatomical region; and

transfer the ROI to the plurality of further first images.

49. The apparatus of claim 45 , wherein determining corresponding seed points includes:

determining a first point of the first deformed model and a second point of the second deformed model that correspond to the same point on the reference model;

determining a first location in the first image corresponding to the first point of the first deformed model; and

determining a second location in the second image corresponding to the second point of the first deformed model,

wherein the corresponding seed points are at the first and second locations.

50. The apparatus of claim 45 , wherein the storage device contains instructions for execution by the processor to:

obtain a third image of the subject anatomical region corresponding to the reference anatomical region, the third image having been captured after the second image;

determine a correspondence relationship between at least one of the first and second images and the third image by using the reference model; and

51. The apparatus of claim 45 , wherein the storage device contains instructions for execution by the processor to generate at least one measurement indicative of the correspondence relationship between the first and second images.

52. A method performed by an anatomical imaging apparatus comprising: