System and method for machine-learning based sensor analysis and vascular tree segmentation
Methods for automated identification of vascular features are described. In some embodiments, one or more machine learning (ML)-based vascular classifiers are used, with their results being combined to with results of at least one other vascular classifier in order to produce the final results. Potentially advantages of this approach include the ability to combine certain strengths of ML classifiers with segmentation approaches based on more classical (“formula-based”) methods. These strengths may include particularly the identification of anatomically identified targets mixed within an image also showing similar looking but anatomically distinct targets.
1. A method of identifying vascular portions of a vascular image, the method comprising:
accessing the vascular image comprising a 2-D angiographic image;
applying a first ML-based vascular identifier to the vascular image to produce a first data structure identifying a path that extends along at least some vascular portions, wherein a vascular portion is an image location associated with part of a blood vessel depicted in the vascular image;
applying a second vascular identifier to the vascular image to produce a second data structure identifying one or more pixel masks, wherein the pixel masks comprise respective blobs of vascular portions;
wherein the path
includes some vascular portions not in the one or more pixels masks; and
combining the first and second data structures to produce a combined identification of vascular portions that form a particular blood vessel depicted in the vascular image, wherein combining is based on:
connection information between the one or more pixel masks and the path, or
similarity of directionality between an individual pixel mask and the path.
2. The method of claim 1 , wherein:
the first ML-based vascular identifier is configured is applied to recognize a first anatomically defined vascular type; and
the second vascular identifier is also an ML-based vascular identifier is applied to recognize a second anatomically defined vascular type.
3. The method of claim 2 , wherein the first and second anatomically defined vascular types are selected from among the group consisting of: a LAD, other vessels of a LAD sub-tree, a LCX, other vessels of a LCX sub-tree, a RCA, and other vessels of a RCA sub-tree.
4. The method of claim 2 , wherein the first and second ML-based vascular identifiers are generated based on training inputs comprising images of and image location indications for the first anatomically defined vascular types and the second anatomically defined vascular types, respectively.
5. The method of claim 1 , wherein the first ML-based vascular identifier comprises a network whereof an output layer is a regression layer.
6. The method of claim 1 , wherein the second vascular identifier is an ML-based vascular identifier, whereof an output layer is a classification layer.
7. The method of claim 1 , wherein the combining comprises:
identifying first and second regions identified by both the first ML-based vascular identifier and the second vascular identifier, and
selecting for inclusion in the combined identification a third region identified by one of the first ML-based vascular identifier and the second vascular identifier, based on the third region extending between the first and second regions.
8. The method of claim 1 , wherein at least one of the first and second data structures includes a vascular portion identified erroneously; the method comprising:
determining whether a difference in vascular portion identification between the first and second data structures corresponds to a known error tendency of one of the first ML-based vascular identifier and the second vascular identifier; and
wherein the combining includes including identified vascular portions in the combined identification selectively, according to the identified known error tendency.
9. The method of claim 1 , wherein the second data structure omits some vascular portions identified in the first data structure.
10. The method of claim 1 , wherein the one or more pixel masks include a first pixel mask and a second pixel mask, and wherein the connection information indicates that the path bridges the first pixel mask and the second pixel mask.
11. The method of claim 1 , wherein the connection information indicates that at least a portion of the path has a similar directionality as a skeletonization of the one or more pixel masks, the skeletonization comprising inputting the one or more pixel masks in an erosion filter.
12. The method of claim 1 , wherein an image location is a pixel of the vascular image.
13. A system for identifying vascular portions of a vascular image, the system comprising a processor and memory, the memory encoding instructions which cause the processor to:
access the vascular image comprising a 2-D angiographic image;
apply a first ML-based vascular identifier to the vascular image to produce and store in the memory a first data structure identifying a path that extends along at least some vascular portions, wherein a vascular portions is an image location associated with part of a blood vessel depicted in the vascular image;
apply a second vascular identifier to the vascular image to produce and store in the memory a second data structure identifying one or more pixel masks, wherein the pixel masks comprise respective blobs of vascular portions;
wherein the path
includes some vascular portions not in the one or more pixel masks; and
combine the first and second data structures to produce and store in the memory a combined identification of vascular portions that form a particular blood vessel depicted in the vascular image, wherein the combination is based on:
connection information between the one or more pixel masks and the path, or
similarity of directionality between an individual pixel mask and the path.
14. The system of claim 13 , wherein the first ML-based vascular identifier is applied to recognize a first anatomically defined vascular type, and wherein the second vascular identifier is also an ML-based vascular identifier, and applied to recognize a second anatomically defined vascular type.
15. The system of claim 14 , wherein the first and second anatomically defined vascular types are selected from among the group consisting of: a LAD, other vessels of a LAD sub-tree, a LCX, other vessels of a LCX sub-tree, a RCA, and other vessels of a RCA sub-tree.
16. The system of claim 14 , wherein the first and second ML-based vascular identifiers are generated based on training inputs comprising images of and image location indications for the first anatomically defined vascular types and the second anatomically defined vascular types, respectively.
17. The system of claim 13 , wherein the first ML-based vascular identifier comprises a network whereof an output layer is a regression layer.
18. The system of claim 13 , wherein the second vascular identifier is an ML-based vascular identifier, whereof an output layer is a classification layer.
19. The system of claim 13 , wherein the instructions furthermore instruct the processor to:
identify first and second regions identified by both the first ML-based vascular identifier and the second vascular identifier, and
select for inclusion in the combined identification a third region identified by one of the first ML-based vascular identifier and the second vascular identifier, based on the third region extending between the first and second regions.
20. The system of claim 13 , wherein the instructions furthermore instruct the processor to:
determine a condition wherein at least one of the first and second data structures includes vascular portions identified erroneously;
determine whether a difference in vascular portion identification between the first and second data structures corresponds to a known error tendency of one of the first ML-based vascular identifier and the second vascular identifier; and
combine the first and second data structures selectively according to the identified known error tendency.
21. The system of claim 13 , wherein the second data structure omits some vascular portions identified in the first data structure.
22. The system of claim 13 , wherein the one or more pixel masks include a first pixel mask and a second pixel mask, and wherein the connection information indicates that the path bridges the first pixel mask and the second pixel mask.
23. The system of claim 13 , wherein the connection information indicates that at least a portion of the path has a similar directionality as a skeletonization of the one or more pixel masks, the skeletonization comprising inputting the one or more pixel masks in an erosion filter.
24. The system of claim 13 , wherein an image location is a pixel of the vascular image.