Dual domain tracking of target structures

1. A computer-implemented method of location prediction using an end-to-end target structure tracking system comprising:

executing, by a computer, a machine learning model to extract a set of features from imaging projection data associated with a target structure of a patient's anatomy;

executing, by the computer, a recurrent neural network to obtain three-dimensional feature map data associated with the target structure, the recurrent neural network configured to sequence the imaging projection data using the set of features in three-dimensional space; and

indicating, by the computer, a probability of a three-dimensional point in a template feature map matching a location of a three-dimensional point in the three-dimensional feature map data, the template feature map comprising the target structure.

2. The computer-implemented method according to claim 1 , further comprising:

receiving, by the computer, segment information associated with the target structure; and

extracting, by the computer, the template feature map from the three-dimensional simulation image using the segment information and the three-dimensional simulation image.

3. The computer-implemented method according to claim 1 , further comprising:

executing, by the computer, a forward projection algorithm to transform the set of features in three-dimensional space into a set of features in two-dimensional space; and

feeding, by the computer, the set of features in two-dimensional space into the machine learning model.

4. The computer-implemented method according to claim 1 , wherein the computer executes a second machine learning model to transform the extracted set of features into a set of features in three-dimensional space.

5. The computer-implemented method according to claim 1 , further comprising:

determining, by the computer, a deformation field associated with the target structure; and

indicating, by the computer, the probability of a three-dimensional point in the template feature map, using the deformation field, matching the location of a three-dimensional point in the three-dimensional feature map data.

6. The computer-implemented method according to claim 1 , further comprising:

determining, by the computer, a confidence value based on peaks and sidelobes associated with a probability distribution corresponding to the probability of a three-dimensional point in the template feature map matching a location of a three-dimensional point in the three-dimensional feature map data.

7. The computer-implemented method according to claim 1 , further comprising:

determining, by the computer, a classification for a point in the three-dimensional feature map data based on the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data satisfying a threshold.

8. The computer-implemented method according to claim 1 , wherein the imaging projection data is based on at least one of a stereo projection pair or a projection set generated using a multi-view imaging system.

9. The computer-implemented method according to claim 1 , further comprising:

extracting, by the computer, an additional template feature map from the three-dimensional simulation image associated with an additional target structure;

comparing, by the computer, the additional template feature map to the three-dimensional feature map data; and

generating, by the computer, a multi-channel probability distribution indicating the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data and the probability of a three-dimensional point in the additional template feature map matching the location of a three-dimensional point in the three-dimensional feature map data.

10. The computer-implemented method according to claim 1 , wherein comparing the template feature map to the three-dimensional feature map data comprises convolving each point of the template feature map with each point of the three-dimensional feature map data.

11. A system comprising:

a server comprising a processor and a non-transitory computer-readable medium containing instructions that when executed by the processor causes the processor to perform operations comprising:

executing a machine learning model to extract a set of features from imaging projection data associated with a target structure of a patient's anatomy;

executing a recurrent neural network to obtain three-dimensional feature map data associated with the target structure, the recurrent neural network configured to sequence the imaging projection data using the set of features in three-dimensional space; and

indicating a probability of a three-dimensional point in a template feature map matching a location of a three-dimensional point in the three-dimensional feature map data, the template feature map comprising the target structure.

12. The system according to claim 11 , wherein the instructions further cause the server to:

receive segment information associated with the target structure; and

extract the template feature map from the three-dimensional simulation image using the segment information and the three-dimensional simulation image.

13. The system according to claim 11 , wherein the instructions further cause the server to:

execute a forward projection algorithm to transform the set of features in three-dimensional space into a set of features in two-dimensional space; and

feed the set of features in two-dimensional space into the machine learning model.

14. The system according to claim 11 , wherein the computer executes a second machine learning model to transform the extracted set of features into a set of features in three-dimensional space.

15. The system according to claim 11 , wherein the instructions further cause the server to:

determine a deformation field associated with the target structure; and

indicate the probability of a three-dimensional point in the template feature map, using the deformation field, matching the location of a three-dimensional point in the three-dimensional feature map data.

16. The system according to claim 11 , wherein the instructions further cause the server to:

determine a confidence value based on peaks and sidelobes associated with a probability distribution corresponding to the probability of a three-dimensional point in the template feature map matching a location of a three-dimensional point in the three-dimensional feature map data.

17. The system according to claim 11 , wherein the instructions further cause the server to:

determine a classification for a point in the three-dimensional feature map data based on the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data satisfying a threshold.

18. The system according to claim 11 , wherein the imaging projection data is based on at least one of a stereo projection pair or a projection set generated using a multi-view imaging system.

19. The system according to claim 11 , wherein the instructions further cause the server to:

extract an additional template feature map from the three-dimensional simulation image associated with an additional target structure;

compare the additional template feature map to the three-dimensional feature map data; and

generate a multi-channel probability distribution indicating the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data and the probability of a three-dimensional point in the additional template feature map matching the location of a three-dimensional point in the three-dimensional feature map data.

20. The system according to claim 11 , wherein comparing the template feature map to the three-dimensional feature map data comprises convolving each point of the template feature map with each point of the three-dimensional feature map data.