Generative AI to create time series prediction radiotherapy treatment planning systems

1 . A method comprising:

monitoring, by at least one processor, a sequence of screen captures generated from a user interface of a radiotherapy treatment planning platform operated by a set of medical professionals;

generating, by the at least one processor, a training dataset comprising a time series dataset corresponding to the sequence of screen captures and a visual difference between at least two screen captures based on a pixel difference between the at least two screen captures, wherein at least one visual attribute within at least one screen capture within the training dataset is redacted;

training, by the at least one processor, a machine learning model using the training dataset such that the machine learning model is configured to predict a visual attribute of a next screen capture of the user interface of the radiotherapy treatment planning platform based on a previous screen capture of the radiotherapy treatment planning platform;

executing, by the at least one processor, the machine learning model to predict a future visual attribute of a future screen capture of the radiotherapy treatment planning platform for a current screen capture of the radiotherapy treatment planning platform being interacted with by a user; and

displaying, by the at least one processor, a warning notification when the future screen capture deviates from the future visual attribute predicted by the machine learning model.

2 . The method of claim 1 , wherein the machine learning model is further trained using a video recording a medical professional interacting with the radiotherapy treatment planning platform.

3 . The method of claim 1 , wherein the machine learning model is further trained using an auditory instruction by a medical professional interacting with the radiotherapy treatment planning platform.

4 . The method of claim 1 , further comprising:

displaying, by the at least one processor, the at least one future visual attribute.

5 . The method of claim 1 , wherein the machine learning model is trained for a particular clinic.

6 . A non-transitory computer readable medium comprising a set instructions that when executed cause a processor to:

monitor a sequence of screen captures generated from a user interface of a radiotherapy treatment planning platform operated by a set of medical professionals;

generate a training dataset comprising a time series dataset corresponding to the sequence of screen captures and a visual difference between at least two screen captures based on a pixel difference between the at least two screen captures, wherein at least one visual attribute within at least one screen capture within the training dataset is redacted;

train a machine learning model using the training dataset, such that the machine learning model is configured to predict a visual attribute of a next screen capture of the user interface of the radiotherapy treatment planning platform based on a previous screen capture of the radiotherapy treatment planning platform;

execute the machine learning model to predict a future visual attribute of a future screen capture of the radiotherapy treatment planning platform for a current screen capture of the radiotherapy treatment planning platform being interacted with by a user; and

display a warning notification when the future screen capture deviates from the future visual attribute predicted by the machine learning model.

7 . The computer readable medium of claim 6 , wherein the machine learning model is further trained using a video recording a medical professional interacting with the radiotherapy treatment planning platform.

8 . The computer readable medium of claim 6 , wherein the machine learning model is further trained using an auditory instruction by a medical professional interacting with the radiotherapy treatment planning platform.

9 . The computer readable medium of claim 6 , wherein the set of instructions further cause the processor to:

display the at least one future visual attribute.

10 . The computer readable medium of claim 6 , wherein the machine learning model is trained for a particular clinic.

11 . A system comprising a server configured to:

monitor a sequence of screen captures generated from a user interface of a radiotherapy treatment planning platform operated by a set of medical professionals;

generate a training dataset comprising a time series dataset corresponding to the sequence of screen captures and a visual difference between at least two screen captures based on a pixel difference between the at least two screen captures, wherein at least one visual attribute within at least one screen capture within the training dataset is redacted;

train a machine learning model using the training dataset, such that the machine learning model is configured to predict a visual attribute of a next screen capture of the user interface of the radiotherapy treatment planning platform based on a previous screen capture of the radiotherapy treatment planning platform;

execute the machine learning model to predict a future visual attribute of a future screen capture of the radiotherapy treatment planning platform for a current screen capture of the radiotherapy treatment planning platform being interacted with by a user; and

display a warning notification when the future screen capture deviates from the future visual attribute predicted by the machine learning model.

12 . The system of claim 11 , wherein the machine learning model is further trained using a video recording a medical professional interacting with the radiotherapy treatment planning platform.

13 . The system of claim 11 , wherein the machine learning model is further trained using an auditory instruction by a medical professional interacting with the radiotherapy treatment planning platform.

14 . The system of claim 11 , the server is further configured to:

display the at least one future visual attribute.