IP Library Granted Patent US 11,612,761
Granted Patent B2
US 11,612,761 · App. 17/124,249 · Granted Mar 28, 2023

Training artificial intelligence models for radiation therapy

Inventors: Shahab Basiri (Siuntio, FI); Mikko Hakala (Rajamäki, FI); Esa Kuusela (Espoo, FI); Elena Czeizler (Helsinki, FI)
Assignee: VARIAN MEDICAL SYSTEMS INTERNATIONAL AG
A61N5/1038G06N3/08A61N2005/1041
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,612,761
App. No.
17/124,249
Granted
Mar 28, 2023
Kind
B2
Abstract

Disclosed herein are systems and methods for iteratively training artificial intelligence models using reinforcement learning techniques. With each iteration, a training agent applies a random radiation therapy treatment attribute corresponding to the radiation therapy treatment attribute associated with previously performed radiation therapy treatments when an epsilon value indicative of a likelihood of exploration and exploitation training of the artificial intelligence model satisfies a threshold. When the epsilon value does not satisfy the threshold, the agent generates, using an existing policy, a first predicted radiation therapy treatment attribute, and generates, using a predefined model, a second predicted radiation therapy treatment attribute. The agent applies one of the first predicted radiation therapy treatment attribute or the second predicted radiation therapy treatment attribute that is associated with a higher reward. The agent iteratively repeats training the artificial intelligence model until the existing policy satisfies an accuracy threshold.

Claims (37)

1. A method of training an artificial intelligence model using reinforcement learning, the method comprising:

iteratively training, by a server, the artificial intelligence model using a training dataset comprising a set of radiation therapy treatment attributes associated with previously performed radiation therapy treatments to predict a corresponding set of radiation therapy treatment attributes, wherein with each iteration the server:

applies a random radiation therapy treatment attribute corresponding to the radiation therapy treatment attribute associated with previously performed radiation therapy treatments when an epsilon value indicative of a likelihood of exploration and exploitation training of the artificial intelligence model satisfies a threshold; and

when the epsilon value does not satisfy the threshold, the server:

generates, using an existing policy, a first predicted radiation therapy treatment attribute, and

generates, using a predefined computer model, a second predicted radiation therapy treatment attribute,

wherein the server applies one of the first predicted radiation therapy treatment attribute or the second predicted radiation therapy treatment attribute that is associated with a higher reward,

wherein the server iteratively repeats training the artificial intelligence model until the existing policy satisfies an accuracy threshold.

2. The method of claim 1 , further comprising:

executing, by the server, the trained artificial intelligence model.

3. The method of claim 1 , wherein the epsilon value is received from a system administrator.

4. The method of claim 1 , wherein the server compares the first predicted radiation therapy treatment attribute with the second predicted radiation therapy treatment attribute.

5. The method of claim 1 , wherein the server trains the existing policy based on the predefined computer model using a supervised training method.

6. The method of claim 1 , wherein the server revises the epsilon value, such that a likelihood of generation of the first or second predicted radiation therapy treatment attribute is higher than generation of the random radiation therapy treatment attribute.

7. The method of claim 1 , wherein the server revises the epsilon value, such that a likelihood of generation of the first or second predicted radiation therapy treatment attribute is lower than generation of the random radiation therapy treatment attribute.

8. The method of claim 1 , wherein the predefined computer model is received from a system administrator.

9. The method of claim 1 , wherein the predefined computer model is specific to a clinic.

10. The method of claim 1 , wherein the predefined computer model is specific to optimizing a dose-histogram volume calculation.

11. A system for training an artificial intelligence model using reinforcement learning comprising:

one or more processors; and

a non-transitory memory to store computer code instructions, the computer code instructions when executed cause the one or more processors to:

iteratively train the artificial intelligence model using a training dataset comprising a set of radiation therapy treatment attributes associated with previously performed radiation therapy treatments to predict a corresponding set of radiation therapy treatment attributes, wherein with each iteration the one or more processors:

apply a random radiation therapy treatment attribute corresponding to the radiation therapy treatment attribute associated with previously performed radiation therapy treatments when an epsilon value indicative of a likelihood of exploration and exploitation training of the artificial intelligence model satisfies a threshold; and

when the epsilon value does not satisfy the threshold, the one or more processors:

generate, using an existing policy, a first predicted radiation therapy treatment attribute, and

generate, using a predefined computer model, a second predicted radiation therapy treatment attribute,

wherein the one or more processors apply one of the first predicted radiation therapy treatment attribute or the second predicted radiation therapy treatment attribute that is associated with a higher reward,

wherein the server iteratively repeats training the artificial intelligence model until the existing policy satisfies an accuracy threshold.

12. The system of claim 11 , wherein the computer code instructions when executed further cause the one or more processors to execute the trained artificial intelligence model.

13. The system of claim 11 , wherein the epsilon value is received from a system administrator.

14. The system of claim 11 , wherein the one or more processors compare the first predicted radiation therapy treatment attribute with the second predicted radiation therapy treatment attribute.

15. The system of claim 14 , wherein the one or more processors train the existing policy based on the predefined computer model using a supervised training method.

16. The system of claim 11 , wherein the one or more processors revise the epsilon value, such that a likelihood of generation of the first or second predicted radiation therapy treatment attribute is higher than generation of the random radiation therapy treatment attribute.

17. The system of claim 11 , wherein the one or more processors revise the epsilon value, such that a likelihood of generation of the first or second predicted radiation therapy treatment attribute is lower than generation of the random radiation therapy treatment attribute.

18. The system of claim 11 , wherein the predefined computer model is received from a system administrator.

19. The system of claim 11 , wherein the predefined computer model is specific to a clinic.

20. The system of claim 11 , wherein the predefined computer model is configured to optimize a dose-histogram volume calculation.

Assignments (6)
MERGER AND CHANGE OF NAME Recorded Mar 15, 2023
From: VARIAN MEDICAL SYSTEMS INTERNATIONAL AG; SIEMENS HEALTHINEERS INTERNATIONAL AG
To: SIEMENS HEALTHINEERS INTERNATIONAL AG
Reel/Frame 063409/0731 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2021
From: VARIAN MEDICAL SYSTEMS INC.
To: VARIAN MEDICAL SYSTEMS INTERNATIONAL AG
Reel/Frame 058410/0512 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 7, 2021
From: VARIAN MEDICAL SYSTEMS, INC.
To: VARIAN MEDICAL SYSTEMS INTERNATIONAL AG
Reel/Frame 057398/0077 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2021
From: VARIAN MEDICAL SYSTEMS INTERNATIONAL AG
To: VARIAN MEDICAL SYSTEMS, INC.
Reel/Frame 057387/0973 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2021
From: VARIAN MEDICAL SYSTEMS INTERNATIONAL AG
To: VARIAN MEDICAL SYSTEMS, INC.
Reel/Frame 057329/0160 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2020
From: BASIRI, SHAHAB; HAKALA, MIKKO; KUUSELA, ESA; CZEIZLER, ELENA
To: VARIAN MEDICAL SYSTEMS, INC.
Reel/Frame 054672/0900 →
Continuity (1)
Related Publication 20220184419A1 · Jun 16, 2022