IP Library Granted Patent US 11,067,786
Granted Patent B2
US 11,067,786 · App. 16/435,430 · Granted Jul 20, 2021

Artifact regulation methods in deep model training for image transformation

Inventors: Shih-Jong James Lee (Bellevue, WA); Hideki Sasaki (Bellevue, WA)
Assignee: Leica Microsystems Inc.
G02B21/365G06N3/08G06N20/00G06T5/001G06T2207/10056G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,067,786
App. No.
16/435,430
Granted
Jul 20, 2021
Kind
B2
Abstract

A computerized method of artifact regulation in deep model training for image transformation first performs one cycle of deep model training by computing means using a training data, a validation data, a similarity loss function, an artifact regulation loss function and a weight of loss functions to generate similarity loss and artifact regulation loss and a deep model. The method then performs a training evaluation using the similarity loss and the artifact regulation loss thus obtained to generate a training readiness output. Then, depending upon the training readiness output, the method may be terminated if certain termination criteria are met, or may perform another cycle of deep model training and training evaluation, with or without updating the weight, until the termination criteria are met. Alternatively, the deep model training in the method may be a deep adversarial model training or a bi-directional deep adversarial training.

Claims (39)

1. A computerized method of artifact regulation in deep model training for image transformation, the method comprising the steps of:

a) inputting a training data, a validation data, a similarity loss function, an artifact regulation loss function and a weight of loss functions and an initial deep model into electronic storage means; and setting a deep model to the initial deep model;

b) performing one cycle of deep model training by computing means to update the deep model using the training data, the validation data, the similarity loss function, the artifact regulation loss function and the weight to generate similarity loss and artifact regulation loss and an updated deep model;

c) performing a training evaluation using the similarity loss and the artifact regulation loss from step b) to generate a training readiness output selected from a group consisting of (1) weight update, (2) continue and (3) stop;

d) in response to the training readiness output from step c) being weight update, performing a weight updating to update the weight, setting the deep model to the updated deep model and repeating steps b) through c);

e) in response to the training readiness output from step c) being continue, setting the deep model to the updated deep model and repeating steps b) through c); and

f) in response to the training readiness output from step c) being stop, terminating the method and outputting the updated deep model generated in step b).

2. The computerized method of artifact regulation of claim 1 , wherein the similarity loss function evaluates likelihood between deep model output and expected output.

3. The computerized method of artifact regulation of claim 1 , wherein the artifact regulation loss function evaluates likelihood of deep model output as artifact.

4. The computerized method of artifact regulation of claim 1 , wherein the training evaluation uses trends of the similarity loss and the artifact regulation loss of the training data and the validation data to generate the training readiness output.

5. The computerized method of artifact regulation of claim 1 , wherein the weight updating balances the similarity loss and the artifact regulation loss by monitoring their ratio.

6. The computerized method of artifact regulation of claim 1 , wherein the electronic storage means is in a cloud platform.

7. The computerized method of artifact regulation of claim 1 , wherein the computing means is in a cloud platform.

8. The computerized method of artifact regulation in deep model training for image transformation of claim 1 , wherein the image transformation is for microscopy image restoration.

9. The computerized method of artifact regulation in deep model training for image transformation of claim 1 , wherein the image transformation is for microscopy image prediction.

10. A computerized method of artifact regulation in deep adversarial model training for image transformation, the method comprising the steps of:

a) inputting a training data, a validation data, a similarity loss function, an artifact regulation loss function and a weight of loss functions and an initial deep model into electronic storage means; and setting a deep model to the initial deep model;

b) performing one cycle of deep adversarial model training by computing means to update the deep model using the training data, the validation data, the similarity loss function, the artifact regulation loss function and the weight to generate similarity loss and artifact regulation loss and an updated deep model;

c) performing a training evaluation using the similarity loss and the artifact regulation loss from step b) to generate a training readiness output selected from a group consisting of (1) weight update, (2) continue and (3) stop;

d) in response to the training readiness output from step c) being weight update, performing a weight updating to update the weight, setting the deep model to the updated deep model and repeating steps b) through c);

e) in response to the training readiness output from step c) being continue, setting the deep model to the updated deep model and repeating steps b) through c); and

f) in response to the training readiness output from step c) being stop, terminating the method and outputting the updated deep model generated in step b).

11. The computerized method of artifact regulation of claim 10 , wherein the similarity loss function evaluates likelihood between deep adversarial model output and expected output.

12. The computerized method of artifact regulation of claim 10 , wherein the artifact regulation loss function evaluates adversarial losses.

13. The computerized method of artifact regulation of claim 10 , wherein the training evaluation uses trends of the similarity loss and the artifact regulation loss of the training data and the validation data to generate the training readiness output.

14. The computerized method of artifact regulation of claim 10 , wherein the weight updating balances the similarity loss and the artifact regulation loss.

15. The computerized method of artifact regulation in deep adversarial model training for image transformation of claim 10 , wherein the image transformation is for microscopy image restoration.

16. The computerized method of artifact regulation in deep adversarial model training for image transformation of claim 10 , wherein the image transformation is for microscopy image prediction.

17. A computerized method of artifact regulation in bi-directional deep adversarial model training for image transformation, the method comprising the steps of:

a) inputting a training data, a validation data, a similarity loss function, an artifact regulation loss function and a weight of loss functions and an initial deep model into electronic storage means; and setting a deep model to the initial deep model;

b) performing one cycle of bi-directional deep adversarial model training by computing means to update the deep model using the training data, the validation data, the similarity loss function, the artifact regulation loss function and the weight to generate similarity loss and artifact regulation loss and an updated deep model;

c) performing a training evaluation using the similarity loss and the artifact regulation loss to generate a training readiness output selected from a group consisting of (1) weight update, (2) continue and (3) stop;

d) in response to the training readiness output from step c) being weight update, performing a weight updating to update the weight, setting the deep model to the updated deep model and repeating steps b) through c);

e) in response to the training readiness output from step c) being continue, setting the deep model to the updated deep model and repeating steps b) through c); and

f) in response to the training readiness output from step c) being stop, terminating the method and outputting the updated deep model generated in step b).

18. The computerized method of artifact regulation of claim 17 , wherein the similarity loss function evaluates bi-directional consistency losses.

19. The computerized method of artifact regulation of claim 17 , wherein the artifact regulation loss function evaluates adversarial losses.

20. The computerized method of artifact regulation of claim 17 , wherein the training evaluation uses trends of the similarity loss and the artifact regulation loss of the training data and the validation data to generate the training readiness output.

21. The computerized method of artifact regulation of claim 17 , wherein the weight updating balances the similarity loss and the artifact regulation loss.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 4, 2021
From: LEICA MICROSYSTEMS INC.
To: LEICA MICROSYSTEMS CMS GMBH
Reel/Frame 057697/0440 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2021
From: SVISION LLC
To: LEICA MICROSYSTEMS INC.
Reel/Frame 055600/0752 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2020
From: DRVISION TECHNOLOGIES LLC
To: SVISION LLC
Reel/Frame 054688/0279 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2019
From: LEE, SHIH-JONG JAMES; SASAKI, HIDEKI
To: DRVISION TECHNOLOGIES LLC
Reel/Frame 049555/0895 →