Technique for retraining operational neural networks using synthetically generated retraining data

1 . A neural network system for retraining at least two operational neural networks by a synthetic retraining data set generated by a synthetic data generator neural network, wherein the synthetic retraining data set, when used as input to each of the at least two operational neural networks prior to retraining produces at least one inconsistent output value of a medical metric and, after retraining, produces, instead of the at least one inconsistent output value, a consistent output value, the neural network system comprising:

a processor;

the synthetic data generator neural network implemented by the processor in communication with a memory, the synthetic data generator neural network comprising:

an input layer configured to receive an input data set;

an output layer configured to output, based on the input data set, the synthetic retraining data set for retraining the at least two operational neural networks; and

a loss function configured to receive, from each of the at least two operational neural networks in relation to a loss function evaluation data set, at least one value of the medical metric, wherein the loss function of the synthetic data generator neural network further comprises at least one of a neuron coverage loss, wherein the neuron coverage loss comprises an inducement for training the synthetic data generator neural network to increase a number of neurons covered in the operational neural networks when receiving the synthetic retraining data set; and a reconstruction loss, wherein the reconstruction loss comprises an inducement for training the synthetic data generator neural network to reduce a difference between the input data set of the synthetic data generator neural network and the generated synthetic retraining data set; and

the at least two operational neural networks, wherein at least two of the at least two operational neural networks are located at different sites and implemented by respective processors, wherein each of the at least two operational neural networks were previously trained locally at a respective site, each of the at least two operational neural networks comprising:

an input layer configured to selectively receive the synthetic retraining data set and the loss function evaluation data set; and

an output layer configured to output at least one value of the medical metric in relation to each data set received at the input layer,

wherein the synthetic data generator neural network with first interconnections between the input layer and the output layer of the synthetic data generator neural network is trained for generating the synthetic retraining data set based on the loss function comprising a function of an absolute value of a difference of the at least one value of the medical metric in relation to the loss function evaluation data set received from each of the at least two operational neural networks, and

wherein each of the at least two operational neural networks with respective second interconnections between the input layer and the output layer of the respective operational neural network is retrained without sharing operational data and without sharing parameters by using the synthetic retraining data set generated by the synthetic data generator neural network.

2 . The neural network system of claim 1 , wherein the function of the absolute value of the difference of the at least one value of the medical metric in relation to the loss function evaluation data set received from each of the at least two operational neural networks comprises a negative of a square of the difference of the at least one value of the medical metric.

3 . The neural network system of claim 1 , wherein the input layer of each of the at least two operational neural networks is further configured to receive an operational data set from a medical imaging device, and

wherein the output layer of each of the at least two operational neural networks is further configured to output at least one value of the medical metric in relation to the received operational data set,

wherein the output related to the operational data set is based on the first interconnections retrained by the synthetic retraining data set generated by the synthetic data generator neural network.

4 . The neural network system of claim 3 , wherein the medical imaging device comprises at least one of:

a magnetic resonance imaging, MRI, device;

a computed tomography, CT, device; and

an ultrasound imaging device.

5 . The neural network system of claim 1 , wherein the input data set of the synthetic data generator neural network and the synthetic retraining data set are of a same type.

6 . The neural network system of claim 5 , wherein the synthetic retraining data set comprises a medical image data set and wherein the input data set of the synthetic data generator neural network comprises at least one of a synthetic medical image set and a medical image set acquired by a medical imaging device.

7 . The neural network system of claim 1 , wherein the input data set received at the input layer of the synthetic data generator neural network comprises the synthetic retraining data set.

8 . The neural network system of claim 1 , wherein the synthetic data generator neural network sequentially generates a plurality of synthetic retraining data sets, by recursively generating a further synthetic retraining data set based on a previous synthetic retraining data set.

9 . The neural network system of claim 1 , wherein the synthetic data generator neural network comprises an auto-encoder-decoder neural network.

10 . The neural network system of claim 1 , wherein the medical metric comprises a cardiac metric.

11 . The neural network system of claim 10 , wherein the cardiac metric comprises at least one of:

(a) a segmentation, a delineation, and/or spatial position of a region of at least one of a cardiac chamber and myocardium;

(b) a scar burden of the myocardium;

(c) a myocardial elasticity;

(d) a motion pattern;

(e) a linear size of a heart chamber

(f) a thickness of a heart wall;

(g) a volumetric size of a heart chamber;

(h) a mass of a heart chamber;

(i) an ejection fraction, EF, of a cardiac chamber; and

(j) a hemodynamic index.

12 . The neural network system of claim 1 , wherein the loss function comprises a sum of:

a negative of a square of the difference of the at least one value of the medical metric in relation to the loss function evaluation data set and/or the synthetic retraining data set received from each of the at least two operational neural networks;

the negative of the neuron coverage loss; and

the reconstruction loss, wherein the reconstruction loss comprises an absolute value of a difference between the input data set and a synthetic retraining data set generated, at the synthetic data generator neural network, based on the input data set,

wherein the loss function is optimized by minimizing the loss function.

13 . The neural network system of claim 1 , wherein the synthetic retraining data set comprises a manual annotation as to the at least one value of the medical metric.

14 . The neural network system of claim 1 , wherein each of the at least two operational neural networks comprises a plurality of hidden layers between the input layer and the output layer of the respective operational neural network.

15 . The neural network system of claim 1 , wherein the at least two operational neural networks comprise an identical layer architecture, wherein the layer architecture comprises a number of neurons at the input layer, a number of neurons at the output layer, a number of hidden layers, and a number of neurons at each hidden layer.

16 . A method of retraining at least two operational neural networks by a synthetic retraining data set generated by a synthetic data generator neural network, wherein the synthetic retraining data set, when used as input to each of the at least two operational neural networks prior to retraining produces at least one inconsistent output value of a medical metric and after retraining produces a consistent output value, the method comprising:

receiving, at an input layer of the synthetic data generator neural network implemented by a processor in communication with a memory, an input data set;

receiving, at a loss function of the synthetic data generator neural network from each of the at least two operational neural networks, at least one value of the medical metric in relation to a loss function evaluation data set provided to an input layer of each of the at least two operational neural networks, wherein at least two of the at least two operational neural networks are located at different sites, wherein each of the at least two operational neural networks were previously trained locally at a respective site, wherein the loss function of the synthetic data generator neural network further comprises at least one of: a neuron coverage loss, wherein the neuron coverage loss comprises an inducement for training the synthetic data generator neural network to increase a number of neurons covered in the operational neural networks when receiving the synthetic retraining data set; and a reconstruction loss, wherein the reconstruction loss comprises an inducement for training the synthetic data generator neural network to reduce a difference between the input data set of the synthetic data generator neural network and the generated synthetic retraining data set;

training, based on the loss function, by the processor, the synthetic data generator neural network with first interconnections between the input layer and an output layer of the synthetic data generator neural network for generating the synthetic retraining data set based on the input data set, wherein the loss function comprises a function of an absolute value of a difference of the at least one value of the medical metric in relation to the loss function evaluation data set received from each of the at least two operational neural networks;

outputting, at the output layer of the synthetic data generator neural network, the synthetic retraining data set for retraining the at least two operational neural networks;

receiving, at the input layer of each of the at least two operational neural networks, the synthetic retraining data set;

retraining, by respective processors, without sharing operational data and without sharing parameters each of the at least two operational neural networks with respective second interconnections between the input layer and an output layer of the respective operational neural network using the synthetic retraining data set generated by the synthetic data generator neural network; and

outputting, at the output layer of each of the at least two operational neural networks, at least one value of the medical metric in relation to each data set received at the input layer of the respective one of the at least two operational neural networks.

17 . The method of claim 16 , further comprising:

receiving, at the input layer of at least one of the at least two operational neural networks, an operational data set from a medical imaging device; and

outputting, at the output layer of the at least one of the at least two operational neural networks, at least one value of the medical metric in relation to the received operational data set, wherein generating the output related to the operational data set is based on the retraining, using the synthetic retraining data set generated by the synthetic data generator neural network, the at least one of the at least two operational neural networks.