Rapid reconstruction of high resolution images from lower resolution images
An approach is disclosed that receives a time sequence of low-resolution images, each of the low-resolution images depicting a physics event. The approach interpolates two adjacent low-resolution images to a higher spatial-resolution interpolated image between a first and a second time. The approach then inputs a previous high-resolution image and the interpolated image to a neural network that includes a physics constraint that corresponds to the physics event. A new high-resolution image is received from the neural network, with the new image corresponding to the inputted previous high-resolution image and the interpolated image. The neural network is trained to minimize the mean squared difference between a smoothed version of the output and the input at the same time plus the difference between the high-resolution output and a second relevant physics equation.
1 . A method, implemented by a processor coupled to a memory, comprising:
receiving a plurality of low-resolution images, each of the low-resolution images depicting a physics event;
interpolating two adjacent low-resolution images to a higher spatial-resolution interpolated image between a first and a second time;
inputting a previous high-resolution image and the interpolated image to a neural network that includes a first physics constraint that corresponds to the physics event;
receiving, from the neural network, a new high-resolution image that corresponds to the inputted previous high-resolution image and the interpolated image;
comparing one of the low-resolution images at a first time with the new high-resolution image; and
evaluating a loss function corresponding to a combination of the compared images and a deviation that is based on the new high-resolution image from a second physics constraint, the evaluated loss function resulting in a loss value.
2 . The method of claim 1 further comprising:
until the loss value reaches a low threshold:
adjusting a weighting used by the neural network; and
reperforming the interpolating, the inputting, the receiving, the comparing, and the evaluating.
3 . The method of claim 2 wherein the neural network is trained when the loss value reaches the low threshold.
4 . The method of claim 3 further comprising:
receiving a second plurality of low-resolution images corresponding to a plurality of time-based events that are affected by the first and second physics constraints;
inputting the second plurality of low-resolution images to the trained neural network; and
receiving, from the trained neural network, a set of high-resolution images that correspond to the second plurality of low-resolution images.
5 . The method of claim 4 wherein the low-resolution images are satellite images of an air pollution disbursement from a pollution source, and wherein the first physics constraint includes one or more advection-diffusion equations.
6 . The method of claim 5 wherein the second physics constraint includes one or more conservation equations.
7 . An information handling system comprising:
one or more processors;
a memory coupled to at least one of the processors; and
a set of instructions stored in the memory and executed by at least one of the processors to perform actions comprising:
receiving a plurality of low-resolution images, each of the low-resolution images depicting a physics event;
interpolating two adjacent low-resolution images to a higher spatial-resolution interpolated image between a first and a second time;
inputting a previous high-resolution image and the interpolated image to a neural network that includes a first physics constraint that corresponds to the physics event;
receiving, from the neural network, a new high-resolution image that corresponds to the inputted previous high-resolution image and the interpolated image;
comparing one of the low-resolution images at a first time with the new high-resolution image; and
evaluating a loss function corresponding to a combination of the compared images and a deviation that is based on the new high-resolution image from a second physics constraint, the evaluated loss function resulting in a loss value.
8 . The information handling system of claim 7 wherein the actions further comprise:
until the loss value reaches a low threshold:
adjusting a weighting used by the neural network; and
reperforming the interpolating, the inputting, the receiving, the comparing, and the evaluating.
9 . The information handling system of claim 8 wherein the neural network is trained when the loss value reaches the low threshold.
10 . The information handling system of claim 9 wherein the actions further comprise:
receiving a second plurality of low-resolution images corresponding to a plurality of time-based events that are affected by the first and second physics constraints;
inputting the second plurality of low-resolution images to the trained neural network; and
receiving, from the trained neural network, a set of high-resolution images that correspond to the second plurality of low-resolution images.
11 . The information handling system of claim 10 wherein the low-resolution images are satellite images of an air pollution disbursement from a pollution source, and wherein the first physics constraint includes one or more advection-diffusion equations.
12 . The information handling system of claim 11 wherein the second physics constraint includes one or more conservation equations.
13 . A computer program product comprising:
a computer readable storage medium comprising a set of computer instructions that, when executed by a processor, are effective to perform actions comprising:
receiving a plurality of low-resolution images, each of the low-resolution images depicting a physics event;
interpolating two adjacent low-resolution images to a higher spatial-resolution interpolated image between a first and a second time;
inputting a previous high-resolution image and the interpolated image to a neural network that includes a first physics constraint that corresponds to the physics event;
receiving, from the neural network, a new high-resolution image that corresponds to the inputted previous high-resolution image and the interpolated image;
comparing one of the low-resolution images at a first time with the new high-resolution image; and
evaluating a loss function corresponding to a combination of the compared images and a deviation that is based on the new high-resolution image from a second physics constraint, the evaluated loss function resulting in a loss value.
14 . The computer program product of claim 13 wherein the actions further comprise:
until the loss value reaches a low threshold:
adjusting a weighting used by the neural network; and
reperforming the interpolating, the inputting, the receiving, the comparing, and the evaluating.
15 . The computer program product of claim 14 wherein the neural network is trained when the loss value reaches the low threshold.
16 . The computer program product of claim 15 wherein the actions further comprise:
receiving a second plurality of low-resolution images corresponding to a plurality of time-based events that are affected by the first and second physics constraints;
inputting the second plurality of low-resolution images to the trained neural network; and
receiving, from the trained neural network, a set of high-resolution images that correspond to the second plurality of low-resolution images.
17 . The computer program product of claim 16 wherein the low-resolution images are satellite images of an air pollution disbursement from a pollution source, wherein the first physics constraint includes one or more advection-diffusion equations wherein the second physics constraint includes one or more conservation equations.