Machine learning processing of contiguous slice image data

Inventors: Jerry Prince (Timonium, MD); Can Zhao (Baltimore, MD); Aaron Carass (Towson, MD)

Assignee: THE JOHNS HOPKINS UNIVERSITY

G06T5/003 G06F18/2148 G06N3/045 G06T7/0012 G06V10/454 G06V10/774 G06V10/82 G06T2207/10088

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,741,580
App. No.: 17/274,901
Granted: Aug 29, 2023
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Disclosed techniques for image processing three-dimensional image data include: obtaining three-dimensional image data representing contiguous slices parallel to a plane, constructing training data from the image data by, for each of a plurality of angles: rotating the image data in the plane to produce rotated image data, blurring the rotated image data in a dimension parallel to the plane to produce low resolution rotated image data, and introducing aliasing into the low resolution rotated image data in the dimension parallel to the plane to produce aliased low resolution rotated image data, training an anti-aliasing neural network with the aliased low resolution image data and the low resolution image data, training a super-resolution neural network with the aliased low resolution image data and the rotated image data, and processing the image data using the trained anti-aliasing neural network and the trained super-resolution neural network to produce processed image data.

Claims (48)

1. A method of electronically image processing three-dimensional contiguous slice image data using machine learning, the method comprising:

obtaining computer readable three-dimensional contiguous slice image data representing a plurality of contiguous slices parallel to a first plane in three-dimensional space;

constructing machine learning training data from the image data, wherein the constructing comprises, for each of a plurality of angles:

rotating the image data in the first plane by a respective angle to produce rotated image data;

blurring the rotated image data in a dimension parallel to the first plane to produce low resolution rotated image data; and

introducing aliasing into the low resolution rotated image data in the dimension parallel to the first plane to produce aliased low resolution rotated image data;

training an anti-aliasing neural network with at least portions of the aliased low resolution image data and portions of the low resolution image data to produce a trained anti-aliasing neural network;

training a super-resolution neural network with at least portions of the aliased low resolution image data and at least portions of the rotated image data to produce a trained super-resolution neural network;

processing the image data using the trained anti-aliasing neural network and the trained super-resolution neural network to produce processed image data, wherein the processing comprises:

applying the trained anti-aliasing neural network to the image data in a first plurality of planes perpendicular to the first plane; and

applying the trained super-resolution neural network to the image data in a second plurality of planes perpendicular to the first plane and perpendicular to the first plurality of planes; and

outputting the processed image data.

2. The method of claim 1 , wherein the processing further comprises, for each of a second plurality of rotations of the image data in the first plane by a respective angle to produce second rotated image data:

applying the trained anti-aliasing neural network to the second rotated image data in the first plurality of planes perpendicular to the first plane;

applying the trained super-resolution neural network to the second rotated image data in the second plurality of planes perpendicular to the first plane and perpendicular to the first plurality of planes to produce rotated processed image data; and

combining the rotated processed image data for each of the second plurality of rotations to produce the processed image data.

3. The method of claim 2 , wherein the combining comprises combining using Fourier burst accumulation.

4. The method of claim 1 , wherein the blurring comprises applying a Gaussian filter.

5. The method of claim 4 , wherein the Gaussian filter is a one-dimensional Gaussian filter with a length and a full-width half-maximum determined according to a quantity b/a, where a represents a resolution of the image data in a dimension parallel to the first plane, and where b represents a resolution of the image data in a dimension perpendicular to the first plane.

6. The method of claim 1 , wherein the image data comprises one of magnetic resonance imaging image data, computed tomography image data, or fluorescence microscopy image data.

7. The method of claim 1 , wherein at least one of the anti-aliasing neural network or the super-resolution neural network comprises an enhanced deep residual network for single image super resolution.

8. The method of claim 1 , further comprising preprocessing the image data using inhomogeneity correction.

9. The method of claim 1 , wherein the outputting comprises displaying at least a portion of the processed image data on a computer monitor.

10. The method of claim 1 , wherein at least one of the anti-aliasing neural network or the super-resolution neural network is pre-trained using pre-training data derived from pre-training image data, wherein the pre-training image data is different from the image data.

11. A system for electronic image processing of three-dimensional contiguous slice image data using machine learning, the system comprising at least one processor that executes instructions to perform operations comprising: