IP Library › Granted Patent US 12,608,828
Granted Patent B2
US 12,608,828 · App. 18/184,690 · Granted Apr 21, 2026

Image processing apparatus, operation method of image processing apparatus, operation program of image processing apparatus, and trained model

Inventor: Caihua Wang (Kanagawa, JP)
Assignee: FUJIFILM Corporation
G06T7/30G06T7/0012G06V10/761G06V10/764G16H30/40G16H50/20G06T2207/10088G06T2207/20081G06T2207/30016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,608,828
App. No.
18/184,690
Granted
Apr 21, 2026
Kind
B2
Abstract

An image processing apparatus includes a processor and a memory connected to or built in the processor. The processor is configured to perform non-linear registration processing on a first medical image and a second medical image among a plurality of medical images, and generate at least one new medical image that is used for training a machine learning model for the medical images by transforming at least one medical image of the first medical image or the second medical image based on a result of the non-linear registration processing.

Claims (55)

1 . An image processing apparatus comprising:

a processor; and

a memory connected to or built in the processor,

wherein the processor is configured to

perform non-linear registration processing on a first medical image and a second medical image among a plurality of medical images to derive transformation amounts as a result of the non-linear registration processing, wherein the medical images are classified into classes,

generate transformation coefficients based on the classes of the first medical image and the second medical image,

multiply the transformation coefficients by the derived transformation amounts to obtain corrected transformation amounts, and

generate at least one new medical image that is used for training a machine learning model for the medical images by transforming at least one medical image of the first medical image or the second medical image using at least one of the corrected transformation amounts.

2 . The image processing apparatus according to claim 1 ,

wherein the transformation coefficients comprise a transformation coefficient α and a transformation coefficient β,

the transformation amounts comprise a transformation amount T_ 12 from the first medical image to the second medical image in the non-linear registration processing, and a transformation amount T_ 21 from the second medical image to the first medical image in the non-linear registration processing, and

the corrected transformation amounts comprise a corrected transformation amount αT_ 12 obtained by multiplying the transformation amount T_ 12 by the transformation coefficient α, and a corrected transformation amount βT_ 21 obtained by multiplying the transformation amount T_ 21 by the transformation coefficient β,

wherein the processor is configured to

set the first medical image as a first new medical image by applying, to the first medical image, the corrected transformation amount αT_ 12 , and

set the second medical image as a second new medical image by applying, to the second medical image, the corrected transformation amount βT_ 21 .

3 . The image processing apparatus according to claim 2 ,

wherein

the processor is configured to change values of the transformation coefficients α and β depending on whether the classes of the first medical image and the second medical image are the same or different from each other.

4 . The image processing apparatus according to claim 3 ,

wherein the processor is configured to

set a class of the first new medical image to be the same as the class of the first medical image, and

set a class of the second new medical image to be the same as the class of the second medical image.

5 . The image processing apparatus according to claim 2 ,

wherein the transformation coefficients α and β are random numbers according to a normal distribution.

6 . The image processing apparatus according to claim 5 ,

wherein

the processor is configured to

change values of the transformation coefficients α and β depending on whether the classes of the first medical image and the second medical image are the same or different from each other, and

a mean of a normal distribution in a case where the classes of the first medical image and the second medical image are different from each other is smaller than a mean of a normal distribution in a case where the classes of the first medical image and the second medical image are the same.

7 . The image processing apparatus according to claim 5 ,

wherein

the processor is configured to

change values of the transformation coefficients α and β depending on whether the classes of the first medical image and the second medical image are the same or different from each other,

set a class of the first new medical image to be the same as the class of the first medical image,

set a class of the second new medical image to be the same as the class of the second medical image, and

a mean of a normal distribution in a case where the classes of the first medical image and the second medical image are different from each other is smaller than a mean of a normal distribution in a case where the classes of the first medical image and the second medical image are the same.

8 . The image processing apparatus according to claim 1 ,

wherein the processor is configured to perform normalization processing of matching the first medical image and the second medical image with a reference medical image prior to the non-linear registration processing.

9 . The image processing apparatus according to claim 1 ,

wherein the medical image is an image in which a head of a patient appears, and

the machine learning model is a model that outputs a dementia opinion on the patient.

10 . An operation method of an image processing apparatus, the method comprising:

performing non-linear registration processing on a first medical image and a second medical image among a plurality of medical images to derive transformation amounts as a result of the non-linear registration processing, wherein the medical images are classified into classes,

generating transformation coefficients based on the classes of the first medical image and the second medical image,

multiplying the transformation coefficients by the derived transformation amounts to obtain corrected transformation amounts; and

generating at least one new medical image that is used for training a machine learning model for the medical images by transforming at least one medical image of the first medical image or the second medical image using at least one of the corrected transformation amounts.

11 . A non-transitory computer-readable storage medium storing an operation program of an image processing apparatus, the program causing a computer to execute a process comprising:

performing non-linear registration processing on a first medical image and a second medical image among a plurality of medical images to derive transformation amounts as a result of the non-linear registration processing, wherein the medical images are classified into classes,

generating transformation coefficients based on the classes of the first medical image and the second medical image,

multiplying the transformation coefficients by the derived transformation amounts to obtain corrected transformation amounts; and

generating at least one new medical image that is used for training a machine learning model for the medical images by transforming at least one medical image of the first medical image or the second medical image using at least one of the corrected transformation amounts.

12 . A non-transitory computer-readable storage medium storing a trained model that is trained by using a new medical image as a learning image, the new medical image being generated by transforming at least one medical image of a first medical image or a second medical image among a plurality of medical images using at least one of corrected transformation amounts, wherein the corrected transformation amounts is obtained by:

performing non-linear registration processing on the first medical image and the second medical image to derive transformation amounts as a result of the non-linear registration processing, wherein the medical images are classified into classes,

generating transformation coefficients based on the classes of the first medical image and the second medical image, and

multiplying the transformation coefficients by the derived transformation amounts to obtain the corrected transformation amounts.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2023
From: WANG, CAIHUA
To: FUJIFILM CORPORATION
Reel/Frame 063027/0785 →
Priority Claims (1)
JP 2020-162678 · Sep 28, 2020 · national
Continuity (2)
Continuation PCTJP2021033189 · Sep 9, 2021
Related Publication 20230222675A1 · Jul 13, 2023
References Cited (26)
US 6915003B2 · Oosawa · 2005 [cited by applicant]
US 10980493B2 · Osawa · 2021 [cited by applicant]
US 11096643B2 · Takei · 2021 [cited by applicant]
US 20200118265A1 · Igarashi · 2020 [cited by applicant]
US 20200327370A1 · Masuda et al. · 2020 [cited by applicant]
US 20210390282A1 · Lin · 2021 [cited by examiner]
US 20230005140A1 · Ferl · 2023 [cited by examiner]
EP 3686805 · 2020 [cited by applicant]
JP 2002032735 · 2002 [cited by applicant]
JP 2018175227 · 2018 [cited by applicant]
JP 2019195426 · 2019 [cited by applicant]
JP 2020058590 · 2020 [cited by applicant]
WO 2019069867 · 2019 [cited by applicant]
Ava Assadi Abolvardi et al., “Registration Based Data Augmentation for Multiple Sclerosis Lesion Segmentation”, 2019 Digital Image Computing: Techniques and Applications (DICTA), Dec. 2, 2019, pp. 1-5. [cited by applicant]
Kevin P. Nguyen et al., “Anatomically-Informed Data Augmentation for functional MRI with Applications to Deep Learning”, Proceedings of SPIE—the International Society for Optical Engineering, Mar. 10, 2020, pp. 1-5, vol… [cited by applicant]
Amy Zhao et al., “Data augmentation using learned transformations for one-shot medical image segmentation”, Computer Vision and Pattern Recognition, Feb. 25, 2019, pp. 1-12. [cited by applicant]
Philip Novosad et al., “Accurate and robust segmentation of neuroanatomy in TI-weighted MRI by combining spatial priors with deep convolutional neural networks”, Human brain mapping, Oct. 21, 2019, pp. 1-22, vol. 41, No… [cited by applicant]
David G. Ellis et al., “Deep Learning Using Augmentation via Registration: 1st Place Solution to the Autolmplant 2020 Challenge”, Topics in Cryptology-CT-RSA2020: The Cryptographers' Track at the RSA Conference, Dec. 4,… [cited by applicant]
“Search Report of Europe Counterpart Application”, issued on Mar. 4, 2024, p. 1-p. 11. [cited by applicant]
“Office Action of Japan Counterpart Application”, issued on Dec. 19, 2023, with English translation thereof, p. 1-p. 6. [cited by applicant]
“International Search Report (Form PCT/ISA/210) of PCT/JP2021/033189”, mailed on Oct. 12, 2021, with English translation thereof, pp. 1-5. [cited by applicant]
“Written Opinion of the International Searching Authority (Form PCT/ISA/237) of PCT/JP2021/033189”, mailed on Oct. 12, 2021, with English translation thereof, pp. 1-6. [cited by applicant]
Connor Shorten et al., “A survey on Image Data Augmentation for Deep Learning,” Journal of Big Data, Jul. 2019, pp. 1-49. [cited by applicant]
Yuji Tokozume et al., “Between-class Learning for Image Classification,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2018, pp. 5486-5494. [cited by applicant]
Gregory Kuling et al., “Data Augmentation with Conditional Generative Adversarial Networks for Improved Medical Image Segmentation,” Proc. Intl. Soc. Mag. Reson. Med., Aug. 2020, pp. 1-3. [cited by applicant]
Ryoma Aoki et al., “Differences in Results provided by Teaching Data on Cartilage Extraction in Knee MR Images by Using Deep Learning,” IEICE Technical Report, Feb. 2019, pp. 1-8. [cited by applicant]