IP Library › Granted Patent US 10,943,347
Granted Patent B2
US 10,943,347 · App. 16/405,928 · Granted Mar 9, 2021

Image processing method, apparatus, and non-transitory readable storage medium

Inventors: Huangjing Lin (Shenzhen, CN); Qi Dou (Shenzhen, CN); Hao Chen (Shenzhen, CN)
Assignee: SHENZHEN IMSIGHT MEDICAL TECHNOLOGY CO. LTD
G06T7/0012G06T5/50G06T7/11G06T2207/20084G06T2207/20101G06T2207/20221G06T2207/30004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,943,347
App. No.
16/405,928
Granted
Mar 9, 2021
Kind
B2
Abstract

Disclosed are an image processing method, an image processing apparatus, and a readable storage medium. First, an image to be processed is received, and the received image to be processed is divided into regions of interest by region segmentation means. Next, the regions of interest are detected by calling a pre-stored full convolution network structure model, to obtain probability image segments. Finally, the probability image segments are synthesized to generate a target probability image. Wherein, the pre-stored full convolution network structure model includes a full convolution structure. A linear regression layer is replaced by an equivalent convolution layer in the full convolution structure. A blank padding operation layer and an up-sampling layer are removed from the full convolution structure.

Claims (84)

1. An image processing method, comprising:

receiving an image to be processed, dividing the image to be processed into regions of interest by region segmentation means;

detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments, wherein, the pre-stored full convolution network structure model comprises a full convolution structure, a linear regression layer is replaced by an equivalent convolution layer in the full convolution structure, a blank padding operation layer and an up-sampling layer are removed from the full convolution structure; and

synthesizing the probability image segments to generate a target probability image.

2. The method of claim 1 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments comprises:

acquiring a defined densification parameter, obtaining an expansion rule of the region of interest and a model transformation rule corresponding to the pre-stored full convolution network structure model, based on the densification parameter;

performing expanded interception on the regions of interest based on the expansion rule, to obtain expansion images;

transforming the pre-stored full convolution network structure model into a full convolution network transformation structure model, based on the model transformation rule; and

performing densification detection on each of the expansion images by the full convolution network transformation structure model, to obtain each of the probability image segments.

3. The method of claim 2 , wherein the model transformation rule comprises a backtracking convolution rule;

the operation of obtaining an expansion rule of the region of interest and a model transformation rule corresponding to the pre-stored full convolution network structure model, based on the densification parameter comprises:

obtaining a densification times and an equivalent scan stride based on the densification parameter, to calculate the expansion rule of the region of interest; and

obtaining an anchor point stride, a convolution kernel moving distance, and a convolution dilation rate required for transformation of each layer in the pre-stored full convolution network structure model based on the densification parameter, to obtain the backtracking convolution rule.

4. The method of claim 1 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments comprises:

obtaining a defined offset densification coefficient and offset densification dimension information, to calculate an offset times and a unit length of a single offset corresponding to each of the regions of interest;

offsetting each of the regions of interest based on the offset times and the unit length of the single offset, to obtain each of offset regions of interest;

detecting the offset regions of interest by calling the pre-stored full convolution network structure model, to obtain the offset probability image segments;

the operation of synthesizing the probability image segments to generate a target probability image comprises:

stitching the probability image segments interleavingly, to reconstruct a densification probability image; and

taking the densification probability image as the target probability image.

5. The method of claim 1 , wherein the operation of receiving an image to be processed, dividing the image to be processed into regions of interest by region segmentation means comprises:

receiving the image to be processed, obtaining a tissue region by retrieving the image to be processed, based on an adaptive threshold maximum inter-class variance algorithm; and

obtaining a current video memory capacity, dividing the tissue region into the regions of interest by using a divide-and-conquer algorithm, according to the current video memory capacity.

6. The method of claim 1 , wherein after the operation of synthesizing the probability image segments to generate a target probability image, the method further comprises:

removing a singular point from the target probability image using a morphological opening operation rule; and

outputting the target probability image after removing the singular point.

7. The method of claim 1 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments comprises:

detecting each of the regions of interest in parallel by calling the pre-stored full convolution network structure model, to obtain each of the probability image segments.

8. An image processing apparatus, wherein, the image processing apparatus comprises a memory, a processor, a communication bus, and an image processing program stored in the memory,

the communication bus is configured to implement communication connection between the processor and the memory;

the processor is configured to execute the image processing program, in order to perform the following operations:

receiving an image to be processed, dividing the image to be processed into regions of interest by region segmentation means;

detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments, wherein, the pre-stored full convolution network structure model comprises a full convolution structure, a linear regression layer is replaced by an equivalent convolution layer in the full convolution structure, a blank padding operation layer and an up-sampling layer are removed from the full convolution structure; and

synthesizing the probability image segments to generate a target probability image.

9. The apparatus of claim 8 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments comprises:

acquiring a defined densification parameter, obtaining an expansion rule of the region of interest and a model transformation rule corresponding to the pre-stored full convolution network structure model, based on the densification parameter;

performing expanded interception on the regions of interest based on the expansion rule, to obtain expansion images;

transforming the pre-stored full convolution network structure model into a full convolution network transformation structure model, based on the model transformation rule; and

performing densification detection on each of the expansion images by the full convolution network transformation structure model, to obtain each of the probability image segments.

10. The apparatus of claim 9 , wherein the model transformation rule comprises a backtracking convolution rule;

the operation of obtaining an expansion rule of the region of interest and a model transformation rule corresponding to the pre-stored full convolution network structure model, based on the densification parameter comprises:

obtaining a densification times and an equivalent scan stride based on the densification parameter, to calculate the expansion rule of the region of interest; and

obtaining an anchor point stride, a convolution kernel moving distance, and a convolution dilation rate required for transformation of each layer in the pre-stored full convolution network structure model based on the densification parameter, to obtain the backtracking convolution rule.

11. The apparatus of claim 8 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments, comprises:

obtaining a defined offset densification coefficient and offset densification dimension information, to calculate an offset times and a unit length of a single offset corresponding to each of the regions of interest;

offsetting each of the regions of interest based on the offset times and the unit length of the single offset, to obtain each of offset regions of interest;

detecting the offset regions of interest by calling the pre-stored full convolution network structure model, to obtain the offset probability image segments;

the operation of synthesizing the probability image segments to generate a target probability image comprises:

stitching the probability image segments interleavingly, to reconstruct a densification probability image; and

taking the densification probability image as the target probability image.

12. The apparatus of claim 8 , wherein the operation of receiving an image to be processed, and region dividing the image to be processed, to obtain regions of interest comprises:

receiving the image to be processed, obtaining a tissue region by retrieving the image to be processed, based on an adaptive threshold maximum inter-class variance algorithm; and

obtaining a current video memory capacity, dividing the tissue region into the regions of interest by using a divide-and-conquer algorithm, according to the current video memory capacity.

13. The apparatus of claim 8 , wherein after the operation of synthesizing the probability image segments to generate a target probability image, the processor is further configured to execute the image processing program to perform the following operations:

removing a singular point from the target probability image using a morphological opening operation rule; and

outputting the target probability image after removing the singular point.

14. The apparatus of claim 8 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model to obtain probability image segments comprises:

detecting each of the regions of interest in parallel by calling the pre-stored full convolution network structure model, to obtain each of the probability image segments.

15. A non-transitory readable storage medium, wherein, the non-transitory readable storage medium stores an image processing program, the image processing program when being executed by a processor performs the following operations:

receiving an image to be processed, dividing the image to be processed into regions of interest by region segmentation means;

detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments, wherein, the pre-stored full convolution network structure model comprises a full convolution structure, a linear regression layer is replaced by an equivalent convolution layer in the full convolution structure, a blank padding operation layer and an up-sampling layer are removed from the full convolution structure; and

synthesizing the probability image segments to generate a target probability image.

16. The non-transitory readable storage medium of claim 15 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments comprises:

acquiring a defined densification parameter, obtaining an expansion rule of the region of interest and a model transformation rule corresponding to the pre-stored full convolution network structure model, based on the densification parameter;

performing expanded interception on the regions of interest based on the expansion rule, to obtain expansion images;

transforming the pre-stored full convolution network structure model into a full convolution network transformation structure model, based on the model transformation rule; and

performing densification detection on each of the expansion images by the full convolution network transformation structure model, to obtain each of the probability image segments.

17. The non-transitory readable storage medium of claim 16 , wherein the model transformation rule comprises a backtracking convolution rule,

the operation of obtaining an expansion rule of the region of interest and a model transformation rule corresponding to the pre-stored full convolution network structure model, based on the densification parameter comprises:

obtaining a densification times and an equivalent scan stride based on the densification parameter, to calculate the expansion rule of the region of interest; and

obtaining an anchor point stride, a convolution kernel moving distance, and a convolution dilation rate required for transformation of each layer in the pre-stored full convolution network structure model based on the densification parameter, to obtain the backtracking convolution rule.

18. The non-transitory readable storage medium of claim 17 , wherein the operation of detecting the regions of interest by calling a pre-stored full convolution network structure model, to obtain probability image segments comprises:

obtaining a defined offset densification coefficient and offset densification dimension information, to calculate an offset times and a unit length of a single offset corresponding to each of the regions of interest;

offsetting each of the regions of interest based on the offset times and the unit length of the single offset, to obtain each of offset regions of interest;

detecting the offset regions of interest by calling the pre-stored full convolution network structure model, to obtain the offset probability image segments;

the operation of synthesizing the probability image segments to generate a target probability image comprises:

stitching the probability image segments interleavingly, to reconstruct a densification probability image; and

taking the densification probability image as the target probability image.

19. The non-transitory readable storage medium of claim 16 , wherein the operation of receiving an image to be processed, and dividing the image to be processed into regions of interest by region segmentation means comprises:

receiving the image to be processed, obtaining a tissue region by retrieving the image to be processed, based on an adaptive threshold maximum inter-class variance algorithm; and

obtaining a current video memory capacity, dividing the tissue region into the regions of interest by using a divide-and-conquer algorithm, according to the current video memory capacity.

20. The non-transitory readable storage medium of claim 16 , wherein after the operation of synthesizing the probability image segments to generate a target probability image, the image processing program when being executed by the processor further performs the following operations:

removing a singular point from the target probability image using a morphological opening operation rule; and

outputting the target probability image after removing the singular point.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2020
From: LIN, HUANGJING; DOU, QI; CHEN, HAO
To: SHENZHEN IMSIGHT MEDICAL TECHNOLOGY CO. LTD
Reel/Frame 052619/0237 →
Priority Claims (1)
CN 201810695269.4 · Jun 28, 2018 · national
Continuity (1)
Related Publication 20200005453A1 · Jan 2, 2020
Cited By (1)
US 12,456,055