IP Library Granted Patent US 11,170,482
Granted Patent B2
US 11,170,482 · App. 16/619,771 · Granted Nov 9, 2021

Image processing method and device

Inventors: Lvwei Wang (Beijing, CN); Zhenglong Li (Beijing, CN); Yingying Li (Beijing, CN)
G06T5/50G06T5/009G06T7/0012G06T7/13G06T2207/20081G06T2207/20084G06T2207/20221G06T2207/30088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,170,482
App. No.
16/619,771
Granted
Nov 9, 2021
Kind
B2
Abstract

The present disclosure relates to an image processing method and a device, wherein the image processing method includes: acquiring a first image, the first image including an image formed with respect to a skin part of a patient to be diagnosed which needs diagnosis; inputting the first image into a neural network, to acquire position information of a pathologic change area in the first image; acquiring a boundary of the pathologic change area in the first image, acquiring an original image and a mask image which include the pathologic change area from the first image; fusing the mask image and the original image, obtaining a target image corresponding to the pathologic change area; wherein the target image is an image for diagnosing the pathologic change area, pixel points in the target image correspond to those in the original image and the mask image one by one.

Claims (62)

1. An image processing method comprising:

acquiring a first image, the first image including an image formed with respect to a skin part of a patient to be diagnosed which needs diagnosis;

inputting the first image into a neural network, to acquire position information of a pathologic change area in the first image;

acquiring a boundary of the pathologic change area in the first image, acquiring an original image and a mask image which include the pathologic change area from the first image; and

fusing the mask image and the original image to obtain a target image corresponding to the pathologic change area; wherein the target image is an image for diagnosing the pathologic change area, and wherein pixel points in the target image have a one-to-one correspondence to pixel points in the original image and to pixel points in the mask image,

wherein fusing the mask image and the original image to obtain the target image corresponding to the pathologic change area includes:

acquiring a first pixel value of each pixel point in the mask image and a second pixel value of each pixel point in the original image;

fusing the first pixel value of each pixel point in the mask image and the second pixel value of a corresponding pixel point in the original image, to acquire a target pixel value of each pixel point; and

forming the target image of the pathologic change area according to target pixel values of all pixel points.

2. The image processing method according to claim 1 , wherein fusing the first pixel value of each pixel point in the mask image and the second pixel value of the corresponding pixel point in the original image to acquire a target pixel value of each pixel point includes:

dividing the first pixel value of each pixel point in the mask image by a maximum pixel value to obtain a ratio, and multiplying the ratio by the second pixel value of the corresponding pixel point in the original image, to obtain a first data;

acquiring a difference value between the maximum pixel value and the first pixel value; and

adding the first data and the difference value to obtain the target pixel value of each pixel point.

3. The image processing method according to claim 1 , wherein acquiring the first pixel value of each pixel point in the mask image and the second pixel value of each pixel point in the original image includes:

extracting the first pixel value of each pixel point in the mask image, and using the first pixel value of each pixel point to constitute a first matrix of the mask image, wherein a position of the first pixel value of each pixel point in the first matrix is determined by a position of the pixel point in the mask image, and

extracting the second pixel value of each pixel point in the original image, and using the second pixel value of each pixel point to constitute a second matrix of the original image, wherein a position of the second pixel value of each pixel point in the second matrix is determined by a position of the pixel point in the original image;

wherein fusing the first pixel value of each pixel point in the mask image and the second pixel value of the corresponding pixel point in the original image to acquire the target pixel value of each pixel point includes:

dividing the first matrix by the maximum pixel value, to obtain a third matrix,

multiplying a pixel value of each pixel point in the third matrix by the second pixel value of a corresponding pixel point in the second matrix, to obtain a fourth matrix,

subtracting the first pixel value of each pixel point in the first matrix from the maximum pixel value respectively, to obtain a fifth matrix, and

adding the fourth matrix and the fifth matrix, to obtain a sixth matrix, wherein a pixel value of each pixel point in the sixth matrix is the target pixel value of each pixel point.

4. The image processing method according to claim 1 , wherein after fusing the mask image and the original image to obtain the target image corresponding to the pathologic change area, the method further includes:

diagnosing the pathologic change area in the target image to acquire a diagnosis result.

5. The image processing method according to claim 1 , wherein before acquiring the first image, the method further includes:

acquiring a sample image;

acquiring mark data of the sample image, the mark data including position information of the pathologic change area in the sample image; and

training a neural network, by the sample image and the mark data, to form the neural network which has a required function.

6. The image processing method according to claim 5 , wherein before training the neural network, by the sample image and the mark data, to form the neural network which has a required function, the method further includes at least one of:

randomly selecting a selected proportion of the sample image to be subjected to a hair supplement process; or,

randomly selecting a selected proportion of the sample image to be subjected to a color enhancement process.

7. The image processing method according to claim 1 , wherein the position information includes center coordinates and a radius value of the pathologic change area.

8. The image processing method according to claim 1 , wherein the position information includes center coordinates, a major axis radius value and a minor axis radius value of the pathologic change area.

9. The image processing method according to claim 1 , before inputting the first image into the neural network to acquire the position information of the pathologic change area in the first image, the method further includes performing pretreatment on the first image.

10. The image processing method according to claim 9 , wherein performing pretreatment on the first image includes:

acquiring a size of the first image;

comparing the size of the first image with an image resolution parameter of an input layer of the neural network to determine whether the size of the first image is more than the image resolution parameter of the input layer;

in response to determining that the size of the first image is more than the image resolution parameter of the input layer, cutting or reducing the first image; and

in response to determining that the size of the first image is less than the image resolution parameter of the input layer, enlarging the first image.

11. The image processing method according to claim 1 , wherein acquiring the boundary of the pathologic change area in the first image to acquire the original image and the mask image which include the pathologic change area from the first image is based on the position information and an image edge detection algorithm.

12. The image processing method according to claim 1 , wherein acquiring the first image includes scanning the skin part of the patient to be diagnosed which needs diagnosis to form the first image.

13. The image processing method according to claim 1 , wherein fusing the mask image and the original image includes performing a bitwise AND operation on the mask image and the original image.

14. A computer device comprising: a processor; a memory;

and computer program instructions stored in the memory, which, when executed by the processor, cause the processor to execute the image processing method according to claim 1 .

15. The computer device of claim 14 , wherein the fusing the mask image and the original image to obtain the target image corresponding to the pathologic change area includes:

acquiring a first pixel value of each pixel point in the mask image and a second pixel value of each pixel point in the original image;

fusing the first pixel value of each pixel point in the mask image and the second pixel value of a corresponding pixel point in the original image, to acquire a target pixel value of each pixel point; and

forming the target image of the pathologic change area according to target pixel values of all pixel points.

16. The computer device of claim 14 , wherein the fusing the first pixel value of each pixel point in the mask image and the second pixel value of the corresponding pixel point in the original image to acquire a target pixel value of each pixel point includes:

dividing the first pixel value of each pixel point in the mask image by a maximum pixel value to obtain a ratio, and multiplying the ratio by the second pixel value of the corresponding pixel point in the original image, to obtain a first data;

acquiring a difference value between the maximum pixel value and the first pixel value; and

adding the first data and the difference value to obtain the target pixel value of each pixel point.

17. The computer device of claim 14 , wherein acquiring the first pixel value of each pixel point in the mask image and the second pixel value of each pixel point in the original image includes:

extracting the first pixel value of each pixel point in the mask image, and using the first pixel value of each pixel point to constitute a first matrix of the mask image, wherein a position of the first pixel value of each pixel point in the first matrix is determined by a position of the pixel point in the mask image, and

extracting the second pixel value of each pixel point in the original image, and using the second pixel value of each pixel point to constitute a second matrix of the original image, wherein a position of the second pixel value of each pixel point in the second matrix is determined by a position of the pixel point in the original image;

wherein fusing the first pixel value of each pixel point in the mask image and the second pixel value of the corresponding pixel point in the original image to acquire the target pixel value of each pixel point includes:

dividing the first matrix by the maximum pixel value, to obtain a third matrix,

multiplying a pixel value of each pixel point in the third matrix by the second pixel value of a corresponding pixel point in the second matrix, to obtain a fourth matrix,

subtracting the first pixel value of each pixel point in the first matrix from the maximum pixel value respectively, to obtain a fifth matrix, and

adding the fourth matrix and the fifth matrix, to obtain a sixth matrix, wherein a pixel value of each pixel point in the sixth matrix is the target pixel value of each pixel point.

18. The computer device of claim 14 , wherein after fusing the mask image and the original image to obtain the target image corresponding to the pathologic change area, the method further includes:

diagnosing the pathologic change area in the target image to acquire a diagnosis result.

19. A non-transient computer-readable storage medium with computer program instructions stored thereon, which, when executed by a processor, cause the processor to execute the image processing method according to claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2019
From: WANG, LVWEI; LI, ZHENGLONG; LI, YINGYING
To: BOE TECHNOLOGY GROUP CO., LTD.
Reel/Frame 051195/0429 →
Priority Claims (1)
CN 201710730705.2 · Aug 23, 2017 · national
Continuity (1)
Related Publication 20200143526A1 · May 7, 2020
Cited By (1)
US 12,555,435