IP Library Granted Patent US 11,373,306
Granted Patent B1
US 11,373,306 · App. 17/410,465 · Granted Jun 28, 2022

Processing pathology images

Inventor: Juha Reunanen (Helsinki, FI)
Assignee: AIFORIA TECHNOLOGIES OYJ
G06T7/0012G06T3/4046G06T2207/10056G06T2207/20081G06T2207/20084G16H30/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,373,306
App. No.
17/410,465
Granted
Jun 28, 2022
Kind
B1
Abstract

A method of facilitating processing of pathology images involves receiving pathology image data representing a pathology image having a plurality of image regions, wherein the pathology image data includes, for each of the plurality of image regions, a respective plurality of representations of the image region including a first representation and a second representation, the second representation having a smaller data size than the first representation. The method involves, for each of the plurality of image regions: determining, based at least in part on the first representation of the image region, a first set of image properties, determining whether the first set of image properties meets first image property criteria, and, if the first set of image properties meets the first image property criteria, producing signals for causing the second representation to be used in place of the first representation. Other methods, systems, and computer-readable media are disclosed.

Claims (105)

1. A method of facilitating processing of pathology images, the method comprising:

receiving pathology image data representing a pathology image having a plurality of image regions, wherein the pathology image data includes, for each of the plurality of image regions, a respective plurality of representations of the image region including a first representation and a second representation, the second representation having a smaller data size than the first representation;

for each of the plurality of image regions:

determining, based at least in part on the first representation of the image region, a first set of image properties, wherein the first set of image properties represents a difference between the first and second representations of the image region; and

wherein determining the first set of image properties comprises determining based on at least the first and second representations, a first image difference representing the difference between the first and second representations of the image region; and

wherein determining the first image difference comprises:

upscaling the second representation of the image region to generate a first upscaled representation of the image region, the first upscaled representation of the image region having the same pixel dimensions as the first representation of the image region; and

for each pixel position in the first representation and the first upscaled representation, determining a pixel value difference between respective pixels of the first representation and the first upscaled representation; and

determining whether the first set of image properties meets first image property criteria; and

if the first set of image properties meets the first image property criteria, producing signals for causing the second representation to be used in place of the first representation.

2. The method of claim 1 wherein the second representation has a pixel width smaller than a pixel width of the first representation and the second representation has a pixel height smaller than a pixel height of the first representation.

3. The method of claim 1 wherein, for each of the plurality of image regions:

determining the first image difference comprises determining from the pixel value differences, a first maximum pixel value difference; and

determining whether the first set of image properties meets the first image property criteria comprises comparing the first maximum pixel value difference with a first maximum pixel difference threshold.

4. The method of claim 1 wherein, for each of the plurality of image regions:

determining the first image difference comprises:

determining from the pixel value differences, an extreme negative pixel value difference; and

determining from the pixel value differences, an extreme positive pixel value difference; and

determining whether the first set of image properties meets the first image property criteria comprises:

comparing the extreme negative pixel value difference with an extreme negative pixel difference threshold; and

comparing the extreme positive pixel value difference with an extreme positive pixel difference threshold, the extreme positive pixel difference threshold being different from the extreme negative pixel difference threshold.

5. The method of claim 1 wherein, for each pixel position in the first representation and the first upscaled representation, determining the pixel value difference comprises scaling the pixel value difference by a pixel value scaling factor, the pixel value scaling factor based at least in part on an overall pixel value intensity of the first representation.

6. The method of claim 1 wherein, for each of the plurality of image regions, the plurality of representations of the image region includes a third representation, the third representation having a smaller data size than the second representation and wherein determining whether the first set of image properties meets the first image property criteria comprises:

determining, based at least in part on the second representation of the image region, a second set of image properties;

determining whether the second set of image properties meets second image property criteria; and

if the second set of image properties meets the second image property criteria, producing signals for causing the third representation to be used in place of the first representation.

7. The method of claim 1 wherein, for each of the plurality of image regions, the plurality of representations of the image region includes a third representation, the third representation having a smaller data size than the second representation and wherein determining whether the first set of image properties meets the first image property criteria comprises:

determining, based at least in part on the second representation of the image region, a second set of image properties;

determining whether the second set of image properties meets second image property criteria; and

if the second set of image properties meets the second image property criteria, producing signals for causing the third representation to be used in place of the first representation.

8. The method of claim 7 wherein the pixel value differences are first pixel value differences and wherein, for each of the plurality of image regions:

determining the first image difference comprises determining from the first pixel value differences, a first maximum pixel value difference; and

determining whether the first set of image properties meets the first image property criteria comprises comparing the first maximum pixel value difference with a first maximum pixel difference threshold; and

wherein, for each of the plurality of image regions, determining the second set of image properties comprises determining based on at least the second and third representations, a second image difference representing a difference between the second and third representations of the image region and wherein, for each of the plurality of image regions, determining the second image difference comprises:

upscaling the third representation of the image region to generate a second upscaled representation of the image region, the second upscaled representation of the image region having the same pixel dimensions as the second representation of the image region;

for each pixel position in the second representation and the second upscaled representation, determining a second pixel value difference between respective pixels of the second representation and the second upscaled representation; and

determining from the second pixel value differences, a second maximum pixel value difference; and

wherein determining whether the second set of image properties meets the second image property criteria comprises comparing the second maximum pixel value difference with a second maximum pixel difference threshold, wherein the second maximum pixel difference threshold has a lower magnitude than the first maximum pixel difference threshold.

9. The method of claim 1 wherein producing signals for causing the second representation to be used in place of the first representation comprises producing signals for causing the second representation to be displayed by at least one display in place of the first representation.

10. The method of claim 1 further comprising receiving a request for the first representation and wherein producing signals for causing the second representation to be used in place of the first representation comprises producing signals for causing the second representation to be provided in response to the request for the first representation.

11. The method of claim 1 wherein, for each of the plurality of image regions, determining the first set of image properties comprises inputting at least the first representation of the image region into a machine learning model to cause the machine learning model to generate at least one machine learning model image property.

12. The method of claim 11 wherein the machine learning model includes a convolutional neural network.

13. The method of claim 1 comprising, for each of the plurality of image regions, whether the first set of image properties meets or does not meet the first image property criteria, causing the second representation to be stored such that the second representation is configured to be provided when requested.

14. The method of claim 1 wherein producing signals for causing the second representation to be used in place of the first representation comprises producing signals identifying the first representation as replaceable by the second representation.

15. The method of claim 14 wherein producing signals identifying the first representation as replaceable by the second representation comprises including an identifier of the first representation in a replaceable first representation record.

16. The method of claim 15 wherein the replaceable first representation record includes an identifying image.

17. The method of claim 16 wherein the identifying image includes a binary image.

18. A system for facilitating processing of pathology images, the system comprising at least one processor configured to:

receive pathology image data representing a pathology image having a plurality of image regions, wherein the pathology image data includes, for each of the plurality of image regions, a respective plurality of representations of the image region including a first representation and a second representation, the second representation having a smaller data size than the first representation;

for each of the plurality of image regions:

determine, based at least in part on the first representation of the image region, a first set of image properties, wherein the first set of image properties represents a difference between the first and second representations of the image region, the at least one process configured to:

determine, based on at least the first and second representations, a first image difference representing the difference between the first and second representations of the image region, the at least one processor configured to:

upscale the second representation of the image region to generate a first upscaled representation of the image region, the first upscaled representation of the image region having the same pixel dimensions as the first representation of the image region; and

for each pixel position in the first representation and the first upscaled representation, determine a pixel value difference between respective pixels of the first representation and the first upscaled representation; and

determine whether the first set of image properties meets first image property criteria; and

if the first set of image properties meets the first image property criteria, produce signals for causing the second representation to be used in place of the first representation.

19. The system of claim 18 wherein the second representation has a pixel width smaller than a pixel width of the first representation and the second representation has a pixel height smaller than a pixel height of the first representation.

20. The system of claim 18 wherein the at least one processor is configured to, for each of the plurality of image regions:

determine from the pixel value differences, a first maximum pixel value difference; and

compare the first maximum pixel value difference with a first maximum pixel difference threshold.

21. The system of claim 18 wherein the at least one processor is configured to, for each of the plurality of image regions:

determine from the pixel value differences, an extreme negative pixel value difference; and

determine from the pixel value differences, an extreme positive pixel value difference; and

compare the extreme negative pixel value difference with an extreme negative pixel difference threshold; and

compare the extreme positive pixel value difference with an extreme positive pixel difference threshold, the extreme positive pixel difference threshold being different from the extreme negative pixel difference threshold.

22. The system of claim 18 wherein, for each of the plurality of image regions, the plurality of representations of the image region includes a third representation, the third representation having a smaller data size than the second representation and wherein the at least one processor is configured to determine whether the first set of image properties meets the first image property criteria by:

determining, based at least in part on the second representation of the image region, a second set of image properties;

determining whether the second set of image properties meets second image property criteria; and

if the second set of image properties meets the second image property criteria, producing signals for causing the third representation to be used in place of the first representation.

23. The system of claim 18 wherein, for each of the plurality of image regions, the plurality of representations of the image region includes a third representation, the third representation having a smaller data size than the second representation and wherein the at least one processor is configured to:

determine, based at least in part on the second representation of the image region, a second set of image properties;

determine whether the second set of image properties meets second image property criteria; and

if the second set of image properties meets the second image property criteria, produce signals for causing the third representation to be used in place of the first representation.

24. The system of claim 23 wherein the pixel value differences are first pixel value differences and wherein the at least one processor is configured to, for each of the plurality of image regions:

determine from the first pixel value differences, a first maximum pixel value difference;

compare the first maximum pixel value difference with a first maximum pixel difference threshold;

determine based on at least the second and third representations, a second image difference representing a difference between the second and third representations of the image region;

upscale the third representation of the image region to generate a second upscaled representation of the image region, the second upscaled representation of the image region having the same pixel dimensions as the second representation of the image region;

for each pixel position in the second representation and the second upscaled representation, determine a second pixel value difference between respective pixels of the second representation and the second upscaled representation; and

determine from the second pixel value differences, a second maximum pixel value difference; and

wherein the at least one processor is configured to compare the second maximum pixel value difference with a second maximum pixel difference threshold, wherein the second maximum pixel difference threshold has a lower magnitude than the first maximum pixel difference threshold.

25. The system of claim 18 wherein the at least one processor is configured to produce signals identifying the first representation as replaceable by the second representation.

26. The system of claim 25 wherein the at least one processor is configured to include an identifier of the first representation in a replaceable first representation record.

27. The system of claim 26 wherein the replaceable first representation record includes an identifying image.

28. The system of claim 27 wherein the identifying image includes a binary image.

29. A non-transitory computer-readable medium having stored thereon codes that when executed by at least one processor cause the at least one processor to:

receive pathology image data representing a pathology image having a plurality of image regions, wherein the pathology image data includes, for each of the plurality of image regions, a respective plurality of representations of the image region including a first representation and a second representation, the second representation having a smaller data size than the first representation;

for each of the plurality of image regions:

determine, based at least in part on the first representation of the image region, a first set of image properties, wherein the first set of image properties represents a difference between the first and second representations of the image region;

wherein determining the first set of image properties comprises determining based on at least the first and second representations, a first image difference representing the difference between the first and second representations of the image region; and

wherein determining the first image difference comprises:

upscaling the second representation of the image region to generate a first upscaled representation of the image region, the first upscaled representation of the image region having the same pixel dimensions as the first representation of the image region; and

for each pixel position in the first representation and the first upscaled representation, determining a pixel value difference between respective pixels of the first representation and the first upscaled representation; and

determine whether the first set of image properties meets first image property criteria; and

if the first set of image properties meets the first image property criteria, produce signals for causing the second representation to be used in place of the first representation.

30. A system for facilitating processing of pathology images, the system comprising:

means for receiving pathology image data representing a pathology image having a plurality of image regions, wherein the pathology image data includes, for each of the plurality of image regions, a respective plurality of representations of the image region including a first representation and a second representation, the second representation having a smaller data size than the first representation;

means for, for each of the plurality of image regions:

determining, based at least in part on the first representation of the image region, a first set of image properties, wherein the first set of image properties represents a difference between the first and second representations of the image region;

wherein determining the first set of image properties comprises determining based on at least the first and second representations, a first image difference representing the difference between the first and second representations of the image region; and

wherein determining the first image difference comprises:

upscaling the second representation of the image region to generate a first upscaled representation of the image region, the first upscaled representation of the image region having the same pixel dimensions as the first representation of the image region; and

for each pixel position in the first representation and the first upscaled representation, determining a pixel value difference between respective pixels of the first representation and the first upscaled representation; and

determining whether the first set of image properties meets first image property criteria; and

means for, if the first set of image properties meets the first image property criteria, producing signals for causing the second representation to be used in place of the first representation.

Assignments (4)
CHANGE OF NAME Recorded Jul 21, 2022
From: AIFORIA TECHNOLOGIES OYJ
To: AIFORIA TECHNOLOGIES OYJ
Reel/Frame 060803/0769 →
CONFIRMATORY ASSIGNMENT Recorded Apr 1, 2022
From: TOMAATTINEN OY
To: AIFORIA TECHNOLOGIES OYJ
Reel/Frame 059568/0671 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2021
From: REUNANEN, JUHA
To: TOMAATTINEN OY
Reel/Frame 057975/0618 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2021
From: TOMAATTINEN OY
To: AIFORIA TECHNOLOGIES OY
Reel/Frame 057975/0622 →
Continuity (1)
Provisional Application 63221109 · Jul 13, 2021