Efficient multi-dimensional storage of very large dense, sparse and semi-sparse image datasets with machine learning
Methods, apparatus, and systems for high-resolution image storage and analysis are disclosed. In one disclosed embodiment, a method includes analyzing a high-resolution image; partitioning the high-resolution image into logical blocks; detecting objects in the high-resolution image within the logical blocks, assigning objects that differ or with differing measures to differing layers, and assigning objects that are similar or with similar measures to the same layer; tabulating the detected objects into a table, wherein each row indicates a spatial position of each detected object and at least one measure of each detected object; storing the spatial position and the at least one measure of each detected object into the table; spatially rendering selected objects over the high-resolution image; and displaying the selected objects at a selected resolution within a portion of the high-resolution image on a display device.
1 . A method of storing and processing a large-scale digital image, the method comprising:
reading a digital image within an image space at a one hundred percent scale, wherein the one hundred percent scale represents a highest resolution of the digital image;
defining the image space of the digital image by using spatial parameters, measure parameters, and resolution parameters, wherein the spatial parameters define the extent of the digital image, and wherein the measure parameters include predefined metrics for detecting imaged objects within the digital image, and wherein the resolution parameters define a plurality of resolutions including the highest resolution for the image space;
partitioning the image space into a plurality of logical blocks that are arbitrarily positioned and arbitrarily sized, wherein at least some of the logical blocks overlap on one or more areas of the image space to generate overlapping blocks;
detecting the imaged objects and associated measures in each of the plurality of logical blocks by using the measure parameters; and
storing the imaged objects associated with the logical blocks at the one hundred percent scale and one or more lower resolutions, wherein the storing includes tabular storage of the associated measures, object types, and object identifiers into a table, and wherein the storing further includes a storage of object spatial data into a semi-sparse matrix associated with the object identifiers.
2 . The method of claim 1 , further comprising:
optimizing storage of the logical blocks, wherein the optimizing includes
fusing overlapping parts of the imaged objects identified in the overlapping blocks,
densifying and re-gridding the digital image at the one or more lower resolutions to generate one or more further lower resolutions, and
using the one or more further lower resolutions to generate a fast visualization of the image space at the further lower resolution levels.
3 . The method of claim 2 , further comprising:
adding new measures to the table by using machine learning inference, wherein the machine learning includes an analysis of data in the table associated with one or more imaged objects; and
iteratively performing the optimizing storage and the adding new measures to further describe additional various characteristics of the imaged objects.
4 . The method of claim 3 , further comprising:
generating a visualization of any scale of the image space, including retrieving planes of one or more measures of interest.
5 . The method of claim 3 , further comprising:
generating a visualization of a sparse layer of the image space by finding overlapping logical blocks in a requested spatial extent in a requested resolution level.
6 . The method of claim 1 , wherein
the digital image is captured by using a microscope.
7 . The method of claim 1 , wherein
the digital image is captured at a highest resolution available for an image capture device.
8 . The method of claim 1 , wherein
the digital image is captured by using an image capture device configured for at least one of:
whole tissue microscopy;
light-sheet microscopy;
large-scale electromagnetic (EM) microscopy;
remote sensing satellite imagery; or
aerial imagery.
9 . The method of claim 1 , wherein
the storing includes down-sampling the imaged objects from one hundred percent to the one or more lower resolutions.
10 . The method of claim 1 , wherein
the lower resolutions are less than or equal to a 50% scale relative to the highest resolution.
11 . The method of claim 1 , wherein
the further lower resolutions are less than or equal to a 25% scale relative to the highest resolution.
12 . The method of claim 1 , further comprising:
displaying, on a display device, selected objects at a selected resolution and within a portion of the image space that is at the one hundred percent scale.
13 . A system comprising:
one or more computers including one or more processors configured to perform:
reading a digital image within an image space at a one hundred percent scale, wherein the one hundred percent scale represents a highest resolution of the digital image;
defining the image space of the digital image by using spatial parameters, measure parameters, and resolution parameters, wherein the spatial parameters define the extent of the digital image, and wherein the measure parameters include predefined metrics for detecting imaged objects within the digital image, and wherein the resolution parameters define a plurality of resolutions including the highest resolution for the image space;
partitioning the image space into a plurality of logical blocks that are arbitrarily positioned and arbitrarily sized, wherein at least some of the logical blocks overlap on one or more areas of the image space to generate overlapping blocks;
detecting the imaged objects and associated measures in each of the plurality of logical blocks by using the measure parameters; and
storing the imaged objects associated with the logical blocks at the one hundred percent scale and one or more lower resolutions, wherein the storing includes tabular storage of the associated measures, object types, and object identifiers into a table, and wherein the storing further includes a storage of object spatial data into a semi-sparse matrix associated with the object identifiers.
14 . The system of claim 13 , wherein the one or more processors are further configured to perform:
optimizing storage of the logical blocks, wherein the optimizing includes
fusing overlapping parts of the imaged objects identified in the overlapping blocks,
densifying and re-gridding the digital image at the one or more lower resolutions to generate one or more further lower resolutions, and
using the one or more further lower resolutions to generate a fast visualization of the image space at the further lower resolution levels.
15 . The system of claim 14 , wherein the one or more processors are further configured to perform:
adding new measures to the table by using machine learning inference, wherein the machine learning includes an analysis of data in the table associated with one or more imaged objects; and
iteratively performing the optimizing storage and the adding new measures to further describe additional various characteristics of the imaged objects.
16 . The system of claim 15 , wherein the one or more processors are further configured to perform:
generating a visualization of any scale of the image space, including retrieving planes of one or more measures of interest.
17 . The system of claim 15 , wherein the one or more processors are further configured to perform:
generating a visualization of a sparse layer of the image space by finding overlapping logical blocks in a requested spatial extent in a requested resolution level.
18 . The system of claim 13 , wherein
the digital image is captured by using a microscope.
19 . The system of claim 13 , wherein
the digital image is captured at a highest resolution available for an image capture device.
20 . The system of claim 13 , wherein
the digital image is captured by using an image capture device configured for at least one of:
whole tissue microscopy;
light-sheet microscopy;
large-scale electromagnetic (EM) microscopy;
remote sensing satellite imagery; or
aerial imagery.
21 . The system of claim 13 , wherein
the storing includes down-sampling the imaged objects from one hundred percent to the one or more lower resolutions.
22 . The system of claim 13 , wherein
the lower resolutions are less than or equal to a 50% scale relative to the highest resolution.
23 . The system of claim 13 , wherein
the further lower resolutions are less than or equal to a 25% scale relative to the highest resolution.
24 . The system of claim 23 , wherein the one or more processors are further configured to perform:
displaying, on a display device, selected objects at a selected resolution and within a portion of the image space that is at the one hundred percent scale.