On-demand metadata extraction of clinical image data
A method and system for providing on-demand updating of metadata information associated with clinical image data files is disclosed. The system allows for extracting, on the fly, additional metadata fields from clinical image data files and re-indexing the metadata to include the additional fields so that an end-user can perform operations (e.g., searching, browsing, etc.) based on the newly indexed fields. The metadata fields may be defined in a configurable schema file (e.g., XML, YML, etc.). The system can gain efficiencies by, for example, reading only a subset of the DICOM file (e.g., reading only the first few kilobytes that contain the header), scanning only a subset of source data folders, etc.
1. A tangible computer-readable storage medium excluding transitory signals, and which contains instructions for performing a method of on-demand metadata extraction, the method comprising:
receive, via an implementation of a function of an API invoked by at least one user device, a list of clinical image data files containing clinical image data, wherein each file in the list of files is associated with a metadata file,
wherein the metadata files are stored in a location remote from the user devices, and
wherein the remote location is a cloud storage device;
receive a list of metadata attributes, via an implementation of a function of the API invoked by the at least one user device, wherein the list of metadata attributes are associated with clinical image data; and
for each clinical image data file in the list of clinical image data files:
retrieve only a header portion of the clinical image data file,
wherein a size of the retrieved header portion of the clinical image data file is smaller than a size of the clinical image data file to thereby gain efficiencies in data transmission, and
wherein the retrieved header portion includes only metadata; and,
for each attribute in the list of metadata attributes:
parse the retrieved header portion of the clinical image data file to identify a value associated with the attribute; and
update the metadata file associated with the clinical image data file to include the attribute and the identified value associated with the attribute.
2. The tangible computer-readable storage medium of claim 1 , wherein the method further comprises:
indexing the updated metadata file associated with at least one clinical image data file in the list of clinical image data files using the attribute and the identified value associated with the attribute; and
based upon the indexing, updating a metadata index associated with the at least one clinical image data file in the list of clinical image data files.
3. The tangible computer-readable storage medium of claim 1 , wherein at least one clinical image data file in the list of clinical image data files is of type:
Digital Imaging and Communications in Medicine (DICOM),
Analyze,
Neuroimaging Informatics Technology Initiative (NIFTI), or
Minc.
4. The tangible computer-readable storage medium of claim 1 , wherein receiving the list of metadata attributes comprises receiving a configurable schema file including the list of metadata attributes, and wherein the configurable schema file is selected from a set of configurable schema files based at least in part on three or more of:
user profile,
security permissions,
one or more locations of the clinical image data files in list of clinical image data files,
timing information, or
news-based events.
5. The tangible computer-readable storage medium of claim 1 , wherein the list of metadata attributes are stored in a data structure.
6. The tangible computer-readable storage medium of claim 1 , wherein retrieving only the header portion of the clinical image data file comprises retrieving a predetermined number of bits of the clinical image data file,
wherein the predetermined number of bits of the clinical image data file is determined based at least in part on two or more of:
type of the clinical image data file,
bandwidth constraints,
data storage constraints, or
response time constraints.
7. The tangible computer-readable storage medium of claim 1 , wherein receiving a list of files containing clinical image data comprises receiving location of at least one folder containing at least one clinical image data file.
8. A computer-implemented method of on-demand metadata extraction, the method comprising:
receive a list of files containing clinical image data, wherein each file in the list of files is associated with a metadata file;
receive a list of metadata attributes, wherein the list of metadata attributes are associated with clinical image data; and
for each file in the list of files:
retrieve only a portion of the file, wherein the retrieved portion of the file is smaller than a size of the file; and
for each attribute in the list of metadata attributes:
parse the retrieved portion of the file to identify a value associated with the attribute; and
update the metadata file associated with the file to include the attribute and the identified value associated with the attribute.
9. The computer-implemented method claim 8 , further comprising:
indexing the updated metadata file associated with at least one file in the list of files; and
based upon the indexing, updating a metadata index associated with the at least one file in the list of files.
10. The computer-implemented method claim 8 , wherein at least one file in the list of files is of type:
Digital Imaging and Communications in Medicine (DICOM),
Analyze,
Neuroimaging Informatics Technology Initiative (NIFTI), or
Minc.
11. The computer-implemented method claim 8 , wherein receiving the list of metadata attributes comprises receiving a configurable schema file including the list of metadata attributes.
12. The computer-implemented method claim 8 , wherein the list of metadata attributes are stored in a data structure.
13. The computer-implemented method claim 8 , wherein retrieving the at least a portion of the file comprises retrieving a header portion of the file.
14. The computer-implemented method claim 8 , wherein retrieving only the portion of the file comprises retrieving a predetermined number of bits of the file.
15. The computer-implemented method claim 8 , wherein receiving a list of files containing clinical image data comprises receiving location of at least one folder containing at least one clinical image data file.
16. An apparatus, coupled to a network and configured to perform on-demand metadata extraction, the apparatus comprising:
at least one hardware processor coupled to the network; and,
at least one memory storing instructions, which when executed by the at least one processor, performs a method to:
receive a list of files containing clinical image data, wherein each file in the list of files is associated with a metadata file;
receive a list of metadata attributes, wherein the list of metadata attributes are associated with clinical image data; and
for each file in the list of files:
retrieve only a header portion of the file,
wherein the retrieved header portion of the is smaller than a size of the file such that retrieving only the header portion results in data transmission efficiency gains, and
wherein the retrieved header portion includes only metadata; and
for each attribute in the list of metadata attributes:
parse the retrieved header portion of the file to identify a value associated with the attribute; and
update the metadata file associated with the file to include the attribute and the identified value associated with the attribute.
17. The apparatus claim 16 , wherein the method further comprises:
indexing the updated metadata file associated with at least one file in the list of files; and
based upon the indexing, updating a metadata index associated with the at least one file in the list of files.
18. The apparatus claim 16 , wherein at least one file in the list of files is of type:
Digital Imaging and Communications in Medicine (DICOM),
Analyze,
Neuroimaging Informatics Technology Initiative (NIFTI), or
Minc.
19. The apparatus claim 16 , wherein receiving the list of metadata attributes comprises receiving a configurable schema file including the list of metadata attributes.