Anatomical location detection of features of a gastrointestinal tract of a patient
Generating a structured medical record from endoscopy data includes obtaining image data including endoscopic images representing portions of a gastrointestinal tract (GI) of a patient; determining features to extract from the image data, the features each representing a physical parameter of the GI tract; extracting the features from the image data; generating anatomical location data specifying a location within the GI tract of a portion of the GI tract represented in the image data; associating the anatomical location data with images that represent the portion of the GI tract; storing, in a node of a data store, data entries including the anatomical location data and the associated one or more images. The data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries.
1 . A data processing system for processing video data representing colonoscopies of a patient, the data processing system comprising:
an interface configured to receive video data from a camera, the video data representing a colonoscopy for a patient;
at least one processor configured to perform operations comprising:
obtaining the video data via the interface;
performing coarse alignment of the video data over a time period, the coarse alignment comprising:
segmenting the video data into segments representing portions of a colon of the patient; and
assigning a time period to each segment;
extracting a set of features from the video data based on a type of disease associated with the video data, wherein the set of features for the video are unique to the type of the disease,
wherein the coarse alignment comprises setting at least one boundary based on a feature of the set of features extracted from the video data;
performing a medium alignment of the video data based on the coarse alignment, the medium alignment comprising:
determining a motion of the camera through a portion of the colon represented by a segment; and
determining a location of the camera within the segment for a given time in the time period of the segment;
performing a fine alignment of the video data based on the medium alignment, the fine alignment comprising:
associating one or more features of the colon detected in the segment with a feature time stamp; and
generating, from the one or more features of the colon detected based on the fine alignment, structured data representing the colon of the patient; and
a user interface configured to present output data representing the structured data.
2 . The data processing system of claim 1 , wherein the structured data comprises a feature vector representing the colon of the patient, wherein the operations further comprise:
generating a cluster comprising the feature vector with one or more other feature vectors representing other colons of other patients, the cluster representing patients that have a similar colon disease.
3 . The data processing system of claim 1 , wherein the features comprise one or more of a detected ulceration, a detected erosion, a detected erythema, a detected reduced vascularization, tumors, polyps, a detected friability, or another disease marker.
4 . The data processing system of claim 1 , wherein at least one feature is associated with additional data specifying one or more of a size of the feature, a density of feature in the colon, and an extent of the feature in the colon.
5 . The data processing system of claim 1 , the operations further comprising performing dimensional grouping of features based on at least one of:
an anatomical section of the colon or segment of the video data;
a frame sequence including the features; or
a time value associated with one or more of the features.
6 . The data processing system of claim 1 , the operations further comprising labeling the features, a label specifying at least one of an ileum average size of erosion, an ileum extent of erosions, an ascending colon average ulceration size, an ascending colon erosion density, a transverse colon average ulceration size, a transverse colon erosion density, a descending colon average ulceration size, or a descending colon erosion density.
7 . The data processing system of claim 1 , wherein the video data represent a scope of a colon of the patient including at least one of a cecum, an ileum, or a rectum of the patient.
8 . The data processing system of claim 1 , wherein performing the coarse alignment comprises:
detecting a chronology of features of a path through the colon of the patient, the chronology comprising an ileum, an ascending colon boundary, a transverse colon boundary, a descending colon boundary, and a rectum boundary; and
based on the detected chronology, assigning a time stamp to each feature of the path through the colon.
9 . The data processing system of claim 1 , wherein a feature represents a cecal valve, an appendix orifice, or a crow-path pattern of a colon of the patient.
10 . The data processing system of claim 1 , wherein a feature represents a colon wall pattern.
11 . The data processing system of claim 1 , wherein a feature represents a triangular pattern of a transverse colon.
12 . The data processing system of claim 1 , wherein performing the medium alignment comprises detecting a movement type of a camera capturing the video data in the colon.
13 . The data processing system of claim 12 , wherein the movement type comprises a hovering the camera in a particular location of the colon.
14 . The data processing system of claim 12 , wherein the movement types comprises a quasi-stationary motion of the camera in a location of the colon.
15 . A data processing system for processing video data representing colonoscopies of a patient, the data processing system comprising:
an interface configured to receive video data from a camera, the video data representing a colonoscopy for a patient;
at least one processor configured to perform operations comprising:
obtaining the video data via the interface;
performing coarse alignment of the video data over a time period, the coarse alignment comprising:
segmenting the video data into segments representing portions of a colon of the patient; and
assigning a time period to each segment;
performing a medium alignment of the video data based on the coarse alignment, the medium alignment comprising:
determining a motion of the camera through a portion of the colon represented by a segment; and
determining a location of the camera within the segment for a given time in the time period of the segment;
performing a fine alignment of the video data based on the medium alignment, the fine alignment comprising:
associating one or more features of the colon detected in the segment with a feature time stamp; and
generating, from the one or more features of the colon detected based on the fine alignment, structured data representing the colon of the patient; and
a user interface configured to present output data representing the structured data,
wherein the structured data comprises a feature vector representing the colon of the patient, wherein the operations further comprise:
generating a cluster comprising the feature vector with one or more other feature vectors representing other colons of other patients, the cluster representing patients that have a similar colon disease.
16 . A data processing system for processing video data representing colonoscopies of a patient, the data processing system comprising:
an interface configured to receive video data from a camera, the video data representing a colonoscopy for a patient;
at least one processor configured to perform operations comprising:
obtaining the video data via the interface;
performing coarse alignment of the video data over a time period, the coarse alignment comprising:
segmenting the video data into segments representing portions of a colon of the patient; and
assigning a time period to each segment;
extracting a set of features from the video data based on a type of disease associated with the video data, wherein the set of features for the video are unique to the type of the disease,
wherein the coarse alignment comprises setting at least one boundary based on a feature of the set of features extracted from the video data;
performing a medium alignment of the video data based on the coarse alignment, the medium alignment comprising:
determining a motion of the camera through a portion of the colon represented by a segment; and
determining a location of the camera within the segment for a given time in the time period of the segment;
performing a fine alignment of the video data based on the medium alignment, the fine alignment comprising:
associating one or more features of the colon detected in the segment with a feature time stamp;
labeling the features, a label specifying at least one of an ileum average size of erosion, an ileum extent of erosions, an ascending colon average ulceration size, an ascending colon erosion density, a transverse colon average ulceration size, a transverse colon erosion density, a descending colon average ulceration size, or a descending colon erosion density; and
generating, from the one or more features of the colon detected based on the fine alignment, structured data representing the colon of the patient; and
a user interface configured to present output data representing the structured data.