Systems and methods for assessing heart and respiratory disorders
The present disclosure describes systems and methods for assessing heart and respiratory disorders in a patient using a video analysis tool to analyze video data taken of the patients neck region. The video analysis tool allows for an accurate, objective assessment of the patient. In some aspects of the present disclosure, a clinician may capture video data of the patients neck region and run the video analysis tool. Additionally or alternatively, the system may be implemented remotely whereby the video analysis tool may be executed remotely from the video recording device. The video analysis tool extracts pulsatile signals in the neck region from the video data and determines pulsatile signals of interest for assessing heart and respiratory disorders in the patient. The video analysis tool generates assessment data based on the pulsatile signals of interest.
1 . A computer-implemented method for assessing heart and respiratory health in a patient, comprising:
receiving video data of a neck region of the patient captured by a video recording device;
generating, with a video analysis tool, assessment data indicative of heart and/or respiratory health in the patient, wherein generating the assessment data comprises:
segmenting the video data into a plurality of sub-regions, each of the plurality of sub-regions depicting a sub-region of the neck region and corresponding to a region in the video data;
extracting a plurality of pulsatile signals corresponding to an artery or a vein in the neck region, wherein a pulsatile signal is extracted from each of the plurality of sub-regions of the video data by tracking variations of pixel color and/or pixel movement in each sub-region of the video data to generate a plurality of times-series data corresponding to the plurality of pulsatile signals, each of the plurality of times-series data corresponding to one of the plurality of pulsatile signals;
extracting features corresponding to pulsatile signal features and comprising statistical features, time domain features, frequency domain features, or combinations thereof from each of the plurality of times-series data;
classifying one or more pulsatile signals of the plurality of pulsatile signals as pulsatile signals of interest indicative of the heart and/or respiratory health in the patient based on the extracted features and classifying one or more of the pulsatile signals of interest as either a jugular venous pulsation or a carotid artery pulsation; and
generating the assessment data indicative of heart and/or respiratory health in the patient based on the pulsatile signals of interest and the extracted features; and
outputting the assessment data indicative of heart and/or respiratory health in the patient,
wherein the assessment data comprises: a heat map of the neck region of the patient highlighting the pulsatile signals of interest and the jugular venous pulsation.
2 . The method of claim 1 , further comprising determining an assessment result of heart and/or respiratory disorder in the patient based on the pulsatile signals of interest, wherein the assessment data comprises the assessment result.
3 . The method of claim 2 , further comprising:
determining at least one of a heart rate and a respiration rate of the patient from the video data based on the pulsatile signals of interest; and
wherein determining the assessment result is further based on the at least one of the heart rate and the respiration rate, and wherein the assessment data comprises one or both of the heart rate and the respiration rate.
4 . The method of claim 2 , further comprising at least one of:
receiving supplemental patient data for the patient, wherein determining the assessment result is further based on the supplemental patient data; and
determining the assessment result further based on historical assessment data generated from previously received video data.
5 . The method of claim 2 , wherein the assessment result is an arrhythmia, determined based on a time difference between consecutive pulse peaks of the pulsatile signals of interest.
6 . The method of claim 2 , further comprising:
determining a recommended action based on the assessment result; and
wherein the assessment data comprises the recommended action.
7 . The method of claim 1 , wherein the features extracted from the pulsatile signal include one or more of: a signal to noise ratio, an amplitude, a frequency, a shape of a pulse, a power level, and a second harmonic of the pulsatile signal.
8 . The method of claim 1 , wherein the video data is received from a remote computing device, and wherein outputting the assessment data comprises transmitting the assessment data to the remote computing device for display.
9 . A non-transitory computer-readable medium having computer-readable instructions stored thereon, which when executed by a processing device, configure the processing device to perform the method of claim 1 .
10 . The method of claim 1 , wherein the heatmap is generated by analyzing signal to noise ratio based on a signal power of the plurality of pulsatile signals and/or based on a ratio of power based on a power of a second harmonic from each of the plurality of pulsatile signals.
11 . The method of claim 1 , wherein the features extracted from the pulsatile signal comprises an envelope of a cardiac signal, the method further comprising, for each sub-region: computing a correlation coefficient between the envelope and a respiratory signal to classify a respective sub-region as affected by breathing or not affected by breathing.
12 . The method of claim 1 , wherein the assessment result comprises a determination of cognitive heart failure, determined based on a frequency of the jugular venous pulsation and heart rate.
13 . A system for assessing heart and respiratory health in a patient, comprising:
a processing unit; and
a memory having computer-readable instructions stored thereon, which when executed by the processing unit, configure the system to:
receive video data of a neck region of the patient captured by a video recording device;
generate, with a video analysis tool, assessment data indicative of heart and/or respiratory heath in the patient, wherein generating the assessment data comprises:
segmenting the video data into a plurality of sub-regions, each of the plurality of sub-regions depicting a sub-region of the neck region and corresponding to a region in the video data;
extracting a plurality of pulsatile signals corresponding to an artery or a vein in the neck region, wherein a pulsatile signal is extracted from each of the plurality of sub-regions of the video data by tracking variations of pixel color and/or pixel movement in each sub-region of the video data to generate a plurality of times-series data corresponding to the plurality of pulsatile signals, each of the plurality of times-series data corresponding to one of the plurality of pulsatile signals;
extracting features corresponding to pulsatile signal features and comprising statistical features, time domain features, frequency domain features, or combinations thereof from each of the plurality of times-series data;
classifying one or more pulsatile signals of the plurality of pulsatile signals as pulsatile signals of interest indicative of the heart and/or respiratory health in the patient based on the extracted features and classifying one or more of the pulsatile signals of interest as either a jugular venous pulsation or a carotid artery pulsation; and
generating the assessment data indicative of heart and/or respiratory health in the patient based on the pulsatile signals of interest and the extracted features; and
output the assessment data indicative of heart and/or respiratory health in the patient,
wherein the assessment data comprises: a heat map of the neck region of the patient highlighting the pulsatile signals of interest and the jugular venous pulsation.
14 . The system of claim 13 , wherein the system is further configured to determine an assessment result of heart and/or respiratory disorder in the patient based on the pulsatile signals of interest, and wherein the assessment data comprises the assessment result.
15 . The system of claim 14 , wherein the system is further configured to:
determine at least one of a heart rate and a respiration rate of the patient from the video data based on the pulsatile signals of interest; and
wherein determining the assessment result is further based on the at least one of the heart rate and the respiration rate, and wherein the assessment data comprises one or both of the heart rate and the respiration rate.
16 . The system of claim 14 , wherein the system is further configured to at least one of:
receive supplemental patient data for the patient,
wherein determining the assessment result is further based on the supplemental patient data; and
determine the assessment result further based on historical assessment data generated from previously received video data.
17 . The system of claim 14 , wherein the assessment result is an arrhythmia, determined based on a time difference between consecutive pulse peaks of the pulsatile signals of interest.
18 . The system of claim 14 , wherein the system is further configured to:
determine a recommended action based on the assessment result; and
wherein the assessment data comprises the recommended action.
19 . The system of claim 13 , wherein the features extracted from the pulsatile signal include one or more of: a signal to noise ratio, an amplitude, a frequency, a shape of a pulse, a power level, and a second harmonic of the pulsatile signal.
20 . The system of claim 13 , wherein the video data is received from a remote computing device, and wherein outputting the assessment data comprises transmitting the assessment data to the remote computing device for display.
21 . The system of claim 13 , wherein the heatmap is generated by analyzing signal to noise ratio based on a signal power of the plurality of pulsatile signals and/or based on a ratio of power based on a power of a second harmonic from each of the plurality of pulsatile signals.
22 . The system of claim 13 , wherein the features extracted from the pulsatile signal comprises an envelope of a cardiac signal, the method further comprising, for each sub-region: computing a correlation coefficient between the envelope and a respiratory signal to classify a respective sub-region as affected by breathing or not affected by breathing.
23 . The system of claim 13 , wherein the assessment result comprises a determination of cognitive heart failure, determined based on a frequency of the jugular venous pulsation and heart rate.