Granted Patent B2

US 12,469,250 · App. 18/559,779 · Granted Nov 11, 2025

Inventors: Daniel Zimmerle (Fort Collins, CO); Marcus Martinez (Fort Collins, CO)

Assignee: Colorado State University Research Foundation

G06V10/36G06V10/30G06V10/34G06V10/766G06V20/40G01M3/04

Abstract

Claims (148)

1 . A method comprising:

receiving video data comprising a plurality of frames representative of infrared radiation (IR) within a scene, each frame of the plurality of frames comprising a plurality of pixels;

identifying high-frequency changes in temperature within the scene due to motion of a gas plume released by a gas source within the scene, wherein identifying the high-frequency changes in temperature includes identifying pixels within the plurality of frames that correspond to the gas plume released by the gas source within the scene based on the IR; and

determining a size of the gas plume within each frame based on the identified pixels;

wherein the method further comprises:

filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames, each filtered frame comprising a plurality of filtered values;

calculating a signal strength value of each pixel based on the plurality of filtered frames according to:

σ

⁢

(

F

i

,

j

,

{

k

}

)

{

k

}

=

k

⁢

…

⁢

k

+

L

in which, σ represents a standard deviation operator, F i,j,(k) represents the filtered values corresponding to current pixels of a subset of the plurality of filtered frames, i represents a horizontal axis index of the current pixels, j represents a vertical axis index of the current pixels, k represents a current filtered frame index, and L represents a number of the plurality of filtered frames to be included in the subset of the filtered frames; and

generating a plurality of strength frames, each strength frame corresponding to a different frame of the plurality of frames and each strength frame comprising the signal strength values of corresponding pixels, wherein the pixels identified within the plurality of frames that correspond to the gas plume are identified based on the corresponding signal strength value.

2 . The method of claim 1 further comprising:

determining a number of pixels within each frame that correspond to the gas plume, wherein the size of the gas plume within each frame is determined based on the number of pixels within each frame that correspond to the gas plume; and

determining a probability of detection of the gas plume within the scene by a user based on the size of the gas plume within each frame.

3 . The method of claim 1 further comprising filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames, wherein the pixels identified within the plurality of frames that correspond to the gas plume comprise the pixels that correspond to high-frequency changes in temperature within the plurality of frames.

4 . The method of claim 1 , further comprising:

applying a filter to the plurality of strength frames to generate a plurality of blurred frames, each blurred frame comprising a plurality of blurred values representative of a smoothed version of corresponding signal strength values;

calculating a signal strength value of each pixel within the plurality of frames;

determining an absolute difference between the signal strength value and the corresponding blurred value for each pixel;

responsive to the absolute difference between the signal strength value and the corresponding blurred value of a pixel being greater than a strength threshold value, identifying the pixel as corresponding to a data spike and setting the corresponding blurred value equal to zero; and

responsive to the signal strength value of a pixel within the plurality of frames being less than a noise threshold value, identifying the pixel as corresponding to the data spike and setting the corresponding blurred value equal to zero, wherein the pixels identified within the plurality of frames that correspond to the gas plume correspond to pixels within the plurality of filtered frames that correspond to a signal strength value greater than zero.

5 . The method of claim 4 , wherein the plurality of frames comprise an OFF subset corresponding to a period of time in which the gas source is not releasing the gas plume, the method further comprising:

comparing the plurality of blurred values to the noise threshold value to generate a plurality of noise frames, each noise frame corresponding to a different frame of the plurality of frames and each noise frame comprising a plurality of noise values;

calculating a mean noise value of the plurality of pixels within the OFF subset based on the corresponding noise values to generate a matrix of mean noise values;

generating a weighting frame comprising a plurality of weight values, wherein each of the weight values is based on the corresponding mean noise value and the corresponding noise values; and

generating a detection frame comprising a plurality of detection values, wherein each detection value of the plurality of detection values is based on the corresponding weight value, wherein the pixels identified within the plurality of frames that correspond to the gas plume comprise pixels that correspond to a detection value greater than zero.

6 . The method of claim 5 further comprising:

determining a number of plume pixels, wherein the plume pixels comprise pixels that correspond to a detection value greater than zero;

determining a number of non-plume pixels, wherein the non-plume pixels comprise pixels that correspond to a detection value equal to zero; and

determining a ratio of the number of plume pixels compared to the number of non-plume pixels, wherein the size of the gas plume is determined based on the ratio of the number of plume pixels compared to the number of non-plume pixels.

7 . A non-transitory computer-readable medium having computer-readable instructions stored thereon that are executable by a processor to perform or control performance of operations comprising:

receiving video data comprising a plurality of frames representative of infrared radiation (IR) within a scene, each frame of the plurality of frames comprising a plurality of pixels;

filtering out, from each frame, pixels that correspond to low-frequency changes in temperature;

identifying pixels within the plurality of frames that correspond to a gas plume released by a gas source within the scene based on the IR, the pixels identified within the plurality of frames that correspond to the gas plume comprising pixels that correspond to high-frequency changes in temperature within the plurality of frames;

determining a size of the gas plume within each frame based on the identified pixels; and

determining a probability of detection of the gas plume within the scene by a user based on the size of the gas plume within each frame;

wherein the filtering out, from each frame, of pixels that correspond to low-frequency changes in temperature generates a plurality of filtered frames, each filtered frame comprising a plurality of filtered values, the operations further comprising:

calculating a signal strength value of each pixel based on the plurality of filtered frames according to:

σ

⁢

(

F

i

,

j

,

{

k

}

)

{

k

}

=

k

⁢

…

⁢

k

+

L

in which, σ represents a standard deviation operator, F i,j,(k) represents the filtered values corresponding to current pixels of a subset of the plurality of filtered frames, i represents a horizontal axis index of the current pixels, j represents a vertical axis index of the current pixels, k represents a current filtered frame index, and L represents a number of the plurality of filtered frames to be included in the subset of the filtered frames; and

generating a plurality of strength frames, each strength frame corresponding to a different frame of the plurality of frames and each strength frame comprising the signal strength values of corresponding pixels, wherein the pixels identified within the plurality of frames that correspond to the gas plume are identified based on the corresponding signal strength value.

8 . The non-transitory computer-readable medium of claim 7 , the operations further comprising determining a number of pixels within each frame that correspond to the gas plume, wherein the size of the gas plume within each frame is determined based on the number of pixels within each frame that correspond to the gas plume.

9 . The non-transitory computer-readable medium of claim 7 , the operations further comprising:

applying a filter to the plurality of strength frames to generate a plurality of blurred frames, each blurred frame comprising a plurality of blurred values representative of a smoothed version of corresponding signal strength values;

calculating a signal strength value of each pixel within the plurality of frames;

determining an absolute difference between the signal strength value and the corresponding blurred value for each pixel;

responsive to the absolute difference between the signal strength value and the corresponding blurred value of a pixel being greater than a strength threshold value, identifying the pixel as corresponding to a data spike and setting the corresponding blurred value equal to zero; and

responsive to the signal strength value of a pixel within the plurality of frames being less than a noise threshold value, identifying the pixel as corresponding to the data spike and setting the corresponding blurred value equal to zero, wherein the pixels identified within the plurality of frames that correspond to the gas plume correspond to pixels within the plurality of filtered frames that correspond to a signal strength value greater than zero.

10 . The non-transitory computer-readable medium of claim 9 , wherein the plurality of frames comprise an OFF subset corresponding to a period of time in which the gas source is not releasing the gas plume, the operations further comprising:

comparing the plurality of blurred values to the noise threshold value to generate a plurality of noise frames, each noise frame corresponding to a different frame of the plurality of frames and each noise frame comprising a plurality of noise values;

calculating a mean noise value of the plurality of pixels within the OFF subset based on the corresponding noise values to generate a matrix of mean noise values;

generating a weighting frame comprising a plurality of weight values, wherein each of the weight values is based on the corresponding mean noise value and the corresponding noise values; and

generating a detection frame comprising a plurality of detection values, wherein each detection value of the plurality of detection values is based on the corresponding weight value, wherein the pixels identified within the plurality of frames that correspond to the gas plume comprise pixels that correspond to a detection value greater than zero.

11 . The non-transitory computer-readable medium of claim 10 , the operations further comprising:

determining a number of plume pixels, wherein the plume pixels comprise pixels that correspond to a detection value greater than zero;

determining a number of non-plume pixels, wherein the non-plume pixels comprise pixels that correspond to a detection value equal to zero; and

determining a ratio of the number of plume pixels compared to the number of non-plume pixels, wherein the size of the gas plume is determined based on the ratio of the number of plume pixels compared to the number of non-plume pixels.

12 . A system comprising:

one or more computer-readable storage media configured to store instructions; and

one or more processors communicatively coupled to the one or more computer-readable storage media and configured to, in response to execution of the instructions, cause the system to perform operations, the operations comprising:

receiving video data comprising a plurality of frames representative of infrared radiation (IR) within a scene, each frame of the plurality of frames comprising a plurality of pixels;

identifying pixels within the plurality of frames that correspond to a gas plume released by a gas source within the scene based on the IR; and

determining a size of the gas plume within each frame based on the identified pixels;

wherein the operations further comprise:

calculating a signal strength value of each pixel based on the plurality of frames according to:

σ

⁢

(

F

i

,

j

,

{

k

}

)

{

k

}

=

k

⁢

…

⁢

k

+

L

in which, σ represents a standard deviation operator, F i,j,(k) represents the values corresponding to current pixels of a subset of the plurality of frames, i represents a horizontal axis index of the current pixels, j represents a vertical axis index of the current pixels, k represents a current frame index, and L represents a number of the plurality of frames to be included in the subset of frames, and

generating a plurality of strength frames, each strength frame corresponding to a different frame of the plurality of frames and each strength frame comprising the signal strength values of corresponding pixels, wherein the pixels identified within the plurality of frames that correspond to the gas plume are identified based on the corresponding signal strength value.

13 . The system of claim 12 , the operations further comprising:

determining a number of pixels within each frame that correspond to the gas plume, wherein the size of the gas plume within each frame is determined based on the number of pixels within each frame that correspond to the gas plume and

determining a probability of detection of the gas plume within the scene by a user based on the size of the gas plume within each frame.

14 . The system of claim 12 , the operations further comprising filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames, wherein the pixels identified within the plurality of frames that correspond to the gas plume comprise the pixels that correspond to high-frequency changes in temperature within the plurality of frames.

15 . The system of claim 12 , the operations further comprising:

applying a filter to the plurality of strength frames to generate a plurality of blurred frames, each blurred frame comprising a plurality of blurred values representative of a smoothed version of corresponding signal strength values;

calculating a signal strength value of each pixel within the plurality of frames;

determining an absolute difference between the signal strength value and the corresponding blurred value for each pixel;

responsive to the absolute difference between the signal strength value and the corresponding blurred value of a pixel being greater than a strength threshold value, identifying the pixel as corresponding to a data spike and setting the corresponding blurred value equal to zero; and

responsive to the signal strength value of a pixel within the plurality of frames being less than a noise threshold value, identifying the pixel as corresponding to the data spike and setting the corresponding blurred value equal to zero, wherein the pixels identified within the plurality of frames that correspond to the gas plume correspond to pixels within the plurality of frames that correspond to a signal strength value greater than zero.

16 . The system of claim 15 , wherein the plurality of frames comprise an OFF subset corresponding to a period of time in which the gas source is not releasing the gas plume, the operations further comprising:

comparing the plurality of blurred values to the noise threshold value to generate a plurality of noise frames, each noise frame corresponding to a different frame of the plurality of frames and each noise frame comprising a plurality of noise values;

calculating a mean noise value of the plurality of pixels within the OFF subset based on the corresponding noise values to generate a matrix of mean noise values;

generating a weighting frame comprising a plurality of weight values, wherein each of the weight values is based on the corresponding mean noise value and the corresponding noise values; and

generating a detection frame comprising a plurality of detection values, wherein each detection value of the plurality of detection values is based on the corresponding weight value, wherein the pixels identified within the plurality of frames that correspond to the gas plume comprise pixels that correspond to a detection value greater than zero.

Assignments (1)

Continuity (2)

References Cited (20)