Granted Patent B2

US 12,475,601 · App. 18/166,394 · Granted Nov 18, 2025

Inventors: Joydeep Acharya (Milpitas, CA); Ravneet Kaur (San Jose, CA); Hidenori Omiya (Tokyo, JP); Yusaku Otsuka (Tokyo, JP); Takahiro Ohira (Tokyo, JP); Toshiki Shimizu (Tokyo, JP)

Assignee: HITACHI, LTD.

G06T7/97G06T7/11

Abstract

Claims (106)

1 . A method for computing and detecting image data drift, the method comprising:

retrieving first segment information of a plurality of segments from a drift database;

receiving a number of images from a sensor;

partitioning each of the received images into segments of a predetermined number;

generating second segment information;

computing drift in values between the first segment information and the second segment information; and

detecting drift based on the computed drift in values by combining the computed drift in segments to generate overall drift, and comparing the overall drift against a drift threshold,

wherein the first segment information comprises a first set of counters, and the first set of counters is derived by:

setting number of segments and shapes of segments based on domain knowledge of applications;

partitioning an image into the plurality of segments, wherein number of the plurality of segments is the number of segments; and

initializing the first set of counters to zero, wherein number of counters in the first set of counters corresponds to the number of segments, and each counter of the first set of counters monitors number of times drift is detected in respective segment of the plurality of segments.

2 . The method of claim 1 , wherein the first segment information comprises a first mean and a first covariance matrix associate with the plurality of segments, and the second segment information comprises a second mean and a second covariance matrix.

3 . The method of claim 1 , wherein the generating the second segment information comprises:

fitting, for each number less than or equal to the predetermined number, a multidimensional distribution computed over a number of instances of segment to generate the second segment information, wherein the number of instances of segment corresponds to number of the number of images.

4 . The method of claim 1 , wherein the predetermined number is same as number of segments of the plurality of segments from the drift database.

5 . The method of claim 1 , wherein the first segment information is derived by:

setting encoder parameters of an encoder, wherein the encoder parameters comprise a length K;

setting a sample window duration;

initializing a second set of counters to zero, wherein number of counters in the second set of counters corresponds to the number of segments, and each counter of the second set of counters monitors number of times drift is no longer detected in respective segment of the plurality of segments;

initializing a set of drift flags to indicate no drift, wherein number of drift flags in the set of drift flags corresponds to the number of segments, and each drift flag is associated with a corresponding segment;

generating a segment vector of size K as output of the encoder and initializing the segment vector, wherein the segment vector represents mean value associated with each of the plurality of segments, and the plurality of segments is provided as input to the encoder;

generating an identity matrix of size K by K as output of the encoder and initializing the identity matrix, wherein the identity matrix represents covariances of output from the encoder in association with the plurality of segments; and

storing the first set of counters, the second set of counters, the set of drift flags, the segment vector, and the identity matrix as the first segment information in the drift database.

6 . The method of claim 5 , wherein the computing drift in values between the first segment information and the second segment information comprises:

generating a second segment vector of size K associated with the second segment information;

generating a second identity matrix of size K by K associated with the second segment information; and

computing drift between the segment vector and the identity matrix against the second segment vector and the second identity matrix.

7 . The method of claim 6 ,

wherein, for detection of drift in a segment between the segment vector and the identity matrix against the second segment vector and the second identity matrix:

updating entry associated with the segment in the segment vector with entry associated with the segment in the second segment vector;

updating entry associated with the segment in the identity matrix with entry associated with the segment in the second identity matrix;

setting counter associated with the segment in the first set of counters to zero;

incrementing counter associated with the segment in the second set of counters by one; and

setting drift flag of the segment as having no drift, and

wherein, for detection of no drift in a segment between the segment vector and the identity matrix against the second segment vector and the second identity matrix:

setting counter associated with the segment in the second set of counters to zero;

incrementing counter associated with the segment in the first set of counters by one;

setting drift flag of the segment as short term; and

for the counter associated with the segment in the first set of counters exceeding a duration threshold, setting the drift flag of the segment as long term.

8 . The method of claim 7 , further comprising:

performing segment combination of segments of the plurality of segments,

wherein the performing segment combination of segments of the plurality of segments comprises:

reading the second set of counters from the drift database;

determining counters of the second set of counters associated with each segment and an adjacent segment of the plurality of segments exceed a counter threshold; and

for counters associated with a segment and an adjacent segment exceeding the counter threshold, combining the segment and the adjacent segment into a combined segment.

9 . The method of claim 7 , further comprising:

performing segment split of segments of the plurality of segments,

wherein the performing segment split of segments of the plurality of segments comprises:

reading the set of drift flags front the drift database;

determining if any drift flag of the set of drift flags is designated as short term over a time threshold; and

for drift flag of the set of drift flags having short term designation over the time threshold, splitting segment associated with drift flag into two separate segments.

10 . The method of claim 1 , further comprising:

performing, on detection of drift, at least one of notification provision to operator of the detected drift, recommendation provision on sensor adjustment, or feedback provision in performing automatic control.

11 . A non-transitory computer readable medium, storing instructions for computing and detecting image data drift, the instructions comprising:

retrieving first segment information of a plurality of segments from a drift database;

receiving a number of images from a sensor;

partitioning each of the received images into segments of a predetermined number;

generating second segment information;

computing drift in values between the first segment information and the second segment information; and

detecting drift based on the computed drift in values by combining the computed drift in segments to generate overall drift, and comparing the overall drift against a drift threshold,

wherein the first segment information comprises a first set of counters, and the first set of counters is derived by:

setting number of segments and shapes of segments based on domain knowledge of applications;

partitioning an image into the plurality of segments, wherein number of the plurality of segments is the number of segments; and

initializing the first set of counters to zero, wherein number of counters in the first set of counters corresponds to the number of segments, and each counter of the first set of counters monitors number of times drift is detected in respective segment of the plurality of segments.

12 . The non-transitory computer readable medium of claim 11 , wherein the first segment information comprises a first mean and a first covariance matrix associate with the plurality of segments, and the second segment information comprises a second mean and a second covariance matrix.

13 . The non-transitory computer readable medium of claim 11 , wherein the generating the second segment information comprises:

fitting, for each number less than or equal to the predetermined number, a multidimensional distribution computed over a number of instances of segment to generate the second segment information, wherein the number of instances of segment corresponds to number of the number of images.

14 . The non-transitory computer readable medium of claim 11 , wherein the predetermined number is same as number of segments of the plurality of segments from the drift database.

15 . The non-transitory computer readable medium of claim 11 , wherein the first segment information is derived by:

setting encoder parameters of an encoder, wherein the encoder parameters comprise a length K;

setting a sample window duration;

initializing a second set of counters to zero, wherein number of counters in the second set of counters corresponds to the number of segments, and each counter of the second set of counters monitors number of times drift is no longer detected in respective segment of the plurality of segments;

initializing a set of drift flags to indicate no drift, wherein number of drift flags in the set of drift flags corresponds to the number of segments, and each drift flag is associated with a corresponding segment;

generating a segment vector of size K as output of the encoder and initializing the segment vector, wherein the segment vector represents mean value associated with each of the plurality of segments, and the plurality of segments is provided as input to the encoder;

generating an identity matrix of size K by K as output of the encoder and initializing the identity matrix, wherein the identity matrix represents covariances of output from the encoder in association with the plurality of segments; and

storing the first set of counters, the second set of counters, the set of drift flags, the segment vector, and the identity matrix as the first segment information in the drift database.

16 . The non-transitory computer readable medium of claim 15 , wherein the computing drift in values between the first segment information and the second segment information comprises:

generating a second segment vector of size K associated with the second segment information;

generating a second identity matrix of size K by K associated with the second segment information; and

computing drift between the segment vector and the identity matrix against the second segment vector and the second identity matrix.

17 . The non-transitory computer readable medium of claim 16 ,

wherein, for detection of drift in a segment between the segment vector and the identity matrix against the second segment vector and the second identity matrix:

updating entry associated with the segment in the segment vector with entry associated with the segment in the second segment vector;

updating entry associated with the segment in the identity matrix with entry associated with the segment in the second identity matrix;

setting counter associated with the segment in the first set of counters to zero;

incrementing counter associated with the segment in the second set of counters by one; and

setting drift flag of the segment as having no drift, and

wherein, for detection of no drift in a segment between the segment vector and the identity matrix against the second segment vector and the second identity matrix:

setting counter associated with the segment in the second set of counters to zero;

incrementing counter associated with the segment in the first set of counters by one;

setting drift flag of the segment as short term; and

for the counter associated with the segment in the first set of counters exceeding a duration threshold, setting the drift flag of the segment as long term.

18 . The non-transitory computer readable medium of claim 17 , further comprising:

performing segment combination of segments of the plurality of segments,

wherein the performing segment combination of segments of the plurality of segments comprises:

reading the second set of counters from the drift database;

determining counters of the second set of counters associated with each segment and an adjacent segment of the plurality of segments exceed a counter threshold; and

for counters associated with a segment and an adjacent segment exceeding the counter threshold, combining the segment and the adjacent segment into a combined segment.

19 . The non-transitory computer readable medium of claim 17 , further comprising:

performing segment split of segments of the plurality of segments,

wherein the performing segment split of segments of the plurality of segments comprises:

reading the set of drift flags from the drift database;

determining if any drift flag of the set of drift flags is designated as short term over a time threshold; and

for drift flag of the set of drift flags having short term designation over the time threshold, splitting segment associated with drift flag into two separate segments.

20 . The non-transitory computer readable medium of claim 11 , further comprising:

Performing, on detection of drift, at least one of notification provision to operator of the detected drift, recommendation provision on sensor adjustment, or feedback provision in performing automatic control.

Assignments (1)

Continuity (1)

References Cited (31)