Method and apparatus for re-training model by recognizing uncertainty in online user behavior detection

1 . A method for training a model related to user behavior detection, the method comprising:

detecting one or more behavior instances having a type of user behavior and a time interval of the user behavior, based on user behavior information recognized in frames;

calculating at least one of a first uncertainty value for the type of user behavior or a second uncertainty value for the time interval of the user behavior, for each detected behavior instance;

generating a first data newly labeling the type of user behavior or a second data newly labeling the time interval of the user behavior, based on a comparison between at least one of the first uncertainty value or the second uncertainty value and a pre-configured threshold; and

training a model that recognizes the user behavior information on a frame-by-frame basis, by refining the first data or the second data.

2 . The method of claim 1 ,

wherein the first data is generated and refined when the first uncertainty value is greater than a first threshold value pre-configured for the type of user behavior.

3 . The method of claim 2 ,

wherein the second uncertainty value is calculated when the first uncertainty value is smaller than or equal to the first threshold value pre-configured for the type of user behavior.

4 . The method of claim 3 ,

wherein the second data is generated and refined when the calculated second uncertainty value is greater than a second threshold value pre-configured for the time interval of the user behavior.

5 . The method of claim 4 , the method further comprising:

generating a third data newly labeled using information on a behavior instance, when the first uncertainty value is less than or equal to the first pre-configured threshold value and the second uncertainty value is less than or equal to the second pre-configured threshold value; and

training a model that extracts vector-type image features from real-time images, by refining the third data.

6 . The method of claim 5 ,

wherein output information of the model that extracts the image features is configured to be input to the model that recognizes the user behavior information on a frame-by-frame basis.

7 . The method of claim 5 ,

wherein only one of the weights of a model that extracts the image features and the weights of a model that recognizes the user behavior information on a frame-by-frame basis is trained, and the other one is set to freeze.

8 . The method of claim 1 ,

wherein a refinement of the first data is performed based on a ratio of the interval in which the first uncertainty value is greater than a certain value in an interval in which a situation that interferes with recognition of user behavior occurs, and

wherein a refinement of the second data is performed based on a ratio of the interval in which the second uncertainty value is greater than a certain value in an interval in which a situation that interferes with recognition of user behavior occurs.

9 . The method of claim 1 ,

wherein the one or more behavior instances are detected when the user behavior information remains greater than or equal to a threshold for the same behavior.

10 . An apparatus for training a model related to user behavior detection, the apparatus comprising:

a processor and a memory,

wherein the processor is configured to:

detect one or more behavior instances having a type of user behavior and a time interval of the user behavior, based on user behavior information recognized in frames;

calculate at least one of a first uncertainty value for the type of user behavior or a second uncertainty value for the time interval of the user behavior, for each detected behavior instance;

generate a first data newly labeling the type of user behavior or a second data newly labeling the time interval of the user behavior, based on a comparison between at least one of the first uncertainty value or the second uncertainty value and a pre-configured threshold; and

train a model that recognizes the user behavior information on a frame-by-frame basis, by refining the first data or the second data.

11 . The apparatus of claim 10 ,

wherein the first data is generated and refined when the first uncertainty value is greater than a first threshold value pre-configured for the type of user behavior.

12 . The apparatus of claim 11 ,

wherein the second uncertainty value is calculated when the first uncertainty value is smaller than or equal to the first threshold value pre-configured for the type of user behavior.

13 . The apparatus of claim 12 ,

wherein the second data is generated and refined when the calculated second uncertainty value is greater than a second threshold value pre-configured for the time interval of the user behavior.

14 . The apparatus of claim 13 , wherein the processor is configured to:

generate a third data newly labeled using information on a behavior instance, when the first uncertainty value is less than or equal to the first pre-configured threshold value and the second uncertainty value is less than or equal to the second pre-configured threshold value; and

train a model that extracts vector-type image features from real-time images, by refining the third data.

15 . The apparatus of claim 14 ,

wherein output information of the model that extracts the image features is configured to be input to the model that recognizes the user behavior information on a frame-by-frame basis.

16 . The apparatus of claim 14 ,

wherein only one of the weights of a model that extracts the image features and the weights of a model that recognizes the user behavior information on a frame-by-frame basis is trained, and the other one is set to freeze.

17 . The apparatus of claim 10 ,

wherein a refinement of the first data is performed based on a ratio of the interval in which the first uncertainty value is greater than a certain value in an interval in which a situation that interferes with recognition of user behavior occurs, and

wherein a refinement of the second data is performed based on a ratio of the interval in which the second uncertainty value is greater than a certain value in an interval in which a situation that interferes with recognition of user behavior occurs.

18 . The apparatus of claim 10 ,

wherein the one or more behavior instances are detected when the user behavior information remains greater than or equal to a threshold for the same behavior.

19 . One or more non-transitory computer readable medium storing one or more instructions,

wherein the one or more instructions are executed by one or more processors and control an apparatus for training a model related to user behavior detection to:

detect one or more behavior instances having a type of user behavior and a time interval of the user behavior, based on user behavior information recognized in frames;

calculate at least one of a first uncertainty value for the type of user behavior or a second uncertainty value for the time interval of the user behavior, for each detected behavior instance;

generate a first data newly labeling the type of user behavior or a second data newly labeling the time interval of the user behavior, based on a comparison between at least one of the first uncertainty value or the second uncertainty value and a pre-configured threshold; and

train a model that recognizes the user behavior information on a frame-by-frame basis, by refining the first data or the second data.

20 . The computer readable medium of claim 19 ,

wherein the second uncertainty value is calculated when the first uncertainty value is smaller than or equal to the first threshold value pre-configured for the type of user behavior.