Video processing method using transfer learning and pre-training server

1 . A video processing method performed by a computing device, the method comprising the steps of:

collecting video from an external device;

generating preprocessed data by extracting two-dimensional or three-dimensional skeleton information from the video;

pre-training a first artificial intelligence model including N transformer blocks from the preprocessed data by applying an attention from a body of an object to a plurality of joints, an attention from each of the plurality of joints to the body, and an attention between persons; and

learning, when parameters determined as a result of the pre-training of the first artificial intelligence model are transferred, a method of recognizing an action from the video received from the external device, using a second artificial intelligence model including the N transformer blocks on the basis of the parameters, wherein

N is a natural number equal to or larger than 2.

2 . The method according to claim 1 , wherein the step of pre-training a first artificial intelligence model includes the steps of:

applying a slice of a positional embedding tensor corresponding to each frame of an output of a previous block, and performing first layer normalization, by an n-th transformer block among the N transformer blocks;

applying a spatial multi-head attention (MHA) to a result of the first layer normalization, by the n-th transformer block; and

deriving a first result by adding a result of applying the slice of a positional embedding tensor corresponding to each frame of an output of a previous block to a result of applying the spatial MHA, by the n-th transformer block, wherein

n is a natural number between 2 and N.

3 . The method according to claim 2 , further comprising the steps of:

applying a matrix with a changed dimension of the positional embedding tensor to a pose sequence matrix in which the first result corresponding to each frame is stacked, and performing second layer normalization;

applying a temporal MHA to a result of the second layer normalization; and

deriving a second result by adding a result of applying the matrix with a changed dimension of the positional embedding tensor to a pose sequence matrix to a result of applying the temporal MHA.

4 . The method according to claim 3 , further comprising the steps of:

performing third layer normalization on the second result;

applying a multi-layer perceptron (MLP) to a result of the third layer normalized; and

deriving a third result by adding the second result to a result of applying the MLP.

5 . The method according to claim 1 , further comprising the step of deriving a motion sequence representation for an input motion sequence according to the video, by an N-th block among the N transformer blocks.

6 . A pre-training server comprising:

a processor;

a memory; and

a computer program loaded on the memory and executed by the processor, wherein

the computer program includes:

an instruction for collecting video from an external device;

an instruction for generating preprocessed data by extracting two-dimensional or three-dimensional skeleton information from the video;

an instruction for pre-training a first artificial intelligence model including N transformer blocks from the preprocessed data by applying an attention from a body of an object to a plurality of joints, an attention from each of the plurality of joints to the body, and an attention between persons; and

an instruction for learning, when parameters determined as a result of the pre-training of the first artificial intelligence model are transferred, a method of recognizing an action from the video received from the external device, using a second artificial intelligence model including the N transformer blocks on the basis of the parameters, wherein

N is a natural number equal to or larger than 2.