Video retrieval method and apparatus using vectorizing segmented videos

1 . A video retrieval method performed by a computing device, the video retrieval method comprising:

identifying, by a machine learning enabled key frame detecting module having one or more encoders, key frame information for one or more video data based on one or more encoded vectors respectively generated from one or more unit video data included in the one or more video data;

segmenting the one or more video data into one or more target retrieval video data based on the identified key frame information; and

generating, by a machine learning enabled retrieval vector generating module having one or more retrieval video encoding modules, one or more retrieval video vectors respectively representing the one or more target retrieval video data, wherein the generating further comprising:

for each target retrieval data, generating, by the one or more retrieval video encoding modules, two or more different domain-based retrieval video embedding tokens corresponding to the target retrieval video data, wherein the two or more different domain-based retrieval video embedding tokens are generated based on two or more data domains included in the target retrieval video data, and

generating the one or more retrieval video vectors from the two or more different domain-based retrieval video embedding tokens of the one or more target retrieval video data, comprising:

generating a time embedding token for a position,

combining the two or more different domain-based retrieval video embedding tokens and the time embedding token to generate a combined token, and

generating a retrieval video vector in the one or more retrieval vectors by applying a retrieval token encoder to the combined token.

2 . The video retrieval method of claim 1 , wherein the one or more retrieval video encoding modules generates one or more retrieval video embedding tokens corresponding to one or more retrieval video data, respectively, for the one or more segmented target retrieval video data.

3 . The video retrieval method of claim 2 , wherein the one or more retrieval video encoding modules generate two or more retrieval video embedding tokens based on a same data domain included in the one or more target retrieval video data.

4 . The video retrieval method of claim 2 , wherein the generating of the retrieval video vector includes dividing each of the one or more segmented target retrieval video data into a plurality of unit retrieval video data.

5 . The video retrieval method of claim 4 , wherein the generating of the retrieval video vector includes generating a temporal embedding token based on time information included in each of the plurality of unit retrieval video data.

6 . The video retrieval method of claim 1 , further comprising:

storing one or more retrieval video vectors corresponding to the retrieval video data in a video retrieval index database.

7 . A non-transitory computer-readable storage medium storing a computer program, in which when the computer program is executed in one or more processors, the computer program causes the one or more processors to perform operations for performing a video retrieval method, the video retrieval method comprising:

identifying, by a machine learning enabled key frame detecting module having one or more encoders, key frame information for one or more video data based on one or more encoded vectors respectively generated from one or more unit video data included in the one or more video data;

segmenting the one or more video data into one or more target retrieval video data based on the identified key frame information; and

generating, by a machine learning enabled retrieval vector generating module having one or more retrieval video encoding modules, one or more retrieval video vectors respectively representing the one or more target retrieval video data, wherein the generating further comprising:

for each target retrieval data, generating, by the one or more retrieval video encoding modules, two or more different domain-based retrieval video embedding tokens corresponding to the target retrieval video data, wherein the two or more different domain-based retrieval video embedding tokens are generated based on two or more data domains included in the target retrieval video data, and

generating the one or more retrieval video vectors from the two or more different domain-based retrieval video embedding tokens of the one or more target retrieval video data, comprising:

generating a time embedding token for a position,

combining the two or more different domain-based retrieval video embedding tokens and the time embedding token to generate a combined token, and

generating a retrieval video vector in the one or more retrieval video vectors by applying a retrieval token encoder to the combined token.

8 . A computing device performing a video retrieval method, the computing device comprising:

a processor including at least one core; and

a memory including program codes executable in the processor,

wherein the processor:

identifies, by a machine learning enabled key frame detecting module having one or more encoders, key frame information for one or more video data based on one or more encoded vectors respectively generated from one or more unit video data included in the one or more video data,

segments the one or more video data into one or more target retrieval video data based on the identified key frame information, and

generates by a machine learning enabled retrieval vector generating module having one or more retrieval video encoding modules, one or more retrieval video vectors respectively representing the one or more target retrieval video data, wherein to generate the one or more retrieval video vectors, the processor:

for each target retrieval data, generate, by the one or more retrieval video encoding modules, two or more different domain-based retrieval video embedding tokens corresponding to the target retrieval video data, wherein the two or more different domain-based retrieval video embedding tokens are generated based on two or more data domains included in the target retrieval video data, and

generate the one or more retrieval vectors from the two or more different domain-based retrieval video embedding tokens of the one or more target retrieval video data, wherein the processor further:

generates a time embedding token for a position,

combines the two or more different domain-based retrieval video embedding tokens and the time embedding token to generate a combined token, and

generates a retrieval video vector in the one or more retrieval video vectors by applying a retrieval token encoder to the combined token.