Method and system for searching for media message using keyword extracted from media file

1 . A non-transitory computer-readable recording medium storing instructions that, when executed by a processor, cause the processor to computer-implement a media message search method in conjunction with a computer apparatus, the media message search method comprising:

generating a specific session of an instant messaging service or participating in the specific session;

analyzing a media message to be sent through the specific session or a media message received through the specific session;

extracting a keyword of media included in the analyzed media message;

storing the extracted keyword in a local database of the computer apparatus in association with an identifier of the analyzed media message that identifies the analyzed media message from any other media message sent through or received through the specific session;

providing a search interface for an instant message sent and received through the specific session;

retrieving an identifier from the local database based on a keyword input through the search interface having been stored in the local database in association with the identifier; and

providing, for display, a media message having the retrieved identifier,

wherein the extracting of the keyword comprises:

when the media includes a video, extracting a first category of thumbnail images of the video using a vision-based first machine learning model that is trained to classify a category of an input image,

extracting a second category of audio of the video using a sound analysis-based second machine learning model that is trained to classify a category of input audio, and

determining at least one of the first category and the second category as the keyword.

2 . The non-transitory computer-readable recording medium of claim 1 , wherein the providing, for display, the media message comprises:

displaying a search result list that includes a text message including the keyword input through the search interface and the media message having the retrieved identifier, and

based on a search result selected from the search result list, displaying a sequence of instant messages sent and received through the specific session and that includes an instant message corresponding to the selected search result in chronological order in the sequence of instant messages.

3 . The non-transitory computer-readable recording medium of claim 1 , wherein the providing, for display, the media message comprises:

displaying a sequence of instant messages sent and received through the specific session and that includes an instant message corresponding to the media message having the retrieved identifier.

4 . The non-transitory computer-readable recording medium of claim 1 , wherein

the providing, for display, the media message comprises:

displaying an instant message corresponding to the keyword input through the search interface in a sequence of instant messages sent and received through the specific session, and

the instant message displayed includes a text message including the keyword input through the search interface and the media message having the retrieved identifier.

5 . The non-transitory computer-readable recording medium of claim 1 , wherein

the analyzing analyzes the media message received through the specific session, and the media message received through the specific session includes a keyword, and

the extracting of the keyword comprises:

extracting the keyword from the received media message.

6 . The non-transitory computer-readable recording medium of claim 1 , wherein the extracting of the keyword comprises:

when the media includes an image, extracting a category of the image as the keyword, using a vision based-machine learning model that is trained to classify a category of an input image.

7 . The non-transitory computer-readable recording medium of claim 1 , wherein the extracting of the keyword comprises:

when the media includes audio, extracting a category of the audio as the keyword, using a sound analysis-based machine learning model that is trained to classify a category of input audio.

8 . A media message search method of a computer apparatus comprising at least one processor, the media message search method comprising:

generating, by the at least one processor, a specific session of an instant messaging service or participating in the specific session;

analyzing, by the at least one processor, a media message to be sent through the specific session or a media message received through the specific session;

extracting a keyword of media included in the analyzed media message;

storing, by the at least one processor, the extracted keyword in a local database of the computer apparatus in association with an identifier of the analyzed media message that identifies the analyzed media message from any other media message sent through or received through the specific session;

providing, by the at least one processor, a search interface for an instant message sent and received through the specific session;

retrieving an identifier from the local database based on a keyword input through the search interface having been stored in the local database in association with the identifier; and

providing, for display, by the at least one processor, a media message having the retrieved identifier,

wherein the extracting of the keyword comprises:

when the media includes a video, extracting a first category of thumbnail images of the video using a vision-based first machine learning model that is trained to classify a category of an input image,

extracting a second category of audio of the video using a sound analysis-based second machine learning model that is trained to classify a category of input audio, and

determining at least one of the first category and the second category as the keyword.

9 . The media message search method of claim 8 , wherein the providing, for display, the media message comprises:

displaying a search result list that includes a text message including the keyword input through the search interface and the media message having the retrieved identifier and

based on a search result selected from the search result list, displaying a sequence of instant messages sent and received through the specific session and that includes an instant message corresponding to the selected search result in chronological order in the sequence of instant messages.

10 . The media message search method of claim 8 , wherein the providing, for display, the media message comprises:

displaying a sequence of instant messages sent and received through the specific session and that includes an instant message corresponding to the media message having the retrieved identifier.

11 . The media message search method of claim 8 , wherein the providing, for display, the media message comprises:

displaying an instant message corresponding to the keyword input through the search interface in a sequence of instant messages sent and received through the specific session, and

the instant message displayed includes a text message including the keyword input through the search interface and the media message having the retrieved identifier.

12 . The media message search method of claim 8 , wherein

the analyzing analyzes the media message received through the specific session, and the media message received through the specific session includes a keyword, and

the extracting of the keyword comprises:

extracting the keyword from the received media message.

13 . The media message search method of claim 8 , wherein the extracting of the keyword comprises:

when the media includes an image, extracting a category of the image as the keyword, using a vision based-machine learning model that is trained to classify a category of an input image.

14 . The media message search method of claim 8 , wherein the extracting of the keyword comprises:

when the media includes audio, extracting a category of the audio as the keyword, using a sound analysis-based machine learning model that is trained to classify a category of input audio.

15 . A non-transitory computer-readable recording medium storing instructions that, when executed by a processor, cause the processor to computer-implement a media message search method in conjunction with a computer apparatus, the media message search method comprising:

generating a specific session of an instant messaging service or participating in the specific session;

analyzing a media message to be sent through the specific session or a media message received through the specific session;

extracting a keyword of media included in the analyzed media message;

storing the extracted keyword in a local database of the computer apparatus in association with an identifier of the analyzed media message;

providing a search interface for an instant message sent and received through the specific session;

retrieving an identifier from the local database based on a keyword input through the search interface having been stored in the local database in association with the identifier; and

providing, for display, a media message having the retrieved identifier,

wherein the extracting of the keyword comprises:

when the media includes a video, extracting a first category of thumbnail images of the video using a vision-based first machine learning model that is trained to classify a category of an input image,

extracting a second category of audio of the video using a sound analysis-based second machine learning model that is trained to classify a category of input audio, and

determining at least one of the first category and the second category as the keyword.