Method and system for providing personalized multimodal objects in real time
The present disclosure is related to the field of digital communication and provides a method and system for providing personalized multimodal objects in real-time. An object predicting system receives a text input from at least one application installed in a user device associated with a user. Thereafter, the object predicting system determines an intent of the user by analyzing the text input, which is then correlated with contextual data to generate a query. Subsequently, the object predicting system performs a unified search in a universal database, based on the query, wherein the universal database comprises multimodal data. Further, a plurality of multimodal objects predicted in response to the unified search are ranked based on at least one of the contextual data and user preferences. Finally, at least one of the predicted plurality of multimodal objected related to the text input are provided to the user based on the ranking.
1 . A method of providing personalized multimodal objects, the method comprising:
obtaining a text input through at least one application installed in an electronic device;
determining an intent of a user of the electronic device based on the text input;
obtaining a query by correlating the determined intent of the user and contextual data;
based on the query, performing a search in a database to generate a prediction of a plurality of multimodal objects based on the determined intent of the user and the contextual data, wherein the database comprises multimodal data related to ideographs, one or more applications configured in the electronic device and one or more internet of things (IoT) devices connected to the electronic device, and wherein the plurality of multimodal objects are predicted using one or more deep neural network (DNN) techniques and a document term matrix (DTM) stored in the database, the DTM comprising the ideographs and mapping of the ideographs with metadata description corresponding to the ideographs;
in response to the search, ranking the plurality of multimodal objects predicted based on at least one of the contextual data and user preferences; and
based on the ranking, displaying multimodal objects related to the text input among the predicted plurality of multimodal objects, the displayed multimodal objects comprising a first object indicating an image or text and a second object indicating a list of items associated with the one or more IoT devices.
2 . The method of claim 1 , wherein the intent of the user is determined using one or more long short-term memory (LSTM) techniques.
3 . The method of claim 1 , wherein the predicted plurality of multimodal objects are related to at least one of ideographs, image content, video content, audio content, documents, vouchers, text content and stored contacts of the user.
4 . The method of claim 1 , further comprising:
initiating a self-learning process based on at least one of selection or rejection of the predicted plurality of multimodal objects by the user, feedback of the user for the predicted plurality of multimodal objects, and indication of user preferences by the user.
5 . The method of claim 1 , wherein the ideographs comprise at least one of emojis, stickers and graphics interchange formats (GIFs).
6 . The method of claim 1 , further comprising:
updating the DTM in real-time, upon detecting one or more new ideographs.
7 . An electronic device for providing personalized multimodal objects, the electronic device comprising:
a memory for storing instructions; and
at least one processor configured to execute the stored instructions to at least:
obtain a text input through at least one application installed in the electronic device;
determine an intent of a user of the electronic device based on the text input;
obtain a query by correlating the determined intent of the user and contextual data;
based on the query, perform a search in a database to generate a prediction of a plurality of multimodal objects based on the determined intent of the user and the contextual data, wherein the database comprises multimodal data related to ideographs, one or more applications configured in the electronic device and one or more internet of things (IoT) devices connected to the electronic device, and wherein the plurality of multimodal objects are predicted using one or more deep neural network (DNN) techniques and a document term matrix (DTM) stored in the database, the DTM comprising the ideographs and mapping of the ideographs with metadata description corresponding to the ideographs;
in response to the search, rank the plurality of multimodal objects predicted based on at least one of the contextual data and user preferences; and
based on the ranking, display multimodal objects related to the text input among the predicted plurality of multimodal objects, the displayed multimodal objects comprising a first object indicating an image or text and a second object indicating a list of items associated with the one or more IoT devices.
8 . The electronic device of claim 7 , wherein the processor is further configured to:
determine the intent of the user using one or more long short-term memory (LSTM) techniques.
9 . The electronic device of claim 7 , wherein:
the predicted plurality of multimodal objects are related to at least one of ideographs, image content, video content, audio content, documents, vouchers, text content and stored contacts of the user, and
the ideographs comprise at least one of emojis, stickers and graphics interchange formats (GIFs).
10 . The electronic device of claim 7 , wherein the at least one processor is further configured to initiate a self-learning process based on at least one of: selection or rejection of the predicted plurality of multimodal objects by the user, feedback of the user for the predicted plurality of multimodal objects, and indication of user preferences by the user.
11 . The electronic device of claim 7 ,
wherein
the at least one processor is further configured to update the DTM in real-time, upon detecting one or more new ideographs.
12 . A non-transitory computer readable medium containing instructions that when executed cause a processor to:
obtain a text input through at least one application installed in an electronic device;
determine an intent of a user of the electronic device based on the text input;
obtain a query by correlating the determined intent of the user and contextual data;
based on the query, perform a search in a database to generate a prediction of a plurality of multimodal objects based on the determined intent of the user and the contextual data, wherein the database comprises multimodal data related to ideographs, one or more applications configured in the electronic device and one or more internet of things (IoT) devices connected to the electronic device, and wherein the plurality of multimodal objects are predicted using one or more deep neural network (DNN) techniques and a document term matrix (DTM) stored in the database, the DTM comprising the ideographs and mapping of the ideographs with metadata description corresponding to the ideographs;
in response to the search, rank the plurality of multimodal objects predicted based on at least one of the contextual data and user preferences; and
based on the ranking, display multimodal objects related to the text input among the predicted plurality of multimodal objects, the displayed multimodal objects comprising a first object indicating an image or text and a second object indicating a list of items associated with the one or more IoT devices.
13 . The non-transitory computer readable medium of claim 12 , wherein the processor is further configured to:
determine the intent of the user using one or more long short-term memory (LSTM) techniques.
14 . The non-transitory computer readable medium of claim 12 , wherein:
the predicted plurality of multimodal objects are related to at least one of ideographs, image content, video content, audio content, documents, vouchers, text content and stored contacts of the user, and
the ideographs comprise at least one of emojis, stickers and graphics interchange formats (GIFs).
15 . The non-transitory computer readable medium of claim 12 , wherein the processor is further configured to initiate a self-learning process based on at least one of, selection or rejection of the predicted plurality of multimodal objects by the user, feedback of the user for the predicted plurality of multimodal objects, and indication of user preferences by the user.