Method and system for multimodal recommendations

1. A method for generating a product recommendation in a retail system, the method comprising:

collecting, by the retail system, a dataset containing a plurality of entities and attributes for the entities, wherein a first entity and a second entity are grouped into a tuple, by performing steps comprising:

generating, using a first neural network, a vector representation of a first input for a first entity in the plurality of entities;

generating, using a second neural network, a vector representation of a second input of the first entity in the plurality of entities;

combining the vector representation of the first input and the vector representation of the second input into a first entity vector representation; and

combining the first entity vector representation with a second entity vector representation into a combined entity vector;

storing, by the retail system, the combined entity vector in a knowledge graph;

learning, by the retail system, a representation of the plurality of entities, attributes and relationships stored in the knowledge graph;

performing, by the retail system, zero-shot learning for a new entity and attributes for the new entity;

storing, by the retail system, the new entity and attributes for the new entity in the knowledge graph;

generating, by the retail system, a first recommendation for a user based on the knowledge graph;

receiving, by the retail system, data from the user;

inferring, by the retail system, a new rule between the first entity and the second entity in the plurality of entities; and

generating, by the retail system, a second recommendation for the user based on the knowledge graph, including the new rule and the data from the user.

2. The method according to claim 1 , wherein performing zero-shot learning further comprises:

generating relationships between the new entity and the plurality of entities.

3. The method according to claim 1 , wherein the entities include at least a user and a product.

4. The method according to claim 1 , wherein the attributes comprise multimodal data.

5. The method according to claim 4 , wherein the multimodal data includes at least one of text, an image, and an audio clip.

6. The method according to claim 1 , wherein generating a recommendation for a user further comprises:

automatically generating the recommendation for the user based on the knowledge graph.

7. The method according to claim 1 , further comprising learning a second representation of the plurality of entities, attributes and relationships stored in the knowledge graph which includes the new entity.

8. The method according to claim 1 further comprising storing a customer profile for the user.

9. The method according to claim 1 wherein generating a recommendation for a user based on the knowledge graph further comprises analyzing a shopping basket associated with the user.

10. The method of claim 1 , wherein providing the recommendation to the user further comprises transmitting the recommendation to a user device.

11. A recommendation system comprising one or more processors which, alone or in combination, are configured to provide for performance of the following steps:

collecting, by the retail system, a dataset containing a plurality of entities and attributes for the entities, wherein a first entity and a second entity are grouped into a tuple, by performing steps comprising:

generating, using a first neural network, a vector representation of a first input for a first entity in the plurality of entities;

generating, using a second neural network, a vector representation of a second input of the first entity in the plurality of entities;

combining the vector representation of the first input and the vector representation of the second input into a first entity vector representation; and

combining the first entity vector representation with a second entity vector representation into a combined entity vector;

storing, by the retail system, the combined entity vector in a knowledge graph;

learning, by the retail system, a representation of the plurality of entities, attributes and relationships stored in the knowledge graph;

performing, by the retail system, zero-shot learning for a new entity and attributes for the new entity;

storing, by the retail system, the new entity and attributes for the new entity in the knowledge graph;

generating, by the retail system, a first recommendation for a user based on the knowledge graph;

receiving, by the retail system, data from the user;

inferring, by the retail system, a new rule between the first entity and the second entity in the plurality of entities; and

generating, by the retail system, a second recommendation for the user based on the knowledge graph, including the new rule and the data from the user.

12. The recommendation system according to claim 11 further configured to generate relationships between the new entity and the plurality of entities.

13. The recommendation system according to claim 11 , wherein the entities include at least a user and a product.

14. The recommendation system according to claim 11 , wherein generating a recommendation for a user further comprises:

automatically generating the recommendation for the user based on the knowledge graph.

15. The recommendation system according to claim 11 , wherein generating a recommendation for a user based on the knowledge graph further comprises analyzing a shopping basket associated with the user.