Granted Patent B2

US 12,346,951 · App. 17/970,666 · Granted Jul 1, 2025

Inventors: Jayavardhana Rama Gubbi Lakshminarasimha (Bangalore, IN); Gaurab Bhattacharya (Bangalore, IN); Balamuralidhar Purushothaman (Bangalore, IN); Bagyalakshmi Vasudevan (Chennai, IN); Nikhil Kilari (Bangalore, IN)

Assignee: Tata Consultancy Services Limited

G06Q30/0631G06F16/56G06Q30/0621

Abstract

Claims (50)

1. A processor implemented method for personalized substitute product recommendation, the method comprising:

receiving, by one or more hardware processors, a query image of a product and optionally one or more personalization instructions comprising one or more attributes of the product and values corresponding to the one or more attributes;

generating, by the one or more hardware processors, a plurality of feature vectors of the query image using a plurality of Personalized Attribute Search Networks (PAtSNets), wherein each of the plurality of PAtSNets generate one feature vector corresponding to an attribute of the query image, wherein each of the plurality of PAtSNets comprises a plurality of neural network layers interleaved with a plurality of Attentive Style Embedding (ASE) modules, and wherein each of the plurality of ASE modules includes a multi-scale feature extraction sub-network, and a concurrent axial attention sub-network;

concatenating, by the one or more hardware processors, the plurality of feature vectors into a feature representation vector, wherein the feature vectors corresponding to the one or more attributes in the one or more personalization instructions, if present, are replaced by generic vectors before concatenation; and

recommending, by the one or more hardware processors, one or more images of the product from a database of images based on distance between the feature representation vector and vectors of each image in the database of images.

2. The method of claim 1 , wherein the one or more personalization instructions specify user requirements with respect to the one or more attributes of the product in the query image.

3. The method of claim 1 , wherein the generic vectors are feature vectors representing values corresponding to the one or more attributes in the one or more personalization instructions.

4. The method of claim 1 , wherein the multi-scale feature extraction sub-network extracts fine-grained attribute features of an input image across multiple scales by:

generating a tensor by concatenating responses of—(i) performing a plurality of convolution operations on the input image by considering different kernel sizes for each of the plurality of convolution operations, and (ii) performing a convolution operation followed by a pooling operation on the input image;

passing the tensor through a plurality of convolution layers to obtain a plurality of feature maps in a plurality of spatial dimensions, wherein feature map generated at a convolution layer is down scaled by a factor of two before passing to next convolution layer; and

performing transposed convolution operation on the plurality of feature maps to generate a multi-scale image feature vector.

5. The method of claim 1 , wherein the concurrent axial attention sub-network generates feature representation of the input image by:

accepting the multi-scale image feature vector as input;

generating a height embedding, a width embedding and a plurality of channel embeddings of the input by processing the input across height, width, and plurality of channel dimensions respectively;

computing a height attention mask, a width attention mask, and a channel attention mask by applying a sigmoid activation function on the height embedding, width embedding, and a dense operation followed by the sigmoid activation on the plurality of channel embeddings; and

multiplying the multi-scale image feature vector with each of the height attention mask, the width attention mask, and the channel attention mask, and summing up the resultant products to obtain axial attentive image feature.

6. A system comprising:

a memory storing instructions;

one or more communication interfaces; and

one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:

receive a query image of a product and optionally one or more personalization instructions comprising one or more attributes of the product and values corresponding to the one or more attributes;

generate a plurality of feature vectors of the query image using a plurality of Personalized Attribute Search Networks (PAtSNets), wherein each of the plurality of PAtSNets generate one feature vector corresponding to an attribute of the query image, wherein each of the plurality of PAtSNets comprises a plurality of neural network layers interleaved with a plurality of Attentive Style Embedding (ASE) modules, and wherein each of the plurality of ASE modules includes a multi-scale feature extraction sub-network, and a concurrent axial attention sub-network;

concatenate the plurality of feature vectors into a feature representation vector, wherein the feature vectors corresponding to the one or more attributes in the one or more personalization instructions, if present, are replaced by generic vectors corresponding to the one or more attributes in the one or more personalization instructions before concatenation; and

recommend one or more images of the product from a database of images based on distance between the feature representation vector and vectors of each image in the database of images.

7. The system of claim 6 , wherein the one or more personalization instructions specify user requirements with respect to the one or more attributes of the product in the query image.

8. The system of claim 6 , wherein the generic vectors corresponding to the one or more attributes in the one or more personalization instructions are feature vectors representing values corresponding to the one or more attributes in the one or more personalization instructions.

9. The system of claim 6 , wherein the multi-scale feature extraction sub-network extracts fine-grained attribute features of an input image across multiple scales by:

generating a tensor by concatenating responses of—(i) performing a plurality of convolution operations on the input image by considering different kernel sizes for each of the plurality of convolution operations, and (ii) performing a convolution operation followed by a pooling operation on the input image;

passing the tensor through a plurality of convolution layers to obtain a plurality of feature maps in a plurality of spatial dimensions, wherein feature map generated at a convolution layer is down scaled by a factor of two before passing to next convolution layer; and

performing transposed convolution operation on the plurality of feature maps to generate a multi-scale image feature vector.

10. The system of claim 6 , wherein the concurrent axial attention sub-network generates feature representation of the input image by:

accepting the multi-scale image feature vector as input;

generating a height embedding, a width embedding and a plurality of channel embeddings of the input by processing the input across height, width, and plurality of channel dimensions respectively;

computing a height attention mask, a width attention mask, and a channel attention mask by applying a sigmoid activation function on the height embedding, width embedding, and a dense operation followed by the sigmoid activation on the plurality of channel embeddings; and

multiplying the multi-scale image feature vector with each of the height attention mask, the width attention mask, and the channel attention mask, and summing up the resultant products to obtain axial attentive image feature.

11. One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

receiving a query image of a product and optionally one or more personalization instructions comprising one or more attributes of the product and values corresponding to the one or more attributes;

generating a plurality of feature vectors of the query image using a plurality of Personalized Attribute Search Networks (PAtSNets), wherein each of the plurality of PAtSNets generate one feature vector corresponding to an attribute of the query image, wherein each of the plurality of PAtSNets comprises a plurality of neural network layers interleaved with a plurality of Attentive Style Embedding (ASE) modules, and wherein each of the plurality of ASE modules includes a multi-scale feature extraction sub-network, and a concurrent axial attention sub-network;

concatenating the plurality of feature vectors into a feature representation vector, wherein the feature vectors corresponding to the one or more attributes in the one or more personalization instructions, if present, are replaced by generic vectors before concatenation; and

recommending one or more images of the product from a database of images based on distance between the feature representation vector and vectors of each image in the database of images.

12. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the one or more personalization instructions specify user requirements with respect to the one or more attributes of the product in the query image, and wherein the generic vectors are feature vectors representing values corresponding to the one or more attributes in the one or more personalization instructions.

13. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the multi-scale feature extraction sub-network extracts fine-grained attribute features of an input image across multiple scales by:

generating a tensor by concatenating responses of—(i) performing a plurality of convolution operations on the input image by considering different kernel sizes for each of the plurality of convolution operations, and (ii) performing a convolution operation followed by a pooling operation on the input image;

passing the tensor through a plurality of convolution layers to obtain a plurality of feature maps in a plurality of spatial dimensions, wherein feature map generated at a convolution layer is down scaled by a factor of two before passing to next convolution layer; and

performing transposed convolution operation on the plurality of feature maps to generate a multi-scale image feature vector.

14. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the concurrent axial attention sub-network generates feature representation of the input image by:

accepting the multi-scale image feature vector as input;

generating a height embedding, a width embedding and a plurality of channel embeddings of the input by processing the input across height, width, and plurality of channel dimensions respectively;

computing a height attention mask, a width attention mask, and a channel attention mask by applying a sigmoid activation function on the height embedding, width embedding, and a dense operation followed by the sigmoid activation on the plurality of channel embeddings; and

multiplying the multi-scale image feature vector with each of the height attention mask, the width attention mask, and the channel attention mask, and summing up the resultant products to obtain axial attentive image feature.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (11)