Systems and methods for harmonized product classification
This disclosure provides systems, methods, and devices for automatic product classification using deep learning and generative artificial intelligence (AI) models for a harmonized system (HS) product classification. A method includes generating embeddings based on an input dataset. The method includes applying a deep learning model to the embeddings to produce a prediction set including classifications corresponding to the embeddings. The method includes converting the classifications to a first set of similarity metrics. The method includes determining a second set of similarity metrics based on the embeddings using a semantic similarity model. The method includes generating a third set of similarity metrics based on an output of the semantic similarity model. The method includes outputting a ranked set of predictions corresponding to the input dataset based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics.
1 . A method comprising:
generating, by one or more processors, a set of embeddings based on an input dataset;
applying, by the one or more processors, a deep learning model, a generative artificial intelligence model, or both, to the set of embeddings to produce a prediction set, wherein the prediction set comprises one or more classifications corresponding to the set of embeddings;
converting, by the one or more processors, the one or more classifications to a first set of similarity metrics;
determining, by the one or more processors, a second set of similarity metrics based on the set of embeddings using a semantic similarity model;
generating, by the one or more processors, a third set of similarity metrics based at least in part on an output of the semantic similarity model; and
outputting, by the one or more processors, a ranked set of predictions corresponding to the input dataset based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics.
2 . The method of claim 1 , wherein the generative artificial intelligence model comprises a large language model, the method further comprising generating a prompt based on the output of the semantic similarity model, wherein the third set of semantic similarity metrics is generated based on the prompt.
3 . The method of claim 1 , wherein the deep learning model comprises a convolutional neural network.
4 . The method of claim 1 , further comprising determining the ranked set of predictions based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics using a voting algorithm.
5 . The method of claim 4 , wherein each of the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics comprise at least one prediction, a similarity score associated with each prediction, and information providing explainability with respect to the similarity score and corresponding prediction.
6 . The method of claim 5 , wherein the at least one prediction comprises a classification corresponding to the input dataset.
7 . The method of claim 1 , wherein the first set of similarity metrics, the second set of similarity metrics, the third set of similarity metrics, or a combination thereof, comprises multiple classifications for at least a portion of the input dataset.
8 . An apparatus, comprising:
one or more memories storing processor-executable code; and
one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
generate a set of embeddings based on an input dataset;
apply a deep learning model, a generative artificial intelligence model, or both, to the set of embeddings to produce a prediction set, wherein the prediction set comprises one or more classifications corresponding to the set of embeddings;
convert the one or more classifications to a first set of similarity metrics;
determine a second set of similarity metrics based on the set of embeddings using a semantic similarity model;
generate a third set of similarity metrics based at least in part on an output of the semantic similarity model; and
output a ranked set of predictions corresponding to the input dataset based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics.
9 . The apparatus of claim 8 , wherein the generative artificial intelligence model comprises a large language model, the one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
comprise generating a prompt based on the output of the semantic similarity model, wherein the third set of semantic similarity metrics is generated based on the prompt.
10 . The apparatus of claim 8 , wherein the deep learning model comprises a convolutional neural network.
11 . The apparatus of claim 8 , the one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
determine the ranked set of predictions based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics using a voting algorithm.
12 . The apparatus of claim 8 , wherein each of the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics comprise at least one prediction, a similarity score associated with each prediction, and information providing explainability with respect to the similarity score and corresponding prediction.
13 . The apparatus of claim 8 , wherein the at least one prediction comprises a classification corresponding to the input dataset.
14 . The apparatus of claim 8 , wherein the first set of similarity metrics, the second set of similarity metrics, the third set of similarity metrics, or a combination thereof, comprises multiple classifications for at least a portion of the input dataset.
15 . A non-transitory computer-readable medium having instructions that, when executed by one or more processors, causes the one or more processors to:
generate a set of embeddings based on an input dataset;
apply a deep learning model, a generative artificial intelligence model, or both, to the set of embeddings to produce a prediction set, wherein the prediction set comprises one or more classifications corresponding to the set of embeddings;
convert the one or more classifications to a first set of similarity metrics;
determine a second set of similarity metrics based on the set of embeddings using a semantic similarity model;
generate a third set of similarity metrics based at least in part on an output of the semantic similarity model; and
output a ranked set of predictions corresponding to the input dataset based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics.
16 . The non-transitory computer-readable medium of claim 15 , wherein the generative artificial intelligence model comprises a large language model, wherein the instructions are further executable by the one or more processors to:
comprise generating a prompt based on the output of the semantic similarity model, wherein the third set of semantic similarity metrics is generated based on the prompt.
17 . The non-transitory computer-readable medium of claim 15 , wherein the deep learning model comprises a convolutional neural network.
18 . The non-transitory computer-readable medium of claim 15 , wherein the instructions are further executable by the one or more processors to:
determine the ranked set of predictions based on the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics using a voting algorithm.
19 . The non-transitory computer-readable medium of claim 15 , wherein each of the first set of similarity metrics, the second set of similarity metrics, and the third set of similarity metrics comprise at least one prediction, a similarity score associated with each prediction, and information providing explainability with respect to the similarity score and corresponding prediction.
20 . The non-transitory computer-readable medium of claim 15 , wherein the at least one prediction comprises a classification corresponding to the input dataset.