Hierarchical deep convolutional neural network for image classification

1. A system comprising:

a memory that stores instructions; and

one or more processors configured by the instructions to perform operations comprising:

accessing a dataset comprising categorized data having a plurality of fine categories;

dividing the dataset into a training set and a value set;

training a first convolutional neural network (CNN) model using the training set;

generating a confusion matrix of the first CNN model using the value set;

applying a clustering algorithm to the confusion matrix to determine, for each fine category, an associated coarse category of a plurality of coarse categories, each coarse category being associated with more than one fine category;

training a base CNN to distinguish between the coarse categories;

training a single fine CNN for each coarse category, the single fine CNN for said coarse category to distinguish between the fine categories associated with said coarse category;

receiving a request to classify data;

using the base CNN, determining a coarse category of the data;

using the single fine CNN for the determined coarse category, determining a single fine category of the data; and

responsive to the request, transmitting the single fine category of the data.

2. The system of claim 1 , wherein the clustering algorithm comprises an affinity propagation algorithm.

3. The system of claim 2 , wherein the operations further comprise:

obtaining low-dimensional feature representations for the fine categories using a Laplacian eigenmap.

4. The system of claim 2 , wherein the training of the fine CNN for each coarse category comprises:

training a second CNN model using the training set;

generating the fine CNN for each coarse category from the second CNN; and

training the fine CNN for each coarse category using a subset of the training set, the subset excluding data having fine categories not associated with said coarse category.

5. The system of claim 1 , wherein the operations further comprise:

combining the base CNN with each of the fine CNNs to form a hierarchically deep CNN (HD-CNN); and

fine-tuning the HD-CNN.

6. The system of claim 5 , wherein the fine-tuning of the HD-CNN comprises:

beginning the fine-tuning with a learning factor;

training the HD-CNN by iterating over a series of training batches using the learning factor;

after each iteration, comparing a training error for the training batch to a threshold;

determining, based on the comparison, that the training error for the training batch is below the threshold; and

in response to the determination that the training error for the training batch is below the threshold, modifying the learning factor.

7. The system of claim 5 , wherein the fine-tuning of the HD-CNN comprises:

applying a temporal sparsity penalty term in evaluating each of the fine CNNs.

8. The system of claim 1 , wherein the dataset comprising the categorized data comprises categorized images.

9. A method comprising:

accessing, by one or more processors, a dataset comprising categorized data having a plurality of fine categories;

dividing, by the one or more processors, the dataset into a training set and a value set;

training, by the one or more processors, a first CNN model using the training set;

generating, by the one or more processors, a confusion matrix of the first CNN model using the value set;

applying, by the one or more processors, a clustering algorithm to the confusion matrix to determine, for each fine category, an associated coarse category of a plurality of coarse categories, each coarse category being associated with more than one fine category;

training, by the one or more processors, a base convolutional neural network (CNN) to distinguish between the coarse categories, the base CNN implemented by a processor of a machine;

training, by the one or more processors, a single fine CNN for each coarse category, the single fine CNN for said coarse category to distinguish between the fine categories associated with said coarse category;

receiving, by the one or more processors, a request to classify data;

using the base CNN, determining, by the one or more processors, a coarse category of the data;

using the single fine CNN for the determined coarse category, determining, by the one or more processors, a single fine category of the data; and

responsive to the request, transmitting, by the one or more processors, the single fine category of the data.

10. The method of claim 9 , wherein the clustering algorithm comprises an affinity propagation algorithm.

11. The method of claim 10 , further comprising obtaining low-dimensional feature representations for the fine categories using a Laplacian eigenmap.

12. The method of claim 10 , wherein the training of the fine CNN for each coarse category comprises:

training a second CNN model using the training set;

generating the fine CNN for each coarse category from the second CNN; and

training the fine CNN for each coarse category using a subset of the training set, the subset excluding data having fine categories not associated with said coarse category.

13. The method of claim 9 , further comprising:

combining the base CNN with each of the fine CNNs to form a hierarchically deep CNN (HD-CNN); and

fine-tuning the HD-CNN.

14. The method of claim 13 , wherein the fine-tuning of the HD-CNN includes:

beginning the fine-tuning with a learning factor;

training the HD-CNN by iterating over a series of training batches using the learning factor:

after each iteration, comparing a training error for the training batch to a threshold;

determining, based on the comparison, that the training error for the training batch is below the threshold; and

in response to the determination that the training error for the training batch is below the threshold, modifying the learning factor.

15. The method of claim 13 , wherein the fine-tuning of the HD-CNN includes:

applying a temporal sparsity penalty term in evaluating each of the fine CNNs.

16. The system of claim 1 , wherein:

the using of the base CNN to determine the coarse category of the data comprises:

providing an input to the base CNN; and

receiving a first output from the base CNN; and

the using of the fine CNN for the determined coarse category to determine the single fine category of the data comprises:

providing the input to the fine CNN; and

receiving a second output from the fine CNN.

17. The system of claim 1 , wherein:

the receiving of the request to classify data comprises receiving, from a computing device via a network, an image; and

the operations further comprise:

generating a listing template based on the single fine category; and

causing the listing template to be presented on the computing device.

18. A non-transitory machine-readable medium having instructions embodied thereon, the instructions executable by a processor of a machine to perform operations comprising:

accessing a dataset comprising categorized data having a plurality of fine categories;

dividing the dataset into a training set and a value set;

training a first CNN model using the training set;

generating a confusion matrix of the first CNN model using the value set;

applying a clustering algorithm to the confusion matrix to determine, for each fine category, an associated coarse category of a plurality of coarse categories, each coarse category being associated with more than one fine category;

training a base convolutional neural network (CNN) to distinguish between the coarse categories, the CNN implemented by a processor of a machine;

training a single fine CNN for each coarse category, the single fine CNN for said coarse category to distinguish between the fine categories associated with said coarse category;

receiving a request to classify data;

using the base CNN, determining a coarse category of the data;

using the single fine CNN for the determined coarse category, determining a single fine category of the data; and

responsive to the request, transmitting the single fine category of the data.

19. The non-transitory machine-readable medium of claim 18 , wherein the clustering algorithm comprises an affinity propagation algorithm.

20. The non-transitory machine-readable medium of claim 19 , wherein the training of the fine CNN for each coarse category comprises:

training a second CNN model using the training set;

generating the fine CNN for each coarse category from the second CNN model; and

training the fine CNN for each coarse category using a subset of the training set, the subset excluding data having fine categories not associated with said coarse category.