TECHNOLOGIES FOR DEEP MACHINE LEARNING WITH CONVOLUTIONAL NEURAL NETWORKS AND REDUCED SET SUPPORT VECTOR MACHINES
Technologies for machine learning with convolutional neural networks (CNNs) and support vector machines (SVMs) include a computing device that may train a deep CNN on a training data set to recognize features of the training data set. The computing device processes the training data set with the CNN after training to extract feature vectors. The computing device trains a multiclass SVM on the feature vectors. The computing device may train a CNN on a training data set to classify the training data set. After training, the computing device may exchange a layer of the CNN with a multiclass SVM. The computing device may use the weights of the exchanged layer to generate the SVM. The computing device may convert the multiclass SVM to a series of binary SVMs. The computing device may generate a reduced set model for each of the binary SVMs. Other embodiments are described and claimed.
1 . A computing device for machine learning, the computing device comprising:
a feature trainer to (i) train a deep convolutional neural network (CNN) on a training data set to recognize features of the training data set, and (ii) process the training data set with the CNN to extract a plurality of feature vectors based on the training data set;
a supervised trainer to train a multiclass support vector machine (SVM) on the plurality of feature vectors to classify the training data set; and
an SVM manager to convert the multiclass SVM to a series of binary SVMs, wherein each binary SVM comprises a model based on the plurality of feature vectors.
2 . The computing device of claim 1 , further comprising a classifier to:
process a test data item with the CNN to extract a test feature vector based on the test data item in response to training of the deep CNN;
process the test feature vector with the series of binary SVMs; and
classify the test data item in response to processing of the test feature vector.
3 . The computing device of claim 1 , wherein to train the deep CNN comprises to perform unsupervised feature learning on the training data set.
4 . The computing device of claim 1 , wherein:
the SVM manager is further to reduce a size of each of the feature vectors; and
to train the multiclass SVM comprises to train the multiclass SVM in response to a reduction of the size of each of the feature vectors.
5 . The computing device of claim 1 , wherein the SVM manager is further to generate a reduced set of vectors for each binary SVM, wherein the model of each binary SVM includes the reduced set of vectors, and wherein the reduced set includes a smaller number of vectors than a corresponding plurality of support vectors.
6 . The computing device of claim 5 , wherein to generate the reduced set of vectors comprises to perform a Burges Reduced Set Vector Method (BRSM).
7 . One or more computer-readable storage media comprising a plurality of instructions that in response to being executed cause a computing device to:
train a deep convolutional neural network (CNN) on a training data set to recognize features of the training data set;
process the training data set with the CNN to extract a plurality of feature vectors based on the training data set;
train a multiclass support vector machine (SVM) on the plurality of feature vectors to classify the training data set; and
convert the multiclass SVM to a series of binary SVMs, wherein each binary SVM comprises a model based on the plurality of feature vectors.
8 . The one or more computer-readable storage media of claim 7 , further comprising a plurality of instructions that in response to being executed cause the computing device to:
process a test data item with the CNN to extract a test feature vector based on the test data item in response to training the deep CNN;
process the test feature vector with the series of binary SVMs; and
classify the test data item in response to processing the test feature vector.
9 . The one or more computer-readable storage media of claim 7 , wherein to train the deep CNN comprises to perform unsupervised feature learning on the training data set.
10 . The one or more computer-readable storage media of claim 7 , further comprising a plurality of instructions that in response to being executed cause the computing device to reduce a size of each of the feature vectors, wherein to train the multiclass SVM comprises to train the multiclass SVM in response to reducing the size of each of the feature vectors.
11 . The one or more computer-readable storage media of claim 7 , further comprising a plurality of instructions that in response to being executed cause the computing device to generate a reduced set of vectors for each binary SVM, wherein the model of each binary SVM includes the reduced set of vectors, and wherein the reduced set includes a smaller number of vectors than a corresponding plurality of support vectors.
12 . A computing device for machine learning, the computing device comprising:
a supervised trainer to train a deep convolutional neural network (CNN) on a training data set to classify the training data set, wherein the deep CNN comprises a plurality of network layers;
a layer exchanger to exchange a layer of the plurality of network layers of the deep CNN with a multiclass support vector machine (SVM) to generate an exchanged CNN in response to training of the deep CNN; and
an SVM manager to convert the multiclass SVM to a series of binary SVMs, wherein each binary SVM comprises a model based on the training data set.
13 . The computing device of claim 12 , further comprising a classifier to:
process a test data item with the exchanged CNN and the series of binary SVMs; and
classify the test data item in response to processing of the test data item.
14 . The computing device of claim 12 , wherein the layer comprises a fully connected layer.
15 . The computing device of claim 12 , wherein the layer comprises a convolution layer.
16 . The computing device of claim 12 , wherein to exchange the layer with the multiclass SVM comprises to generate the multiclass SVM with a plurality of weights of the layer.
17 . The computing device of claim 12 , wherein to exchange the layer with the multiclass SVM comprises to train the multiclass SVM on an output of the exchanged CNN from the training data set to classify the training data set.
18 . The computing device of claim 12 , wherein the SVM manager is further to generate a reduced set of vectors for each binary SVM, wherein the model of each binary SVM includes the reduced set of vectors, and wherein the reduced set includes a smaller number of vectors than a corresponding plurality of support vectors.
19 . The computing device of claim 18 , wherein to generate the reduced set of vectors comprises to perform a Burges Reduced Set Vector Method (BRSM).
20 . One or more computer-readable storage media comprising a plurality of instructions that in response to being executed cause a computing device to:
train a deep convolutional neural network (CNN) on a training data set to classify the training data set, wherein the deep CNN comprises a plurality of network layers;
exchange a layer of the plurality of network layers of the deep CNN with a multiclass support vector machine (SVM) to generate an exchanged CNN in response to training the deep CNN; and
convert the multiclass SVM to a series of binary SVMs, wherein each binary SVM comprises a model based on the training data set.
21 . The one or more computer-readable storage media of claim 20 , further comprising a plurality of instructions that in response to being executed cause the computing device to:
process a test data item with the exchanged CNN and the series of binary SVMs; and
classify the test data item in response to processing the test data item.
22 . The one or more computer-readable storage media of claim 20 , wherein the layer comprises a fully connected layer.
23 . The one or more computer-readable storage media of claim 20 , wherein to exchange the layer with the multiclass SVM comprises to generate the multiclass SVM with a plurality of weights of the layer.
24 . The one or more computer-readable storage media of claim 20 , wherein to exchange the layer with the multiclass SVM comprises to train the multiclass SVM on an output of the exchanged CNN from the training data set to classify the training data set.
25 . The one or more computer-readable storage media of claim 20 , further comprising a plurality of instructions that in response to being executed cause the computing device to generate a reduced set of vectors for each binary SVM, wherein the model of each binary SVM includes the reduced set of vectors, and wherein the reduced set includes a smaller number of vectors than a corresponding plurality of support vectors.