COGNITIVE INDEXING OF IMAGES IN DIGITAL VIDEO CONTENT
A method for indexing and cognitively labeling image elements from a plurality of video frames is implemented in a special purpose computer system. Video files are ingested and stored on a storage device within the computer system. The video files comprise the plurality of video frames. A subset of the video frames indicative of substantial motion in context with adjacent video frames is identified. The subset of the video frames is separated as a sample data set. The subset of video frames is filtered to identify discrete image elements within the video frames. The image elements are segmented from the video frames in the subset. A multi-dimensional feature vector is calculated for each image element. The feature vectors are clustered based upon similarities in feature vector values. The clusters of feature vectors are processed with a support vector classifier machine to refine clusters into predicted classes.
1 . A computer-implemented image classification system comprising
a storage device configured to ingest and store one or more video files thereon, wherein the one or more video files comprise a plurality of video frames;
one or more processors configured with instructions to
identify a subset of the video frames from the one or more video files indicative of substantial motion in context with adjacent video frames;
separate the subset of the video frames as a sample data set;
filter the subset of video frames to identify discrete image elements within the video frames;
segment the image elements from the video frames in the subset of video frames;
calculate a multi-dimensional feature vector for each image element;
cluster the feature vectors into a plurality of clusters based upon similarities in feature vector values; and
process the clusters of feature vectors with a support vector classifier machine to refine clusters into predicted classes.
2 . The computer-implemented image classification system of claim 1 , wherein the one or more processors are further configured with instructions to remove substantially duplicate video frames from the subset.
3 . The computer-implemented image classification system of claim 1 , wherein the one or more processors are further configured with instructions to train the support vector classifier machine using one or more groups of feature vectors from the plurality of clusters.
4 . The computer-implemented image classification system of claim 3 , wherein the one or more processors are further configured with instructions to
separate the feature vectors in each cluster into one of three groups defined across all the clusters;
instantiate three support vector classifier machines;
train each support vector classifier machine on one of the three groups of feature vectors to define predictive classes; and
for each support vector classifier machine, process feature vectors from two of the three groups not previously used to train the respective support vector classifier machine predictively sort the feature vectors into the predicted classes.
5 . The computer-implemented image classification system of claim 4 , wherein the one or more processors are further configured with instructions to
sample a subset of the feature vectors sorted into the predicted classes; and
train a fourth support vector classifier machine by processing the subset of feature vectors sampled from the predicted classes.
6 . The computer-implemented image classification system of claim 1 , wherein the one or more processors are further configured with instructions to
store metadata associated with each of the video frames in the sample data set in a database on the storage device;
at least temporarily maintain an association in the database between each video frame in the sample data set and the respective image elements segmented from each of the video frames in the sample data set;
store feature vectors calculated from the image elements in the database on the storage device; and
record a linked relationship in the database on the storage device between respective feature vectors and the metadata associated with the video frames in the sample set based upon the association of the image elements and the video frames in the sample data set previously maintained.
7 . The computer-implemented image classification system of claim 1 , wherein the one or more processors are further configured with instructions to
process the feature vectors in the predicted classes in a general support vector classifier machine previously trained on known images; and
assign cognizable labels to each of the predicted classes based upon a predictive determination of the general support vector classifier machine of correspondence of the predicted classes to a particular known image.
8 . A method implemented in a computer system for indexing and cognizably labeling image elements from a plurality of video frames, wherein one or more processors in the computer system is particularly configured to perform a number of processing steps comprising
ingesting and storing one or more video files on a storage device within the computer system, wherein the one or more video files comprise the plurality of video frames;
identifying a subset of the video frames from the one or more video files indicative of substantial motion in context with adjacent video frames;
separating the subset of the video frames as a sample data set;
filtering the subset of video frames to identify discrete image elements within the video frames;
segmenting the image elements from the video frames in the subset of video frames;
calculating a multi-dimensional feature vector for each image element;
clustering the feature vectors into a plurality of clusters based upon similarities in feature vector values; and
processing the clusters of feature vectors with a support vector classifier machine to refine clusters into predicted classes.
9 . The method of claim 8 , wherein the step of separating the subset of the video frames further comprises removing substantially duplicate video frames from the subset.
10 . The method of claim 8 comprising a further step of training the support vector classifier machine using one or more groups of feature vectors from the plurality of clusters.
11 . The method of claim 10 comprising further steps of
separating the feature vectors in each cluster into one of three groups defined across all the clusters;
instantiating three support vector classifier machines;
training each support vector classifier machine on one of the three groups of feature vectors to define predictive classes; and
for each support vector classifier machine, processing feature vectors from two of the three groups not previously used to train the respective support vector classifier machine predictively sort the feature vectors into the predicted classes.
12 . The method of claim 11 comprising further steps of
sampling a subset of the feature vectors sorted into the predicted classes; and
training a fourth support vector classifier machine by processing the subset of feature vectors sampled from the predicted classes.
13 . The method of claim 8 comprising further steps of
storing metadata associated with each of the video frames in the sample data set in a database;
at least temporarily maintaining an association in the database between each video frame in the sample data set and the respective image elements segmented from each of the video frames in the sample data set;
storing feature vectors calculated from the image elements in the database; and
recording a linked relationship in the database between respective feature vectors and the metadata associated with the video frames in the sample set based upon the association of the image elements and the video frames in the sample data set previously maintained.
14 . The method of claim 8 comprising further steps of
processing the feature vectors in the predicted classes in a general support vector classifier machine previously trained on known images; and
assigning cognizable labels to each of the predicted classes based upon a predictive determination of the general support vector classifier machine of correspondence of the predicted classes to a particular known image.
15 . A non-transitory computer readable storage medium containing instructions for instantiating a special purpose computer to index and cognizably label image elements from a plurality of video frames, wherein the instructions implement a computer process comprising the steps of
ingesting and storing one or more video files on a storage device within a computer system, wherein the one or more video files comprise the plurality of video frames;
identifying a subset of the video frames from the one or more video files indicative of substantial motion in context with adjacent video frames;
separating the subset of the video frames as a sample data set;
filtering the subset of video frames to identify discrete image elements within the video frames;
segmenting the image elements from the video frames in the subset of video frames;
calculating a multi-dimensional feature vector for each image element;
clustering the feature vectors into a plurality of clusters based upon similarities in feature vector values; and
processing the clusters of feature vectors with a support vector classifier machine to refine clusters into predicted classes.
16 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions implement a further process step comprising separating the subset of the video frames further comprises removing substantially duplicate video frames from the subset.
17 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions implement a further process step comprising training the support vector classifier machine using one or more groups of feature vectors from the plurality of clusters.
18 . The non-transitory computer readable storage medium of claim 17 , wherein the instructions implement further process steps comprising
separating the feature vectors in each cluster into one of three groups defined across all the clusters;
instantiating three support vector classifier machines;
training each support vector classifier machine on one of the three groups of feature vectors to define predictive classes; and
for each support vector classifier machine, processing feature vectors from two of the three groups not previously used to train the respective support vector classifier machine predictively sort the feature vectors into the predicted classes.
19 . The non-transitory computer readable storage medium of claim 18 , wherein the instructions implement further process steps comprising
sampling a subset of the feature vectors sorted into the predicted classes; and
training a fourth support vector classifier machine by processing the subset of feature vectors sampled from the predicted classes.
20 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions implement further process steps comprising
storing metadata associated with each of the video frames in the sample data set in a database;
at least temporarily maintaining an association in the database between each video frame in the sample data set and the respective image elements segmented from each of the video frames in the sample data set;
storing feature vectors calculated from the image elements in the database; and
recording a linked relationship in the database between respective feature vectors and the metadata associated with the video frames in the sample set based upon the association of the image elements and the video frames in the sample data set previously maintained.
21 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions implement further process steps comprising
processing the feature vectors in the predicted classes in a general support vector classifier machine previously trained on known images; and
assigning cognizable labels to each of the predicted classes based upon a predictive determination of the general support vector classifier machine of correspondence of the predicted classes to a particular known image.