Training an object detector using raw and unlabeled videos and extracted speech
An example system includes a processor to receive raw and unlabeled videos. The processor is to extract speech from the raw and unlabeled videos. The processor is to extract positive frames and negative frames from the raw and unlabeled videos based on the extracted speech for each object to be detected. The processor is to extract region proposals from the positive frames and negative frames. The processor is to extract features based on the extracted region proposals. The processor is to cluster the region proposals and assign a potential score to each cluster. The processor is to train a binary object detector to detect objects based on positive samples randomly selected based on the potential score.
1. A system, comprising a processor to:
receive raw and unlabeled videos;
extract speech from the raw and unlabeled videos;
extract positive frames and negative frames from the raw and unlabeled videos based on the extracted speech for each object to be detected;
extract region proposals from the positive frames and negative frames;
extract features based on the extracted region proposals;
cluster the region proposals and assign a potential score to each cluster; and
train a binary object detector to detect objects based on positive samples randomly selected based on the potential score.
2. The system of claim 1 , wherein the potential score is based on a positive ratio.
3. The system of claim 1 , wherein the potential score is based on a cluster variance.
4. The system of claim 1 , wherein the potential score is based on cluster member variety.
5. The system of claim 1 , wherein the potential score is based on a positive ratio, a cluster variance, and cluster member variety.
6. A computer-implemented method, comprising:
receiving, via a processor, raw and unlabeled videos;
extracting, via the processor, speech from the raw and unlabeled videos;
extracting, via the processor, positive frames and negative frames from the raw and unlabeled videos based on the extracted speech for each object to be detected;
extracting, via the processor, region proposals from the positive frames and negative frames;
extracting, via the processor, features based on the extracted region proposals;
clustering, via the processor, the region proposals and assign a potential score to each cluster; and
training, via the processor, a binary object detector to detect objects based on positive samples selected based on potential score.
7. The computer-implemented method of claim 6 , comprising generating, via the processor, hard negative samples using dense subgraph discovery and uniformly sampling the negative frames, the processor to train the binary object detector to detect non-object regions based on the combined hard negative samples and sampled negative frames.
8. The computer-implemented method of claim 6 , wherein extracting the region proposals comprises using a selective search.
9. The computer-implemented method of claim 6 , wherein clustering the region proposals comprises performing a weighted deep embedded clustering.
10. The computer-implemented method of claim 6 , comprising iteratively refining the clusters based on the potential score of the clusters.
11. A computer program product for training object detectors, the computer program product comprising a computer-readable storage medium having program code embodied therewith, the program code executable by a processor to cause the processor to:
receive raw and unlabeled videos;
extract speech from the raw and unlabeled videos;
extract positive frames and negative frames from the raw and unlabeled videos based on the extracted speech for each object to be detected;
extract region proposals from the positive frames and negative frames;
extract features based on the extracted region proposals;
cluster the region proposals and assign a potential score to each cluster; and
train a binary object detector to detect objects based on positive samples randomly selected using the potential score.
12. The computer program product of claim 11 , further comprising program code executable by the processor to generate hard negative samples using dense subgraph discovery and uniformly sample the negative frames and train the binary object detector to detect non-object regions based on the combined hard negative samples and sampled negative frames.
13. The computer program product of claim 11 , further comprising program code executable by the processor to extract the region proposals comprises using a selective search.
14. The computer program product of claim 11 , further comprising program code executable by the processor to detect a number of times that the object is shown in a raw and unlabeled video comprising the image.
15. The computer program product of claim 11 , further comprising program code executable by the processor to detect a number of times the object appeared when mentioned in one of the raw and unlabeled videos.