Media content analysis system and method

1. A method comprising:

using, by a computing device, a trained model as an agent to analyze a media object using a number of rounds of analysis, the trained model comprising a number of state layers to store an outcome from each round of analysis of the media object by the agent;

making, by the computing device and using the agent, a determination in a current round of analysis of a next action to take in the analysis of the media object, the determination being made using a stored state from the number of state layers and results of the current round of analysis by the agent; and

providing, by the computing device and using the agent, an output from the number of rounds of analysis of the media object, the output comprising a plurality of labels corresponding to the media object's content.

2. The method of claim 1 , the agent's trained model being a recurrent neural network comprising an input layer, a number of inner layers, an output layer and the number of state layers.

3. The method of claim 2 , the recurrent neural network being trained from a convolutional neural network comprising the input layer, the number of inner layers and the output layer and using training data comprising a plurality of media objects and metadata associated with the plurality of media objects, the recurrent neural network comprising a set of parameters updated from the set of parameters learned for the convolutional neural network.

4. The method of claim 3 , the training data further comprising a number of saliency maps, each saliency map corresponding to a media object of the plurality of media objects, the saliency maps being used to train the agent in identifying a portion of the media object to analyze.

5. The method of claim 1 , providing an output from the number of rounds of analysis of the media object further comprising:

identifying, by the computing device and using the agent, the plurality of labels using a number of semantic vectors mapped to a semantic vector space, the number of semantic vectors being identified by the model in the number of rounds of analysis of the media object.

6. The method of claim 1 , making a determination in a current round of analysis of a next action to take in the analysis of the media object further comprising:

determining, by the computing device and using the agent, whether or not to make a label prediction;

determining, by the computing device and using the agent, whether or not to identify a new region of the media object to be analyzed in a next round of analysis of the media object; and

determining, by the computing device and using the agent, whether or not to end the analysis of the media object.

7. The method of claim 6 , determining whether or not to identify a new region of the media object to be analyzed in a next round of analysis further comprising:

determining, by the computing device and using the agent, the new region of the media object to be analyzed in the next round of analysis of the media object using a saliency map generated using the media object.

8. A system comprising:

at least one computing device, each computing device comprising a processor and a storage medium for tangibly storing thereon program logic for execution by the processor, the stored program logic comprising:

using logic executed by the processor for using a trained model as an agent to analyze a media object using a number of rounds of analysis, the trained model comprising a number of state layers to store an outcome from each round of analysis of the media object by the agent;

making logic executed by the processor for making, using the agent, a determination in a current round of analysis of a next action to take in the analysis of the media object, the determination being made using a stored state from the number of state layers and results of the current round of analysis by the agent; and

providing logic executed by the processor for providing, using the agent, an output from the number of rounds of analysis of the media object, the output comprising a plurality of labels corresponding to the media object's content.

9. The system of claim 8 , the agent's trained model being a recurrent neural network comprising an input layer, a number of inner layers, an output layer and the number of state layers.

10. The system of claim 9 , the recurrent neural network being trained from a convolutional neural network comprising the input layer, the number of inner layers and the output layer and using training data comprising a plurality of media objects and metadata associated with the plurality of media objects, the recurrent neural network comprising a set of parameters updated from the set of parameters learned for the convolutional neural network.

11. The system of claim 10 , the training data further comprising a number of saliency maps, each saliency map corresponding to a media object of the plurality of media objects, the saliency maps being used to train the agent in identifying a portion of the media object to analyze.

12. The system of claim 8 , the providing logic executed by the processor for providing an output from the number of rounds of analysis of the media object further comprising:

identifying logic executed by the processor for identifying, using the agent, the plurality of labels using a number of semantic vectors mapped to a semantic vector space, the number of semantic vectors being identified by the model in the number of rounds of analysis of the media object.

13. The system of claim 8 , the making logic executed by the processor for making a determination in a current round of analysis of a next action to take in the analysis of the media object further comprising:

determining logic executed by the processor for determining, using the agent, whether or not to make a label prediction;

determining logic executed by the processor for determining, using the agent, whether or not to identify a new region of the media object to be analyzed in a next round of analysis of the media object; and

determining logic executed by the processor for determining, using the agent, whether or not to end the analysis of the media object.

14. The system of claim 13 , the determining logic executed by the processor for determining whether or not to identify a new region of the media object to be analyzed in a next round of analysis further comprising:

determining logic executed by the processor for determining, using the agent, the new region of the media object to be analyzed in the next round of analysis of the media object using a saliency map generated using the media object.

15. A computer readable non-transitory storage medium for tangibly storing thereon computer readable instructions that when executed cause at least one processor to:

use a trained model as an agent to analyze a media object using a number of rounds of analysis, the trained model comprising a number of state layers to store an outcome from each round of analysis of the media object by the agent;

make a determination, using the agent, in a current round of analysis of a next action to take in the analysis of the media object, the determination being made using a stored state from the number of state layers and results of the current round of analysis by the agent; and

provide, using the agent, an output from the number of rounds of analysis of the media object, the output comprising a plurality of labels corresponding to the media object's content.

16. The computer readable non-transitory storage medium of claim 15 , the agent's trained model being a recurrent neural network comprising an input layer, a number of inner layers, an output layer and the number of state layers.

17. The computer readable non-transitory storage medium of claim 16 , the recurrent neural network being trained from a convolutional neural network comprising the input layer, the number of inner layers and the output layer and using training data comprising a plurality of media objects and metadata associated with the plurality of media objects, the recurrent neural network comprising a set of parameters updated from the set of parameters learned for the convolutional neural network.

18. The computer readable non-transitory storage medium of claim 17 , the training data further comprising a number of saliency maps, each saliency map corresponding to a media object of the plurality of media objects, the saliency maps being used to train the agent in identifying a portion of the media object to analyze.

19. The computer readable non-transitory storage medium of claim 15 , the computer readable instructions that when executed further cause at least one processor to provide an output from the number of rounds of analysis of the media object further comprising computer readable instructions that when executed further cause at least one processor to:

identify, using the agent, the plurality of labels using a number of semantic vectors mapped to a semantic vector space, the number of semantic vectors being identified by the model in the number of rounds of analysis of the media object.

20. The computer readable non-transitory storage medium of claim 15 , the computer readable instructions that when executed further cause at least one processor to make a determination in a current round of analysis of a next action to take in the analysis of the media object further comprising computer readable instructions that when executed further cause at least one processor to:

determine, using the agent, whether or not to make a label prediction;

determine, using the agent, whether or not to identify a new region of the media object to be analyzed in a next round of analysis of the media object; and

determine, using the agent, whether or not to end the analysis of the media object.

21. The computer readable non-transitory storage medium of claim 20 , the computer readable instructions that when executed further cause at least one processor to determine whether or not to identify a new region of the media object to be analyzed in a next round of analysis further comprising:

determine, using the agent, the new region of the media object to be analyzed in the next round of analysis of the media object using a saliency map generated using the media object.