IP Library › Granted Patent US 12,374,101
Granted Patent B2
US 12,374,101 · App. 17/656,391 · Granted Jul 29, 2025

Error-based explanations for artificial intelligence behavior

Inventors: Arijit Ray (Princeton, NJ); Michael A. Cogswell (West Windsor, NJ); Ajay Divakaran (Monmouth Junction, NJ); Yi Yao (Princeton, NJ); Giedrius T. Burachas (Princeton, NJ); Kamran Alipour (San Diego, CA)
Assignee: SRI International
G06V10/98G06T11/001G06V10/7715G06V10/776
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,374,101
App. No.
17/656,391
Granted
Jul 29, 2025
Kind
B2
Abstract

A computing system comprising a memory configured to store an artificial intelligence (AI) model and an image, and a computation engine executing one or more processors may be configured to perform the techniques for error-based explanations for AI behavior. The computation engine may execute the AI model to analyze the image to output a result. The AI model may, when analyzing the image to output the result, process, based on data indicative of the result, the image to assign an error score to each image feature extracted from the image, and obtain, based on the error scores, an error map. The AI model may next update, based on the error map and to obtain a first updated image, the image to visually indicate the error score assigned to each of the image features, and output one or more of the error scores, the error map, and the first updated image.

Claims (75)

1. A computing system comprising:

a memory configured to store a trained artificial intelligence model and an image; and

a computation engine executing one or more processors,

wherein the computation engine is configured to execute the trained artificial intelligence model to analyze the image to output a result,

wherein the trained artificial intelligence model is configured to, when analyzing the image to output the result:

process, based on data indicative of the result, the image to assign an error score to each of one or more image features extracted from the image, each of the error scores indicative of a likelihood the trained artificial intelligence model is having difficulty in analyzing the corresponding image feature extracted from the image;

obtain, based on the error scores, an error map;

update, based on the error map and to obtain a first updated image, the image to visually indicate the error score assigned to each of the one or more image features; and

output one or more of the error scores, the error map, and the first updated image.

2. The computing system of claim 1 , wherein the trained artificial intelligence model is further configured to, when analyzing the image to output the result:

process the image to identify an attention score for each of the one or more image features extracted from the image, each of the attention scores indicative of a basis by which the trained artificial intelligence model reached the result; and

determine, based on the attention scores, an attention map.

3. The computing system of claim 2 , wherein the trained artificial intelligence model is further configured to, when analyzing the image to output the result:

update, based on the attention map and to obtain a second updated image, the image to visually indicate which of the one or more image features extracted from the image represent the basis by which the trained artificial intelligence model reached the result to a query; and

output one or more of the attention scores, the attention map, and the second updated image.

4. The computing system of claim 2 , wherein the trained artificial intelligence model is further configured to, when analyzing the image to output the result:

determine, based on the error map and the attention map, a helpfulness metric that identifies a relevance of the attention map in explaining the result output by the trained artificial intelligence model; and

output the helpfulness metric.

5. The computing system of claim 4 , wherein the trained artificial intelligence model is configured to determine the helpfulness metric as a negative correlation between the attention map and the error map.

6. The computing system of claim 2 , wherein the trained artificial intelligence model is configured to, when processing the image to identify the attention score, apply a transformer model to identify the attention score for each of the one or more image features extracted from the image.

7. The computing system of claim 1 , wherein the trained artificial intelligence model is configured to, when determining the error map, apply a Gradient-weighted Class Activation Mapping model to process the image to assign the error score to each of the one or more image features extracted from the image and obtain, based on the error scores, the error map.

8. The computing system of claim 1 , wherein the trained artificial intelligence model is configured to:

generate, based on the error map, a heatmap that visually indicates which of the one or more image features are difficult for the trained artificial intelligence model to analyze; and

overlay the heatmap over the image to obtain the first updated image.

9. The computing system of claim 1 ,

wherein the computation engine is configured to receive a query regarding the image,

wherein the computation engine is configured to execute the trained artificial intelligence model to analyze the image to output the result to the query, and

wherein each of the error scores comprises a respective error score that is indicative of the likelihood the trained artificial intelligence model having difficulty in analyzing the corresponding image feature extracted from the image to output the result to the query.

10. A method of explaining a result output by a trained artificial intelligence model configured to analyze an image, the method comprising:

executing, by a computing system, the trained artificial intelligence model to analyze the image in order to output a result,

wherein analyzing, by the trained artificial intelligence model, the image to output the result comprises:

processing, based on data indicative of the result, the image to assign an error score to each of one or more image features extracted from the image, each of the error scores indicative of the trained artificial intelligence model having difficulty in analyzing the corresponding image feature extracted from the image;

obtaining, based on the error scores, an error map;

updating, based on the error map and to obtain a first updated image, the image to visually indicate the error score assigned to each of the one or more image features; and

outputting one or more of the error scores, the error map, and the first updated image.

11. A device configured to explain a result output by a trained artificial intelligence model configured to analyze an image, the device comprising:

a memory configured to store the trained artificial intelligence model and the image; and

a computation engine executing one or more processors,

wherein the computation engine is configured to execute the trained artificial intelligence model to analyze the image in order to output a result,

wherein the trained artificial intelligence model is configured to, when analyzing the image to output the result:

process the image to extract one or more image features;

determine an error map for the image in which corresponding error scores are assigned to one or more of the image features;

determine an attention map for the image in which corresponding attention scores are assigned to one or more of the image features;

determine a helpfulness metric as a negative correlation between the attention map and the error map, wherein the helpfulness metrics identifies a relevance of the attention map in explaining the result output by the trained artificial intelligence model; and

output one or more of the error scores, the error map, the attention scores, the attention map, and the helpfulness metric.

12. The device of claim 11 , wherein the trained artificial intelligence model is further configured to:

update, based on the error map and to obtain a first updated image, the image to visually indicate a likelihood the trained artificial intelligence model had difficulty when analyzing the one or more image features extracted from the image; and

output the first updated image.

13. The device of claim 12 , wherein the trained artificial intelligence model is configured to:

generate, based on the error map, a first heatmap that visually indicates the likelihood the trained artificial intelligence model had difficulty when analyzing the one or more image features extracted from the image; and

overlay the first heatmap over the image to obtain the first updated image.

14. The device of claim 11 , wherein the trained artificial intelligence model is further configured to:

update, based on the attention map and to obtain a second updated image, the image to visually indicate which of the one or more image features formed a basis for the result output by the trained artificial intelligence model; and

output the second updated image.

15. The device of claim 14 , wherein the trained artificial intelligence model is configured to:

generate, based on the attention map, a second heatmap that visually indicates which of the one or more image features form the basis by which the trained artificial intelligence model reached the result; and

overlay the second heatmap over the image to obtain the second updated image.

16. The device of claim 11 ,

wherein the computation engine is configured to receive a query regarding the image,

wherein the computation engine is configured to execute the trained artificial intelligence model to analyze the image to output the result to the query, and

wherein each of the error scores comprises a respective error score that is indicative of a likelihood the trained artificial intelligence model is having difficulty in analyzing the corresponding image feature extracted from the image to output the result to the query.

17. A method of explaining a result output by a trained artificial intelligence model configured to analyze an image, the method comprising:

receiving, by a computing system, a query regarding the image; and

executing, by the computing system, the trained artificial intelligence model to analyze the image in order to output the result to the query,

wherein analyzing, by the trained artificial intelligence model, the image to output the result:

determining, based on the query, an error map for the image in which an error score is assigned to one or more image features extracted from the image;

determining, based on the query, an attention map for the image in which an attention score is assigned to one or more of each of the image features;

determining a helpfulness metric as a negative correlation between the attention map and the error map, wherein the helpfulness metric identifies a relevance of the attention map in explaining the result output by the trained artificial intelligence model; and

outputting the helpfulness metric.

18. The method of claim 17 , further comprising:

updating, based on the error map and to obtain a first updated image, the image to visually indicate a likelihood the trained artificial intelligence model had difficulty when analyzing the one or more image features extracted from the image; and

outputting the first updated image.

19. The method of claim 18 , wherein updating the image comprises:

generating, based on the error map, a first heatmap that visually indicates the likelihood the trained artificial intelligence model had difficulty when analyzing the one or more image features extracted from the image; and

overlaying the first heatmap over the image to obtain the first updated image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2022
From: RAY, ARIJIT; COGSWELL, MICHAEL A.; DIVAKARAN, AJAY; YAO, YI; BURACHAS, GIEDRIUS T.; ALIPOUR, KAMRAN
To: SRI INTERNATIONAL
Reel/Frame 059395/0326 →
Continuity (2)
Provisional Application 63166201 · Mar 25, 2021
Related Publication 20240005654A1 · Jan 4, 2024
References Cited (143)
US 8213709B1 · Medasani · 2012 [cited by examiner]
US 10909401B2 · Burachas et al. · 2021 [cited by applicant]
US 12067794B2 · Kwon · 2024 [cited by examiner]
US 20190370587A1 · Burachas · 2019 [cited by examiner]
US 20200175377A1 · Iio · 2020 [cited by examiner]
US 20210133485A1 · Kubota · 2021 [cited by examiner]
US 20210232773A1 · Wang · 2021 [cited by examiner]
US 20230206661A1 · Choi · 2023 [cited by examiner]
Ziaeefard Maryam; Method and Processing Device for Training a Neural Network; 2020 (Year: 2020). [cited by examiner]
Adadi et al., “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)”, Sep. 16, 2018, pp. 52138-52160. [cited by applicant]
Agrawal et al., “Analyzing the Behavior of Visual Question Answering Models”, arXiv:1606.07356v2, Jun. 23, 2016, 13 pp. [cited by applicant]
Agrawal et al., “CloudCV: Large Scale Distributed Computer Vision as a Cloud Service”, arXiv:1506.04130v3, Jun. 12, 2015, 26 pp. [cited by applicant]
Alipour et al., “The Impact of Explanations on AI Competency Prediction in VQA”, IEEE, Sep. 21, 2020, 8 pp. [cited by applicant]
Anderson et al., “Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering”, IEEE, Mar. 14, 2018, pp. 6077-6086. [cited by applicant]
Antol et al., “VQA: Visual Question Answering”, IEEE, May 3, 2015, pp. 2425-2433. [cited by applicant]
Bansal et al., “Towards Transparent Systems: Semantic Characterization of Failure Modes”, vol. 8694, in European Conference on Computer Vision, 2014, p. 366-381., (Applicant points out, in accordance with MPEP 609.04(a)… [cited by applicant]
Bazzani et al., “Self-Taught Object Localization with Deep Networks”, IEEE, Mar. 7, 2016, 9 pp. [cited by applicant]
Bengio et al., “Representation Learning: A Review and New Perspectives”, vol. 35, No. 8, IEEE Transactions on Pattern Analysis and Machine Intelligence, Apr. 23, 2014, pp. 1798-1828. [cited by applicant]
Cadene et al., “MUREL: Multimodal Relational Reasoning for Visual Question Answering”, Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2019, pp. 1989-1998. [cited by applicant]
Chen et al., “Microsoft COCO Captions: Data Collection and Evaluation Server”, arXiv:1504.00325v2, Jun. 1, 2015, 7 pp. [cited by applicant]
Chen et al., “Static correlative filter based convolutional neural network for visual question answering”, IEEE, Jan. 15, 2018, pp. 526-529. [cited by applicant]
Cinbis et al., “Weakly Supervised Object Localization with Multi-fold Multiple Instance Learning”, IEEE, Mar. 3, 2015, 15 pp. [cited by applicant]
Clark et al., “Simple and effective multi- paragraph reading comprehension”, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Jul. 2018,… [cited by applicant]
Clark et al., “What Does BERT Look At? An Analysis of BERT's Attention”, arXiv:1906.04341v1, Jun. 11, 2019, 11 pp. [cited by applicant]
Conneau et al., “Supervised Learning of Universal Sentence Representations from Natural Language Inference Data”, arXiv:1705.02364v5, Jun. 2016, 12 pp. [cited by applicant]
Das et al., “Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?”, vol. 163, Elsevier, Jun. 2016, 23 pp. [cited by applicant]
De Marneffe et al., “Stanford typed dependencies manual”, Sep. 2008, p. 28. [cited by applicant]
Deng et al., “ImageNet: A large-scale hierarchical image database”, IEEE, Jun. 20, 2009, 8 pp. [cited by applicant]
Desta et al., “Object-Based Reasoning in VQA”, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, Jan. 29, 2018, pp. 1814-1823. [cited by applicant]
Devlin et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, arXiv:1810.04805v2, May 24, 2019, 16 pp. [cited by applicant]
Dosovitskiy et al., “Inverting Visual Representations with Convolutional Networks”, CVPR, Jun. 9, 2015, pp. 4829-4837. [cited by applicant]
Dozat et al., “Deep Biaffine Attention for Neural Dependency Parsing”, arXiv:1611.01734v3, Nov. 4, 2016, 8 pp. [cited by applicant]
Erhan et al., “Visualizing Higher-Layer Features of a Deep Network”, Jun. 9, 2009, 12 pp. [cited by applicant]
Fang et al., “From Captions to Visual Concepts and Back”, IEEE, Nov. 18, 2014, pp. 1473-1482. [cited by applicant]
Gan et al., “DevNet: A Deep Event Network for Multimedia Event Detection and Evidence Recounting”, IEEE, Jun. 7, 2015, pp. 2568-2577. [cited by applicant]
Gao et al., “Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering”, May 21, 2015, 9 pp. [cited by applicant]
Garcia et al., “KnowIT VQA: Answering knowledge-based questions about videos”, arXiv:1910.10706v3, Oct. 23, 2019, 10 pp. [cited by applicant]
Gardner et al., “AllenNLP: A Deep Semantic Natural Language Processing Platform”, arXiv:1803.07640v2, Mar. 20, 2018, 6 pp. [cited by applicant]
Girshick et al., “Detectron”, facebookresearch / Detectron, Retrieved from:https://github.com/facebookresearch/detectron, Mar. 27, 2021, 5 pp. [cited by applicant]
Girshick et al., “Rich feature hierarchies for accurate object detection and semantic segmentation”, IEEE, Nov. 11, 2013, 8 pp. [cited by applicant]
Gong et al., “Efficient training of BERT by progressively stacking”, Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, May 24, 2019, pp. 2337-2346. [cited by applicant]
Gordon et al., “IQA: Visual Question Answering in Interactive Environments”, IEEE, Dec. 9, 2017, 10 pp. [cited by applicant]
Goyal et al., “Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering”, IEEE, Dec. 2, 2016, pp. 6904-6913. [cited by applicant]
Gunning et al., “DARPA's Explainable Artificial Intelligence Program”, vol. 40, No. 2, AI Magazine, 2019, p. 44-58, (Applicant points out, in accordance with MPEP 609.04(a), that the year of publication, 2019, is suffic… [cited by applicant]
Gurari et al., “VizWiz Grand Challenge: Answering Visual Questions from Blind People”, IEEE, Feb. 22, 2018, 13 pp. [cited by applicant]
He et al., “Deep Residual Learning for Image Recognition”, arXiv:1512.03385v1, Dec. 10, 2015, 12 pp. [cited by applicant]
Hendricks et al., “Generating Visual Explanations”, vol. 9908, in European Conference on Computer Vision, 2016, pp. 3-19, (Applicant points out, in accordance with MPEP 609.04(a), that the year of publication, 2016, is … [cited by applicant]
Hochreiter et al., “Long Short-Term Memory”, vol. 9, No. 8, Neural Computation, Nov. 15, 1997, pp. 1735-1780. [cited by applicant]
Hoiem et al., “Diagnosing Error in Object Detectors”, European conference on computer vision, ECCV, Oct. 7, 2012, pp. 340-353. [cited by applicant]
Hudson et al., “Learning by abstraction: The neural state machine”, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Jul. 9, 2019, 17 pp. [cited by applicant]
Jia et al., “Caffe: Convolutional Architecture for Fast Feature Embedding”, arXiv:1408.5093v1, Jun. 20, 2014, 4 pp. [cited by applicant]
Jiang et al., “Pythia v0.1: the Winning Entry to the VQA Challenge 2018”, arXiv:1807.09956v2, Jul. 26, 2018, 3 pp. [cited by applicant]
Johns et al., “Becoming the Expert—Interactive Multi-Class Machine Teaching”, IEEE, Apr. 28, 2015, pp. 2616-2624. [cited by applicant]
Johnson et al., “DenseCap: Fully Convolutional Localization Networks for Dense Captioning”, IEEE, Nov. 24, 2015, pp. 4565-4574. [cited by applicant]
Johnson et al., “Image Retrieval using Scene Graphs”, IEEE, Jun. 7, 2015, pp. 3668-3678. [cited by applicant]
Karpathy et al., “Deep Visual-Semantic Alignments for Generating Image Descriptions”, IEEE, Dec. 6, 2014, pp. 3128-3167. [cited by applicant]
Karpathy et al., “What I learned from competing against a ConvNet on ImageNet”, Sep. 2, 2014, 8 pp. [cited by applicant]
Kazemzadeh et al., “ReferItGame: Referring to Objects in Photographs of Natural Scenes”, Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Oct. 1, 2014, pp. 787-798. [cited by applicant]
Kim et al., “Bilinear Attention Networks”, 32nd Conference on Neural Information Processing Systems (NeuriPS 2018), May 21, 2018, 11 pp. [cited by applicant]
Kim et al., “DeepStory: Video Story QA by Deep Embedded Memory Networks”, ArXiv, Jul. 4, 2017, 7 pp. [cited by applicant]
Kim et al., “Multi-modal dual attention memory for video story question answering”, arXiv:1809.07999v1, Sep. 8, 2018, 16 pp. [cited by applicant]
Kim et al., “Progressive attention memory network for movie story question answering”, IEEE, Apr. 18, 2019, 10 pp. [cited by applicant]
Kingma et al., “Adam: a Method for Stochastic Optimization”, arXiv:1412.6980v9, Dec. 22, 2014, 15 pp. [cited by applicant]
Kiros et al., “Skip-thought vectors”, arXiv:1506.06726v1, Jun. 22, 2015, 11 pp. [cited by applicant]
Krishna et al., “Dense-captioning events in videos”, IEEE, May 2, 2017, 16 pp. [cited by applicant]
Krishna et al., “Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations”, International Journal of Computer Vision, Feb. 23, 2016, pp. 32-73. [cited by applicant]
Krizhevsky et al., “ImageNet Classification with Deep Convolutional Neural Networks”, Dec. 3, 2012, 9 pp. [cited by applicant]
Lei et al., “TVQA+: Spatio-Temporal Grounding for Video Question Answering”, arXiv:1904.11574v2, Apr. 25, 2019, 15 pp. [cited by applicant]
Lei et al., “TVQA: Localized, Compositional Video Question Answering”, arXiv:1809.01696v2, Sep. 5, 2018, 13 pp. [cited by applicant]
Li et al., “Relation-aware graph attention network for visual question answering”, arXiv:1903.12314v3, Mar. 29, 2019, 10 pp. [cited by applicant]
Li et al., “Story Ending Prediction by Transferable BERT”, arXiv:1905.07504v2, May 17, 2019, 7 pp. [cited by applicant]
Li et al., “Tell-and-answer: Towards explainable visual question answering using attributes and captions”, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computat… [cited by applicant]
Li et al., “Visualbert: a Simple and Performant Baseline for Vision and Language”, arXiv:1908.03557v1, Aug. 9, 2019, 14 pp. [cited by applicant]
Lin et al., “Microsoft COCO: Common objects in context”, arXiv:1405.0312v3, May 1, 2014, 15 pp. [cited by applicant]
Lin et al., “Network in network”, Dec. 16, 2013, 10 pp. [cited by applicant]
Lipton, “The Mythos of Model Interpretability”, arXiv:1606.03490v1, Jun. 10, 2016, pp. 96-100. [cited by applicant]
Liu et al., “Multimodal cross-guided attention networks for visual question answering”, Advances in Intelligent Systems Research (AISR), vol. 151, Atlantis Press, Apr. 1, 2018, pp. 347-353. [cited by applicant]
Long et al., “Fully Convolutional Networks for Semantic Segmentation”, IEEE, Nov. 14, 2014, pp. 3431-3440. [cited by applicant]
Lu et al., “Deeper LSTM+ normalized CNN for Visual Question Answering”, GT-Vision-Lab / VQA_LSTM_CNN, Retrieved from: GT-Vision-Lab / VQA_LSTM_CNN, Jun. 14, 2016, 5 pp. [cited by applicant]
Lu et al., “Hierarchical Question-Image Co-Attention for Visual Question Answering”, 30th Conference on Neural Information Processing Systems (NIPS 2016), May 31, 2016, 9 pp. [cited by applicant]
Lu et al., “R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering”, arXiv:1805.09701v2, May 24, 2018, 10 pp. [cited by applicant]
Lu et al., “ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks”, 33th Conference on Neural Information Processing Systems (NIPS 2019), Aug. 6, 2019, 11 pp. [cited by applicant]
Mahendran et al., “Salient deconvolutional networks”, European Conference on Computer Vision, Oct. 11, 2016, pp. 120-135. [cited by applicant]
Mahendran et al., “Visualizing deep convolutional neural networks using natural pre-images”, arXiv:1512.02017v3, Apr. 15, 2016, 25 pp. [cited by applicant]
Malinowski et al., “Ask Your Neurons: A Neural-based Approach to Answering Questions about Images”, IEEE, May 5, 2015, 9 pp. [cited by applicant]
McCann et al., “Learned in Translation: Contextualized Word Vectors”, 31st Conference on Neural Information Processing Systems (NIPS 2017), Aug. 1, 2017, 12 pp. [cited by applicant]
Mun et al., “MarioQA: Answering Questions by Watching Gameplay Videos”, IEEE, Dec. 6, 2016, 9 pp. [cited by applicant]
Narasimhan et al., “Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering”, 32nd Conference on Neural Information Processing Systems (NIPS 2018), Nov. 1, 2018, 12 pp. [cited by applicant]
Narasimhan et al., “Straight to the facts: Learning knowledge base retrieval for factual visual question answering”, arXiv:1809.01124v1, Sep. 4, 2018, 18 pp. [cited by applicant]
Norcliffe-Brown et al., “Learning Conditioned Graph Structures for Interpretable Visual Question Answering”, 32nd Conference on Neural Information Processing Systems (NIPS 2018), Jun. 1, 2018, 13 pp. [cited by applicant]
Oquab et al., “Is object localization for free?—Weakly-supervised learning with convolutional neural networks”, Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, Jun. 7, 2015, pp. 685-… [cited by applicant]
Oquab et al., “Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks”, Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, Jun. 23, 2014, pp. 1717… [cited by applicant]
Patro et al., “U-CAM: Visual Explanation Using Uncertainty Based Class Activation Maps”, IEEE, 2019, p. 7444-7453, (Applicant points out, in accordance with MPEP 609.04(a), that the year of publication, 2019, is suffici… [cited by applicant]
Pennington et al., “GloVe: Global Vectors for Word Representation”, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Oct. 25, 201… [cited by applicant]
Peters et al., “Deep contextualized word representations.”, arXiv:1802.05365v2, Feb. 15, 2018, 15 pp. [cited by applicant]
Petsiuk et al., “RISE: Randomized Input Sampling for Explanation of Black-box Models”, arXiv preprint arXiv:1806.07421, Sep. 25, 2018, 13 pp. [cited by applicant]
Pinheiro et al., “From Image-level to Pixel-level Labeling with Convolutional Networks”, IEEE, Nov. 23, 2014, pp. 1713-1721. [cited by applicant]
Plummer et al., “Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models”, Proceedings of the IEEE international conference on computer vision, May 19, 2015, pp. 2641-2649. [cited by applicant]
Radford et al., “Improving language understanding by generative pre-training”, OpenAI blog, Jun. 11, 2018, 12 pp. [cited by applicant]
Radford et al., “Language models are unsupervised multitask learners”, OpenAI blog, Feb. 14, 2019, 24 pp. [cited by applicant]
Ren et al., “Exploring Models and Data for Image Question Answering”, Advances in Neural Information Processing Systems 28 (NIPS 2015), May 8, 2015, 9 pp. [cited by applicant]
Ren et al., “Faster R-CNN: Towards real-time object detection with region proposal networks”, IEEE, Jun. 4, 2015, 10 pp. [cited by applicant]
Ribeiro et al., “”Why Should I Trust You?“ Explaining the Predictions of Any Classifier”, Proceedings of the 22nd Acm Sigkdd international conference on knowledge discovery and data mining, ACM, Feb. 16, 2016, pp. 1135-… [cited by applicant]
Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge”, International Journal of Computer Vision, vol. 115, No. 3, Springer, Apr. 2015, pp. 211-252. [cited by applicant]
Ruwa et al., “Mood-aware visual question answering”, ScienceDirect, Feb. 22, 2019, pp. 305-316. [cited by applicant]
Santoro et al., “A simple neural network module for relational reasoning”, arXiv:1706.01427v1, Jun. 5, 2017, 16 pp. [cited by applicant]
Selvaraju et al., “Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizations”, IEEE, Oct. 7, 2017, pp. 618-626. [cited by applicant]
Selvaraju et al., “Grad-CAM: Why did you say that?”, arXiv:1611.07450v2, Nov. 22, 2016, 4 pp. [cited by applicant]
Sharma et al., “Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset for Automatic Image Captioning”, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), … [cited by applicant]
Silver et al., “Mastering the game of Go with deep neural networks and tree search”, Nature, vol. 529, No. 7587, Macmillan Publisher Limited, Jan. 28, 2016, pp. 484-489. [cited by applicant]
Simonyan et al., “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps”, arXiv:1312.6034v2, Dec. 20, 2013, 8 pp. [cited by applicant]
Simonyan et al., “Very Deep Convolutional Networks for Large-Scale Image Recognition”, ICLR, Sep. 4, 2014, 14 pp. [cited by applicant]
Singh et al., “Pythia—A platform for vision & language research”, Computer Science, 2018, 4 pp., (Applicant points out, in accordance with MPEP 609.04(a), that the year of publication, 2018, is sufficiently earlier than… [cited by applicant]
Singh et al., “Towards VQA models that can read”, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, Apr. 18, 2019, pp. 8317-8326. [cited by applicant]
Socher et al., “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank”, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Association for Computational Lingu… [cited by applicant]
Springenberg et al., “Striving for Simplicity: the All Convolutional Net”, ICLR, Dec. 21, 2014, 14 pp. [cited by applicant]
Suhr et al., “A corpus for reasoning about natural language grounded in photographs”, Nov. 1, 2018, 11 pp. [cited by applicant]
Sun et al., “VideoBert: A joint model for video and language representation learning”, IEEE, Apr. 3, 2019, 13 pp. [cited by applicant]
Tapaswi et al., “MovieQA: Understanding Stories in Movies through Question-Answering”, IEEE, Dec. 9, 2015, 10 pp. [cited by applicant]
Turian et al., “Word Representations: A Simple and General Method for Semi-Supervised Learning”, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Association for Computational Lin… [cited by applicant]
Vaswani et al., “Attention is all you need”, 31st Conference on Neural Information Processing Systems (NIPS 2017), Jun. 12, 2017, 15 pp. [cited by applicant]
Mnyals et al., “Show and Tell: A Neural Image Caption Generator”, Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, Nov. 17, 2014, pp. 3156-3164. [cited by applicant]
Voita et al., “Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned”, arXiv:1905.09418v2, May 23, 2019, 12 pp. [cited by applicant]
Vondrick et al., “HOGgles: Visualizing Object Detection Features”, IEEE, Dec. 1, 2013, 8 pp. [cited by applicant]
Wu et al., “A Robust Passage Retrieval Algorithm for Video Question Answering”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, No. 10, IEEE, Oct. 1, 2008, pp. 1411-1421. [cited by applicant]
Wu et al., “CLVQ: cross-language video question/answering system”, Proceedings of the IEEE Sixth International Symposium on Multimedia Software Engineering (ISMSE'04), IEEE, Dec. 13, 2004, 9 pp. [cited by applicant]
Wu et al., “Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”, arXiv:1609.08144v2, Sep. 26, 2016, 23 pp. [cited by applicant]
Xu et al., “Ask, attend and answer: Exploring question-guided spatial attention for visual question answering”, arXiv:1511.05234v2, Nov. 17, 2015, 16 pp. [cited by applicant]
Xu et al., “Show, attend and tell: Neural image caption generation with visual attention”, arXiv:1502.03044v3, Feb. 10, 2015, 22 pp. [cited by applicant]
Yang et al., “BERT Representations for Video Question Answering”, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, IEEE, Mar. 1, 2020, pp. 1556-1565. [cited by applicant]
Yang et al., “Stacked attention networks for image question answering”, IEEE, Nov. 7, 2015, 11 pp. [cited by applicant]
Yang et al., “XLNet: Generalized Autoregressive Pretraining for Language Understanding”, 33rd Conference on Neural Information Processing Systems (NeurlPS 2019), Jun. 19, 2019, 18 pp. [cited by applicant]
Yin et al., “Obj2Text: Generating Visually Descriptive Language from Object Layouts”, arXiv:1707.07102v1, Jul. 22, 2017, 11 pp. [cited by applicant]
Yu et al., “Deep modular co-attention networks for visual question answering”, IEEE, Jun. 1, 2019, 11 pp. [cited by applicant]
Yu et al., “Multimodal transformer with multi-view visual representation for image captioning”, Journal of Latex Class Files, vol. 14, No. 8, IEEE, Aug. 2015, 12 pp. [cited by applicant]
Yu et al., “Structured semantic representation for visual question answering”, IEEE, Oct. 1, 2018, p. 2286-2290. [cited by applicant]
Zeiler et al., “Visualizing and Understanding Convolutional Networks”, European conference on computer vision, Springer International Publishing, Nov. 12, 2013, pp. 818-833. [cited by applicant]
Zellers et al., “From recognition to cognition: Visual commonsense reasoning”, IEEE, Nov. 27, 2018, 29 pp. [cited by applicant]
Zeng et al., “Leveraging video descriptions to learn video question answering”, Thirty-First AAAI Conference on Artificial Intelligence, Feb. 12, 2017, 7 pp. [cited by applicant]
Zhang et al., “Top-Down Neural Attention by Excitation Backprop”, Springer International Publishing, Aug. 1, 2016, pp. 543-559. [cited by applicant]
Zhao et al., “Multi-turn video question answering via multi-stream hierarchical attention context Network”, Proceedings of the Twenty-Seventh International Conference on Artificial Intelligence (IJCAI-18), Jul. 13, 2018… [cited by applicant]
Zhou et al., “Learning Deep Features for Discriminative Localization”, Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, Dec. 14, 2015, pp. 2921-2929. [cited by applicant]
Zhu et al., “Uncovering the temporal context for video question answering”, arXiv:1511.04670v1, Sep. 1, 2017, 10 pp. [cited by applicant]