Granted Patent B2

US 12,293,294 · App. 17/316,503 · Granted May 6, 2025

Inventors: Elham Saraee (Jamaica Plain, MA); Jehan Hamedi (South Boston, MA); Zachary Halloran (Braintree, MA); Arsenii Mustafin (Boston, MA)

Assignee: VIZIT LABS, INC.

G06N3/084G06N3/08G06Q30/0277G06T11/001G06T11/60G06V10/764G06V10/776G06V20/46

Abstract

Claims (78)

1. A method for improved explainability of a neural network, comprising:

receiving, by one or more processors, an image or video comprising a set of pixels;

extracting, by the one or more processors, a plurality of features from the image or video;

generating, by the one or more processors, a feature vector including the plurality of features each at different index values of the feature vector, each of the plurality of features corresponding to a different pixel or subset of pixels of the set of pixels of the image or video;

executing, by the one or more processors, a neural network using the plurality of features to obtain a performance score for the image or video, the neural network comprising an input layer, a plurality of intermediate layers subsequent to the input layer, and a regression layer or a classification layer subsequent to the plurality of intermediate layers;

extracting, by the one or more processors, one or more values from one or more signals between an intermediate layer of the plurality of intermediate layers and the regression layer or the classification layer, the one or more values generated responsive to executing the neural network using the extracted plurality of features;

back-propagating, by the one or more processors, a set of the extracted one or more values through the plurality of intermediate layers and the input layer;

extracting, by the one or more processors, an impact score for each of the plurality of features from the input layer and generated from the back-propagation of the set of the extracted one or more values through the plurality of intermediate layers and the input layer; and

assigning, by the one or more processors, an impact score of the extracted impact scores to each pixel and/or subset of pixels of the set of pixels of the image or video based on the pixel or subset of pixels mapping to the same index value of the feature vector as the impact score;

determining, by the one or more processors, (i) a first plurality of pixels of the image or video that correspond to a first color indicating a positive impact to the performance score, and (ii) a second plurality of pixels of the image or video that correspond to a second color indicating a negative impact to the performance score, based on an assignment of a positive impact score to each of the first plurality of pixels and a negative impact score to each of the second plurality of pixels;

generating, by the one or more processors, a heat map overlay comprising the first color at locations of the image or video corresponding to the first plurality of pixels and the second color at locations of the image or video corresponding to the second plurality of pixels; and

presenting, at a display of a client device and by the one or more processors, the heat map overlay over the image or video.

2. The method of claim 1 , comprising:

generating, by the one or more processors, an evaluation image or video by placing the heat map overlay over the image or video.

3. The method of claim 1 , further comprising:

determining, by the one or more processors, whether each of the plurality of features had a positive impact on the performance score, a negative impact on the performance score, or no impact on the performance score.

4. The method of claim 3 , wherein determining whether each of the plurality of features had a positive impact on the performance score or a negative impact on the performance score comprises:

determining, by the one or more processors, a magnitude of a positive impact on the performance score or a negative impact on the performance score of a feature of the image or video; and

wherein generating the heat map overlay further comprises:

generating, by the one or more processors, the heat map overlay to include a first shade of the first color or a second shade of the second color for the feature selected based on (1) the determined positive impact or negative impact, and (2) the determined magnitude of the determined positive impact or negative impact.

5. The method of claim 1 , wherein the neural network comprises the classification layer, the classification layer configured to output a probability for each of a plurality of performance score ranges based on signals received from an intermediate layer of the plurality of intermediate layers, the signals generated based on executing the neural network using the extracted plurality of features;

wherein extracting the one or more values comprises:

extracting, by the one or more processors, the probabilities for the plurality of performance score ranges; and

wherein back-propagating the set of the extracted one or more values comprises back-propagating, by the one or more processors, the probabilities through the plurality of intermediate layers and the input layer.

6. The method of claim 1 , further comprising:

training, by the one or more processors, the neural network using training data labeled based on feedback from a first target audience;

training, by the one or more processors, a first neural network using training data labeled based on feedback from a second target audience;

executing, by the one or more processors, the first neural network using the plurality of features to obtain a first performance score for the image or video, the first neural network comprising a first input layer, a first plurality of intermediate layers, and a first regression layer or a first classification layer;

extracting, by the one or more processors, one or more values from one or more first signals between a first intermediate layer of the first plurality of intermediate layers and the first regression layer or the first classification layer, the one or more first values generated responsive to executing the first neural network using the extracted plurality of features;

for each of the plurality of features, calculating, by the one or more processors and based on at least one of the one or more first values, a first impact score indicating a first impact the feature had on the first performance score; and

generating, by the one or more processors and based on the first impact scores of the plurality of features, first indications indicating an impact different features of the first image or video had on the first performance score.

7. The method of claim 6 , wherein generating the first indications comprises generating a heat map overlay by generating the heat map overlay to have a first pattern of colors indicating an impact different portions of the image or video had on the performance score, and wherein generating the heat map overlay comprises generating the first heat map overlay to have a second pattern of colors indicating a first impact different portions of the image or video had on the first performance score.

8. The method of claim 6 , further comprising:

presenting, by the one or more processors, a first identifier associated with the first target audience and a second identifier associated with the second target audience on a display,

wherein executing the first neural network is performed responsive to receiving a user selection of the second identifier.

9. The method of claim 6 , wherein the first target audience comprises individuals having a first common characteristic, and wherein the second target audience comprises individuals having a second common characteristic, and further comprising:

presenting, by the one or more processors, the first characteristic of the first target audience and the second characteristic of the second target audience on a display,

wherein executing the first neural network is performed responsive to receiving a user selection of the second characteristic.

10. The method of claim 1 , wherein executing the neural network using the extracted plurality of features to obtain the performance score for the image or video comprises:

obtaining, by the one or more processors, the performance score from an output of the regression layer.

11. A system for improved explainability of a neural network, the system comprising:

one or more hardware processors configured by machine-readable instructions to:

receive an image or video comprising a set of pixels;

extract a plurality of features from the image or video;

generate a feature vector including the plurality of features each at different index values of the feature vector, each of the plurality of features corresponding to a different pixel or subset of pixels of the set of pixels of the image or video;

execute a neural network using the plurality of features to obtain a performance score for the image or video, the neural network comprising an input layer, a plurality of intermediate layers subsequent to the input layer, and a regression layer or a classification layer subsequent to the plurality of intermediate layers;

extract one or more values from one or more signals between an intermediate layer of the plurality of intermediate layers and the regression layer or the classification layer, the one or more values generated responsive to executing the neural network using the extracted plurality of features;

back-propagate a set of the extracted one or more values through the plurality of intermediate layers and the input layer;

extract an impact score for each of the plurality of features from the input layer and generated from the back-propagation of the set of the extracted one or more values through the plurality of intermediate layers and the input layer;

assign an impact score of the extracted impact scores to each pixel and/or subset of pixels of the set of pixels of the image or video based on the pixel or subset of pixels mapping to the same index value of the feature vector as the impact score;

determine (i) a first plurality of pixels of the image or video that correspond to a first color indicating a positive impact to the performance score, and (ii) a second plurality of pixels of the image or video that correspond to a second color indicating a negative impact to the performance score, based on an assignment of a positive impact score to each of the first plurality of pixels and a negative impact score to each of the second plurality of pixels;

generate a heat map overlay comprising the first color at locations of the image or video corresponding to the first plurality of pixels and the second color at locations of the image or video corresponding to the second plurality of pixels; and

present, at a display of a client device the heat map overlay over the image or video.

12. The system of claim 11 ,

wherein the one or more hardware processors are further configured by machine-readable instructions to:

generate an evaluation image or video by placing the heat map overlay over the image or video.

13. The system of claim 11 , wherein the one or more hardware processors are further configured by machine-readable instructions to:

determine whether each of the plurality of features had a positive impact on the performance score or a negative impact on the performance score.

14. The system of claim 13 , wherein the one or more hardware processors are configured by machine-readable instructions to determine whether each of the plurality of features had a positive impact on the performance score or a negative impact on the performance score by:

determining a magnitude of a positive impact on the performance score or a negative impact on the performance score of the feature; and

wherein the one or more hardware processors are further configured by machine-readable instructions to generate the heat map overlay by:

generating, by the one or more processors, the heat map overlay to include a first shade of the first color or a second shade of the second color for the feature selected based on (1) the determined positive impact or negative impact, and (2) the determined magnitude of the determined positive impact or negative impact.

15. A non-transitory computer-readable storage medium for improved explainability of a neural network, the non-transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by one or more processors to perform a method, the method comprising:

receiving an image or video comprising a set of pixels;

extracting a plurality of features from the image or video;

generating a feature vector including the plurality of features each at different index values of the feature vector, each of the plurality of features corresponding to a different pixel or subset of pixels of the set of pixels of the image or video;

executing a neural network using the plurality of features to obtain a performance score for the image or video, the neural network comprising an input layer, a plurality of intermediate layers subsequent to the input layer, and a regression layer or a classification layer subsequent to the plurality of intermediate layers;

extracting one or more values from one or more signals between an intermediate layer of the plurality of intermediate layers and the regression layer, the one or more values generated responsive to executing the neural network using the extracted plurality of features;

back-propagating a set of the extracted one or more values through the plurality of intermediate layers and the input layer;

extracting an impact score for each of the plurality of features from the input layer and generated from the back-propagation of the set of the extracted one or more values through the plurality of intermediate layers and the input layer;

assigning an impact score of the extracted impact scores to each pixel and/or subset of pixels of the set of pixels of the image or video based on the pixel or subset of pixels mapping to the same index value of the feature vector as the impact score;

determining (i) a first plurality of pixels of the image or video that correspond to a first color indicating a positive impact to the performance score, and (ii) a second plurality of pixels of the image or video that correspond to a second color indicating a negative impact to the performance score, based on an assignment of a positive impact score to each of the first plurality of pixels and a negative impact score to each of the second plurality of pixels;

generating a heat map overlay comprising the first color at locations of the image or video corresponding to the first plurality of pixels and the second color at locations of the image or video corresponding to the second plurality of pixels; and

presenting, at a display of a client device, the heat map overlay over the image or video.

16. The non-transitory computer-readable storage medium of claim 15 , further comprising:

generating, by the one or more processors, an evaluation image or video by placing the heat map overlay over the image or video.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the method comprises:

determining whether each of the plurality of features had a positive impact on the performance score or a negative impact on the performance score.

Assignments (1)

Continuity (2)

References Cited (50)