CHARACTERIZING MODEL PERFORMANCE USING HIERARCHICAL FEATURE GROUPS
The disclosed embodiments provide a system for processing data. During operation, the system uses a hierarchical structure of features inputted into a statistical model to obtain a set of groups of the features. Next, the system uses the groups as input to a set of view models for estimating an output of the statistical model. The system then applies the view models to the features to generate a set of view model outputs, wherein each view model output in the set of view model outputs represents an effect of a group in the set of groups on an output of the statistical model. Finally, the system outputs the view model outputs for use in characterizing a performance of the statistical model.
1 . A method, comprising:
using a hierarchical structure of features inputted into a statistical model to obtain a set of groups of the features;
using the groups as input to a set of view models for estimating an output of the statistical model;
applying, by one or more computer systems, the view models to the features to generate a set of view model outputs, wherein each view model output in the set of view model outputs represents an effect of a group in the set of groups on the output of the statistical model; and
outputting the view model outputs for use in characterizing a performance of the statistical model.
2 . The method of claim 1 , further comprising:
aggregating the view model outputs as input to a secondary statistical model for estimating the output of the statistical model; and
using one or more attributes of the secondary statistical model to further characterize the effect of the groups on the output of the statistical model.
3 . The method of claim 2 , further comprising:
when a difference between a secondary output of the secondary statistical model and the output of the statistical model exceeds a threshold, adjusting one or more of the view model outputs to compensate for the difference.
4 . The method of claim 2 , wherein the one or more attributes comprise a set of weights associated with the set of groups.
5 . The method of claim 1 , further comprising:
generating the hierarchical structure of features.
6 . The method of claim 5 , wherein the hierarchical structure is generated based on correlations among the features.
7 . The method of claim 6 , wherein the hierarchical structure is generated to increase correlations of features within a group and decrease correlations among the groups.
8 . The method of claim 5 , wherein the hierarchical structure is generated based on semantic groupings of the features.
9 . The method of claim 1 , wherein using the hierarchical structure to obtain the set of groups of the features comprises:
selecting a level of granularity associated with the hierarchical structure; and
using the level of granularity to obtain the set of groups of the features.
10 . The method of claim 1 , further comprising:
using a set of attributes of the view models to further characterize an effect of the features on the output of the statistical model.
11 . The method of claim 1 , wherein outputting the view model outputs for use in characterizing the performance of the statistical model comprises:
displaying a visualization comprising representations of the groups; and
adjusting, in the visualization, the representations to reflect the view model outputs.
12 . An apparatus, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
use a hierarchical structure of features inputted into a statistical model to obtain a set of groups of the features;
use the groups as input to a set of view models for estimating an output of the statistical model;
apply the view models to the features to generate a set of view model outputs, wherein each view model output in the set of view model outputs represents an effect of a group in the set of groups on the output of the statistical model; and
output the view model outputs for use in characterizing a performance of the statistical model.
13 . The apparatus of claim 12 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
aggregate the view model outputs as input to a secondary statistical model for estimating the output of the statistical model; and
use one or more attributes of the secondary statistical model to further characterize the effect of the groups on the output of the statistical model.
14 . The apparatus of claim 13 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
when a difference between a secondary output of the secondary statistical model and the output of the statistical model exceeds a threshold, adjust one or more of the view model outputs to compensate for the difference.
15 . The apparatus of claim 12 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
generate the hierarchical structure of features.
16 . The apparatus of claim 15 , wherein the hierarchical structure is generated based on at least one of:
correlations among the features; and
semantic groupings of the features.
17 . The apparatus of claim 12 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
use a set of attributes of the view models to further characterize an effect of the features on the output of the statistical model.
18 . The apparatus of claim 12 , wherein using the hierarchical structure to obtain the set of groups of the features comprises:
selecting a level of granularity associated with the hierarchical structure; and
using the level of granularity to obtain the set of groups of the features.
19 . A system, comprising:
an analysis module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to:
use a hierarchical structure of features used inputted into a statistical model to obtain a set of groups of the features;
use the groups to as input to a set of view models for estimating an output of the statistical model; and
apply the view models to the features to generate a set of view model outputs, wherein each view model output in the set of view model outputs represents an effect of a group in the set of groups on the output of the statistical model; and
a management module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to output the view model outputs for use in characterizing a performance of the statistical model.
20 . The system of claim 19 , wherein the non-transitory computer-readable medium of the analysis module further stores instructions that, when executed, cause the system to:
aggregate the view model outputs as input to a secondary statistical model for estimating the output of the statistical model; and
use one or more attributes of the secondary statistical model to further characterize the effect of the groups on the output of the statistical model.