Efficiently constructing regression models for selectivity estimation
A model generator constructs a model for estimating selectivity of database operations by determining a number of training examples necessary for the model to achieve a target accuracy and by generating approximate selectivity labels for the training examples. The model generator may train the model on an initial number of training examples using cross-validation. The model generator may determine whether the model satisfies the target accuracy and iteratively and geometrically increase the number of training examples based on an optimized geometric step size (which may minimize model construction time) until the model achieves the target accuracy based on a defined confidence level. The model generator may generate labels using a subset of tuples from an intermediate query expression. The model generator may iteratively increase a size of the subset of tuples used until a relative error of the generated labels is below a target threshold.
1. A system comprising:
one or more processors;
memory in electronic communication with the one or more processors; and
instructions stored in the memory, the instructions being executable by the one or more processors to:
train a model on a set of labeled training examples using cross-validation, the set of labeled training examples having a size;
determine that the model does not satisfy an accuracy target;
determine a geometric step size for increasing the size of the set of labeled training examples;
increase the size of the set of labeled training examples to a new size based on the geometric step size;
generate labels for the additional training examples needed to increase the size of the set of labeled training examples to the new size;
add the additional training examples and the generated labels to the set of labeled training examples; and
train the model on the set of labeled training examples using cross-validation.
2. The system of claim 1 , wherein the instructions are further executable by the one or more processors to determine the geometric step size by:
determining a labeling cost, wherein the labeling cost is a time required to compute a label for one example;
determining a cross-validation cost, wherein the cross-validation cost is a time required to run a cross-validation for one example;
computing a ratio of the labeling cost to the cross-validation cost; and
determining the geometric step size based on the ratio.
3. The system of claim 1 , wherein determining that the model does not satisfy the accuracy target comprises:
computing a confidence interval for the model, wherein the confidence interval has a lower bound and an upper bound and represents that, with a confidence significance level, a percentile of predictions of the model that have a q-error less than a target q-error is between the lower bound and the upper bound; and
determining that a target percentile is between the lower bound and the upper bound.
4. The system of claim 3 , wherein generating the labels for the additional training examples comprises:
obtaining a set of random tuples from an intermediate query expression, wherein the set of random tuples is less than a total number of tuples from the intermediate query expression;
determining approximate labels for the additional training examples based on the set of random tuples;
determining confidence probabilities for the additional training examples to achieve an error threshold;
identifying one or more examples of the additional training examples that have a confidence probability above a probability threshold;
obtaining a second set of random tuples from the intermediate query expression, wherein the second set of random tuples does not overlap with the set of random tuples; and
updating the approximate labels for the one or more examples of the additional training examples.
5. The system of claim 4 , wherein the approximate labels are approximate selectivity labels.
6. The system of claim 1 , wherein determining that the model does not satisfy the accuracy target comprises:
determining that a target percentile is less than or equal to a lower bound of a confidence interval.
7. The system of claim 6 , wherein the confidence interval is a percentile of predictions of the model that have a q-error less than a target q-error.
8. The system of claim 1 , wherein the geometric step size minimizes a total time spent on model construction.
9. The system of claim 1 , wherein the instructions being further executable by the one or more processors to:
iteratively increase the size of the set of labeled training examples, wherein the size of the set of labeled training examples is less than or equal to a maximum size of labeled training examples.
10. The system of claim 1 , wherein generating the labels for the additional training examples comprises:
obtaining a set of random tuples from an intermediate query expression, wherein the set of random tuples is less than a total number of tuples from the intermediate query expression;
determining approximate labels for the additional training examples based on the set of random tuples; and
updating the approximate labels for the one or more examples of the additional training examples.
11. A method, comprising:
training a model on a set of labeled training examples using cross-validation, the set of labeled training examples having a size;
determining that the model does not satisfy an accuracy target;
determining a geometric step size for increasing the size of the set of labeled training examples;
increasing the size of the set of labeled training examples to a new size based on the geometric step;
generating labels for additional training examples needed to increase the size of the set of labeled training examples to the new size;
adding the additional training examples and the generated labels to the set of labeled training examples; and
training the model on the set of labeled training examples using cross-validation.
12. The method of claim 11 , wherein determining the geometric step size further comprises:
determining a labeling cost, wherein the labeling cost is a time required to compute a label for one example;
determining a cross-validation cost, wherein the cross-validation cost is a time required to run a cross-validation for one example;
computing a ratio of the labeling cost to the cross-validation cost; and
determining the geometric step size based on the ratio.
13. The method of claim 11 , wherein determining that the model does not satisfy the accuracy target comprises:
computing a confidence interval for the model, wherein the confidence interval has a lower bound and an upper bound and represents that, with a confidence significance level, a percentile of predictions of the model that have a q-error less than a target q-error is between the lower bound and the upper bound; and
determining that a target percentile is between the lower bound and the upper bound.
14. The method of claim 13 , wherein generating the labels for the additional training examples comprises:
obtaining a set of random tuples from an intermediate query expression, wherein the set of random tuples is less than a total number of tuples from the intermediate query expression;
determining approximate labels for the additional training examples based on the set of random tuples;
determining confidence probabilities for the additional training examples to achieve an error threshold;
identifying one or more examples of the additional training examples that have a confidence probability above a probability threshold;
obtaining a second set of random tuples from the intermediate query expression, wherein the second set of random tuples does not overlap with the set of random tuples; and
updating the approximate labels for the one or more examples of the additional training examples.
15. The method of claim 14 , wherein the approximate labels are approximate selectivity labels.
16. The method of claim 11 , wherein determining that the model does not satisfy the accuracy target comprises:
determining that a target percentile is less than or equal to a lower bound of a confidence interval.
17. The method of claim 16 , wherein the confidence interval is a percentile of predictions of the model that have a q-error less than a target q-error.
18. The method of claim 11 , wherein the geometric step size minimizes a total time spent on model construction.
19. The method of claim 11 , wherein the instructions being further executable by the one or more processors to:
iteratively increase the size of the set of labeled training examples, wherein the size of the set of labeled training examples is less than or equal to a maximum size of labeled training examples.
20. The method of claim 11 , wherein generating the labels for the additional training examples comprises:
obtaining a set of random tuples from an intermediate query expression, wherein the set of random tuples is less than a total number of tuples from the intermediate query expression;
determining approximate labels for the additional training examples based on the set of random tuples; and
updating the approximate labels for the one or more examples of the additional training examples.