Learned validation metric for evaluating autonomous vehicle motion planning performance

1. A computer-implemented method for validating a trajectory generated by an autonomous vehicle control system (“the AV trajectory”) in a driving scenario, comprising:

(a) obtaining the AV trajectory and a reference trajectory, wherein the reference trajectory describes a desired motion of a vehicle in the driving scenario;

(b) determining a plurality of component divergence values for a plurality of divergence metrics, wherein a respective divergence value characterizes a respective difference between the AV trajectory and the reference trajectory;

(c) providing the plurality of component divergence values to a machine-learned model to generate a score that indicates an aggregate divergence between the AV trajectory and the reference trajectory, wherein the machine-learned model comprises a plurality of learned parameters defining an influence of the plurality of component divergence values on the score; and

(d) validating the AV trajectory based on the score.

2. The computer-implemented method of claim 1 , further comprising, prior to (c), determining the plurality of learned parameters based on a plurality of training examples, wherein a respective training example comprises (i) a training trajectory and a reference trajectory and (ii) a label indicating a validation state between the training trajectory and the reference trajectory.

3. The computer-implemented method of claim 2 , wherein the label indicates whether there is a material divergence between the training trajectory and the reference trajectory.

4. The computer-implemented method of claim 1 , wherein the plurality of learned parameters respectively correspond to the plurality of divergence metrics.

5. The computer-implemented method of claim 1 , comprising:

weighting a respective contribution of the respective component divergence value using a context value obtained using a context metric, wherein the context value indicates a relationship between the AV trajectory and an environment of the driving scenario.

6. The computer-implemented method of claim 5 , comprising:

determining, using the context metric and based on an attribute of the AV trajectory or the reference trajectory, a context domain for the respective component divergence value; and

weighting the respective contribution of the respective component divergence value based on a weighting parameter associated with the context domain.

7. The computer-implemented method of claim 5 , wherein the context metric maps a plurality of context domains over a decision tree.

8. The computer-implemented method of claim 5 , wherein the context metric measures an interval between:

a validation time; and

a time associated with the respective component divergence value.

9. The computer-implemented method of claim 1 , wherein the score comprises a weighted combination of the plurality of component divergence values.

10. The computer-implemented method of claim 9 , wherein the weighted combination is a linear combination.

11. The computer-implemented method of claim 1 , wherein at least one divergence metric corresponds to a divergence encoder that is configured to generate, based on the AV trajectory and using one or more machine-learned parameters, the respective component divergence.

12. The computer-implemented method of claim 1 , wherein validating the AV trajectory comprises comparing the score to a threshold.

13. The computer-implemented method of claim 1 , wherein the machine-learned model is optimized based on constraints indicated by a curated set of AV trajectories and reference trajectories.

14. The computer-implemented method of claim 13 , wherein:

the curated set of trajectory matches comprises positive match examples and negative match examples; and

the machine-learned model defines a decision boundary for classifying the positive match examples and the negative match examples.

15. The computer-implemented method of claim 14 , wherein the machine-learned model comprises a support vector machine.

16. The computer-implemented method of claim 14 , wherein:

a positive match example is an example in which a first aggregate divergence between a first respective AV trajectory and a first respective reference trajectory is below a threshold; and

a negative match example is an example in which a second aggregate divergence between a second respective AV trajectory and a second respective reference trajectory is above the threshold.

17. The computer-implemented method of claim 1 , wherein the AV trajectory describes a planned trajectory output by a motion planner of the autonomous vehicle control system operating in open loop.

18. The computer-implemented method of claim 1 , wherein the AV trajectory describes executed behavior of an autonomous vehicle controlled by the autonomous vehicle control system operating in closed loop.

19. One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations for validating a trajectory generated by an autonomous vehicle control system (“the AV trajectory”) in a driving scenario, the operations comprising:

(a) obtaining the AV trajectory and a reference trajectory, wherein the reference trajectory describes a desired motion of a vehicle in the driving scenario;

(b) determining a plurality of component divergence values for a plurality of divergence metrics, wherein a respective divergence value characterizes a respective difference between the AV trajectory and the reference trajectory;

(c) providing the plurality of component divergence values to a machine-learned model to generate a score that indicates an aggregate divergence between the AV trajectory and the reference trajectory, wherein the machine-learned model comprises a plurality of learned parameters defining an influence of the plurality of component divergence values on the score; and

(d) validating the AV trajectory based on the score.

20. A computing system, comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that are executable by the one or more processors to cause the computing system to perform operations for validating a trajectory generated by an autonomous vehicle control system (“the AV trajectory”) in a driving scenario, the operations comprising:

(a) obtaining the AV trajectory and a reference trajectory, wherein the reference trajectory describes a desired motion of a vehicle in the driving scenario;

(b) determining a plurality of component divergence values for a plurality of divergence metrics, wherein a respective divergence value characterizes a respective difference between the AV trajectory and the reference trajectory;

(c) providing the plurality of component divergence values to a machine-learned model to generate a score that indicates an aggregate divergence between the AV trajectory and the reference trajectory, wherein the machine-learned model comprises a plurality of learned parameters defining an influence of the plurality of component divergence values on the score; and

(d) validating the AV trajectory based on the score.