Method of machine-learned verification and advance notice oracles for autonomous systems

1 . A method of training and verification of an autonomous system, comprising:

by a machine learning classifier, wherein an adversary vehicle, and a primary vehicle are on a same path;

creating a path position probability transition matrix, wherein the path position probability transition matrix comprises:

an acceleration parameters array of one or more acceleration parameters for a potential position, and

equal probabilities for creating a samples array, evaluating a sample path, sorting the sample path, and using an N e elite path;

creating the samples array, wherein the samples array is one or more paths, wherein the one or more paths are a sequence of grid points of a two-dimensional grid, wherein the sequence of grid points comprise: a sampled position and a sampled acceleration, wherein the sampled position is from a probability distribution of the path position probability transition matrix, and wherein the sampled acceleration is from an acceleration probability distribution associated with the sampled position;

evaluating the sample path, wherein a custom score function asserts the potential position and a potential acceleration to identify a collision path of a perturbation-adversary and the primary vehicle, a no collision path of the adversary vehicle and the primary vehicle, a position distance and an acceleration distance between the adversary vehicle and the primary vehicle;

sorting the sample path, based at least in part a custom score of a custom score function and selecting a subset of the sample path as the N e elite path;

using the N e elite paths to update the path position probability transition matrix and the acceleration parameters array;

repeating all steps until the N e elite path stabilizes; and

wherein a high-variance dataset trains a machine learning model;

wherein the custom score function prioritizes a high priority sample path as a deviation from the no collision path that results in a collision in the N e elite paths; and

wherein the custom score function prioritizes successively the collision path or the no collision paths to generate the high variance dataset.

2 . The method of claim 1 , further comprising:

updating the path position probability transition matrix by the number of times the N e elite paths transitions from location i to location j, divided by N 2 ; and

updating Gaussian parameters pairs of the sampled acceleration parameters array as follows:

μ

i

=

1

N

e

⁢

∑

K

=

0

N

e

⁢

accek

[

k

,

i

]

⁢

σ

i

2

=

1

N

e

⁢

∑

k

=

0

N

e

⁢

(

μ

i

-

accel

[

k

,

i

]

)

2

wherein accel[k, i] is the sampled acceleration in cell i of the two-dimensional grid according to a k elite path.

3 . The method of claim 1 , further comprising an independent adversary vehicle, wherein the independent adversary vehicle, a vanilla adversary vehicle, the adversary vehicle, and the primary vehicle are on the same path; and

wherein the custom score function asserts the potential position and the potential acceleration to identify a no collision path between the adversary vehicle and the independent adversary vehicle, and the position distance and the acceleration distance between the adversary vehicle and independent adversary vehicle.

4 . The method of claim 3 , wherein the sample path of the independent adversary vehicle and the primary vehicle are configured not to collide.

5 . The method of claim 1 , further comprising:

simulating scenarios for one or more autonomous vehicle reporting databases.

6 . The method of claim 1 , further comprising:

adding one or more random sample paths the high-variance dataset; and

wherein the one or more random sample paths represent one or more naturalistic paths.

7 . The method of claim 1 , wherein one or more different collision paths and the no collision paths contribute to model robustness.

8 . A system for training, verifying, and providing advance notice, the system comprising a non-transitory computer readable medium storing instruction executable by a processor, wherein the instructions comprise instructions to:

by a machine learning classifier, wherein an adversary vehicle, and a primary vehicle are on a same path;

creating a path position probability transition matrix, wherein the path position probability transition matrix comprises:

an acceleration parameters array of one or more acceleration parameters for a potential position, and

equal probabilities for creating a samples array, evaluating a sample path, sorting the sample path, and using an N e elite path;

creating the samples array, wherein the samples array is one or more paths, wherein the one or more paths are a sequence of grid points of a two-dimensional grid, wherein the sequence of grid points comprise: a sampled position and a sampled acceleration, wherein the sampled position is from a probability distribution of the path position probability transition matrix, and wherein the sampled acceleration is from an acceleration probability distribution associated with the sampled position;

evaluating the sample path, wherein a custom score function asserts the potential position and a potential acceleration to identify a collision path of a perturbation-adversary and the primary vehicle, a no collision path of the adversary vehicle and the primary vehicle, a position distance and an acceleration distance between the adversary vehicle and the primary vehicle;

sorting the sample path, based at least in part a custom score of a custom score function and selecting a subset of the sample path as the N e elite path;

using the N e elite paths to update the path position probability transition matrix and the acceleration parameters array;

repeating all steps until the N e elite path stabilizes; and

wherein a high-variance dataset trains a machine learning model;

wherein the custom score function prioritizes a high priority sample path as a deviation from the no collision path that results in a collision in the N e elite paths; and

wherein the custom score function prioritizes successively the collision path or the no collision paths to generate the high variance dataset.

9 . The system of claim 8 , the instructions further comprising:

updating the path position probability transition matrix by the number of times the N e elite paths transitions from location i to location j, divided by N 2 ; and

updating Gaussian parameters pairs of the sampled acceleration parameters array as follows:

μ

i

=

1

N

e

⁢

∑

K

=

0

N

e

⁢

accek

[

k

,

i

]

⁢

σ

i

2

=

1

N

e

⁢

∑

k

=

0

N

e

⁢

(

μ

i

-

accel

[

k

,

i

]

)

2

wherein accel[k, i] is the sampled acceleration in cell i of the two-dimensional grid according to a k elite path.

10 . The system of claim 8 , further comprising an independent adversary vehicle, wherein the independent adversary vehicle, the primary vehicle, and the adversary vehicle are on the same path; and

wherein the custom score function asserts the potential position and the potential acceleration to identify a no collision path between adversary and independent adversary vehicles, and the position distance and the acceleration distance between adversary vehicle and independent adversary vehicle.

11 . The system of claim 10 , wherein a sample path of the independent-adversary and the primary vehicle are configured not to collide.

12 . The system of claim 8 , wherein one or more autonomous vehicle reporting databases are configured to simulate a scenario.

13 . The system of claim 8 , further comprising:

adding one or more random sample paths to the high-variance dataset; and

wherein the one or more random sample paths represent one or more naturalistic paths.

14 . The system of claim 8 , wherein one or more different collision paths and the no collision paths contribute to model robustness.

15 . A Hybrid-Pair-Cross Entropy method of training and verification for an autonomous system, the method comprising:

by an adversary vehicle and a primary vehicle, wherein the adversary vehicle, and the primary vehicle are on a same path;

creating a path position probability transition matrix, wherein the path position probability transition matrix comprises:

an acceleration parameters array of one or more acceleration parameters for a potential position, and

equal probabilities for creating a samples array, evaluating a sample path, sorting the sample path, and using an N e elite path;

creating the samples array, wherein the samples array is one or more paths, wherein the one or more paths are a sequence of grid points of a two-dimensional grid, wherein the sequence of grid points comprise: a sampled position and a sampled acceleration, wherein the sampled position is from a probability distribution of the path position probability transition matrix, and wherein the sampled acceleration is from an acceleration probability distribution associated with the sampled position;

evaluating the sample path, wherein a custom score function asserts a potential position and a potential acceleration to identify a collision path of a perturbation-adversary and the primary vehicle, a no collision path of adversary vehicle and the primary vehicle, a position distance and an acceleration distance between adversary and primary vehicle, wherein the custom score function prioritizes a high priority sample path as a deviation from a non-colliding path that result in a collision in an N e elite paths, wherein the custom score function prioritizes successively different the collision path or the no collision paths to generate a high-variance dataset, wherein the high-variance data set trains a machine learning model;

sorting the sample path based on a corresponding score from the custom score function and selecting a subset of the sample path as the N e elite path;

using the N e elite paths to update the path position probability transition matrix and the acceleration parameters array;

repeating the steps above until the N e elite path stabilizes;

updating the path position probability transition matrix by the number of times the N e elite paths transitions from location i to location j, divided by N 2 ;

updating Gaussian parameters pairs of the acceleration parameters array as follows:

μ

i

=

1

N

e

⁢

∑

K

=

0

N

e

⁢

accek

[

k

,

i

]

⁢

σ

i

2

=

1

N

e

⁢

∑

k

=

0

N

e

⁢

(

μ

i

-

accel

[

k

,

i

]

)

2

wherein accel[k, i] is the acceleration in cell i of the two-dimensional grid according to a k elite path;

wherein the high-variance data set trains the machine learning model wherein the custom score function prioritizes a high priority sample path as a deviation from the no collision path that results in a collision in the N e elite paths; and

wherein the custom score function prioritizes successively the collision path or the no collision paths to generate the high variance dataset.

16 . The method of claim 15 , further comprising an independent adversary vehicle, wherein the independent adversary vehicle, the adversary vehicle, and the primary vehicle are on the same path; and

wherein the custom score function asserts the potential position and the potential acceleration to identify a no collision path between the adversary vehicle and independent adversary vehicle, and the position distance and the acceleration distance between the adversary vehicle and the independent adversary vehicle.

17 . The method of claim 16 , wherein a sample path of the independent adversary vehicle and the primary vehicle are configured not to collide.

18 . The method of claim 17 , further comprising:

simulating scenarios for one or more autonomous vehicle reporting databases; and

wherein one or more different collision paths and the no collision paths contribute to model robustness.