Using divergence to conduct log-based simulations

1. A method of testing software for operating a vehicle in an autonomous driving mode, the method comprising:

running a first simulation using log data collected by a vehicle operating in an autonomous driving mode, wherein the first simulation is run using the software to control a first simulated vehicle;

comparing one or more characteristics of the first simulated vehicle with one or more characteristics of the vehicle from the log data in order to determine a divergence point;

using the divergence point to determine a handover time for a second simulation, the handover time corresponding to a time in the second simulation at which the software is allowed to take control of a second simulated vehicle of the second simulation;

running the second simulation using the log data, wherein the second simulation is run using the handover time and the software to control the second simulated vehicle; and

determining whether the software is able to complete the second simulation without a particular type of event occurring in the second simulation.

2. The method of claim 1 , wherein the particular type of event is a collision, and the method further comprises, when the collision is determined to have occurred, flagging the second simulation for further review.

3. The method of claim 1 , wherein determining the divergence point includes comparing a planned trajectory of the first simulated vehicle with a planned trajectory of the vehicle from the log data.

4. The method of claim 3 , wherein determining the divergence point includes determining when one or more of a location, speed or change in speed of the planned trajectory of the first simulated vehicle and the planned trajectory of the vehicle from the log data diverge more than some threshold amount over some period of time.

5. The method of claim 1 , wherein determining the divergence point includes determining when a location of the first simulated vehicle and a location of the vehicle from the log data diverge more than a threshold amount.

6. The method of claim 1 , wherein determining the divergence point includes determining when a location of the first simulated vehicle and a location of the vehicle from the log data diverge more than a threshold amount in a lateral direction relative to a direction of traffic in a lane in which the simulated vehicle is traveling in the first simulation.

7. The method of claim 1 , wherein determining the divergence point includes determining when a location of the first simulated vehicle and a location of the vehicle from the log data diverge more than a threshold amount in a longitudinal direction relative to a direction of traffic in a lane in which the simulated vehicle is traveling in the first simulation.

8. The method of claim 7 , wherein determining the divergence point includes determining when the location of the first simulated vehicle and the location of the vehicle from the log data diverge more than a threshold amount in a lateral direction relative to the direction of traffic in the lane.

9. The method of claim 1 , further comprising:

at the divergence point in the first simulation, starting a timer that expires during the first simulation; and

flagging the first simulation for review when the particular type of event occurs before the timer expires.

10. The method of claim 9 , further comprising, not flagging the first simulation for review if the particular type of event only occurs during the first simulation after the timer expires.

11. The method of claim 9 , wherein the particular type of event is a collision between the first simulated vehicle and an object in the first simulation.

12. The method of claim 9 , wherein the particular type of event includes the simulated vehicle exhibiting a particular type of maneuvering behavior.

13. A method of testing software for operating a vehicle in an autonomous driving mode, the method comprising:

running a simulation using log data collected by a vehicle operating in an autonomous driving mode, wherein the simulation is run using the software to control a simulated vehicle;

comparing one or more characteristics of the simulated vehicle with one or more characteristics of the vehicle from the log data in order to determine a divergence point;

at the divergence point in the simulation, starting a timer that expires during the simulation; and

determining whether the software is able to continue through the simulation without a particular event occurring in the simulation prior to the timer expiring.

14. The method of claim 13 , wherein the particular event is a collision between the simulated vehicle and an object in the simulation.

15. The method of claim 13 , further comprising, flagging the simulation for review if the particular event occurs before the timer expires.

16. The method of claim 13 , further comprising, not flagging the simulation for review if the particular event only occurs during the simulation after the timer expires.

17. The method of claim 15 , wherein the particular event is a collision between the simulated vehicle and an object in the simulation.

18. The method of claim 13 , wherein the particular event includes the simulated vehicle exhibiting a particular type of maneuvering behavior.

19. The method of claim 13 , wherein determining the divergence point includes determining when a location of the simulated vehicle and a location of the vehicle from the log data diverge more than a threshold amount.

20. The method of claim 13 , wherein determining the divergence point includes determining when a location of the simulated vehicle and a location of the vehicle from the log data diverge more than a threshold amount in at least one of a lateral direction or a longitudinal direction relative to a direction of traffic in a lane in which the simulated vehicle is traveling in the simulation.