Controlling an automated vehicle using visual anchors

1. A method for controlling an automated vehicle using visual anchors, comprising:

receiving, from one or more cameras of an autonomous vehicle, first video data;

identifying one or more visual anchors in the first video data;

determining one or more predicted visual anchors based on sensor data comprising second video data and data from one or more non-camera sensors, wherein the one or more predicted visual anchors comprises a predicted location of the one or more visual anchors, wherein the second video data occurs before the first video data;

determining one or more differentials between the one or more visual anchors and the one or more predicted visual anchors;

determining, based on the one or more differentials, one or more control operations for the autonomous vehicle to reduce the one or more differentials; and

applying the one or more control operations.

2. The method of claim 1 , wherein the one or more differentials comprise one or more pixel width differentials, one or more pixel height differentials, one or more pixel area differentials, or one or more vector differentials.

3. The method of claim 1 , wherein determining the one or more predicted visual anchors comprises providing the sensor data as input to a machine learning model.

4. The method of claim 1 , wherein the sensor data comprises sensor data received within a time window ending at a time offset relative to a current time.

5. An apparatus for controlling an automated vehicle using visual anchors, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out steps comprising:

receiving, from one or more cameras of an autonomous vehicle, first video data;

identifying one or more visual anchors in the first video data;

determining one or more predicted visual anchors based on sensor data comprising second video data and data from one or more non-camera sensors, wherein the one or more predicted visual anchors comprises a predicted location of the one or more visual anchors, wherein the second video data occurs before the first video data;

determining one or more differentials between the one or more visual anchors and the one or more predicted visual anchors;

determining, based on the one or more differentials, one or more control operations for the autonomous vehicle to reduce the one or more differentials; and

applying the one or more control operations.

6. The apparatus of claim 5 , wherein the one or more differentials comprise one or more pixel width differentials, one or more pixel height differentials, one or more pixel area differentials, or one or more vector differentials.

7. The apparatus of claim 5 , wherein determining the one or more predicted visual anchors comprises providing the sensor data as input to a machine learning model.

8. The apparatus of claim 5 , wherein the sensor data comprises sensor data received within a time window ending at a time offset relative to a current time.

9. An autonomous vehicle for detecting out-of-model scenarios for an autonomous vehicle, comprising:

an apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out steps comprising:

receiving, from one or more cameras of an autonomous vehicle, first video data;

identifying one or more visual anchors in the first video data;

determining one or more predicted visual anchors based on sensor data comprising second video data and data from one or more non-camera sensors, wherein the one or more predicted visual anchors comprises a predicted location of the one or more visual anchors, wherein the second video data occurs before the first video data;

determining one or more differentials between the one or more visual anchors and the one or more predicted visual anchors;

determining, based on the one or more differentials, one or more control operations for the autonomous vehicle to reduce the one or more differentials; and

applying the one or more control operations.

10. The autonomous vehicle of claim 9 , wherein the one or more differentials comprise one or more pixel width differentials, one or more pixel height differentials, one or more pixel area differentials, or one or more vector differentials.

11. The autonomous vehicle of claim 9 , wherein determining the one or more predicted visual anchors comprises providing the sensor data as input to a machine learning model.

12. The autonomous vehicle of claim 9 , wherein the sensor data comprises sensor data received within a time window ending at a time offset relative to a current time.

13. A computer program product disposed upon a non-transitory computer readable medium, the computer program product comprising computer program instructions for detecting out-of-model scenarios for an autonomous vehicle that, when executed, cause a computer system of the autonomous vehicle to perform a method comprising:

receiving, from one or more cameras of an autonomous vehicle, first video data;

identifying one or more visual anchors in the first video data;

determining one or more predicted visual anchors based on sensor data comprising second video data and data from one or more non-camera sensors, wherein the one or more predicted visual anchors comprises a predicted location of the one or more visual anchors, wherein the second video data occurs before the first video data;

determining one or more differentials between the one or more visual anchors and the one or more predicted visual anchors;

determining, based on the one or more differentials, one or more control operations for the autonomous vehicle to reduce the one or more differentials; and

applying the one or more control operations.