Video normalization for an autonomous vehicle

1 . A method, comprising:

receiving, from one or more cameras of an autonomous vehicle, video data;

applying a color normalization to the video data by:

converting the video data to a lighting-invariant color space including

determining, for one or more pixels of the video data, a relative lighting change; and

scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination;

providing the color normalized video data to a machine learning model; and

training the machine learning model to determine control operations for the autonomous vehicle based on the color normalized video data.

2 . The method of claim 1 , further comprising applying, based on a registration point, an image stabilization to the video data.

3 . The method of claim 2 , wherein the registration point comprises a vanishing point.

4 . The method of claim 3 , further comprising determining, based on an inertial measurement of the autonomous vehicle, the vanishing point.

5 . The method of claim 2 , wherein applying the image stabilization comprises centering the video data on the registration point.

6 . The method of claim 1 , further comprising applying a respective spherical reprojection to each frame of the video data.

7 . A method, comprising:

receiving, from one or more cameras of an autonomous vehicle, video data;

processing the video data by applying a respective spherical reprojection to each frame of the video data;

scaling, for one or more pixels of the video data, a relative lighting change relative to one or more of a local illumination or a total illumination;

providing the processed video data to a machine learning model; and

training the machine learning model to determine control operations for the autonomous vehicle based on the processed video data.

8 . The method of claim 7 , further comprising applying a color normalization to the video data.

9 . The method of claim 8 , wherein applying the color normalization to the video data comprises:

converting the video data to a lighting-invariant color space;

determining, for one or more pixels of the video data, a relative lighting change; and

scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination.

10 . The method of claim 7 , further comprising applying, based on a registration point, an image stabilization to the video data.

11 . The method of claim 10 , wherein the registration point comprises a vanishing point.

12 . The method of claim 11 , further comprising determining, based on an inertial measurement of the autonomous vehicle, the vanishing point.

13 . The method of claim 10 , wherein applying the image stabilization comprises centering the video data on the registration point.

14 . A method, comprising:

receiving, from one or more cameras of an autonomous vehicle, video data;

processing the video data by applying, based on a registration point, an image stabilization to the video data;

scaling, for one or more pixels of the video data, a relative lighting change relative to one or more of a local illumination or a total illumination;

providing the processed video data to a machine learning model; and

training the machine learning model to determine control operations for the autonomous vehicle based on the processed video data.

15 . The method of claim 14 , wherein the registration point comprises a vanishing point.

16 . The method of claim 15 , further comprising determining, based on an inertial measurement of the autonomous vehicle, the vanishing point.

17 . The method of claim 14 , wherein applying the image stabilization comprises centering the video data on the registration point.

18 . The method of claim 14 , further comprising applying a color normalization to the video data.

19 . The method of claim 18 , wherein applying the color normalization to the video data comprises:

converting the video data to a lighting-invariant color space;

determining, for one or more pixels of the video data, a relative lighting change; and

scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination.

20 . The method of claim 14 , further comprising applying a respective spherical reprojection to each frame of the video data.