Video normalization for an autonomous vehicle
Video normalization for an autonomous vehicle, including: receiving, from one or more cameras of an autonomous vehicle, video data; applying a color normalization to the video data by: converting the video data to a lighting-invariant color space; determining, for one or more pixels of the video data, a relative lighting change; and scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination; and providing the video data to a machine learning model configured to determine control operations for the autonomous vehicle.
1 . A method, comprising:
receiving, from one or more cameras of an autonomous vehicle, video data;
applying a color normalization to the video data by:
converting the video data to a lighting-invariant color space including
determining, for one or more pixels of the video data, a relative lighting change; and
scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination;
providing the color normalized video data to a machine learning model; and
training the machine learning model to determine control operations for the autonomous vehicle based on the color normalized video data.
2 . The method of claim 1 , further comprising applying, based on a registration point, an image stabilization to the video data.
3 . The method of claim 2 , wherein the registration point comprises a vanishing point.
4 . The method of claim 3 , further comprising determining, based on an inertial measurement of the autonomous vehicle, the vanishing point.
5 . The method of claim 2 , wherein applying the image stabilization comprises centering the video data on the registration point.
6 . The method of claim 1 , further comprising applying a respective spherical reprojection to each frame of the video data.
7 . A method, comprising:
receiving, from one or more cameras of an autonomous vehicle, video data;
processing the video data by applying a respective spherical reprojection to each frame of the video data;
scaling, for one or more pixels of the video data, a relative lighting change relative to one or more of a local illumination or a total illumination;
providing the processed video data to a machine learning model; and
training the machine learning model to determine control operations for the autonomous vehicle based on the processed video data.
8 . The method of claim 7 , further comprising applying a color normalization to the video data.
9 . The method of claim 8 , wherein applying the color normalization to the video data comprises:
converting the video data to a lighting-invariant color space;
determining, for one or more pixels of the video data, a relative lighting change; and
scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination.
10 . The method of claim 7 , further comprising applying, based on a registration point, an image stabilization to the video data.
11 . The method of claim 10 , wherein the registration point comprises a vanishing point.
12 . The method of claim 11 , further comprising determining, based on an inertial measurement of the autonomous vehicle, the vanishing point.
13 . The method of claim 10 , wherein applying the image stabilization comprises centering the video data on the registration point.
14 . A method, comprising:
receiving, from one or more cameras of an autonomous vehicle, video data;
processing the video data by applying, based on a registration point, an image stabilization to the video data;
scaling, for one or more pixels of the video data, a relative lighting change relative to one or more of a local illumination or a total illumination;
providing the processed video data to a machine learning model; and
training the machine learning model to determine control operations for the autonomous vehicle based on the processed video data.
15 . The method of claim 14 , wherein the registration point comprises a vanishing point.
16 . The method of claim 15 , further comprising determining, based on an inertial measurement of the autonomous vehicle, the vanishing point.
17 . The method of claim 14 , wherein applying the image stabilization comprises centering the video data on the registration point.
18 . The method of claim 14 , further comprising applying a color normalization to the video data.
19 . The method of claim 18 , wherein applying the color normalization to the video data comprises:
converting the video data to a lighting-invariant color space;
determining, for one or more pixels of the video data, a relative lighting change; and
scaling, for the one or more pixels, the relative lighting change relative to one or more of a local illumination or a total illumination.
20 . The method of claim 14 , further comprising applying a respective spherical reprojection to each frame of the video data.