Rolling shutter compensation

1 . A method comprising:

causing, by at least one processor, a rolling shutter camera to capture an image of an environment;

determining, by the at least one processor, a first position in the environment, the first position captured in the image; and

determining, by the at least one processor, a second position in the image that corresponds to the first position in the environment, wherein determining the second position comprises:

applying a first approximation that relative movement between the rolling shutter camera and the first position is linear during the capture of the image,

applying a second approximation that a distortion function characterizing the rolling shutter camera is linear, and

based at least on the first approximation and the second approximation, determining a particular time at which light representative of the first position in the environment is detected by the rolling shutter camera,

wherein determining the second position is based at least on the particular time.

2 . The method of claim 1 , wherein the rolling shutter camera is mounted in a vehicle, and wherein determining the second position comprises:

determining movement of the vehicle during the capture of the image; and

determining the relative movement between the rolling shutter camera and the first position based at least on the movement of the vehicle.

3 . The method of claim 1 , wherein the rolling shutter camera comprises a rolling shutter that causes successive pixels of the rolling shutter camera to detect light for the capture of the image, and

wherein determining the second position comprises determining a pixel of the rolling shutter camera that detects the light representative of the first position.

4 . The method of claim 3 , wherein, based at least on operation of the rolling shutter, a first group of pixels arranged in a first row of the rolling shutter camera detects light at a different time than a second ground of pixels arranged in a second row of the rolling shutter camera.

5 . The method of claim 1 , wherein the particular time is determined based at least on one or more of a shutter speed of a rolling shutter of the rolling shutter camera, a projection function of the rolling shutter camera, the distortion function, or the relative movement between the rolling shutter camera and the first position.

6 . The method of claim 1 , wherein the first approximation comprises a linear interpolation between (i) relative positions of the rolling shutter camera and the first position at a first time and (ii) relative positions of the rolling shutter camera and the first position at a second time.

7 . The method of claim 1 , wherein the second approximation comprises a planar expression for distortion as a function of two projected coordinates.

8 . The method of claim 1 , wherein the rolling shutter camera is mounted in a vehicle, and wherein the method comprises:

based at least on determining the second position, performing object recognition, by the at least one processor, on a portion of the image, the portion of the image including the second position in the image;

identifying, by the at least one processor, an object in the environment as a result of the object recognition, the object located at the first position in the environment; and

causing, by the at least one processor, navigation of the vehicle based at least on identifying the object.

9 . The method of claim 1 , wherein the distortion function characterizes effects of a lens of the rolling shutter camera.

10 . The method of claim 1 , comprising determining the second position in parallel with determining a plurality of third positions in the image, the third positions corresponding to respective second positions in the environment,

wherein the parallel determination is free of branching.

11 . The method of claim 1 , wherein determining the second position comprises:

based at least on the first approximation and the second approximation, obtaining a closed-form expression for the particular time, and

determining the particular time using the closed-form expression.

12 . A system, comprising:

a rolling shutter camera configured to be mounted in a vehicle;

at least one processor; and

at least one memory storing instructions thereon that, when executed by the at least one processor, cause the at least one processor to:

cause the rolling shutter camera to capture an image of an environment of the vehicle;

determine a first position in the environment, the first position captured in the image; and

determine a second position in the image that corresponds to the first position in the environment, wherein determining the second position comprises:

applying a first approximation that relative movement between the rolling shutter camera and the first position is linear during capture of the image, wherein the relative movement between the rolling shutter camera and the first position is based at least on relative movement between the vehicle and the first position,

applying a second approximation that a distortion function characterizing the rolling shutter camera is linear, and

based at least on the first approximation and the second approximation, determining a particular time at which light representative of the first position in the environment is detected by the rolling shutter camera,

wherein determining the second position is based at least on the particular time.

13 . The system of claim 12 , wherein the rolling shutter camera comprises a rolling shutter that causes successive pixels of the rolling shutter camera to detect light for the capture of the image, and

wherein determining the second position comprises determining a pixel of the rolling shutter camera that detects the light representative of the first position.

14 . The system of claim 12 , wherein the particular time is determined based at least on one or more of a shutter speed of a rolling shutter of the rolling shutter camera, a projection function of the rolling shutter camera, the distortion function, or the relative movement between the rolling shutter camera and the first position.

15 . The system of claim 12 , wherein the first approximation comprises a linear interpolation between (i) relative positions of the rolling shutter camera and the first position at a first time and (ii) relative positions of the rolling shutter camera and the first position at a second time.

16 . The system of claim 12 , wherein the second approximation comprises a planar expression for distortion as a function of two projected coordinates.

17 . The system of claim 12 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to,

based at least on determining the second position, perform object recognition on a portion of the image, the portion of the image including the second position in the image;

identify an object in the environment as a result of the object recognition, the object located at the first position in the environment; and

cause navigation of the vehicle based at least on identifying the object.

18 . A non-transitory computer-readable medium comprising instructions stored thereon that, when executed by at least one processor, cause the at least one processor to carry out operations comprising:

causing a rolling shutter camera to capture an image of an environment;

determining a first position in the environment, the first position captured in the image; and

determining a second position in the image that corresponds to the first position in the environment, wherein determining the second position comprises:

applying a first approximation that relative movement between the rolling shutter camera and the first position is linear during the capture of the image,

applying a second approximation that a distortion function characterizing the rolling shutter camera is linear, and

based at least on the first approximation and the second approximation, determining a particular time at which light representative of the first position in the environment is detected by the rolling shutter camera, and

wherein determining the second position is based at least on the particular time.

19 . The non-transitory computer-readable medium of claim 18 , wherein the rolling shutter camera is mounted in a vehicle, and wherein determining the second position comprises:

determining movement of the vehicle during capture of the image; and

determining the relative movement between the rolling shutter camera and the first position based at least on the movement of the vehicle.

20 . The non-transitory computer-readable medium of claim 18 , wherein the rolling shutter camera comprises a rolling shutter that causes successive pixels of the rolling shutter camera to detect light for the capture of the image, and

wherein determining the second position comprises determining a pixel of the rolling shutter camera that detects the light representative of the first position.

21 . The non-transitory computer-readable medium of claim 18 , wherein the rolling shutter camera is mounted in a vehicle, and wherein the operations comprise:

based at least on determining the second position, performing object recognition on a portion of the image, the portion of the image including the second position in the image;

identifying an object in the environment as a result of the object recognition, the object located at the first position in the environment; and

causing navigation of the vehicle based at least on identifying the object.