Extrinsic parameter prediction for image sensor(s)

1 . A method for relative pose estimation, comprising:

obtaining a first frame for a first time period, wherein the first frame is associated with a first image sensor;

warping the first frame;

after warping the first frame, processing, with a first sub-model of a machine learning (ML) model, at least the first frame; and

obtaining a relative pose of the first image sensor with respect to an object or a second image sensor, wherein the relative pose of the first image sensor is provided as output from the first sub-model and is based on the processing.

2 . The method of claim 1 , further comprising:

obtaining a second frame for the first time period, wherein the second frame is associated with the second image sensor, wherein:

processing at least the first frame comprises processing, with the first sub-model of the ML model, the first frame and the second frame; and

obtaining the relative pose comprises obtaining the relative pose of the first image sensor with respect to the second image sensor.

3 . The method of claim 2 , wherein:

the ML model is trained to perform first frame prediction based on a first loss function, and

the first loss function is configured to adjust a first loss based on a first error between a first input frame for a second time period associated with the first image sensor and a first output frame for the second time period predicted by the ML model.

4 . The method of claim 1 , wherein:

the first image sensor is associated with the object, and

obtaining the relative pose comprises obtaining the relative pose of the first image sensor with respect to the object.

5 . The method of claim 4 , wherein the object comprises a moving object.

6 . The method of claim 4 , wherein:

the ML model is trained to perform first frame prediction based on a first loss function, and

the first loss function is configured to adjust first loss based on an error between a first input frame, for a second time period, associated with the first image sensor and a first output frame for the second time period predicted by the ML model.

7 . The method of claim 6 , wherein the error between the first input frame and the first output frame is predicted by the ML model based on:

an estimated relative pose between the first image sensor and the object based on a second input frame, for a third time period, associated with the first image sensor;

an estimated depth for the second input frame; and

an estimated motion of the object from the second time period to the third time period.

8 . The method of claim 7 , wherein:

the first sub-model of the ML model is trained to determine the estimated relative pose; and

a second sub-model of the ML model is trained to determine the estimated depth.

9 . The method of claim 7 , wherein the estimated motion of the object is obtained via one or more sensors associated with the object.

10 . The method of claim 1 , wherein the first frame comprises a raw image.

11 . The method of claim 1 , further comprising:

removing one or more dynamic objects from the first frame, excluding the object if the object comprises a moving object, prior to processing, with the first sub-model of the ML model, at least the first frame.

12 . The method of claim 1 , further comprising:

obtaining multiple relative poses of the first image sensor with respect to the object or the second image sensor, wherein:

the multiple relative poses of the first image sensor are provided as multiple output from the first sub-model, and

the multiple relative poses comprise the relative pose of the first image sensor; and

determining a final relative pose of the first image sensor based on the multiple relative poses.

13 . A method for relative pose estimation, comprising:

obtaining a first frame for a first time period, wherein the first frame is associated with a first image sensor;

obtaining a second frame for the first time period, wherein the second frame is associated with a second image sensor;

processing, with a first sub-model of a machine learning (ML) model, the first frame and the second frame, wherein:

the ML model is trained to perform first frame prediction based on a first loss function, and

the first loss function is configured to adjust a first loss based on a first error between a first input frame for a second time period associated with the first image sensor and a first output frame for the second time period predicted by the ML model; and

obtaining a relative pose of the first image sensor with respect to the second image sensor, wherein the relative pose of the first image sensor is provided as output from the first sub-model and is based on the processing.

14 . The method of claim 13 , wherein the first error between the first input frame and the first output frame is predicted by the ML model based on:

a first estimated relative pose between the first image sensor and the second image sensor, based on the first input frame and a second input frame, for the second time period, associated with the second image sensor;

a first estimated depth for the first input frame; and

a first estimated motion of the second image sensor between the second input frame, for the second time period, associated with the second image sensor and a third input frame, for a third time period, associated with the second image sensor.

15 . The method of claim 14 , wherein:

the first sub-model of the ML model has been trained to determine the first estimated relative pose;

a second sub-model of the ML model has been trained to determine the first estimated depth; and

a third sub-model of the ML model has been trained to determine the first estimated motion.

16 . The method of claim 14 , wherein the first output frame predicted by the ML model is further based on:

a second estimated relative pose between the first image sensor and the second image sensor based on the third input frame and a fourth input frame, for the third time period, associated with the first image sensor.

17 . The method of claim 14 , wherein the first output frame predicted by the ML model is further based on:

a second estimated motion of the first image sensor between the first input frame and a fourth input frame, for the third time period, associated with the first image sensor.

18 . The method of claim 13 , wherein the first error between the first input frame and the first output frame is predicted by the ML model based on:

a first estimated relative pose between the first image sensor and a scene, based on the first input frame, wherein the first input frame is associated with the scene;

a first estimated depth for the first input frame; and

a first estimated motion of the second image sensor between a second input frame, for the second time period, associated with the second image sensor and a third input frame, for a third time period, associated with the second image sensor.

19 . The method of claim 18 , wherein:

the ML model is trained to perform second frame prediction based on a second loss function, and

the second loss function is configured to adjust a second loss based on a second error between the second input frame and a second output frame for the second time period predicted by the ML model.

20 . The method of claim 19 , wherein the second error between the second input frame and the second output frame is predicted by the ML model based on:

a second estimated relative pose between the second image sensor and the scene, based on the second input frame, wherein the second input frame is associated with the scene;

a second estimated depth for the second input frame; and

a second estimated motion of the first image sensor between the first input frame and a fourth input frame, for the third time period, associated with the first image sensor.