Method and system for performing convolutional image transformation estimation

1. A method for generating inputs for a neural network based on an image, the method comprising:

receiving the image;

identifying a position within the image;

defining a subset of the image at the position, the subset of the image being defined by a first set of corners;

perturbing at least one of the first set of corners to form a second set of corners, the second set of corners defining a modified subset of the image, wherein perturbing the at least one of the first set of corners to form the second set of corners includes:

defining a first perturbation region surrounding a first corner of the first set of corners in which a first perturbed corner of the second set of corners may be formed;

defining a second perturbation region surrounding a second corner of the first set of corners in which a second perturbed corner of the second set of corners may be formed;

defining a third perturbation region surrounding a third corner of the first set of corners in which a third perturbed corner of the second set of corners may be formed;

defining a fourth perturbation region surrounding a fourth corner of the first set of corners in which a fourth perturbed corner of the second set of corners may be formed;

forming the first perturbed corner in a first random position within the first perturbation region;

forming the second perturbed corner in a second random position within the second perturbation region;

forming the third perturbed corner in a third random position within the third perturbation region; and

forming the fourth perturbed corner in a fourth random position within the fourth perturbation region;

determining a homography based on a comparison between the subset of the image and the modified subset of the image;

generating a transformed image by applying the homography to the image; and

defining a subset of the transformed image at the position.

2. The method of claim 1 , further comprising:

sending the subset of the image, the subset of the transformed image, and the homography to the neural network.

3. The method of claim 1 , further comprising:

determining that the subset of the transformed image does not include bordering artifacts.

4. The method of claim 1 , wherein identifying the position within the image includes:

randomly selecting the position within the image such that the first perturbation region, the second perturbation region, the third perturbation region, and the fourth perturbation region do not extend outside the image.

5. The method of claim 1 , wherein determining the homography based on the comparison includes:

determining one or more displacement vectors between the first set of corners and the second set of corners, wherein each of the one or more displacement vectors indicates an offset between corresponding corners in a first dimension and in a second dimension.

6. The method of claim 1 , wherein generating the transformed image by applying the homography to the image includes:

applying an inverse of the homography to the image to generate the transformed image.

7. A method for generating an estimated homography based on two images, the method comprising:

receiving a first image based on a first camera pose;

receiving a second image based on a second camera pose;

inputting the first image and the second image into a neural network; and

generating, by the neural network, an estimated homography based on the first image and the second image, wherein the neural network was previously trained by:

receiving a plurality of images; and

for each individual image of the plurality of images:

identifying a position within the individual image;

identifying a subset of the individual image at the position, the subset of the individual image being defined by a first set of corners;

perturbing at least one of the first set of corners to form a second set of corners, the second set of corners defining a modified subset of the individual image;

determining a homography based on a comparison between the subset of the individual image and the modified subset of the individual image;

generating a transformed image by applying the homography to the individual image;

identifying a subset of the transformed image at the position;

generating a particular estimated homography based on the subset of the individual image and the subset of the transformed image;

comparing the particular estimated homography to the homography; and

modifying the neural network based on the comparison.

8. The method of claim 7 , further comprising:

sending the subset of the individual image, the subset of the transformed image, and the homography to the neural network.

9. The method of claim 7 , further comprising:

determining that the subset of the transformed image does not include bordering artifacts.

10. The method of claim 7 , wherein perturbing the at least one of the first set of corners to form the second set of corners includes:

defining a first perturbation region surrounding a first corner of the first set of corners in which a first perturbed corner of the second set of corners may be formed;

defining a second perturbation region surrounding a second corner of the first set of corners in which a second perturbed corner of the second set of corners may be formed;

defining a third perturbation region surrounding a third corner of the first set of corners in which a third perturbed corner of the second set of corners may be formed;

defining a fourth perturbation region surrounding a fourth corner of the first set of corners in which a fourth perturbed corner of the second set of corners may be formed;

forming the first perturbed corner in a first random position within the first perturbation region;

forming the second perturbed corner in a second random position within the second perturbation region;

forming the third perturbed corner in a third random position within the third perturbation region; and

forming the fourth perturbed corner in a fourth random position within the fourth perturbation region.

11. The method of claim 10 , wherein identifying the position within the individual image includes:

randomly selecting the position within the individual image such that the first perturbation region, the second perturbation region, the third perturbation region, and the fourth perturbation region do not extend outside the individual image.

12. The method of claim 7 , wherein generating the transformed image by applying the homography to the individual image includes:

applying an inverse of the homography to the individual image to generate the transformed image.

13. A method for generating inputs for a neural network based on an image, the method comprising:

receiving the image;

identifying a position within the image;

defining a first sub-image of the image at the position, the first sub-image being defined by a first set of corners;

perturbing at least one of the first set of corners to form a second set of corners, the second set of corners defining a modified sub-image, wherein perturbing the at least one of the first set of corners to form the second set of corners includes:

defining a first perturbation region surrounding a first corner of the first set of corners in which a first perturbed corner of the second set of corners may be formed;

defining a second perturbation region surrounding a second corner of the first set of corners in which a second perturbed corner of the second set of corners may be formed;

defining a third perturbation region surrounding a third corner of the first set of corners in which a third perturbed corner of the second set of corners may be formed;

defining a fourth perturbation region surrounding a fourth corner of the first set of corners in which a fourth perturbed corner of the second set of corners may be formed;

forming the first perturbed corner in a first random position within the first perturbation region;

forming the second perturbed corner in a second random position within the second perturbation region;

forming the third perturbed corner in a third random position within the third perturbation region; and

forming the fourth perturbed corner in a fourth random position within the fourth perturbation region;

determining a homography based on a comparison between the first sub-image and the modified sub-image;

generating a transformed image by applying the homography to the image;

defining a second sub-image of the transformed image at the position; and

sending the first sub-image, the second sub-image, and the homography to the neural network.

14. The method of claim 13 , further comprising determining that the second sub-image does not include bordering artifacts.

15. The method of claim 13 , wherein identifying the position within the image includes randomly selecting the position within the image such that the first perturbation region, the second perturbation region, the third perturbation region, and the fourth perturbation region do not extend outside the image.

16. The method of claim 13 , wherein determining the homography based on the comparison includes determining one or more displacement vectors between the first set of corners and the second set of corners, wherein each of the one or more displacement vectors indicates an offset between corresponding corners in a first dimension and in a second dimension.

17. The method of claim 13 , wherein generating the transformed image by applying the homography to the image includes applying an inverse of the homography to the image to generate the transformed image.