Methods, storage media, and systems for generating a three-dimensional line segment
Methods, storage media, and systems for generating a three-dimensional line segment are disclosed. Exemplary implementations may: receive a plurality of images; generate a point cloud based on the plurality of images; detect a two-dimensional line segment in a first image; project a set of 3d points of the plurality of 3d points as 2d points in the first image; select projected 3d points that are proximate to 2d points along the 2d line segment; and generate a 3d line segment by connecting 3d points of the point cloud represented by the selected projected 3d points.
1 . A method for generating a three-dimensional (3d) line segment, the method comprising:
receiving a plurality of images;
generating a point cloud based on the plurality of images, wherein the point cloud comprises a plurality of 3d points;
detecting a two-dimensional (2d) line segment in a first image;
projecting a set of 3d points of the plurality of 3d points as 2d points in the first image;
selecting projected 3d points that are geometrically proximate to 2d points of the 2d line segment;
generating a 3d line segment by connecting the selected projected 3d points of the point cloud.
2 . The method of claim 1 , wherein projecting the set of 3d points is based on a visibility direction of a first camera associated with the first image.
3 . The method of claim 2 , wherein the visibility direction of the first camera is based on an orientation of the first camera.
4 . The method of claim 3 , wherein the visibility direction of the first camera is calculated by casting a ray from an optical center of the first camera through a center of the first image.
5 . The method of claim 2 ,
wherein the plurality of 3d points comprise associated visibility directions; and
wherein projecting the set of 3d points is further based on the visibility directions of the plurality of 3d points.
6 . The method of claim 5 , wherein projecting the set of 3d point comprises:
comparing the visibility directions of the plurality of 3d points to the visibility direction of the first camera; and
projecting a 3d point of the plurality of 3d points responsive to a visibility direction of the 3d point corresponding to the visibility direction of the first camera.
7 . The method of claim 6 , wherein the visibility direction of the 3d point corresponds to the visibility direction of the first camera when the visibility direction of the 3d point is less than orthogonal to the visibility direction of the first camera.
8 . The method of claim 5 , wherein a visibility direction of a 3d point of the plurality of 3d points is based in visibility directions of cameras associated with images of the plurality of images that were used to generate the 3d point of the plurality of 3d points.
9 . The method of claim 8 , wherein the visibility direction of the 3d point of the plurality of 3d points is calculated based on a mean of normalized differences between the 3d point and poses associated with the images of the plurality of images that were used to generate the 3d point of the plurality of 3d points.
10 . The method of claim 1 , wherein projecting the set of 3d points comprises:
computing the distances between the plurality of 3d points and the location of a first camera; and
projecting a 3d point of the plurality of 3d points responsive to a distance between the 3d point of the plurality of 3d points and a location of the first camera being less than a threshold distance.
11 . The method of claim 1 , wherein projecting the set of 3d points is based on scene understanding.
12 . The method of claim 11 , wherein each 3d point of the plurality of 3d points comprises metadata describing classifications of the 3d point, and wherein each 2d point of the first image comprises metadata describing classifications of the 2d point.
13 . The method of claim 12 , wherein projecting the set of 3d points comprises:
comparing the classifications of the 3d points of the plurality of 3d points to the classifications of the 2d points of the first image; and
projecting a 3d point of the plurality of 3d points responsive to a classification of the 3d point matching a classification of a 2d point of the image that the 3d point would be projected onto.
14 . The method of claim 1 , wherein selecting projected 3d points that are geometrically proximate comprises selecting projected 3d points that are within a threshold distance of the points of the 2d line segment.
15 . The method of claim 14 , wherein selecting projected 3d points that are geometrically proximate comprises:
calculating a mean or a median distance of the distances between the projected 3d points within the predetermined threshold distance and the 2D point along the 2D line segment.
16 . The method of claim 15 , further comprising selecting projected 3d points that are closest to the mean or the median distance from the 2d points of the 2D line segment.
17 . The method of claim 15 , further comprising selecting one or more new 3d points in the point cloud generated based on the mean or the median distance.
18 . The method of claim 17 , wherein generating the 3d line segment comprises connecting the new 3d points of the point cloud.
19 . The method of claim 1 , wherein selecting the projected 3d points that are geometrically proximate comprises selecting projected 3d points that are proximate to end points of the 2d line segment.
20 . The method of claim 1 , further comprising:
projecting the 3d line segment as a 2d line segment in the first image; and
displaying the first image, wherein displaying the first image comprises displaying the projected 3d line segment.
21 . The method of claim 1 , wherein selecting projected 3d points that are geometrically proximate comprises selecting projected 3d points that are closest to points of the 2D line segment.