Associating two dimensional label data with three-dimensional point cloud data

1 . An image processing system comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to:

obtain a two-dimensional image including an object;

obtain three-dimensional point clouds including the object;

obtain a three-dimensional container which identifies a cluster among the three-dimensional point clouds to associate label data with the cluster;

associate the label data with the cluster, the cluster including a plurality of three-dimensional points, the plurality of three-dimensional points being a part of the three-dimensional point clouds, and the label data being associated with the object included in the two-dimensional image; and

associate the label data with two-dimensional point clouds that are obtained by projecting the cluster onto a plane, wherein the two-dimensional point clouds are identified by a two-dimensional container which is associated with the three-dimensional container.

2 . The image processing system according to claim 1 , wherein the label data indicates a class of the object.

3 . The image processing system according to claim 1 ,

wherein obtaining a three-dimensional container which identifies the cluster among the three-dimensional point clouds to associate label data with the cluster includes identifying a plurality of clusters among the three-dimensional point clouds, each of the plurality of clusters including ones of other pluralities of three-dimensional points,

wherein the label data is obtained from a plurality of label data corresponding to a plurality of objects included in the two-dimensional image, and

wherein associating the label data with the cluster includes associating the plurality of label data with the plurality of clusters.

4 . The image processing system according to claim 1 , wherein the at least one processor is further configured to execute the instructions to:

obtain a two-dimensional container by projecting the three-dimensional container onto an image, wherein the two-dimensional container covers the two-dimensional point clouds.

5 . The image processing system according to claim 1 , wherein obtaining a three-dimensional container which identifies the cluster among the three-dimensional point clouds to associate label data with the cluster includes determining a three-dimensional container on the three-dimensional point clouds.

6 . The image processing system according to claim 1 , wherein the at least one processor is further configured to execute the instructions to:

obtain the label data corresponding to the object included in the two-dimensional image of the object.

7 . The image processing system according to claim 1 , wherein the three-dimensional point clouds indicate a structure of an environment.

8 . The image processing system according to claim 1 , wherein the three-dimensional point clouds indicate a structure of an actual environment.

9 . The image processing system according to claim 1 , wherein the cluster includes the object.

10 . The image processing system according to claim 1 , wherein the three-dimensional point clouds are obtained by a sensor.

11 . The image processing system according to claim 10 , wherein the sensor uses a 3D LiDAR.

12 . The image processing system according to claim 1 , wherein the two-dimensional image is a part of a captured image captured by a camera.

13 . The image processing system according to claim 1 , wherein identifying the cluster comprises distinguishing between the cluster and other three-dimensional points by applying clustering to the three-dimensional point clouds.

14 . The image processing system according to claim 1 , wherein associating the label data with the cluster comprises associating the label data with the cluster in response to identifying the cluster.

15 . An image processing method comprising:

obtaining a two-dimensional image including an object;

obtaining three-dimensional point clouds including the object;

obtaining a three-dimensional container which identifies a cluster among the three-dimensional point clouds to associate label data with the cluster;

associating the label data with the cluster, the cluster including a plurality of three-dimensional points, the plurality of three-dimensional points being a part of the three-dimensional point clouds, and the label data being associated with the object included in the two-dimensional image; and

associating the label data with two-dimensional point clouds that are obtained by projecting the cluster onto a plane, wherein the two-dimensional point clouds are identified by a two-dimensional container which is associated with the three-dimensional container.

16 . The image processing method according to claim 15 , wherein the label data indicates a class of the object.

17 . The image processing method according to claim 15 ,

wherein obtaining a three-dimensional container which identifies the cluster among the three-dimensional point clouds to associate label data with the cluster includes identifying a plurality of clusters among the three-dimensional point clouds, each of the plurality of clusters including ones of other pluralities three-dimensional points,

wherein the label data is obtained from a plurality of label data corresponding to a plurality of objects included in the two-dimensional image, and

wherein associating the label data with the cluster includes associating the plurality of label data with the plurality of clusters.

18 . The image processing method according to claim 15 , further comprising:

obtaining a three-dimensional container covering the cluster to associate the label data with the cluster.

19 . The image processing method according to claim 18 , further comprising:

obtaining a two-dimensional container by projecting the three-dimensional container onto an image, wherein the two-dimensional container covers the two-dimensional point clouds.

20 . The image processing method according to claim 15 , wherein the three-dimensional point clouds indicate a structure of an environment.

21 . The image processing method according to claim 15 , wherein the at least one cluster includes the object.

22 . The image processing method according to claim 15 , wherein the two-dimensional image is a part of a captured image captured by a camera.

23 . The image processing method according to claim 15 , wherein identifying the cluster comprises distinguishing between the cluster and other three-dimensional points by applying clustering to the three-dimensional point clouds.

24 . The image processing method according to claim 15 , wherein associating the label data with the cluster comprises associating the label data with the cluster in response to identifying the cluster.

25 . A non-transitory computer readable medium storing a program for causing a computer to:

obtain a two-dimensional image including an object;

obtain three-dimensional point clouds including the object;

obtain a three-dimensional container which identifies a cluster among the three-dimensional point clouds to associate label data with the cluster;

associate the label data with the cluster, the cluster including a plurality of three-dimensional points, the plurality of three-dimensional points being a part of the three-dimensional point clouds, and the label data being associated with the object included in the two-dimensional image; and

associate the label data with two-dimensional point clouds that are obtained by projecting the cluster onto a plane, wherein the two-dimensional point clouds are identified by a two-dimensional container which is associated with the three-dimensional container.

26 . The non-transitory computer readable medium according to claim 25 , wherein the label data indicates a class of the object.

27 . The non-transitory computer readable medium according to claim 25 ,

wherein the obtaining of a three-dimensional container which identifies the cluster among the three-dimensional point clouds to associate label data with the cluster includes identifying a plurality of clusters among the three-dimensional point clouds, each of the plurality of clusters including a plurality of three-dimensional points,

wherein the obtaining the label data includes obtaining a plurality of label data corresponding to a plurality of objects included in the two-dimensional image, and

wherein associating the label data with the cluster includes associating the plurality of label data with the plurality of clusters.

28 . The non-transitory computer readable medium according to claim 25 , wherein the program further causes the computer to:

obtain a two-dimensional container by projecting the three-dimensional container onto an image, wherein the two-dimensional container covers the two-dimensional point clouds.

29 . The non-transitory computer readable medium according to claim 25 , wherein the three-dimensional point clouds indicate a structure of an environment.

30 . The non-transitory computer readable medium according to claim 25 , wherein the cluster includes the object.

31 . The non-transitory computer readable medium according to claim 25 , wherein the two-dimensional image is a part of a captured image captured by a camera.

32 . The non-transitory computer readable medium according to claim 25 , wherein identifying the cluster comprises distinguishing between the cluster and other three-dimensional points by applying clustering to the three-dimensional point clouds.

33 . The non-transitory computer readable medium according to claim 25 , wherein associating the label data with the cluster comprises associating the label data with the cluster in response to identifying the cluster.