Detection of prohibited objects concealed in an item, using a three-dimensional image of the item
Systems and methods are provided for enabling detection of prohibited objects using three-dimensional imaging data. The system obtains a three-dimensional representation of at least part of an item. The 3D representation includes multiple points with associated intensity values determined based on data acquired by an imaging device. A cluster of points corresponding to an object of interest is identified within the 3D representation. From this cluster, data indicating the position and the orientation of the object is determined. The data indicating the position and the orientation of the object is used to generate a two-dimensional representation of the object with an orientation which is chosen to facilitate detectability of the object as a prohibited item by a detection algorithm (such as a machine learning model).
1 . A system comprising one or more processing circuitries configured to:
obtain a three-dimensional (3D) representation of at least part of an item, wherein the three-dimensional representation includes a plurality of points each associated with an intensity obtained based on an acquisition of the at least part of the item by an acquisition device,
determine one or more given clusters of points in the 3D representation, each given cluster of points being informative of a given object present in the at least part of the item,
for each given cluster:
use the given cluster of points to determine data D position/orientation informative of at least one of a position or an orientation of the given object,
determine a single region using at least part of the given cluster of points and the data D position/orientation informative of at least one of a position or an orientation of the given object, and
use at least the single region to generate a two-dimensional image displaying the given object with a viewing angle selected to enhance detectability of the given object as a prohibited object, wherein the viewing angle is selected depending on a type of prohibited object to be detected.
2 . The system of claim 1 , configured to use the data D position/orientation and the single region to generate the two-dimensional image.
3 . The system of claim 1 , wherein the single region is a three-dimensional region, wherein the system is configured to use D position/orientation to convert intensity of points of the three-dimensional region into a two-dimensional image displaying the given object with the viewing angle selected to enhance detectability of the given object as a prohibited object.
4 . The system of claim 1 , wherein D position/orientation includes one or more weights informative of a geometric distribution of the given object along one or more directions, wherein the system is configured to use the one or more weights to determine the region.
5 . The system of claim 1 , wherein the single region is determined based on D position/orientation for a first type of prohibited objects differently than for a second type of prohibited objects, wherein the second type is different from the first type.
6 . The system of claim 1 , configured to:
obtain one or more weights informative of a geometric distribution of the given object along one or more directions, and
use one or more of the weights to determine whether the given cluster of points is informative of a given object which has a shape matching a shape of a prohibited object.
7 . The system of claim 1 , wherein D position/orientation is informative of three axes X obj , Y obj , Z obj informative of axes of the given object, wherein the system is configured to:
determine a region of the 3D representation comprising at least part of a plane which comprises two axes X obj , Y obj among the axes X obj , Y obj , Z obj , or
determine a region of the 3D representation comprising at least part of a plane which is orthogonal to the axis Z obj .
8 . The system of claim 1 , wherein D position/orientation is informative of three axes X obj , Y obj , Z obj informative of axes of the given object, wherein the system is configured to:
obtain, for the axis X obj , a weight W X informative of geometric distribution along said axis X,
obtain, for the axis Y obj , a weight W Y informative of geometric distribution along said axis Y,
obtain, for the axis Z obj , a weight W Z informative of geometric distribution along said axis Z,
perform (i) or (ii):
(i) determining a region of the 3D representation comprising at least part of a plane which includes two axes X obj , Y obj , wherein the weights W X , W Y and W Z indicate that geometric distribution is greater along the two axes X obj , Y obj than along the axis Z obj ,
(ii) determining a region of the 3D representation along a plane which is orthogonal to axis Z obj , wherein the weights W X , W Y and W Z indicate that geometric distribution along the axis Z obj is smaller than along the two axes X obj , Y obj .
9 . The system of claim 1 , configured to determine a region of the 3D representation which includes:
at least part of a plane of the 3D representation determined using D position/orientation ,
a volume of the 3D representation, wherein a position of the volume with respect to the plane is determined using D position/orientation .
10 . The system of claim 9 , wherein at least one of (i) or (ii) is met:
(i) the volume has a height H+ above the plane along a direction Z obj , or has a height H− below the plane along the direction Z obj , wherein Z obj has been determined using D position/orientation , or
(ii) one or more dimensions of the volume is selected based on an estimate of one or more dimensions of a prohibited object.
11 . The system of claim 9 , configured to use said region to generate a two-dimensional image informative of the given object, wherein one or more dimensions of the volume are selected to include mostly or only points which are informative of the given object, wherein a minority or none of these points are informative of other neighbouring objects of the given object in the item.
12 . The system of claim 1 , configured to:
obtain an initial three-dimensional (3D) representation of the at least part of the item, select a fraction of the points of the initial three-dimensional (3D) representation which are associated with an intensity which is a predefined range, and
use the selected points to generate the three-dimensional (3D) representation of the at least part of the item.
13 . The system of claim 1 , configured to divide the three-dimensional representation of the at least part of the item into a plurality of clusters, wherein points which belong to a same cluster meet a proximity criterion.
14 . The system of claim 13 , configured to perform at least one of (i) or (ii):
(i) selecting the given cluster as a cluster of the plurality of clusters which includes a number of points which is above a threshold; or
(ii) determining that one or more given clusters of the plurality of clusters do not represent a prohibited object, based on a detection that one or more geometrical parameters of each of the one or more given clusters do not comply with an expected range.
15 . The system of claim 1 , configured to:
determine data D flatness informative of a flatness of areas of the three-dimensional representation,
use the data D flatness to determine a given cluster of points in the three-dimensional representation, the given cluster of points being informative of at least one given object present in the item,
use the given cluster of points to determine data D position/orientation informative of at least one of a position or an orientation of the given object, and
use data D position/orientation to generate a two-dimensional image informative of the given object.
16 . The system of claim 1 , configured to:
determine, for each given point of a plurality of points of the three-dimensional representation, data D flatness informative of a flatness of said given point,
determine a given three-dimensional area of the 3D representation, said given three-dimensional area including said given point,
determine a given cluster of points within said given three-dimensional area, use the given cluster of points to determine data D position/orientation informative of an orientation of the given object,
determine a region of the 3D representation, wherein the region is determined using at least part of the given cluster of points and D position/orientation , and
use at least said region to generate a two-dimensional image informative of the given object, wherein the two-dimensional image is usable to detect whether the given object is a prohibited object.
17 . The system of claim 1 , configured to:
feed the two-dimensional image of the given object to a trained machine learning model, and
use the trained machine learning model to determine whether the given object is a prohibited object,
wherein at least one of (i) or (ii) is met:
(i) most of the images of prohibited objects used to train the machine learning model include prohibited objects observed with a viewing angle which matches an optimal viewing angle according to a criterion; or
(ii) the trained machine learning model has been trained with images of prohibited objects, wherein most of the images of prohibited objects used to train the machine learning model include prohibited objects observed with a viewing angle which matches a viewing angle of the given object in the two-dimensional image according to a criterion.
18 . A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:
obtain an initial three-dimensional (3D) representation of at least part of an item, wherein the initial three dimensional representation includes a plurality of points each associated with an intensity obtained based on an acquisition of the at least part of the item by an acquisition device,
select a fraction of the points of the initial three-dimensional (3D) representation which are associated with an intensity which is a predefined range,
use the selected points to generate a three-dimensional (3D) representation of the at least part of the item,
determine at least one given cluster of points in the 3D representation, the given cluster of points being informative of at least one given object present in the at least part of the item,
use the given cluster of points to determine data D position/orientation informative of at least one of a position or an orientation of the given object, and
use data D position/orientation to generate a two-dimensional image displaying the given object with a viewing angle selected to enhance detectability of the given object as a prohibited object, wherein the viewing angle is selected depending on a type of prohibited object to be detected.
19 . A system comprising one or more processing circuitries configured to:
obtain a three-dimensional (3D) representation of at least part of an item, wherein the three-dimensional representation includes a plurality of points each associated with an intensity obtained based on an acquisition of the at least part of the item by an acquisition device,
determine at least one given cluster of points in the 3D representation, the given cluster of points being informative of at least one given object present in the at least part of the item,
use the given cluster of points to determine data D position/orientation informative of at least one of a position or an orientation of the given object,
determine a region of the 3D representation which includes:
at least part of a plane of the 3D representation determined using D position/orientation , and
a volume of the 3D representation,
wherein a position of the volume with respect to the plane is determined using D position/orientation , wherein one or more dimensions of the volume are selected to include mostly or only points which are informative of the given object, wherein a minority or none of these points are informative of other neighbouring objects of the given object in the item, and
use said region to generate a two-dimensional image displaying the given object with a viewing angle selected to enhance detectability of the given object as a prohibited object.