Separation of objects in images from three-dimensional cameras
Methods, systems, and programs are presented for simultaneous recognition of objects within a detection space utilizing three-dimensional (3D) cameras configured for capturing 3D images of the detection space. One system includes the 3D cameras, calibrated based on a pattern in a surface of the detection space, a memory, and a processor. The processor combines data of the 3D images to obtain pixel data and removes, from the pixel data, background pixels of the detection space to obtain object pixel data associated with objects in the detection space. Further, the processor creates a geometric model of the object pixel data, the geometric model including surface information of the objects in the detection space, generates one or more cuts in the geometric model to separate objects and obtain respective object geometric models, and performs object recognition to identify each object in the detection space based on the respective object geometric models.
1 . A method comprising:
capturing a set of images depicting a set of objects;
determining a point cloud associated with the set of objects, wherein the point cloud comprises a set of points;
determining a set of planar cuts based on a topographical representation from a single predetermined perspective of the point cloud;
determining a set of distinct point clusters within the point cloud using the set of planar cuts, each point cluster of the set of distinct point clusters comprising a subset of the set of points;
identifying object pixel data, from the set of images, based on each point cluster within the set of distinct point clusters; and
determining an object identity based on the respective object pixel data associated with each point cluster of the set of distinct point clusters.
2 . The method of claim 1 , wherein determining an object identity based on the respective object pixel data associated with each point cluster further comprises matching the point cluster with a stored 3D representation of a generic object, the generic object associated with the object identity.
3 . The method of claim 2 , wherein the stored 3D representation of the generic object is stored within a database additionally storing a 3D representation of a generic object bundle comprising multiple copies of the generic object.
4 . The method of claim 1 , wherein determining an object identity based on the respective object pixel data associated with each point cluster comprises extracting a set of image features from the object pixel data and determining the object identity based on the set of image features.
5 . The method of claim 1 , wherein identifying the set of point clusters is performed at least in part with a classifier.
6 . The method of claim 1 , wherein determining an object identity based on the respective object pixel data associated with each point cluster comprises identifying text from the respective object pixel data associated with each point cluster and using the identified text to determine the object identity.
7 . The method of claim 1 , wherein a planar cut is determined by identifying a cut line based on geometric features associated with the topographical representation and orthogonally extending the cut line through the point cloud.
8 . The method of claim 1 , wherein each pair of abutting point clusters is separated by a single planar cut.
9 . A system comprising:
a set of cameras, configured to capture a set of images of a set of objects; and
a processing system, configured to:
determine a point cloud based on the set of images, wherein the point cloud comprises a set of points;
determine a set of distinct point clusters within the point cloud using a set of planar cuts, wherein each of the set of distinct point clusters comprises a subset of the set of points, wherein the set of planar cuts are determined based on a topographical representation from a single predetermined perspective of the point cloud;
identify object pixel data from the set of images based on each point cluster, wherein each point cluster is within the set of distinct point clusters; and
determine an object identity based on the respective object pixel data associated with each point cluster of the set of distinct point clusters.
10 . The system of claim 9 , wherein the processing system is further configured to remove a set of background points from the point cloud based on 3D locations of the background points within the set of background points.
11 . The system of claim 10 , wherein the system further comprises a base element associated with a visual pattern, and wherein removing the set of background points from the point cloud comprises removing points corresponding to the visual pattern.
12 . The system of claim 9 , wherein the set of cameras comprises at least three cameras.
13 . The system of claim 9 , wherein the processing system is further configured to generate an invoice based on the object identity.
14 . The system of claim 9 , wherein determining an object identity based on the respective object pixel data associated with each point cluster comprises matching a 3D object model generated from the object pixel data with a stored 3D representation of a generic object, the generic object associated with the object identity.
15 . The system of claim 9 , wherein determining an object identity based on the respective object pixel data associated with each point cluster comprises extracting a set of features from the object pixel data and determining the object identity based on the set of features.
16 . The system of claim 9 , wherein determining the object identity is performed at least in part with a classifier.
17 . The system of claim 9 , wherein a planar cut from the set of planar cuts is determined by identifying a cut line in the single predetermined perspective from a direction parallel to a vertical axis of the point cloud, and orthogonally extending the cut line through the point cloud.
18 . The system of claim 9 , wherein each pair of point clusters is separated by one vertical planar cut.
19 . The method of claim 4 , wherein determining an object identity based on the respective object pixel data associated with each point cluster further comprises extracting a set of volumetric features based on the point cluster and determining the object identity based on the set of volumetric features.
20 . The method of claim 1 , wherein the topographical representation consists of representations of a set of top surfaces of the set of objects.