Methods and apparatus for machine learning system for edge computer vision and active reality
In some embodiments, a method includes: receiving, at a first time at a user device, a first set of image frames of an inventory. The method further includes identifying a first control point in the first set of image frames. The method further includes locating, using a light ranging sensor of the user device, a first set of storage units in an area associated with the first control point in the inventory. The method further includes storing a representation of the first set of storage units in the area in a memory of the user device. The method further includes receiving, at a second time after the first time, a second set of image frames of the inventory. The method further includes locating, based on the representation and without identifying the first control point, a second set of storage units in the area at the second time.
1 . A non-transitory processor-readable medium storing instructions that when executed by a processor, cause the processor to:
receive, from an image sensor operatively coupled to the processor of a user device, a plurality of image frames;
locate an object in the plurality of image frames to define a control point in a virtual space, the virtual space being representative of an inventory;
generate a mesh in the virtual space, the mesh including a first set of points defined by contours of at least one storage unit and a second set of points defined by contours of a shelf that supports the at least one storage unit;
detect a set of storage units on the shelf, based on the first set of points of the mesh and the second set of points of the mesh, the set of storage units including at least a first storage unit and a second storage unit, the first storage unit being disposed between the user device and the second storage unit;
identify in the virtual space a form for the first storage unit from the set of storage units;
determine, based on the form for the first storage unit from the set of storage units, a plurality of identifying techniques for the first storage unit from the set of storage units;
execute each identifying technique from the plurality of identifying techniques to define a set of potential unit types for the first storage unit from the set of storage units;
assign, based on the set of potential unit types, a unit type identity to (1) the first storage unit from the set of storage units and (2) the second storage unit from the set of storage units;
calculate a storage unit count for the set of storage units, based on the mesh and the unit type identity; and
send a signal to output the storage unit count on a display of the user device.
2 . The non-transitory processor-readable medium of claim 1 , wherein the plurality of identifying techniques includes at least one of a direct match, an optical character recognition (OCR) match, or a label match.
3 . The non-transitory processor-readable medium of claim 1 , wherein the form is at least one of a shape or a size and is based on the first set of points.
4 . The non-transitory processor-readable medium of claim 1 , wherein the set of potential unit types is displayed on the display of the user device.
5 . The non-transitory processor-readable medium of claim 1 , wherein the unit type identity is assigned in response to a selection from a user of the user device.
6 . The non-transitory processor-readable medium of claim 1 , wherein the image sensor is not fixed and is configured to capture the plurality of image frames in substantially real-time.
7 . The non-transitory processor-readable medium of claim 1 , wherein the unit type identity includes (1) a first confidence value associated with the first storage unit from the set of storage units and (2) a second confidence value associated with the second storage unit from the set of storage units, the first confidence value being greater than or equal to the second confidence value.
8 . The non-transitory processor-readable medium of claim 1 , wherein the unit type identity is assigned to the second storage unit from the set of storage units in response to a selection from a user of the user device.
9 . A non-transitory, processor-readable medium storing instructions that, when executed by a processor, cause the processor to:
receive, from an image sensor operatively coupled to the processor of a user device, a plurality of image frames;
locate an object in the plurality of image frames to define a control point in a virtual space, the virtual space being representative of an inventory;
generate a mesh in the virtual space, the mesh including a first set of points defined by contours of at least one storage unit and a second set of points defined by contours of a shelf that supports the at least one storage unit;
detect a set of storage units on the shelf, based on the first set of points of the mesh and the second set of points of the mesh, the set of storage units including at least a first storage unit and a second storage unit, the first storage unit being disposed between the user device and the second storage unit;
identify in the virtual space a form for the first storage unit from the set of storage units;
determine, based on the form for the first storage unit from the set of storage units, at least a first identifying technique and a second identifying technique;
execute the first identifying technique to define a first potential unit type with a first confidence value and the second identifying technique to define a second potential unit type with a second confidence value;
assign, based on at least one of the first potential unit type with the first confidence value or the second potential unit type with the second confidence value, a unit type identity to each storage unit from the set of storage units;
calculate a storage unit count for the set of storage units, based on the mesh and the unit type identity; and
send a signal to output the storage unit count on a display of the user device.
10 . The non-transitory, processor-readable medium of claim 9 , wherein at least one of the first identifying technique or the second identifying technique includes at least one of a size classifier, a location classifier, or a shape classifier.
11 . The non-transitory, processor-readable medium of claim 9 , wherein:
the storage unit count is at least partially based on a configuration for the set of storage units.
12 . The non-transitory, processor-readable medium of claim 9 , wherein the form is at least one of a shape or a size and is based on the first set of points of the mesh.
13 . The non-transitory, processor-readable medium of claim 9 , wherein a difference in the first confidence value and the second confidence value satisfies a threshold value.
14 . The non-transitory, processor-readable medium of claim 9 , wherein the image sensor is not fixed and is configured to capture the plurality of image frames in substantially real-time.
15 . The non-transitory, processor-readable medium of claim 9 , wherein:
the first confidence value is a first weighted confidence value,
the first weighted confidence value is weighted according to a first predefined weight associated with the first identifying technique,
the second confidence value is a second weighted confidence value, and
the second weighted confidence value is weighted according to a second predefined weight associated with the second identifying technique.