Vacuum-based end effector, system, and method for detecting parcel engagement and classifying parcels using audio
A system for detecting parcel engagement and classifying parcels includes: a robot configured to engage and transfer parcels; one or more microphones configured to obtain sound recordings (audio data) and positioned to obtain audio data corresponding to the robot engaging a parcel for transfer; and a control subsystem. The control subsystem includes a controller which receives and processes the audio data obtained by the one or more microphones to detect engagement of the robot with the parcel and to classify a characteristic of the parcel, such as parcel type. The robot can be a vacuum-based end effector, with the one or more microphones a component thereof. A method for classifying parcels is also disclosed.
1. A system for engaging and transferring parcels, comprising:
a robot configured to engage and transfer parcels;
one or more microphones configured and positioned to obtain audio data corresponding to the robot engaging a parcel when the system is in use; and
a control subsystem, including
a controller operably connected to the robot, the controller including a processor for executing instructions stored in a memory component to (i) receive and process the audio data obtained by the one or more microphones, with the audio data corresponding to the robot engaging the parcel, and (ii) identify a characteristic of the parcel based on the audio data obtained by the one or more microphones.
2. The system as recited in claim 1 , wherein the controller is configured to identify the parcel as being one of a box-type parcel, a flexible plastic bag, and a flat parcel.
3. A system for engaging and transferring parcels, comprising:
a robot configured to engage and transfer parcels, the robot including a framework and a vacuum-based end effector mounted to the framework, the vacuum-based end effector including
a base plate mounted to the framework,
one or more vacuum cups connected to the base plate and configured to be placed in fluid communication with a vacuum source, and
one or more microphones configured and positioned about the vacuum-based end effector, such that the one or more microphones obtain audio data corresponding to the one or more vacuum cups engaging a parcel when the system is in use; and
a control subsystem, including
a controller operably connected to the robot, the controller including a processor for executing instructions stored in a memory component to (i) receive and process the audio data obtained by the one or more microphones, with the audio data corresponding to the one or more vacuum cups engaging the parcel, and (ii) at least one of (A) detect whether the one or more vacuum cups are pneumatically engaged with the parcel based on the audio data obtained by the one or more microphones and (B) identify a characteristic of the parcel based on the audio data obtained by the one or more microphones.
4. The system according to claim 3 ,
wherein the controller detects whether the one or more vacuum cups are pneumatically engaged with the parcel based on the audio data obtained by the one or more microphones when the system is in use; and
wherein the memory component further include instructions, which, when executed by the processor, cause the controller to (iii) communicate instructions which affect operation of the robot and/or the vacuum source based on the audio data obtained by the one or more microphones.
5. The system according to claim 3 ,
wherein the controller identifies the characteristic of the parcel based on the audio data obtained by the one or more microphones when the system is in use; and
wherein the characteristic of the parcel is parcel type.
6. The system according to claim 5 , wherein the controller identifies the characteristic of the parcel by inputting the audio data obtained by the one or more microphones into an audio classifier model configured to identify parcel type based on audio data.
7. The system as recited in claim 6 , wherein the audio classifier model is a neural network pre-trained on a collection of audio samples obtained by the one or more microphones, the collection of audio samples including audio samples corresponding to the one or more vacuum cups engaging a plurality of parcels of different types at different times.
8. The system according to claim 7 , wherein the neural network is a convolutional neural network.
9. The system according to claim 5 , wherein the memory component further include instructions, which, when executed by the processor, cause the controller to (iii) communicate instructions which cause the robot to transfer the parcel to one of multiple potential locations based on the particular parcel type of the parcel.
10. The system as recited in claim 3 ,
wherein the one or more vacuum cups includes a plurality of vacuum cups; and
wherein the one or more microphones includes a microphone positioned between at least two vacuum cups of the plurality of vacuum cups.
11. The system as recited in claim 3 ,
wherein the one or more vacuum cups includes a plurality of vacuum cups; and
wherein the one or more microphones includes a microphone mounted to the base plate in a position centrally positioned relative to the plurality of vacuum cups.
12. The system as recited in claim 3 ,
wherein the one or more vacuum cups includes a plurality of vacuum cups; and
wherein the one or more microphones is a single microphone positioned to obtain sound recordings of each of the plurality of vacuum cups as the plurality of vacuum cups engage the parcel.
13. The system as recited in claim 3 ,
wherein the controller detects whether the one or more vacuum cups are pneumatically engaged with the parcel based on the audio data obtained by the one or more microphones when the system is in use; and
wherein the controller identifies the characteristic of the parcel based on the audio data obtained by the one or more microphones when the system is in use.