IP Library Granted Patent US 11,875,301
Granted Patent B2
US 11,875,301 · App. 16/523,903 · Granted Jan 16, 2024

Refuse contamination analysis

Inventors: Anthony Romano (Havertown, PA); Michael Dean Geary (Toronto, CA); John Forrest Smith (Flat Rock, AL); Garrett Eckerl (Fort Payne, AL)
Assignee: The Heil Co.
G06Q10/0832B65F1/0033G06F18/24G06N20/00G06V20/00G06V20/20B65F2001/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,875,301
App. No.
16/523,903
Granted
Jan 16, 2024
Kind
B2
Abstract

A computer-implemented method for analyzing refuse includes operations of receiving sensor data indicating an operational state of a vehicle body component of a refuse collection vehicle (RCV); analyzing the sensor data to detect a presence of a triggering condition based at least partly on a particular operational state of the vehicle body component, as indicated by the sensor data; in response to detecting the triggering condition, accessing image data indicating a physical state of refuse collected by the RCV; providing the image data as input to at least one contaminant detection model trained, using at least one machine learning (ML) algorithm, to output a classification of the image data, the classification indicating a degree of contamination of the refuse; and storing, in a machine-readable medium, the classification of the image data.

Claims (63)

1. A computer-implemented method for analyzing refuse, the method comprising:

receiving, by one or more processors, a plurality of images of refuse, each image of the plurality of images being labeled as either including contaminated refuse or not including contaminated refuse;

training, by the one or more processors and based on the labeling, at least one machine learning model using the plurality of images; and

receiving, by at least one processor, sensor data indicating an operational state of a vehicle body component of a refuse collection vehicle (RCV), the sensor data generated by a sensor device configured to detect the operational state of the vehicle body component;

detecting, by the at least one processor, a presence of a triggering condition based at least partly on a particular operational state of the vehicle body component, as indicated by the sensor data;

in response to detecting the triggering condition, accessing, by the at least one processor, image data indicating a physical state of refuse collected by the RCV, the image data being generated by a camera mounted on the RCV and configured to generate digitized images of the refuse;

providing, by the at least one processor, the image data as input to the trained at least one machine learning model, to output a classification of the image data, wherein the classification indicates a degree of contamination of the refuse;

storing, by the at least one processor in a machine-readable medium, the classification of the image data;

receiving, by the at least one processor from a user, feedback regarding the accuracy of the classification of the image data output by at least one the machine learning model; and

retraining, by the at least one processor, the at least one machine learning model based on the image data, the classification, and the feedback regarding the accuracy of the classification.

2. The method of claim 1 , further comprising:

determining, by the at least one processor, that the degree of contamination of the refuse exceeds a contamination threshold; and

in response to determining the degree of contamination of the refuse exceeds the contamination threshold, routing, by the at least one processor, the RCV to a recycling facility,

wherein the degree of contamination indicates a degree of recyclable material in the refuse.

3. The method of claim 1 , further comprising:

determining, by the at least one processor, that the degree of contamination of the refuse exceeds a contamination threshold; and

in response to determining the degree of contamination of the refuse exceeds the contamination threshold, routing, by the at least one processor, the RCV to a landfill facility,

wherein the degree of contamination indicates a degree of non-recyclable material in the refuse.

4. The method of claim 1 , further comprising

determining, by the at least one processor, that the degree of contamination of the refuse exceeds a contamination threshold; and

in response to determining the degree of contamination of the refuse exceeds the contamination threshold, transmitting, by the at least one processor, a notification to a customer associated with the refuse exhibiting a degree of contamination above the contamination threshold.

5. The method of claim 1 , wherein the at least one machine learning model further outputs boundary information that describes one or more boundaries of contaminant objects identified in the refuse, the boundary information including object segmentation information for each of the contaminant objects.

6. The method of claim 1 , wherein,

the vehicle body component includes a lifting component that operates to empty a container into a receptacle of the RCV; and

the triggering condition comprises an operational state in which the lifting component is at a predetermined point in its operational cycle to empty the container.

7. The method of claim 1 , wherein the classification indicates a percentage of the refuse that is recyclable material and a percentage of the refuse that is non-recyclable material.

8. The method of claim 1 , further comprising routing the RCV to a waste receiving facility selected at least in part on the classification of the data indicating the degree of contamination of the refuse.

9. The method of claim 1 , wherein the at least one processor comprises an onboard computing device located in the RCV.

10. The method of claim 1 , wherein the RCV further comprises a light source, wherein the light source is configured to illuminate the refuse during collection of digitized images of the refuse by the camera.

11. A system comprising:

a refuse collection vehicle (RCV) comprising:

a hopper configured to receive refuse;

a body sensor device configured to detect an operational state of a vehicle body component of the RCV; and

a camera configured to generate digitized images of refuse collected by the RCV; and

at least one processor communicably coupled to the body sensor device and the camera, the at least one processor configured to perform operations comprising:

receiving a plurality of images of refuse, each image of the plurality of images being labeled as either including contaminated refuse or not including contaminated refuse;

training, based on the labeling, at least one machine learning model using the plurality of images; and

detecting, based on sensor data generated by the body sensor device, a presence of a triggering condition based at least partly on a particular operational state of the vehicle body component, the sensor data indicating an operational state of the vehicle body component of the RCV;

in response to detecting the triggering condition, accessing image data generated by the camera, the image data indicating a physical state of refuse collected by the RCV;

providing the image data as input to the trained at least one machine learning model, to output a classification of the data, wherein the classification indicates a degree of contamination of the refuse;

storing, in a machine-readable medium, the classification of the image data;

receiving, from a user, feedback regarding the accuracy of the classification of the image data output by the at least one machine learning model; and

retraining the at least one machine learning model based on the image data, the classification, and the feedback regarding the accuracy of the classification.

12. The system of claim 11 , wherein the at least one processor is also configured to:

route the RCV to a waste receiving facility selected at least in part on the classification of the data indicating the degree of contamination of the refuse.

13. The system of claim 11 , wherein the at least one processor is also configured to:

determine that the degree of contamination of the refuse exceeds a contamination threshold; and

in response to determining the degree of contamination of the refuse exceeds the contamination threshold, transmit a notification to a customer associated with the refuse exhibiting a degree of contamination above the contamination threshold.

14. The system of claim 11 , wherein the at least one processor comprises an onboard computing device located in the RCV.

15. The system of claim 11 , wherein the RCV further comprises a light source, wherein the light source is configured to illuminate the refuse during collection of digitized images of the refuse by the camera.

16. The system of claim 11 , wherein the at least one processor is also configured to:

determining, by the at least one processor, that the degree of contamination of the refuse exceeds a contamination threshold; and

in response to determining the degree of contamination of the refuse exceeds the contamination threshold, routing, by the at least one processor, the RCV to a recycling facility,

wherein the degree of contamination indicates a degree of recyclable material in the refuse.

17. The system of claim 11 , wherein the at least one processor is also configured to:

determining, by the at least one processor, that the degree of contamination of the refuse exceeds a contamination threshold; and

in response to determining the degree of contamination of the refuse exceeds the contamination threshold, routing, by the at least one processor, the RCV to a landfill facility,

wherein the degree of contamination indicates a degree of non-recyclable material in the refuse.

18. The system of claim 11 , wherein the at least one machine learning model further outputs boundary information that describes one or more boundaries of contaminant objects identified in the refuse, the boundary information including object segmentation information for each of the contaminant objects.

19. The system of claim 11 , wherein,

the vehicle body component includes a lifting component that operates to empty a container into a receptacle of the RCV; and

the triggering condition comprises an operational state in which the lifting component is at a predetermined point in its operational cycle to empty the container.

20. The system of claim 11 , wherein the classification indicates a percentage of the refuse that is recyclable material and a percentage of the refuse that is non-recyclable material.

Assignments (2)
SECURITY INTEREST Recorded Oct 9, 2024
From: THE HEIL CO.; TEREX USA, LLC; TEREX SOUTH DAKOTA, INC.
To: UBS AG, STAMFORD BRANCH
Reel/Frame 069177/0271 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2020
From: ROMANO, ANTHONY; GEARY, MICHAEL DEAN; SMITH, JOHN FORREST; ECKERL, GARRETT
To: THE HEIL CO.
Reel/Frame 052799/0825 →