IP Library Granted Patent US 11,727,698
Granted Patent B2
US 11,727,698 · App. 16/871,293 · Granted Aug 15, 2023

Identifying elements in an environment

Inventors: Syed Mohammed Jerrar Bukhari (Chelmsford, MA); Tyler Barron (Chelmsford, MA); Justin Holwell (Chelmsford, MA)
Assignee: AUTOGUIDE, LLC
G05D1/0251G05D1/0219G05D1/0221G05D1/0253G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,727,698
App. No.
16/871,293
Granted
Aug 15, 2023
Kind
B2
Abstract

An example method of detecting an element using an autonomous vehicle includes the following operations: using a sensor on the autonomous vehicle to capture image data in a region of interest containing the element, where the image data represents components of the element; filtering the image data to produce filtered data having less of an amount of data than the image data; identifying the components of the element by analyzing the filtered data using a deterministic process; and detecting the element based on the components.

Claims (72)

1. A method performed for an autonomous vehicle to detect a volume in which to place an element, the method comprising:

detecting a rack containing the volume;

controlling a sensor to detect three-dimensional (3D) data above a bottom of the rack in a region of interest;

detecting the volume based on at least part of the 3D data within a first part of the region of interest and less than a predetermined amount of the 3D data within a second part of the region of interest that is expected to contain the volume based on the at least part of the 3D data within the first part of the region; and

determining; based on the volume and dimensions of the element, whether the volume is large enough to fit the element;

wherein detecting the volume comprises:

detecting less than the predetermined amount of 3D data within the second part of the region of interest;

detecting boundaries of the second part of the region of interest, the boundaries being based on where there is less than the predetermined amount of 3D data; and

determining the volume based on the boundaries.

2. The method of claim 1 , further comprising:

detecting the element by performing operations comprising:

using the sensor on the autonomous vehicle to capture image data in a region of interest containing the element, the image data representing components of the element;

filtering the image data to produce filtered data having less of an amount of data than the image data;

identifying candidates for the components of the element by analyzing the filtered data using a deterministic process, where identifying the candidates for the components comprises locating multiple structures represented by the filtered data;

voting on whether each of the multiple structures meets one or more predefined tolerances, where voting comprises assigning one or more points to each of the multiple structures so that at least two of the multiple structures have scores that exceed a threshold; and

detecting the element by comparing the least two of the multiple structures to a predefined configuration.

3. The method of claim 1 , wherein detecting the volume comprises:

detecting the at least part of the 3D data within the first part of the region of interest by outputting light and detecting reflection of the light; and

wherein detecting less than the predetermined amount of 3D data comprises detecting less than a predetermined amount of light reflection.

4. The method of claim 3 , wherein detecting less than the predetermined amount of light reflection comprises detecting no light reflection.

5. The method of claim 1 , further comprising:

aligning the element within the volume.

6. The method of claim 1 , wherein detecting the rack comprises:

using the sensor to capture image data in a region of interest containing the rack, the image data representing components of the rack;

identifying the components of the rack by analyzing the image data using a deterministic process; and

detecting the rack based on the components.

7. The method of claim 6 , wherein the components comprise pillars; and

wherein identifying the pillars comprises locating one or more structures represented by the image data that extend along a same dimension and that are separated by a predefined space.

8. The method of claim 7 , wherein the components comprise beams; and

wherein identifying the beams comprises locating one or more structures represented by the image data that extend along a same dimension and that are separated by a predefined space.

9. The method of claim 7 , wherein the components comprise beams and the same dimension is a first dimension; and

wherein identifying the beams comprises locating multiple structures represented by the image data that extend along a second dimension that is different from the first dimension; and

wherein the first dimension is a vertical dimension in 3D space and the second dimension is a horizontal dimension in the 3D space.

10. The method of claim 6 , further comprising:

storing a library containing attributes of different types of racks;

wherein the components are identified based, at least in part; on one or more attributes from the library.

11. One or more non-transitory machine-readable media storing instructions that are executable to perform operations that implement the method of claim 1 .

12. A system comprising:

an autonomous vehicle comprising a sensor to capture data; and

a control system to perform operations comprising:

detecting a rack containing a volume based on data obtained from the sensor about the rack;

controlling the sensor to detect three-dimensional (3D) data above a bottom of the rack in a region of interest;

detecting the volume based on at least part of the 3D data within a first part of the region of interest and less than a predetermined amount of the 3D data within a second part of the region of interest that is expected to contain the volume based on the at least part of the 3D data within the first part of the region; and

determining, based on the volume and dimensions of an element, whether the volume is large enough to fit the element;

wherein detecting the volume comprises:

detecting less than the predetermined amount of 3D data within the second part of the region of interest;

detecting boundaries of the second part of the region of interest, the boundaries being based on where there is less than the predetermined amount of 3D data; and

determining the volume based on the boundaries.

13. The system of claim 12 , wherein the operations comprise detecting the element, wherein detecting the element comprises:

a controlling the sensor to capture image data in a region of interest containing the element, the image data representing components of the element;

filtering the image data to produce filtered data having less of an amount of data than the image data;

identifying candidates for the components of the element by analyzing the filtered data using a deterministic process, where identifying the candidates for the components comprises locating multiple structures represented by the filtered data;

voting on whether each of the multiple structures meets one or more predefined tolerances, where voting comprises assigning one or more points to each of the multiple structures so that at least two of the multiple structures have scores that exceed a threshold; and

detecting the element by comparing the least two of the multiple structures to a predefined configuration.

14. The system of claim 12 wherein detecting the volume comprises:

detecting the at least part of the 3D data within the first part of the region of interest by outputting light and detecting reflection of the light; and wherein detecting less than the predetermined amount of 3D data comprises detecting less than a predetermined amount of light reflection.

15. The system of claim 14 , wherein detecting less than the predetermined amount of light reflection comprises detecting no light reflection.

16. The system of claim 12 , wherein the control system is configured to control the autonomous vehicle to align the element within the volume.

17. The system of claim 12 , wherein detecting the rack comprises:

controlling the sensor to capture image data in a region of interest containing the rack, the image data representing components of the rack;

identifying the components of the rack by analyzing the image data using a deterministic process; and

detecting the rack based on the components.

18. The system of claim 17 , wherein the components comprise pillars; and

wherein identifying the pillars comprises locating one or more structures represented by the image data that extend along a same dimension and that are separated by a predefined space.

19. The system of claim 18 , wherein the components comprise beams; and

wherein identifying the beams comprises locating one or more structures represented by the image data that extend along a same dimension and that are separated by a predefined space.

20. The system of claim 18 , wherein the components comprise beams and the same dimension is a first dimension; and

wherein identifying the beams comprises locating multiple structures represented by the image data that extend along a second dimension that is different from the first dimension; and

wherein the first dimension is a vertical dimension in 3D space and the second dimension is a horizontal dimension in the 3D space.

21. The system of claim 17 , further comprising:

memory storing a library containing attributes of different types of racks;

wherein the components are identified based, at least in part, on one or more attributes from the library.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE NAME OF THE FIRST INVENTOR PREVIOUSLY RECORDED ON REEL 52985 FRAME 667. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 22, 2024
From: BUKHARI, SYED MOHAMMED JERRAR; BARRON, TYLER; HOLWELL, JUSTIN
To: AUTOGUIDE, LLC
Reel/Frame 067160/0204 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2023
From: AUTOGUIDE, LLC
To: MOBILE INDUSTRIAL ROBOTS INC.
Reel/Frame 064009/0759 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 19, 2020
From: BUKHARI, JERRAR; BARRON, TYLER; HOLWELL, JUSTIN
To: AUTOGUIDE, LLC
Reel/Frame 052985/0667 →
Continuity (1)
Related Publication 20210349468A1 · Nov 11, 2021