IP Library Granted Patent US 11,592,826
Granted Patent B2
US 11,592,826 · App. 16/719,541 · Granted Feb 28, 2023

Method, system and apparatus for dynamic loop closure in mapping trajectories

Inventors: Ziehen Huang (Richmond Hill, CA); Harsoveet Singh (Mississauga, CA); Jonathan Kelly (Toronto, CA); Mahyar Vajedi (Mississauga, CA)
Assignee: Zebra Technologies Corporation
G05D1/0221G05D1/0274G05D2201/0216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,592,826
App. No.
16/719,541
Granted
Feb 28, 2023
Kind
B2
Abstract

A method for dynamic loop closure in a mobile automation apparatus includes: obtaining mapping trajectory data defining a plurality of trajectory segments traversing a facility to be mapped; controlling a locomotive mechanism of the apparatus to traverse a current segment; generating a sequence of keyframes for the current segment using sensor data captured via a navigational sensor of the apparatus; and, for each keyframe: determining an estimated apparatus pose based on the sensor data and a preceding estimated pose corresponding to a preceding keyframe; and, determining a noise metric defining a level of uncertainty associated with the estimated pose relative to the preceding estimated pose; determining, for a selected keyframe, an accumulated noise metric based on the noise metrics for the selected keyframe and each previous keyframe; and when the accumulated noise metric exceeds a threshold, updating the mapping trajectory data to insert a repetition of one of the segments.

Claims (47)

1. A method for dynamic loop closure in a mobile automation apparatus, the method comprising:

at a navigational controller of the mobile automation apparatus, obtaining mapping trajectory data defining a plurality of trajectory segments traversing a facility to be mapped;

at the navigational controller, controlling a locomotive mechanism of the mobile automation apparatus to traverse a current one of the trajectory segments;

at the navigational controller, generating a sequence of keyframes for the current trajectory segment using sensor data captured via a navigational sensor of the mobile automation apparatus; and, for each keyframe:

determining an estimated pose of the mobile automation apparatus based on (i) the sensor data and (ii) a preceding estimated pose corresponding to a preceding one of the keyframes; and,

determining a noise metric defining a level of uncertainty associated with the estimated pose relative to the preceding estimated pose;

determining, for a selected keyframe, an accumulated noise metric based on the noise metrics for the selected keyframe and each previous keyframe in the sequence; and

when the accumulated noise metric exceeds a threshold, updating the mapping trajectory data to insert a repetition of one of the trajectory segments.

2. The method of claim 1 , further comprising:

at the navigational controller, responsive to obtaining the mapping trajectory data, selecting one of the segments as the current segment, and presenting the current segment via an output assembly.

3. The method of claim 2 , wherein presenting the current segment comprises at least one of (i) rendering the current segment on a display coupled to the navigational controller, and (ii) transmitting the current segment to a client device.

4. The method of claim 1 , further comprising:

receiving operational commands via an input assembly connected to the navigational controller; and

controlling the locomotive mechanism according to the operational commands.

5. The method of claim 1 , wherein the navigational sensor includes at least one of a motion sensor and an environmental sensor.

6. The method of claim 1 , further comprising determining respective accumulated noise metrics for each keyframe.

7. The method of claim 1 , wherein updating the mapping trajectory data includes identifying a previously completed trajectory segment, and inserting the previously completed trajectory segment into the trajectory data.

8. The method of claim 7 , further comprising inserting the previously completed trajectory segment into the trajectory data following the current trajectory segment.

9. The method of claim 1 , further comprising, for each keyframe:

determining whether the keyframe matches any one of the previous keyframes; and,

when the determination is affirmative, storing a loop closure noise metric linking the keyframe and the previous keyframe.

10. The method of claim 9 , wherein the loop closure noise metric has a predefined uncertainty value.

11. A mobile automation apparatus comprising:

a memory;

a locomotive mechanism;

a navigational sensor;

a navigational controller connected to the memory, the locomotive mechanism and the navigational sensor; the navigational controller configured to:

obtain, from the memory, mapping trajectory data defining a plurality of trajectory segments traversing a facility to be mapped;

control the locomotive mechanism to traverse a current one of the trajectory segments;

generate a sequence of keyframes for the current trajectory segment using sensor data captured via the navigational sensor; and, for each keyframe:

determine an estimated pose of the mobile automation apparatus based on (i) the sensor data and (ii) a preceding estimated pose corresponding to a preceding one of the keyframes; and,

determine a noise metric defining a level of uncertainty associated with the estimated pose relative to the preceding estimated pose;

determine, for a selected keyframe, an accumulated noise metric based on the noise metrics for the selected keyframe and each previous keyframe in the sequence; and

when the accumulated noise metric exceeds a threshold, update the mapping trajectory data to insert a repetition of one of the trajectory segments.

12. The mobile automation apparatus of claim 11 , wherein the navigational controller is further configured, responsive to obtaining the mapping trajectory data, to select one of the segments as the current segment, and present the current segment via an output assembly.

13. The mobile automation apparatus of claim 12 , wherein the navigational controller is further configured to present the current segment by at least one of (i) rendering the current segment on a display coupled to the navigational controller, and (ii) transmitting the current segment to a client device.

14. The mobile automation apparatus of claim 11 , wherein the navigational controller is further configured to:

receive operational commands via an input assembly connected to the navigational controller; and

control the locomotive mechanism according to the operational commands.

15. The mobile automation apparatus of claim 11 , wherein the navigational sensor includes at least one of a motion sensor and an environmental sensor.

16. The mobile automation apparatus of claim 11 , wherein the navigational controller is further configured to determine respective accumulated noise metrics for each keyframe.

17. The mobile automation apparatus of claim 11 , wherein the navigational controller is further configured, to update the mapping trajectory data, to identify a previously completed trajectory segment, and insert the previously completed trajectory segment into the trajectory data.

18. The mobile automation apparatus of claim 17 , wherein the navigational controller is further configured to insert the previously completed trajectory segment into the trajectory data following the current trajectory segment.

19. The mobile automation apparatus of claim 11 , wherein the navigational controller is further configured, for each keyframe, to:

determine whether the keyframe matches any one of the previous keyframes; and,

when the determination is affirmative, store a loop closure noise metric in the memory linking the keyframe and the previous keyframe.

20. The mobile automation apparatus of claim 19 , wherein the loop closure noise metric has a predefined uncertainty value.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2022
From: HUANG, ZICHEN; SINGH, HARSOVEET; KELLY, JONATHAN; VAJEDI, MAHYAR
To: ZIH CORP.
Reel/Frame 062106/0502 →
MERGER Recorded Dec 15, 2022
From: ZIH CORP.
To: ZEBRA TECHNOLOGIES CORPORATION
Reel/Frame 062106/0607 →
SECURITY INTEREST Recorded Apr 12, 2021
From: ZEBRA TECHNOLOGIES CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 056471/0906 →
RELEASE OF SECURITY INTEREST - 364 - DAY Recorded Mar 5, 2021
From: JPMORGAN CHASE BANK, N.A.
To: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
Reel/Frame 056036/0590 →
SECURITY INTEREST Recorded Sep 1, 2020
From: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053841/0212 →
Cited By (1)
US 12,314,055