Granted Patent B2

US 12,736,345 · App. 18/843,509 · Granted Sep 15, 2026

Inventor: Konstantin Stulov (Redwood City, CA)

Assignee: Bear Robotics, Inc.

G01C21/005B25J9/1664B25J9/1697G06T1/0014G06T7/70G06T7/73G06T2207/10016G06T2207/20021G06T2207/20081G06T2207/20084

Abstract

Claims (57)

1 . A method to generate an image-based localization model for mobile robot navigation, the method comprising:

performing data collection at a plurality of different service locations, to which a fleet of mobile robots is deployable, to generate collected data;

dividing the collected data into a plurality of blocks of consecutive portions of the collected data;

generating a first image-based localization model for a first service location of the plurality of different service locations, using the collected data;

generating a second image-based localization model for a second service location of the plurality of different service locations, using the collected data;

deploying the first image-based localization model to a first mobile robot of the fleet of mobile robots, the first mobile robot being deployed at the first service location of the plurality of different service locations, the first mobile robot to use the first image-based localization model to navigate the first service location;

deploying a second image-based localization to a second mobile robot of the fleet of mobile robots, the second mobile robot being deployed at the second service location of the plurality of different service locations, the second mobile robot to use the second image-based localization model to navigate the second service location, wherein:

the collected data comprises image data, and the dividing comprises dividing the image data into equally-sized blocks of consecutive images;

the generating of the first image-based localization model comprises performing training, development, and testing of the first image-based localization model using different ones of the equally-sized blocks of consecutive images;

the method further comprises shuffling the different ones of the equally-sized blocks of consecutive images before allocating the different ones of the equally-sized blocks of consecutive images to each of the training, the development, and the testing of the image-based localization model; and

at least one of the first mobile robot and the second mobile robot executes a localization service module that uses the respective image-based localization model to serve image-based localization responses to localization requests;

at the first mobile robot, performing a reboot operation; and

responsive to the reboot operation and at the first mobile robot, automatically checking remote storage to determine that a new image-based localization model has been generated and stored at the remote storage.

2 . The method of claim 1 , comprising randomly allocating the different ones of the equally-sized blocks of consecutive images to each of the training, the development, and the testing of the first image-based localization model.

3 . The method of claim 1 , comprising automatically determining a size of each of the equally-sized blocks of consecutive images by balancing the size based on a balanced distribution across grid cells of a map grid of the first service location.

4 . The method of claim 1 , wherein:

the image data comprises image timestamps, each timestamp indicating a capture time period for a corresponding image;

the image timestamps for images in each of the equally-sized blocks of consecutive images indicate a same capture time period for all images in the corresponding block; and

the method further comprises allocating different ones of the equally-sized blocks of consecutive images to each of the training, the development and the testing of the first image-based localization model based on capture time periods for images in the equally-sized blocks.

5 . The method of claim 4 , wherein the allocating of the different ones of the equally-sized blocks of consecutive images to each of the training, the development and the testing of the first image-based localization model comprises allocating one or more blocks of consecutive images for a first capture time period to only one of the training, development and testing of the first image-based localization model.

6 . The method of claim 1 , wherein:

the image data comprises mission identifiers, each image in the image data being associated with a mission identifier;

images in each of the equally-sized blocks of consecutive images being associated with a unique mission identifier of a plurality of mission identifiers; and

the method further comprises allocating different ones of the equally-sized blocks of consecutive images to each of the training, the development and the testing of the first image-based localization model based on mission identifiers for images in the equally-sized blocks.

7 . The method of claim 5 , wherein the allocating of the different ones of the equally-sized blocks of consecutive images to each of the training, the development and the testing of the first image-based localization model comprises allocating one or more blocks of consecutive images for a first mission identifier to only one of the training, development and testing of the first image-based localization model.

8 . The method of claim 1 , wherein the generating of the first image-based localization model for the first service location of the plurality of different service locations further comprises:

retrieving first location data specific to the first service location from the collected data;

generating a plurality of online model performance metrics based on the first location data related to a current version of the first image-based localization model;

using at least a portion of the first location data, performing an offline evaluation of the current version of the first image-based localization model; and

automatically generating a new version of the first image-based localization model for the first service location, based on the offline evaluation.

9 . The method of claim 8 , wherein the plurality of online model performance metrics is reported by a navigation stack of the first mobile robot.

10 . The method of claim 8 , comprising retraining the image-based localization model using the plurality of online model performance metrics.

11 . The method of claim 1 , further comprising:

responsive to determining that the new image-based localization model has been generated and is stored at the remote storage, storing the new image-based localization model to local memory at the first mobile robot; and

at the first mobile robot, serving image-based localization responses to localization requests.

12 . The method of claim 1 , wherein the automatic checking of the remote storage to determine that the new image-based localization model has been generated and stored at the remote storage comprises checking that a retrained version of a current image-based localization model has been generated.

13 . The method of claim 1 , wherein the automatic checking of the remote storage to determine that the new image-based localization model has been generated and stored at the remote storage comprises checking whether a new image-based localization model type has been generated.

14 . The method of claim 1 , further comprising:

at a cloud storage, maintaining a plurality of image-based localization model types, and a plurality of versions of each of the plurality of image-based localization model types; and

at the first mobile robot, implementing fallback logic to allow the first mobile robot to use the plurality of image-based localization model types and the plurality of versions of each of the plurality of image-based localization model types.

15 . The method of claim 14 , wherein the maintaining comprises maintaining a file structure to store the plurality of image-based localization model types and the plurality of versions of each of the plurality of image-based localization model types within the cloud storage.

16 . The method of claim 15 , wherein the fallback logic is included in at a robotics stack of the first mobile robot and accesses the file structure in order to access at least one of the plurality of image-based localization model types or the plurality of versions of each of the plurality of image-based localization model types within the cloud storage.

17 . A computing apparatus comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, configure the apparatus to:

perform data collection at a plurality of different service locations, to which a fleet of mobile robots is deployable, to generate collected data;

divide the collected data into a plurality of blocks of consecutive portions of the collected data;

generate a first image-based localization model for a first service location of the plurality of different service locations, using the collected data;

generate a second image-based localization model for a second service location of the plurality of different service locations, using the collected data;

deploy the first image-based localization model to a first mobile robot of the fleet of mobile robots, the first mobile robot being deployed at the first service location of the plurality of different service locations, the first mobile robot to use the first image-based localization model to navigate the first service location;

deploy a second image-based localization to a second mobile robot of the fleet of mobile robots, the second mobile robot being deployed at a second service location of the plurality of different service locations, the second mobile robot to use the second image-based localization model to navigate the second service location, wherein:

the collected data comprises image data, and dividing the collected data comprises dividing the image data into equally-sized blocks of consecutive images;

generating the first image-based localization model comprises performing training, development, and testing of the first image-based localization model using different ones of the equally-sized blocks of consecutive images;

the instructions further configure the apparatus to shuffle the different ones of the equally-sized blocks of consecutive images before allocating the different ones of the equally-sized blocks of consecutive images to each of the training, the development, and the testing of the image-based localization model; and

at least one of the first mobile robot and the second mobile robot executes a localization service module that uses the respective image-based localization model to serve image-based localization responses to localization requests;

at the first mobile robot, performing a reboot operation; and

responsive to the reboot operation and at the first mobile robot, automatically checking remote storage to determine that a new image-based localization model has been generated and stored at the remote storage.

Assignments (1)

Continuity (2)

References Cited (77)