IP Library Granted Patent US 12,094,195
Granted Patent B2
US 12,094,195 · App. 16/877,680 · Granted Sep 17, 2024

Identifying stairs from footfalls

Inventor: Adam Komoroski (Waltham, MA)
Assignee: Boston Dynamics, Inc.
G06V20/10B62D57/024B62D57/032G06F18/23G06V10/44G06V10/762
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,094,195
App. No.
16/877,680
Filed
May 19, 2020
Granted
Sep 17, 2024
Kind
B2
Art Unit
3667
USPC
701/400
Abstract

A method of identifying stairs from footfalls includes receiving a plurality of footfall locations of a robot traversing an environment. Each respective footfall location indicates a location where a leg of the robot contacted a support surface. The method also includes determining a plurality of candidate footfall location pairs based on the plurality of footfall locations. The candidate footfall location pair includes a first and a second candidate footfall location. The method further includes clustering the first candidate footfall location into a first cluster group based on a height of the first candidate footfall location and clustering the second candidate footfall location into a second cluster group based on a height of the second candidate footfall location. The method additionally includes generating a stair model by representing each of the cluster groups as a corresponding stair and delineating each stair based on a respective midpoint between each adjacent cluster group.

Claims (55)

1. A method comprising:

receiving, at data processing hardware of a robot, sensor data representing a plurality of footfall locations of the robot, each footfall location of the plurality of footfall locations indicating a location where a distal end of a leg of the robot contacted a support surface beneath the robot;

selecting, by the data processing hardware, a footfall location pair of the plurality of footfall locations, the footfall location pair comprising a first footfall location and a second footfall location, wherein selecting the footfall location pair is based on determining that at least one of a vertical distance between the first footfall location and the second footfall location satisfies a first threshold associated with one or more heights of one or more stair structures, or a horizontal distance between the first footfall location and the second footfall location satisfies a second threshold associated with one or more depths of one or more stair structures;

clustering, by the data processing hardware, each footfall location of the footfall location pair into a respective cluster group of footfall locations of a plurality of cluster groups of footfall locations based on a height of the respective footfall location and based on selecting the footfall location pair;

generating, by the data processing hardware, a stair model of a set of stairs based on the received sensor data by representing each of the plurality of cluster groups of footfall locations as a corresponding stair of the set of stairs; and

instructing, by the data processing hardware, traversal of the set of stairs by at least one robot.

2. The method of claim 1 , wherein:

the first threshold corresponds to a height of a stair riser; and

first footfall location and the second footfall location satisfies a stair depth threshold, the

the second threshold corresponds to a depth of a stair.

3. The method of claim 1 , wherein the plurality of cluster groups of footfall locations indicates an orientation associated with the stair model of the set of stairs, the orientation corresponding to a vector direction that the set of stairs ascends or descends within an environment.

4. The method of claim 1 , further comprising:

identifying, by the data processing hardware, from the plurality of cluster groups of footfall locations, a first cluster group of footfall locations and a second cluster group of footfall locations adjacent to the first cluster group of footfall locations, each of the first cluster group of footfall locations and the second cluster group of footfall locations comprising one or more footfall locations;

selecting, by the data processing hardware, a third footfall location from the one or more footfall locations of the first cluster group of footfall locations and a fourth footfall location from the one or more footfall locations of the second cluster group of footfall locations, wherein the third footfall location and the fourth footfall location are separated by a particular horizontal distance; and

generating, by the data processing hardware, a stair edge for the stair model of the set of stairs based on a horizontal midpoint between the third footfall location and the fourth footfall location.

5. The method of claim 4 , wherein:

the plurality of cluster groups of footfall locations indicates an orientation associated with the stair model of the set of stairs, the orientation corresponding to a vector direction that the set of stairs ascends or descends within an environment, and the stair edge extends in a direction perpendicular to the vector direction.

6. The method of claim 1 , wherein instructing traversal of the set of stairs by the at least one robot comprises communicating the stair model of the set of stairs to a control system of the robot to navigate the set of stairs in an autonomous drive mode, wherein the at least one robot comprises the robot.

7. The method of claim 1 , further comprising:

detecting, by the data processing hardware, that the robot is approaching a location associated with the stair model of the set of stairs; and

instructing, by the data processing hardware, movement by the robot to an orientation associated with the stair model of the set of stairs, the orientation corresponding to a vector direction that the set of stairs ascends or descends within an environment.

8. The method of claim 7 , wherein instructing movement by the robot to the orientation comprises instructing movement by the robot such that one or more sensors of the robot face the vector direction.

9. The method of claim 1 , wherein the set of stairs comprise a set of stairs within an environment, the method further comprising augmenting, by the data processing hardware, a perception map of the environment with the stair model of the set of stairs.

10. The method of claim 1 , wherein the robot is a quadruped robot.

11. The method of claim 1 , wherein instructing traversal of the set of stairs by the at least one robot comprises instructing traversal of the set of stairs by the at least one robot according to the stair model of the set of stairs.

12. The method of claim 1 , wherein instructing traversal of the set of stairs by the at least one robot comprises providing the stair model of the set of stairs to a control system.

13. The method of claim 1 , wherein instructing traversal of the set of stairs by the at least one robot comprises providing the stair model of the set of stairs to a control system wherein the control system comprises a control system of the robot, wherein the at least one robot comprises the robot.

14. The method of claim 1 , wherein instructing traversal of the set of stairs by the at least one robot comprises providing the stair model of the set of stairs to a control system, wherein the control system comprises a control system of a remote system.

15. A robot comprising:

a body;

two or more legs coupled to the body and configured to traverse an environment; and

a stair modeling system in communication with the robot, the stair modeling system comprising data processing hardware and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:

receiving sensor data representing a plurality of footfall locations of at least one leg of the two or more legs, each footfall location of the plurality of footfall locations indicating a location where a distal end of a corresponding leg of the two or more legs contacted a support surface beneath the robot;

selecting a footfall location pair of the plurality of footfall locations, the footfall location pair comprising a first footfall location and a second footfall location, wherein selecting the footfall location pair is based on determining that at least one of a vertical distance between the first footfall location and the second footfall location satisfies a first threshold associated with one or more heights of one or more stair structures, or a horizontal distance between the first footfall location and the second footfall location satisfies a second threshold associated with one or more depths of one or more stair structures;

clustering each footfall location of the footfall location pair into a respective cluster group of footfall locations of a plurality of cluster groups of footfall locations based on a height of the respective footfall location and based on selecting the footfall location pair;

generating a stair model of a set of stairs based on the received sensor data by representing each of the plurality of cluster groups of footfall locations as a corresponding stair of the set of stairs; and

instructing traversal of the set of stairs by at least one robot.

16. The robot of claim 15 , wherein:

the first threshold corresponds to a height of a stair riser; and

the second threshold corresponds to a depth of a stair tread.

17. The robot of claim 15 , wherein the plurality of cluster groups of footfall locations indicates an orientation associated with the stair model of the set of stairs, the orientation corresponding to a vector direction that the set of stairs ascends or descends within an environment.

18. The robot of claim 15 , wherein the operations further comprise:

identifying from the plurality of cluster groups of footfall locations, a first cluster group of footfall locations and a second cluster group of footfall locations adjacent to the first cluster group of footfall locations, each of the first cluster group of footfall locations and the second cluster group of footfall locations comprising one or more footfall locations;

selecting a third footfall location from the one or more footfall locations of the first cluster group of footfall locations and a fourth footfall location from the one or more footfall locations of the second cluster group of footfall locations, wherein the third footfall location and the fourth footfall location are separated by a particular horizontal distance; and

generating a stair edge for the stair model of the set of stairs based on a horizontal midpoint between the third footfall location and the fourth footfall location.

19. The robot of claim 18 , wherein:

the plurality of cluster groups of footfall locations indicates an orientation associated with the stair model of the set of stairs, the orientation corresponding to a vector direction that the set of stairs ascends or descends within the environment, and

the stair edge extends in a direction perpendicular to the vector direction.

20. The robot of claim 15 , wherein instructing traversal of the set of stairs by the at least one robot comprises communicating the stair model of the set of stairs to a control system of the robot to navigate the set of stairs in an autonomous drive mode, wherein the at least one robot comprises the robot.

21. The robot of claim 15 , wherein the operations further comprise:

detecting that the robot is approaching a location associated with the stair model of the set of stairs; and

instructing movement by the robot to an orientation associated with the stair model of the set of stairs, the orientation corresponding to a vector direction that the set of stairs ascends or descends within the environment.

22. The robot of claim 21 , wherein instructing movement by the robot to the orientation comprises instructing movement by the robot such that one or more sensors of the robot face the vector direction.

23. The robot of claim 15 , wherein the operations further comprise augmenting a perception map of the environment with the stair model of the set of stairs.

24. The robot of claim 15 , wherein the two or more legs comprise four legs defining a quadruped robot.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE APPLICATIONS NUMBERS 63127573 AND 11/302759 AND THE CITY OF THE ASSIGNEE PREVIOUSLY RECORDED AT REEL: 057111 FRAME: 0202. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 27, 2021
From: BOSTON DYNAMICS, INC.
To: BOSTON DYNAMICS, INC.
Reel/Frame 057964/0415 →
CHANGE OF NAME Recorded Oct 5, 2021
From: BOSTON DYNAMICS, INC.
To: BOSTON DYNAMICS, INC.
Reel/Frame 057711/0202 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 15, 2020
From: KOMOROSKI, ADAM
To: BOSTON DYNAMICS, INC.
Reel/Frame 052937/0318 →
Continuity (2)
Provisional Application 63012614 · Apr 20, 2020
Related Publication 20210323618A1 · Oct 21, 2021
Cited By (8)
US 1,056,983 US 1,064,015 US 1,094,505 US 1,098,230 US 12,585,292 US 12,632,051 US 12,637,158 US 12,705,880