Granted Patent B2

US 12,372,966 · App. 18/260,697 · Granted Jul 29, 2025

Inventors: Ryoichi Tsuzaki (Tokyo, JP); Noriaki Takasugi (Tokyo, JP); Toshimitsu Kai (Tokyo, JP)

Assignee: SONY GROUP CORPORATION

G05D1/0223B62D57/02G05D1/0246G05D1/0274

Abstract

Claims (67)

1. A moving apparatus, comprising:

a plurality of visual sensors configured to detect information; and

a central processing unit (CPU) configured to:

acquire the information of the plurality of visual sensors;

generate, based on the information of the plurality of visual sensors, a three-dimensional point cloud data corresponding to each visual sensor of the plurality of visual sensors;

synthesize the generated three-dimensional point cloud data corresponding to each visual sensor of the plurality of visual sensors to generate three-dimensional point cloud synthesis data;

generate a plurality of time-series map images based on the generated three-dimensional point cloud synthesis data and the information of the plurality of visual sensors at different timings;

integrate the generated plurality of time-series map images to generate a piece of time-series map integrated map data;

regenerate the three-dimensional point cloud data based on the generated piece of time-series map integrated map data;

generate, based on the regenerated three-dimensional point cloud data, traveling surface shape data that enables analysis of a shape of a traveling surface of the moving apparatus;

acquire target movement route information of the moving apparatus;

select, based on the acquired target movement route information of the moving apparatus, generation target region of the traveling surface shape data;

generate the traveling surface shape data in the selected generation target region;

determine one of a movement route or a moving direction of the moving apparatus with reference to the traveling surface shape data in the generation target region; and

control movement of the moving apparatus based on the determined one of the movement route or the moving direction of the moving apparatus.

2. The moving apparatus according to claim 1 , wherein the CPU is further configured to select the generation target region of the traveling surface shape data based on the acquired target movement route information of the moving apparatus and a specific search range region.

3. The moving apparatus according to claim 1 , wherein

the moving apparatus is a walking type moving apparatus that moves by a plurality of legs,

the acquired target movement route information is target foot placement information of the walking type moving apparatus, and

the CPU is further configured to select the generation target region of the traveling surface shape data based on the target foot placement information and a specific search range region.

4. The moving apparatus according to claim 3 , wherein

the target foot placement information includes a target foot placement position,

the specific search range region is centered on the target foot placement position, and

the specific search range region is one of a circular region having a specific radius or a rectangular region having a specific area.

5. The moving apparatus according to claim 4 , wherein

the CPU is further configured to select one of a plurality of circular regions or a plurality of rectangular regions centered on the target foot placement position of each of a plurality of steps of each of a plurality of feet of the walking type moving apparatus as the generation target region of the traveling surface shape data,

the plurality of circular regions includes the circular region, and

the plurality of rectangular regions includes the rectangular region.

6. The moving apparatus according to claim 5 , wherein

the specific radius of the circular region increases as a distance increases from a current position of the moving apparatus, and

the specific area of the rectangular region increases as the distance increases from the current position of the moving apparatus.

7. The moving apparatus according to claim 1 , wherein the CPU is further configured to determine a course of the moving apparatus based on the regenerated three-dimensional point cloud data.

8. The moving apparatus according to claim 1 , wherein

the CPU is further configured to select a sensor acquisition image based on the acquired target movement route information of the moving apparatus, and

the sensor acquisition image corresponds to a generation target of the traveling surface shape data.

9. The moving apparatus according to claim 8 , wherein

the acquired information of the plurality of visual sensors corresponds to a plurality of distance images,

the CPU is further configured to select a distance image from the plurality of distance images based on the acquired target movement route information of the moving apparatus, and

the distance image corresponds to a generation target of the generated three-dimensional point cloud data.

10. The moving apparatus according to claim 1 , wherein

the CPU is further configured to select an image region in a sensor acquisition image based on the acquired target movement route information of the moving apparatus, and

the sensor acquisition image corresponds to a generation target of the traveling surface shape data.

11. The moving apparatus according to claim 1 , wherein

the acquired target movement route information includes target speed information of the moving apparatus, and

the CPU is further configured to select the generation target region of the traveling surface shape data based on the target speed information.

12. The moving apparatus according to claim 11 , wherein the CPU is further configured to select the generation target region of the traveling surface shape data based on the target speed information of the moving apparatus and a specific search range region.

13. The moving apparatus according to claim 12 , wherein

the moving apparatus is one of a walking type moving apparatus that moves by a foot, a wheel driving type moving apparatus that moves by a wheel, or a leg wheel type moving apparatus that is a hybrid of the walking type moving apparatus and the wheel driving type moving apparatus, and

the CPU is further configured to select the generation target region of the traveling surface shape data based on the target speed information of one of the walking type moving apparatus, the wheel driving type moving apparatus, or the leg wheel type moving apparatus, and the specific search range region.

14. The moving apparatus according to claim 1 , wherein the CPU is further configured to:

select a region for the generated three-dimensional point cloud data based on the acquired target movement route information,

generate the three-dimensional point cloud data corresponding to the selected region.

15. A moving apparatus control method, comprising:

in a moving apparatus that includes a plurality of visual sensors and a central processing unit (CPU):

detecting information by the plurality of visual sensors;

acquiring, by the CPU, the information of the plurality of visual sensors;

generating, by the CPU, based on the information of the plurality of visual sensors, a three-dimensional point cloud data corresponding to each visual sensor of the plurality of visual sensors;

synthesizing, by the CPU, the generated three-dimensional point cloud data corresponding to each visual sensor of the plurality of visual sensors to generate three-dimensional point cloud synthesis data;

generating, by the CPU, a plurality of time-series map images based on the generated three-dimensional point cloud synthesis data and the information of the plurality of visual sensors at different timings;

integrating, by the CPU, the generated plurality of time-series map images to generate a piece of time-series map integrated map data;

regenerating, by the CPU, the three-dimensional point cloud data based on the generated piece of time-series map integrated map data;

generating, by the CPU, based on the regenerated three-dimensional point cloud data, traveling surface shape data that enables analysis of a shape of a traveling surface of the moving apparatus;

acquiring, by the CPU, target movement route information of the moving apparatus;

selecting, by the CPU, based on the acquired target movement route information of the moving apparatus, a generation target region of the traveling surface shape data;

generating, by the CPU, the traveling surface shape data in the selected generation target region;

determining, by the CPU, one of a movement route or a moving direction of the moving apparatus with reference to the traveling surface shape data in the generation target region; and

controlling, by the CPU, movement of the moving apparatus based on the determined one of the movement route or the moving direction of the moving apparatus.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (11)