IP Library › Granted Patent US 12,711,632
Granted Patent B2
US 12,711,632 · App. 18/386,915 · Granted Aug 18, 2026

Information processing apparatus, information processing method, program, and flight object

Inventors: Masahiko Toyoshi (Kanagawa, JP); Kohei Urushido (Kanagawa, JP); Shun Lee (Saitama, JP); Shinichiro Abe (Kanagawa, JP); Takuto Motoyama (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
G06T7/194G06T7/30G06V20/647B64U2101/30B64U2201/10G06T2207/10028G06T2207/10032G06T2207/30181G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,711,632
App. No.
18/386,915
Granted
Aug 18, 2026
Kind
B2
Abstract

To enable high-speed autonomous flight of a flight object. A three-dimensional real-time observation result is generated on the basis of self-position estimation information and three-dimensional distance measurement information. A prior map corresponding to a three-dimensional real-time observation result is acquired. The three-dimensional real-time observation result and the prior map are aligned. After the alignment, the three-dimensional real-time observation result is expanded on the basis of the prior map. A flight route is set on the basis of the three-dimensional real-time observation result having been expanded. In the flight object such as a drone, a somewhat long flight route can be accurately calculated at a time in a global behavior plan, which enables high-speed autonomous flight of the flight object.

Claims (63)

1 . An information processing apparatus comprising:

circuitry configured to

generate a three-dimensional real-time observation result based on self-position estimation information and three-dimensional distance measurement information obtained from a sensor,

acquire a prior map corresponding to the three-dimensional real-time observation result,

align the three-dimensional real-time observation result with the prior map,

expand the three-dimensional real-time observation result based on the prior map, and

determine a flight route based on the three-dimensional real-time observation result having been expanded,

wherein the prior map includes information related to recognized environmental structure including at least one of a topography, a wall, or a building,

wherein the circuitry is further configured to expand the three-dimensional real-time observation result corresponding to semantic segmentation of the recognized environmental structure so that an expanded area of the three-dimensional real-time result is continuous for same semantics within and outside a detection range of the sensor, and

wherein the circuitry is further configured to

perform the semantic segmentation on the three-dimensional real-time observation result to determine whether continuity is present in semantics at a connection portion with respect to previously performed semantic segmentation of the prior map in relation to the recognized environmental structure, and

expand, with a result of the semantic segmentation, a plane at the connection portion.

2 . The information processing apparatus according to claim 1 ,

wherein the circuitry is configured to expand the three-dimensional real-time observation result in an unobservable area.

3 . The information processing apparatus according to claim 1 ,

wherein the circuitry is further configured to

perform plane detection on the three-dimensional real-time observation result, and

expand, with a result of the plane detection, the plane based on information regarding the prior map.

4 . The information processing apparatus according to claim 1 ,

wherein the three-dimensional real-time observation result corresponds to a three-dimensional occupancy grid map.

5 . The information processing apparatus according to claim 1 ,

wherein the circuitry is configured to acquire the prior map from a different information processing apparatus through communication.

6 . The information processing apparatus according to claim 5 ,

wherein the prior map corresponds to a map based on the three-dimensional real-time observation result generated by the different information processing apparatus.

7 . The information processing apparatus according to claim 1 ,

wherein the prior map corresponds to a map obtained by processing of cutting the three-dimensional real-time observation result at a certain height and converting the cut three-dimensional real-time observation result into a bird's-eye view.

8 . The information processing apparatus according to claim 5 ,

wherein the prior map corresponds to a map obtained by processing of reducing resolution of the three-dimensional real-time observation result to an extent enabling the communication.

9 . An information processing method comprising:

generating a three-dimensional real-time observation result based on self-position estimation information and three-dimensional distance measurement information obtained from a sensor;

acquiring a prior map corresponding to the three-dimensional real-time observation result;

aligning the three-dimensional real-time observation result with the prior map; and

expanding the three-dimensional real-time observation result based on the prior map; and

determining a flight route based on the three-dimensional real-time observation result having been expanded, and

wherein the prior map includes information related to recognized environmental structure including at least one of a topography, a wall, or a building,

wherein the expanding of the three-dimensional real-time observation result corresponds to semantic segmentation of the recognized environmental structure so that an expanded area of the three-dimensional real-time result is continuous for same semantics within and outside a detection range of the sensor, and

wherein the method further comprises:

performing the semantic segmentation on the three-dimensional real-time observation result to determine whether continuity is present in semantics at a connection portion with respect to previously performed semantic segmentation of the prior map in relation to the recognized environmental structure; and

expanding, with a result of the semantic segmentation, a plane at the connection portion.

10 . A flight object comprising:

circuitry configured to

generate a three-dimensional real-time observation result based on self-position estimation information and three-dimensional distance measurement information obtained from a sensor,

acquire a prior map corresponding to the three-dimensional real-time observation result,

align the three-dimensional real-time observation result with the prior map,

expand the three-dimensional real-time observation result based on the prior map, and

determine a flight route based on the three-dimensional real-time observation result having been expanded,

wherein the prior map includes information related to recognized environmental structure including at least one of a topography, a wall, or a building,

wherein the expanding of the three-dimensional real-time observation result corresponds to semantic segmentation of the recognized environmental structure so that an expanded area of the three-dimensional real-time result is continuous for same semantics within and outside a detection range of the sensor, and

wherein the circuitry is further configured to

perform the semantic segmentation on the three-dimensional real-time observation result to determine whether continuity is present in semantics at a connection portion with respect to previously performed semantic segmentation of the prior map in relation to the recognized environmental structure, and

expand, with a result of the semantic segmentation, a plane at the connection portion.

11 . The flight object according to claim 10 ,

wherein the circuitry is configured to acquire the prior map from a different flight object from the flight object through communication.

12 . The flight object according to claim 11 ,

wherein the prior map corresponds to a map based on the three-dimensional real-time observation result generated by the different flight object.

13 . The flight object according to claim 10 ,

wherein the prior map corresponds to a map obtained by processing of cutting the three-dimensional real-time observation result at a certain height and converting the cut three-dimensional real-time observation result into a bird's-eye view.

14 . The flight object according to claim 10 ,

wherein the prior map corresponds to a map obtained by processing of reducing resolution of the three-dimensional real-time observation result to an extent enabling the communication.

15 . The flight object according to claim 10 ,

wherein the circuitry is further configured to

perform plane detection on the three-dimensional real-time observation result, and

expand, with a result of the plane detection, the plane based on information regarding the prior map.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2023
From: TOYOSHI, MASAHIKO; URUSHIDO, KOHEI; LEE, SHUN; ABE, SHINICHIRO; MOTOYAMA, TAKUTO
To: SONY GROUP CORPORATION
Reel/Frame 065456/0439 →
Priority Claims (1)
JP 2019-193922 · Oct 25, 2019 · national
Continuity (2)
Continuation 17767347 · Oct 14, 2020
Related Publication 20240070873A1 · Feb 29, 2024
References Cited (27)
US 11393114B1 · Ebrahimi Afrouzi · 2022 [cited by examiner]
US 11461912B2 · Matthies et al. · 2022 [cited by applicant]
US 11600022B2 · Momcilovic · 2023 [cited by examiner]
US 20150279083A1 · Pradeep · 2015 [cited by examiner]
US 20160236683A1 · Eggert · 2016 [cited by examiner]
US 20170337824A1 · Chen · 2017 [cited by applicant]
US 20180218533A1 · Millin et al. · 2018 [cited by applicant]
US 20180357796A1 · Bishop et al. · 2018 [cited by applicant]
US 20190050000A1 · Kennedy · 2019 [cited by examiner]
US 20190156684A1 · Pohl · 2019 [cited by examiner]
US 20190212752A1 · Fong et al. · 2019 [cited by applicant]
US 20190241263A1 · Gu · 2019 [cited by examiner]
US 20190311533A1 · Doh · 2019 [cited by examiner]
US 20200025931A1 · Liang et al. · 2020 [cited by applicant]
US 20200025935A1 · Liang · 2020 [cited by examiner]
US 20200082614A1 · Xu · 2020 [cited by examiner]
CN 107145578A · 2017 [cited by applicant]
CN 108124489A · 2018 [cited by applicant]
CN 109163718A · 2019 [cited by applicant]
EP 3056861A1 · 2020 [cited by applicant]
JP H07332980A · 1995 [cited by applicant]
JP H09062353A · 1997 [cited by applicant]
JP 2005092820A · 2005 [cited by applicant]
JP 2007249632A · 2007 [cited by applicant]
JP 2014139538A · 2014 [cited by applicant]
JP 2019045892A · 2019 [cited by applicant]
WO WO2018073879A1 · 2018 [cited by applicant]