IP Library Granted Patent US 12,367,788
Granted Patent B2
US 12,367,788 · App. 17/414,933 · Granted Jul 22, 2025

Operation simulation system and method for unmanned aircraft

Inventors: Mitsuru Nakazawa (Tokyo, JP); Jun Takizawa (Chino, JP)
Assignee: RAKUTEN GROUP, INC.
G09B9/085G09B9/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,367,788
App. No.
17/414,933
Granted
Jul 22, 2025
Kind
B2
Abstract

Provided is a flight simulation system for an unmanned aerial vehicle capable of performing pilot training of the unmanned aerial vehicle in an environment closer to a real environment. The flight simulation system for an unmanned aerial vehicle includes: an operation data acquisition unit configured to acquire operation data of a virtual unmanned aerial vehicle performed by a trainee; a simulator unit configured to calculate, based on geospatial data of a real space and the operation data, a current position of the virtual unmanned aerial vehicle in the real space; and a display unit configured to generate an image of the virtual unmanned aerial vehicle so that the virtual unmanned aerial vehicle is visually recognizable at the current position in the real space, and to output the generated image to the trainee.

Claims (25)

1. A flight simulation system for an unmanned aerial vehicle, comprising:

one or more processor;

a memory storing instructions which, when executed by said one or more processor, cause said one or more processors to perform operations comprising:

acquiring operation data of a virtual unmanned aerial vehicle performed by a trainee;

acquiring environment data of a real space repeatedly detected by a real unmanned aerial vehicle flying in the real space,

calculating, based on geospatial data of the real space, the operation data, and the environment data, a current position of the virtual unmanned aerial vehicle, the current position being in the real space,

repeatedly updating the current position of the virtual unmanned aerial vehicle in real time based on the repeatedly detected environment data,

generating an image of the virtual unmanned aerial vehicle so that the virtual unmanned aerial vehicle is visually recognizable at the current position in the real space, and to output the generated image to a display; and

performing flight control of the real unmanned aerial vehicle,

wherein the real unmanned aerial vehicle and the virtual unmanned aerial vehicle are operated simultaneously in flight.

2. The flight simulation system for an unmanned aerial vehicle according to claim 1 , wherein the display is an optical see-through type or video see-through type head-mounted display configured to display the image so that the image of the virtual unmanned aerial vehicle at the current position is overlapped on a scene of the real space.

3. The flight simulation system for an unmanned aerial vehicle according to claim 1 , wherein the performing comprises controlling a position of the real unmanned aerial vehicle based on the current position of the virtual unmanned aerial vehicle.

4. The flight simulation system for an unmanned aerial vehicle according to claim 3 , wherein the performing comprises limiting an action in accordance with the current position of the virtual unmanned aerial vehicle when the virtual unmanned aerial vehicle is abnormal.

5. The flight simulation system for an unmanned aerial vehicle according to claim 1 , wherein the performing comprises controlling a position of the real unmanned aerial vehicle based on a position of the trainee and a line-of-sight direction of the trainee.

6. A flight simulation method for an unmanned aerial vehicle, comprising:

acquiring operation data of a virtual unmanned aerial vehicle performed by a trainee;

acquiring environment data of a real space repeatedly detected by a real unmanned aerial vehicle flying in the real space,

acquiring geospatial data of the real space;

calculating, based on the geospatial data, the operation data and the environmental data, a current position of the virtual unmanned aerial vehicle, the current position being in the real space,

repeatedly updating the current position of the virtual unmanned aerial vehicle in real time based on the repeatedly detected environment data,

generating an image of the virtual unmanned aerial vehicle so that the virtual unmanned aerial vehicle is visually recognizable at the current position in the real space; and

performing flight control of the real unmanned aerial vehicle,

wherein the real unmanned aerial vehicle and the virtual unmanned aerial vehicle are operated simultaneously in flight.

7. The flight simulation system for an unmanned aerial vehicle according to claim 1 , wherein the current position of the real unmanned aerial vehicle is controlled based on the current position of the virtual unmanned aerial vehicle.

8. The flight simulation system for an unmanned aerial vehicle according to claim 1 , wherein the real unmanned aerial vehicle is never positioned in the line-of-sight direction of the user.

Assignments (2)
CHANGE OF NAME Recorded Jul 13, 2021
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 056845/0831 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2021
From: NAKAZAWA, MITSURU; TAKIZAWA, JUN
To: RAKUTEN, INC.
Reel/Frame 056569/0181 →
Continuity (1)
Related Publication 20220068155A1 · Mar 3, 2022
References Cited (26)
US 8834163B2 · Pollak · 2014 [cited by examiner]
US 9589476B2 · Zhang · 2017 [cited by examiner]
US 10086954B2 · Zhong · 2018 [cited by examiner]
US 10134298B2 · Tao · 2018 [cited by examiner]
US 10134299B2 · Zhang · 2018 [cited by examiner]
US 10290227B2 · Quinn · 2019 [cited by examiner]
US 10614724B2 · Catani · 2020 [cited by examiner]
US 11043138B2 · Nissen · 2021 [cited by examiner]
US 11099558B2 · Huang · 2021 [cited by examiner]
US 11189189B2 · Rosolio · 2021 [cited by examiner]
US 20150364057A1 · Catani · 2015 [cited by examiner]
US 20160275809A1 · Cho · 2016 [cited by examiner]
US 20170010611A1 · Tao · 2017 [cited by examiner]
US 20180121149A1 · Tao et al. · 2018 [cited by applicant]
US 20200394009A1 · Tao et al. · 2020 [cited by applicant]
US 20230386357A1 · Fujisawa · 2023 [cited by examiner]
CN 104029825A · 2014 [cited by examiner]
JP 2017524958A · 2017 [cited by applicant]
WO 2016049924A1 · 2016 [cited by applicant]
WO WO2016050099A1 · 2016 [cited by examiner]
Office Action of Jul. 12, 2022, for corresponding CN Patent Application No. 201980086564.2 with partial English translation pp. 1-14. [cited by applicant]
Office Action of Sep. 21, 2021, for corresponding JP Patent Application No. 2021-523522 with partial English translation pp. 1-4. [cited by applicant]
What is “Mixed Reality (MR)”? How MR differs from VR and AR and how it can be used, Aug. 19, 2019, https://www.mirai-lab.co.jp/info/588, retrieved on Sep. 8, 2021, pp. 1-8 (See partial English translation of the Office … [cited by applicant]
Web Archive in the World, Internet Data Collection and Preservation Project of National Diet Library, Aug. 30, 2018, https://web.archive.org/web/20180830051930 /https://xooms.co.jp/news 170315/, retrieved on Sep. 8, 202… [cited by applicant]
MoguLive—[online], May 21, 2019 [retrieved: Nov. 8, 2019], Internet: <URL: http://web.archive.org/web/20190521200323/ https://www.moguravr.com/epson-ar-flight-simulator/> <URL: https://www.moguravr.com/ epson-ar-flight-… [cited by applicant]
The International Search Report for PCT/JP2019/034135 with English translation, p. 1-4. [cited by applicant]
Cited By (1)
US 12,441,187