IP Library Granted Patent US 11,334,078
Granted Patent B1
US 11,334,078 · App. 17/518,967 · Granted May 17, 2022

Method and apparatus for handling goods by unmanned aerial vehicle and autonomous vehicle

Inventors: Youngjoon Kim (Incheon, KR); Seung Han Lim (Daejeon, KR)
Assignees: PABLO AIR Co., Ltd.; PABLO AIR International, INC.
G05D1/0094B64C39/024B64D1/12B64D1/22B64D47/08G05D1/0276G05D1/101G06K7/10722G06K7/1417G06V20/13B64C2201/027B64C2201/127B64C2201/128B64C2201/16H04W76/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,334,078
App. No.
17/518,967
Granted
May 17, 2022
Kind
B1
Abstract

Provided is a method for an unmanned aerial vehicle to handle goods in cooperation with an autonomous vehicle. The method comprises capturing, by the unmanned aerial vehicle, an image of the autonomous vehicle having a goods storage box, recognizing, by the unmanned aerial vehicle, a marker displayed in the goods storage box by analyzing the captured image, identifying, by the unmanned aerial vehicle, a region occupied by the marker on the captured image, adjusting a relative position of the unmanned aerial vehicle and the autonomous vehicle, wherein the marker displayed in the goods storage box is covered by a lid of the goods storage box and placed in a state that cannot be captured by the unmanned aerial vehicle, and the marker is exposed in a state that can be captured by the unmanned aerial vehicle only when the lid of the storage box is opened by communication between the unmanned aerial vehicle and the autonomous vehicle.

Claims (37)

1. A method for an unmanned aerial vehicle to handle goods in cooperation with an autonomous vehicle comprising:

capturing, by the unmanned aerial vehicle, an image of the autonomous vehicle having a storage box for storing goods;

recognizing, by the unmanned aerial vehicle, a marker displayed in the storage box by analyzing the captured image;

identifying, by the unmanned aerial vehicle, a region occupied by the marker on the captured image;

calculating, by the unmanned aerial vehicle, a distance between center coordinates of the identified region and center coordinates of the captured image;

adjusting a relative position of the unmanned aerial vehicle and the autonomous vehicle based on the calculated distance exceeding a threshold distance;

determining, by the unmanned aerial vehicle, a handling attribute of the goods based on delivery information associated with the goods;

based on a determination that the goods has a first handling attribute,

unwinding a wire connected to a gripping member gripping the goods to lower the gripping member into the storage box while the unmanned aerial vehicle is hovering, and releasing a gripping state of the goods when the lowering of the gripping member is completed; and

based on a determination that the goods has a second handling attribute,

releasing a gripping state of the goods in air above the storage box while the unmanned aerial vehicle is hovering,

wherein the marker displayed in the storage box is covered by a lid of the storage box and placed in a state that is invisible to the unmanned aerial vehicle, and the marker is exposed in a state that is visible to the unmanned aerial vehicle only when the lid of the storage box is opened by communication between the unmanned aerial vehicle and the autonomous vehicle.

2. The method of claim 1 , wherein the adjusting comprises,

moving the unmanned aerial vehicle towards a center of the region to adjust the relative position with the autonomous vehicle when the calculated distance exceeds the threshold distance.

3. The method of claim 1 , wherein capturing the image comprises,

establishing, by the unmanned aerial vehicle, a short-range wireless session with the autonomous vehicle;

transmitting, by the unmanned aerial vehicle, a first signal for opening the lid of the storage box to the autonomous vehicle through the short-range wireless session; and

capturing, by the unmanned aerial vehicle, an image of the autonomous vehicle, wherein the lid of the storage box is opened to expose the marker in response to the first signal.

4. The method of claim 3 further comprises,

transmitting, by the unmanned aerial vehicle, a second signal for closing the lid of the storage box to the autonomous vehicle when unloading of the goods is completed.

5. The method of claim 1 , wherein the marker is located at a center of a bottom surface of the storage box.

6. The method of claim 1 , wherein the marker is a two-dimensional barcode.

7. An unmanned aerial vehicle comprising:

a thrust generating unit for generating a thrust to move an unmanned aerial vehicle to an autonomous vehicle having a storage box for storing goods;

a gripping member for gripping goods;

a camera for capturing the autonomous vehicle; and

a control unit for recognizing a marker displayed in the storage box by analyzing the captured image, controlling the thrust generating unit to adjust a relative position of the autonomous vehicle and the unmanned aerial vehicle based on a recognition state of the marker, and releasing a gripping state of the gripping member to unload the goods into the storage box when position adjustment is completed,

wherein the control unit calculates a distance between center coordinates of a region occupied by the marker in the captured image and center coordinates of the image, determines that the position adjustment is completed when the calculated distance is within a threshold distance, and controls the thrust generating unit so that the relative position of the autonomous vehicle and the unmanned aerial vehicle is adjusted when the calculated distance exceeds the threshold distance,

wherein the marker displayed in the storage box is covered by a lid of the storage box and placed in a state that is invisible to the unmanned aerial vehicle, and the marker is exposed in a state that is visible to the unmanned aerial vehicle only when the lid of the storage box is opened by communication between the unmanned aerial vehicle and the autonomous vehicle.

8. The unmanned aerial vehicle of claim 7 , wherein the control unit determines a handling attribute of the goods based on stored delivery information when the position adjustment is completed,

wherein the control unit, based on a determination that the goods has a first handling attribute, unwinds a wire connected to the gripping member to lower the gripping member into the storage box while the unmanned aerial vehicle is hovering, and controls the gripping member to release a gripping state of the goods when the lowering is completed,

wherein the control unit, based on a determination that the goods has a second handling attribute, controls the gripping member to release the gripping state of the goods in air above the storage box while the unmanned aerial vehicle is hovering.

9. The unmanned aerial vehicle of claim 7 , wherein the control unit controls the thrust generating unit to move the unmanned aerial vehicle towards a center of a region occupied by the marker when the calculated distance exceeds the threshold distance.

10. The unmanned aerial vehicle of claim 7 further comprises,

a wireless communication unit for establishing a short-range wireless session with the autonomous vehicle,

wherein the control unit transmits an first signal for opening a lid of the goods storage box to the autonomous vehicle that establishes the short-range wireless session,

wherein the camera captures an image of the autonomous vehicle, wherein the lid of the goods storage box is opened to expose the marker in response to the first signal.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2021
From: KIM, YOUNGJOON; LIM, SEUNG HAN
To: PABLO AIR CO., LTD.; PABLO AIR INTERNATIONAL, INC.
Reel/Frame 058021/0309 →
Priority Claims (1)
KR 10-2021-0008448 · Jan 21, 2021 · national
Cited By (1)
US 12,391,382