IP Library Granted Patent US 11,827,356
Granted Patent B2
US 11,827,356 · App. 16/893,561 · Granted Nov 28, 2023

Payload-release device position tracking

Inventors: James Ryan Burgess (Redwood City, CA); Joanna Cohen (Mountain View, CA)
Assignee: Wing Aviation LLC
B64D1/12B64C39/024B64D1/22G05D1/0094G05D1/042B64U2101/60B64U2101/64
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,827,356
App. No.
16/893,561
Granted
Nov 28, 2023
Kind
B2
Abstract

An unmanned aerial vehicle (UAV) is disclosed that includes a retractable payload delivery system. The payload delivery system can lower a payload to the ground using a delivery device that secures the payload during descent and releases the payload upon reaching the ground. The location of the delivery device can be determined as it is lowered to the ground using image tracking. The UAV can include an imaging system that captures image data of the suspended delivery device and identifies image coordinates of the delivery device, and the image coordinates can then be mapped to a location. The UAV may also be configured to account for any deviations from a planned path of descent in real time to effect accurate delivery locations of released payloads.

Claims (49)

1. A system comprising:

a payload having a reflector arranged thereon;

a tether coupled to an unmanned aerial vehicle (UAV), wherein the tether suspends the payload below the UAV, and wherein the payload is configured to releasably couple to the tether;

a retraction system operable to use the tether to lower the payload from the UAV;

an imaging system arranged on the UAV such that while the payload is suspended from the UAV via the tether, the imaging system detects light reflected from the reflector on the payload; and

a control system configured to:

(i) while the payload is suspended from the UAV via the tether, receive image data from the imaging system;

(ii) identify, based on the received image data, an image coordinate associated with the reflector situated on the payload;

(iii) determine a position of the payload based at least in part on the identified image coordinate;

(iv) determine, based at least in part on the position of the payload, flight-control commands to navigate the UAV so as to cause the payload to become closer to a path of descent associated with a delivery location; and

(v) cause the UAV to operate in accordance with the determined flight-control commands.

2. The system of claim 1 , wherein the control system is further configured to:

while the UAV hovers over a delivery location, use the retraction system to initiate a delivery operation, wherein initiating the delivery operation comprises lowering the payload secured thereby toward the ground such that the payload descends from the UAV;

determine that the payload is at or near the ground; and

in response to determining that the payload is at or near the ground, release the payload.

3. The system of claim 2 , wherein the control system is further configured to:

determine that the determined position of the payload is within a threshold distance of the delivery location; and

cause the release of the payload in response to both determining that the payload is at or near the ground and that the determined position of the payload is within the threshold distance of the delivery location.

4. The system of claim 1 , wherein the control system is further configured to:

determine, based at least in part on the determined position of the payload, flight-control commands to navigate the UAV, in hover mode, so as to cause the suspended payload to become closer to a path of descent associated with a delivery location; and

cause the UAV to fly, in hover mode, in accordance with the determined flight-control commands.

5. The system of claim 4 , wherein the control system is further configured to:

prior to causing the UAV to fly, in hover mode, in accordance with the determined flight-control commands, determine that the determined position of the payload is greater than a threshold distance from the path of descent associated with the target delivery location.

6. The system of claim 4 , wherein the control system determining the flight-control commands comprises the control system: (a) determining a translational offset of the UAV which compensates for a displacement between the determined position of the payload and the path of descent associated with the target delivery location, wherein the translational offset is based on: the identified image coordinate associated with the reflector, an image coordinate associated with the path of descent, and a current elevation of the UAV, and (b) using a navigational module to determine the flight-control commands which cause the UAV to traverse the determined translational offset in hover mode.

7. The system of claim 1 , wherein the imaging system comprises a downward-facing camera mounted to the UAV via a gimbal mount so as to substantially maintain an orientation of the point of view of the camera.

8. The system of claim 1 , wherein the reflector is a first reflector and a second reflector is also situated on the payload such that, while the payload is suspended from the UAV via the tether, both the first and second reflectors are arranged to reflect light toward the UAV, and

wherein the control system is further configured to:

identify a pattern in the received image data that corresponds to light received from both the first and second reflectors;

identify, in the pattern, respective image coordinates associated with the first and second reflectors, and

determine the position of the payload based at least in part on both the image coordinate associated with the first reflector and the image coordinate associated with the second reflector.

9. The system of claim 8 , wherein the first and second reflectors are situated at respective positions on the payload, and wherein the control system identifying the pattern within the received image data comprises the control system identifying a pattern in which light from the first and second reflectors are received from respective reflection locations that are spatially separated from one another.

10. The system of claim 9 , wherein the control system is further configured to determine an orientation of the payload while the payload is suspended from the UAV based on the image coordinates associated with the first and second reflectors.

11. A method comprising:

while a payload is suspended from an unmanned aerial vehicle (UAV) via a tether, receiving image data from an imaging sensor system, wherein the imaging sensor system comprises an image sensor mounted on the UAV such that, while the payload is suspended from the UAV via the tether, a field of view of the image sensor includes a reflector that is situated on the payload and arranged to reflect light toward the UAV;

identifying, by a computing system, based on the received image data, an image coordinate associated with the reflector situated on the payload;

determining, by the computing system, based at least in part on the identified image coordinate, a position of the payload;

determining, by the computing system, based at least in part on the position of the payload, flight-control commands to navigate the UAV, such that movement of the UAV causes the suspended payload to become closer to a path of descent associated with a delivery location; and

causing, by the computing system, the UAV to fly, in accordance with the determined flight-control commands.

12. The method of claim 11 , wherein the payload 1 s configured to be releasably coupled to the tether, the method further comprising:

while the UAV hovers over a delivery location, using a retractable delivery system to initiate a delivery operation, wherein initiating the delivery operation comprises lowering the payload toward the ground such that the payload descends from the UAV;

determining that the payload is at or near the ground;

determining that the determined position of the payload is within a threshold distance of the delivery location; and

causing the payload to be released in response to both determining that the payload is at or near the ground and that the determined position of the payload is within the threshold distance of the delivery location.

13. The method of claim 11 , further comprising:

determining, based at least in part on the determined position of the payload, flight-control commands to navigate the UAV, in hover mode, so as to cause the payload to become closer to a path of descent associated with a delivery location; and

causing the UAV to fly, in hover mode, in accordance with the determined flight-control commands.

14. The method of claim 13 , wherein determining the flight-control commands comprises:

determining a translational offset of the UAV which compensates for a displacement between the determined position of the payload and the path of descent associated with the delivery location, wherein the translational offset is based on: the identified image coordinate associated with the reflector, an image coordinate associated with the path of descent, and a current elevation of the UAV; and

using a navigational module to determine the flight-control commands which cause the UAV to traverse the determined translational offset in hover mode.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2020
From: BURGESS, JAMES RYAN; COHEN, JOANNA
To: GOOGLE INC.
Reel/Frame 052869/0726 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2020
From: X DEVELOPMENT LLC
To: WING AVIATION LLC
Reel/Frame 052869/0741 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2020
From: GOOGLE INC.
To: X DEVELOPMENT LLC
Reel/Frame 052874/0798 →