IP Library Granted Patent US 11,409,279
Granted Patent B1
US 11,409,279 · App. 16/584,066 · Granted Aug 9, 2022

Autonomously motile device with remote control

Inventors: Amy Solenberg (San Francisco, CA); Anil Kumar Katta (Bellevue, WA); Swati S Rao (Sunnyvale, CA); Swetha Bijoy (Bothell, WA); Anthony George Robson (Los Altos, CA); David Allen Fotland (San Jose, CA); Robert Franklin Ebert (Los Altos Hills, CA); Roger Robert Webster (Los Altos, CA); Adam Fineberg (Saratoga, CA)
Assignee: Amazon Technologies, Inc.
G05D1/0038G05D1/0088G06F3/0488H04N7/185
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,409,279
App. No.
16/584,066
Granted
Aug 9, 2022
Kind
B1
Abstract

Movement of an autonomously motile device may be controlled by a user device. The user device may display image data captured by a camera of the autonomously motile device; a user may provide input, such as a touch gesture on a display screen, indicating a command for the autonomously motile device to move to a location indicated by the input. The autonomously motile device determines a coordinate of the input and a time of the touch input; the autonomously motile device then determines a direction and distance of a corresponding movement.

Claims (90)

1. A computer-implemented method comprising:

displaying, on a screen of a user device, an image corresponding to image data received from an autonomously motile device, the image data representing a portion of a physical space;

determining data representing an input at the screen of the user device, the input corresponding to an x-coordinate of the screen and a y-coordinate of the screen;

determining a first ratio of the x-coordinate to a width of the screen;

determining a horizontal field of view of a camera of the autonomously motile device;

determining, based at least on the horizontal field of view and the first ratio, a travel direction of the autonomously motile device;

determining a second ratio of the y-coordinate to a height of the screen;

determining, based at least on the second ratio, a travel distance of the autonomously motile device;

determining a first destination, in a physical space, corresponding to the travel direction and the travel distance; and

causing the autonomously motile device to move to a location in the physical space corresponding to the first destination.

2. The computer-implemented method of claim 1 , further comprising:

determining a time of capture of image data displayed on the screen at a time of image data of the autonomously motile device corresponding to the input;

determining a position of the autonomously motile device at the time;

determining an orientation of a camera of the autonomously motile device at the time;

determining, based on the first destination, the position, and the orientation, a second destination; and

causing the autonomously motile device to move to the second destination.

3. The computer-implemented method of claim 1 , further comprising:

determining, using a map of the physical space, that a path to the first destination intersects an obstruction; and

at least one of:

shortening the travel distance, or

altering the path.

4. The computer-implemented method of claim 1 , wherein determining the travel distance comprises:

determining, using the first ratio and the second ratio, a ray extending from a present position of the autonomously motile device; and

determining that the ray intersects with a representation of an obstacle or a representation of a floor in the physical space.

5. The computer-implemented method of claim 4 , wherein determining the travel direction comprises:

determining a difference between the present position and a position of the autonomously motile device at a time of image data of the autonomously motile device corresponding to the input; and

determining the travel direction by modifying the second ratio based at least in part on the difference.

6. The computer-implemented method of claim 1 , wherein determining the travel distance comprises:

determining that the travel distance is greater than a first distance, the first distance corresponding to a portion of the physical space proximate to the autonomously motile device and outside a field of view of the camera; and

determining that the travel distance is less than a second distance corresponding to a size of the physical space.

7. The computer-implemented method of claim 6 , further comprising:

determining a height of the camera with respect to ground;

determining a lower limit of a vertical field of view of the camera; and

determining the first distance using the height and the lower limit.

8. The computer-implemented method of claim 1 , further comprising:

determining, using a two-dimensional map of the physical space, that a path to the first destination intersects a first obstruction; or

determining, using a three-dimensional map of the physical space, that the path to the first destination intersects a second obstruction different from the first obstruction.

9. The computer-implemented method of claim 1 , wherein determining the travel direction comprises:

determining a present orientation of the camera;

determining a time corresponding to image data of the autonomously motile device corresponding to the input; and

determining a difference between the present orientation and an orientation of the camera at the time.

10. The computer-implemented method of claim 1 , further comprising:

determining a present time;

determining a time corresponding to the input; and

prior to causing the autonomously motile device to move to the first destination determining that a difference between the present time and the time is less than a threshold.

11. The computer-implemented method of claim 1 , further comprising:

sending, from the user device to the autonomously motile device, the x-coordinate and y-coordinate,

wherein determining the first ratio and determining the second ratio are performed by the autonomously motile device.

12. A system comprising:

at least one processor; and

at least one memory including instructions that, when executed by the at least one processor, cause the system to:

display, on a screen of a user device, an image corresponding to image data received from an autonomously motile device, the image data representing a portion of a physical space;

determine data representing an input at the screen of the user device, the input corresponding to an x-coordinate of the screen and a y-coordinate of the screen;

determine a first ratio of the x-coordinate to a width of the screen;

determine a horizontal field of view of a camera of the autonomously motile device;

determine, based at least on the horizontal field of view and the first ratio, a travel direction of the autonomously motile device;

determine a second ratio of the y-coordinate to a height of the screen;

determine, based at least on the second ratio, a travel distance of the autonomously motile device;

determine a first destination, in a physical space, corresponding to the travel direction and the travel distance; and

cause the autonomously motile device to move to a location in the physical space corresponding to the first destination.

13. The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine a time of capture of image data displayed on the screen at a time of image data of the autonomously motile device corresponding to the input;

determine, a position of the autonomously motile device at the time;

determine, an orientation of a camera of the autonomously motile device at the time;

determine, based on the first destination, the position, and the orientation, a second destination; and

cause the autonomously motile device to move to the second destination.

14. The system of claim 12 , wherein the at least one memory further comprises instructions that determine the travel distance and that, when executed by the at least one processor, further cause the system to:

determine, using a map of the physical space, that a path to the first destination intersects an obstruction; and

at least one of:

shorten the travel distance, or

alter the path.

15. The system of claim 12 , wherein the at least one memory further comprises instructions that determine the travel distance and that, when executed by the at least one processor, further cause the system to:

determine, using the first ratio and the second ratio, a ray extending from a present position of the autonomously motile device; and

determine that the ray intersects with a representation of an obstacle or a representation of a floor in the physical space.

16. The system of claim 15 , wherein the at least one memory further comprises instructions that determine the travel direction and that, when executed by the at least one processor, further cause the system to:

determine a difference between the present position and a position of the autonomously motile device at a time of image data of the autonomously motile device corresponding to the input; and

determine the travel direction by modifying the second ratio based at least in part on the difference.

17. The system of claim 12 , wherein the at least one memory further comprises instructions that determine the travel distance and that, when executed by the at least one processor, further cause the system to:

determine that the travel distance is greater than a first distance, the first distance corresponding to a portion of the physical space proximate to the autonomously motile device and outside a field of view of the camera; and

determine that the travel distance is less than a second distance corresponding to a size of the physical space.

18. The system of claim 17 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine a height of the camera with respect to ground;

determine lower limit of a vertical field of view of the camera; and

determine the first distance using the height and the lower limit.

19. The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine, using a two-dimensional map of the physical space, that a path to the first destination intersects a first obstruction; or

determine, using a three-dimensional map of the physical space, that the path to the first destination intersects a second obstruction different from the first obstruction.

20. The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

send, from the user device to the autonomously motile device, the x-coordinate and y-coordinate,

wherein the instructions that cause the system to determine the first ratio and determine the second ratio are executed by the autonomously motile device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 25, 2021
From: SOLENBERG, AMY; KATTA, ANIL KUMAR; RAO, SWATI S.; BIJOY, SWETHA; ROBSON, ANTHONY GEORGE; FOTLAND, DAVID ALLEN; EBERT, ROBERT FRANKLIN; WEBSTER, ROGER ROBERT; FINEBERG, ADAM
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 056349/0547 →