IP Library › Granted Patent US 10,956,739
Granted Patent B2
US 10,956,739 · App. 15/194,342 · Granted Mar 23, 2021

Augmented reality robotic system visualization

Inventors: David Thomasson (San Rafael, CA); Evan Patrick Atherton (Foster City, CA); Maurice Ugo Conti (Muir Beach, CA); Heather Kerrick (Oakland, CA)
Assignee: AUTODESK, INC.
G06K9/00671B25J9/1664B25J9/1671G02B27/017G06T19/006H04N7/183G05B2219/39451Y10S901/30Y10S901/47
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,956,739
App. No.
15/194,342
Filed
Jun 27, 2016
Granted
Mar 23, 2021
Kind
B2
Art Unit
2486
USPC
348/86
Abstract

A technique for displaying a representative path associated with a robotic device. The technique includes detecting at least one reference point within a first image of a workspace, generating the representative path based on path instructions associated with the robotic device and the at least one reference point, and displaying the representative path within the workspace.

Claims (53)

1. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, configure the one or more processors to perform the steps of:

processing path instructions that comprise coded instructions and specify an operation to be performed by a robotic device;

detecting at least one reference point within a first image of a workspace;

generating a representative path associated with the robotic device based on the processed path instructions and the at least one reference point,

wherein generating the representative path comprises:

detecting the at least one reference point within a second image of the workspace;

determining that the at least one reference point has moved from a first location to a second location; and

modifying the representative path based on the second location; and

displaying, via a visualization application, the representative path within the workspace.

2. The one or more non-transitory computer-readable media of claim 1 , wherein generating the representative path further comprises at least one of translating the representative path based on a location of the at least one reference point, or rotating the representative path based on an orientation of the at least one reference point.

3. The one or more non-transitory computer-readable media of claim 1 , wherein displaying the representative path comprises overlaying, via a head-mounted display device, the representative path on a live view of the workspace.

4. The one or more non-transitory computer-readable media of claim 1 , wherein displaying the representative path comprises overlaying the representative path on substantially real-time images of the workspace acquired via an image sensor.

5. The one or more non-transitory computer-readable media of claim 1 , wherein the representative path comprises a path associated with an end effector of the robotic device.

6. The one or more non-transitory computer-readable media of claim 1 , wherein the representative path comprises an animated three-dimensional representation of the robotic device.

7. The one or more non-transitory computer-readable media of claim 1 , wherein the representative path comprises volumetric boundaries occupied by the robotic device while traveling along the representative path.

8. The one or more non-transitory computer-readable media of claim 1 , further comprising displaying, proximate to the representative path, a graphical indicator that includes at least one of an angle, a speed, and an acceleration associated with a joint of the robotic device.

9. The one or more non-transitory computer-readable media of claim 1 , wherein modifying the representative path comprises:

modifying at least one of a size, an orientation, and a location of the representative path based on the second location.

10. The one or more non-transitory computer-readable media of claim 1 , further comprising:

detecting user input within the second image of the workspace proximate to at least one point on the representative path;

modifying at least one of the path instructions based on the user input; and

updating the representative path based on the at least one path instruction.

11. A method for displaying a representative path associated with a robotic device, the method comprising:

processing path instructions that comprise coded instructions and specify an operation to be performed by a robotic device;

detecting at least one reference point within a first image of a workspace;

generating the representative path based on the processed path instructions and the at least one reference point, wherein generating the representative path comprises:

detecting the at least one reference point within a second image of the workspace;

determining that the at least one reference point has moved from a first location to a second location; and

modifying the representative path based on the second location; and

displaying, via a visualization application, the representative path within the workspace.

12. The method of claim 11 , wherein generating the representative path further comprises at least one of translating the representative path based on a location of the at least one reference point, or rotating the representative path based on an orientation of the at least one reference point.

13. The method of claim 11 , wherein displaying the representative path comprises at least one of overlaying, via a head-mounted display device, the representative path on a live view of the workspace and overlaying the representative path on substantially real-time images of the workspace acquired via an image sensor.

14. The method of claim 11 , wherein the representative path comprises a path associated with an end effector of the robotic device.

15. The method of claim 11 , wherein the representative path comprises an animated three-dimensional representation of the robotic device.

16. The method of claim 11 , wherein the representative path comprises volumetric boundaries occupied by the robotic device while traveling along the representative path.

17. The method of claim 11 , further comprising displaying, proximate to the representative path, a graphical indicator that includes at least one of an angle, a speed, and an acceleration associated with a joint of the robotic device.

18. The method of claim 11 , wherein modifying the representative path comprises:

modifying at least one of a size, an orientation, and a location of the representative path based on the second location.

19. The method of claim 11 , further comprising:

detecting user input within the second image of the workspace proximate to at least one point on the representative path;

modifying at least one of the path instructions based on the user input; and

updating the representative path based on the at least one path instruction.

20. A system for displaying a representative path associated with a robotic device, the system comprising:

a memory storing a path visualization application;

a processor that is coupled to the memory and, when executing the path visualization application, is configured to:

process path instructions that comprise coded instructions and specify an operation to be performed by a robotic device;

detect at least one reference point within a first image of a workspace; and

generate the representative path based on the processed path instructions and the at least one reference point, wherein generating the representative path comprises:

detecting the at least one reference point within a second image of the workspace;

determining that the at least one reference point has moved from a first location to a second location; and

modifying the representative path based on the second location; and

a display coupled to the processor and configured to display the representative path within the workspace.

21. The one or more non-transitory computer-readable media of claim 1 , wherein generating the representative path further comprises scaling the representative path based on a size of the at least one reference point.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2016
From: THOMASSON, DAVID; ATHERTON, EVAN PATRICK; CONTI, MAURICE UGO; KERRICK, HEATHER
To: AUTODESK, INC.
Reel/Frame 040593/0230 →
Continuity (1)
Related Publication 20170372139A1 · Dec 28, 2017
Cited By (2)
US 12,583,104 US 12,691,572