IP Library › Granted Patent US 10,226,869
Granted Patent B2
US 10,226,869 · App. 15/123,394 · Granted Mar 12, 2019

Haptic virtual fixture tools

Inventors: Howard Jay Chizeck (Seattle, WA); Andrew Stewart (Seattle, WA); Fredrik Ryden (Seattle, WA)
Assignee: University of Washington
B25J9/1689G05B19/427G05B2219/40122
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,226,869
App. No.
15/123,394
Filed
Sep 2, 2016
Granted
Mar 12, 2019
Kind
B2
Examiner
TRAN, DALENA
Art Unit
3664
USPC
700/257
Abstract

Apparatus and methods for defining and utilizing virtual fixtures for haptic navigation within real-world environments, including underwater environments, are provided. A computing device can determine a real-world object within a real-world environment. The computing device can receive an indication of the real-world object. The computing device can determine a virtual fixture that corresponds to the real-world object based on the indication, where aspects of the virtual fixture are configured to align with aspects of the real-world object. The computing device can provide a virtual environment for manipulating the robotic tool to operate on the real-world object utilizing the virtual fixture. The virtual fixture is configured to provide haptic feedback based on a position of a virtual robotic tool in the virtual environment that corresponds to the robotic tool in the real-world environment.

Claims (64)

1. A method, comprising:

determining a particular real-world object within a real-world environment using a computing device, wherein the real-world environment also comprises a robotic tool;

receiving a specification of the particular real-world object at the computing device;

determining a corresponding virtual fixture that corresponds to the particular real-world object based on the specification of the particular real-world object using the computing device, wherein one or more corresponding aspects of the corresponding virtual fixture are configured to align with one or more aspects of the particular real-world object; and

providing a virtual environment that is configured for manipulating the robotic tool to operate on the particular real-world object by utilizing the corresponding virtual fixture using the computing device, wherein the corresponding virtual fixture comprises a combination fixture configured to provide haptic feedback based on a position of a virtual robotic tool in the virtual environment by at least:

providing haptic feedback to direct the robotic tool into alignment with the one or more aspects of the particular real-world object, and

after the robotic tool is aligned with the one or more aspects of the particular real-world object, providing haptic feedback to inhibit movement of the robotic tool out of alignment with the one or more aspects of the particular real-world object, and

wherein the virtual robotic tool in the virtual environment corresponds to the robotic tool in the real-world environment.

2. The method of claim 1 , wherein receiving the specification of the particular real-world object comprises receiving one or more scans of the particular real-world object, and wherein the one or more scans are taken using one or more sensors in the real-world environment.

3. The method of claim 2 , wherein receiving the one or more scans comprises:

requesting that the robotic tool be positioned in the real-world environment to enable the one or more sensors to sense the particular real-world object;

after the robotic tool is positioned, receiving the one or more scans of the particular real-world object; and

determining the corresponding virtual fixture based on the one or more scans.

4. The method of claim 3 , wherein the corresponding virtual fixture comprises a first particular surface, and wherein the first particular surface is a corresponding aspect of the one or more corresponding aspects of the corresponding virtual fixture.

5. The method of claim 4 , wherein the particular real-world object comprises a second particular surface, and wherein the first particular surface is configured to align with the second particular surface.

6. The method of claim 2 , wherein determining the corresponding virtual fixture comprises generating the corresponding virtual fixture based on sensor data from the one or more scans using the computing device.

7. The method of claim 1 , wherein determining the corresponding virtual fixture comprises requesting a search for the corresponding virtual fixture from among a predetermined plurality of virtual fixtures.

8. The method of claim 1 , further comprising:

determining a work plan to accomplish one or more tasks in the real-world environment, wherein at least one task of the one or more tasks involves the particular real-world object, and wherein the work plan for the at least one task comprises a reference to the corresponding virtual fixture.

9. The method of claim 8 , further comprising:

determining an action performed by the robotic tool associated with the one or more tasks;

generating a record of the action performed by the robotic tool; and

storing the record in a log.

10. The method of claim 1 , wherein providing the virtual environment comprises:

manipulating the robotic tool to align with the particular real-world object in a first dimension using virtual robotic tool in the virtual environment;

after the robotic tool is aligned with the particular real-world object in the first dimension, attempting to manipulate the robotic tool to not align with the particular real-world object in the first dimension using the virtual robotic tool in the virtual environment; and

while attempting to manipulate the robotic tool to not align with the particular real-world object in the first dimension, receiving at least haptic feedback via the combination fixture to stop manipulating the robotic tool to not align with the particular real-world object.

11. The method of claim 10 , wherein manipulating the robotic tool to align with the particular real-world object in the first dimension using the virtual environment comprises determining that at least one corresponding aspect of the corresponding virtual fixture is aligned with at least one aspect of the particular real-world object in at least the first dimension.

12. A computing device, comprising:

a processor; and

data storage, storing instructions that, upon execution by the processor, cause the computing device to perform functions comprising:

determining a particular real-world object within a real-world environment, wherein the real-world environment also comprises a robotic tool;

receiving a specification of the particular real-world object;

determining a corresponding virtual fixture that corresponds to the particular real-world object based on the specification of the particular real-world object, wherein one or more corresponding aspects of the corresponding virtual fixture are configured to align with one or more aspects of the particular real-world object; and

providing a virtual environment that is configured for manipulating the robotic tool to operate on the particular real-world object by utilizing the corresponding virtual fixture, wherein the corresponding virtual fixture comprises a combination fixture configured to provide haptic feedback based on a position of a virtual robotic tool in the virtual environment by at least:

providing haptic feedback to direct the robotic tool into alignment with the one or more aspects of the particular real-world object, and

after the robotic tool is aligned with the one or more aspects of the particular real-world object, providing haptic feedback to inhibit movement of the robotic tool out of alignment with the one or more aspects of the particular real-world object, and

wherein the virtual robotic tool in the virtual environment corresponds to the robotic tool in the real-world environment.

13. The computing device of claim 12 , wherein receiving the specification of the particular real-world object comprises receiving one or more scans of the particular real-world object, and wherein the one or more scans are taken using one or more sensors in the real-world environment.

14. The computing device of claim 13 , wherein receiving the one or more scans comprises:

requesting that the robotic tool be positioned in the real-world environment to enable the one or more sensors to sense the particular real-world object;

after the robotic tool is positioned, receiving the one or more scans of the particular real-world object; and

determining the corresponding virtual fixture based on the one or more scans.

15. The computing device of claim 13 , wherein the corresponding virtual fixture comprises a first particular surface, and wherein the first particular surface is a corresponding aspect of the one or more corresponding aspects of the corresponding virtual fixture.

16. The computing device of claim 15 , wherein the particular real-world object comprises a second particular surface, and wherein the first particular surface is configured to align with the second particular surface.

17. The computing device of claim 13 , wherein determining the corresponding virtual fixture comprises generating the corresponding virtual fixture based on sensor data from the one or more scans using the computing device.

18. The computing device of claim 12 , wherein the functions further comprise:

determining a work plan to accomplish one or more tasks in the real-world environment, wherein at least one task of the one or more tasks involves the particular real-world object, and wherein the work plan for the at least one task comprises a reference to the corresponding virtual fixture;

determining an action performed by the robotic tool associated with the one or more tasks;

generating a record of the action performed by the robotic tool; and

storing the record in a log.

19. The computing device of claim 12 , wherein providing the virtual environment comprises:

manipulating the robotic tool to align with the particular real-world object in a first dimension using the virtual environment so that at least one corresponding aspect of the corresponding virtual fixture is aligned with at least one aspect of the particular real-world object in at least the first dimension;

after the robotic tool is aligned with the particular real-world object in the first dimension, attempting to manipulate the robotic tool to not align with the particular real-world object in the first dimension using the virtual environment; and

while attempting to manipulate the robotic tool to not align with the particular real-world object in the first dimension, receiving at least haptic feedback via the combination fixture to stop manipulating the robotic tool to not align with the particular real-world object.

20. An article of manufacture, comprising a physical computer-readable storage medium storing instructions that, upon execution by a processor of the article of manufacture, cause the article of manufacture to perform functions comprising:

determining a particular real-world object within a real-world environment, wherein the real-world environment also comprises a robotic tool;

receiving a specification of the particular real-world object;

determining a corresponding virtual fixture that corresponds to the particular real-world object based on the specification of the particular real-world object, wherein one or more corresponding aspects of the corresponding virtual fixture are configured to align with one or more aspects of the particular real-world object; and

providing a virtual environment that is configured for manipulating the robotic tool to operate on the particular real-world object by utilizing the corresponding virtual fixture, wherein the corresponding virtual fixture comprises a combination fixture configured to provide haptic feedback based on a position of a virtual robotic tool in the virtual environment by at least:

providing haptic feedback to direct the robotic tool into alignment with the one or more aspects of the particular real-world object, and

after the robotic tool is aligned with the one or more aspects of the particular real-world object, providing haptic feedback to inhibit movement of the robotic tool out of alignment with the one or more aspects of the particular real-world object, and

wherein the virtual robotic tool in the virtual environment corresponds to the robotic tool in the real-world environment.

21. The article of manufacture of claim 20 , wherein receiving the specification of the particular real-world object comprises receiving one or more scans of the particular real-world object, and wherein the one or more scans are taken using one or more sensors in the real-world environment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 2, 2016
From: CHIZECK, HOWARD JAY; STEWART, ANDREW; RYDEN, FREDRIK
To: UNIVERSITY OF WASHINGTON
Reel/Frame 039623/0311 →
Continuity (2)
Provisional Application 61947181 · Mar 3, 2014
Related Publication 20170106537A1 · Apr 20, 2017