IP Library › Granted Patent US 8,600,761
Granted Patent B2
US 8,600,761 · App. 12/206,902 · Granted Dec 3, 2013

Hands-free and non-visually occluding object information interaction system

Inventors: Joseph C. Marsh (St. Louis, MO); Eric M. Smith (Wildwood, MO)
Assignee: The Boeing Company
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,600,761
App. No.
12/206,902
Granted
Dec 3, 2013
Kind
B2
Abstract

Technologies are described herein for providing a hands-free and non-visually occluding interaction with object information. In one method, a visual capture of a portion of an object is received through a hands-free and non-visually occluding visual capture device. An audio capture is also received from a user through a hands-free and non-visually occluding audio capture device. The audio capture may include a request for information about a portion of the object in the visual capture. The information is retrieved and is transmitted to the user for playback through a hands-free and non-visually occluding audio output device.

Claims (40)

1. A method for providing a hands-free and non-visually occluding object information interface, comprising:

receiving a direct visual capture of a portion of an object through a hands-free and non-visually occluding visual capture device configured for attachment to a user at the same location as the object;

receiving an audio capture from the user through a hands-free and non-visually occluding audio capture device, the audio capture comprising a request for information about a portion of the object in the visual capture;

retrieving the information; and

transmitting the information to the user for playback through a hands-free and non-visually occluding output device.

2. The method of claim 1 , wherein receiving a visual capture of a portion of an object through a hands-free and non-visually occluding visual capture device comprises receiving a video capture of a portion of the object through a video capture device.

3. The method of claim 1 , wherein receiving a visual capture of a portion of an object through a hands-free visual capture device comprises receiving an image capture of a portion of the object through a hands-free image capture device.

4. The method of claim 1 , wherein the hands-free and non-visually occluding visual capture device is operated through voice commands spoken into the hands-free and non-visually occluding audio capture device.

5. The method of claim 1 , wherein the hands-free and non-visually occluding visual capture device mimics an actual view from the user operating the hands-free and non-visually occluding visual capture device.

6. The method of claim 1 , wherein retrieving the information comprises:

converting the audio capture into text comprising the request for information; and

retrieving the information about a portion of the object according to the request for information in the text.

7. The method of claim 1 , wherein transmitting the information to the user for playback through a hands-free and non-visually occluding output device comprises:

converting the information to a spoken dialog through a speech synthesizer; and

transmitting the spoken dialog to the user for playback through the hands-free and non-visually occluding audio output device.

8. The method of claim 1 , further comprising determining a field of view of the visual capture device with respect to the object.

9. The method of claim 8 , further comprising aligning the field of view with a known representation of the object to determine the field of view of the visual capture device with respect to the object.

10. The method of claim 9 , wherein the known representation of the object comprises a computer aided design (CAD) generated representation of the object.

11. The method of claim 8 , further comprising mapping the portion of the object in the visual capture utilizing a simultaneous location and mapping (SLAM) analysis of the visual capture.

12. The method of claim 1 , wherein retrieving the information comprises retrieving information identifying the portion of the object.

13. The method of claim 1 , wherein retrieving the information comprises retrieving instructions for performing a maintenance task on the portion of the object.

14. A method for providing a hands-free and non-visually occluding object interface, comprising:

receiving a video capture of a portion of an object through a hands-free and non-visually occluding video capture device attached to a user operating the hands-free and non-visually occluding video capture device at the same location as the object, the hands-free and non-visually occluding video capture device configured to directly capture video of the portion of the object as actually viewed by the user;

receiving an audio capture from the user through a hands-free and non-visually occluding audio capture device attached to the user, the audio capture comprising a request for information about the portion of the object in the video capture;

converting the audio capture into text comprising the request for information;

retrieving the information according to the request for information in the text;

converting the information into a spoken dialog through a speech synthesizer; and

transmitting the information to the user for playback through a hands-free and non-visually occluding audio output device attached to the user.

15. The method of claim 14 , wherein the hands-free and non-visually occluding video capture device comprises a head-mounted video camera or a helmet-mounted video camera.

16. The method of claim 14 , wherein the hands-free and non-visually occluding audio capture device and the hands-free and non-visually occluding audio output device comprise a headset attached to the user.

17. The method of claim 14 , wherein the request for information about the portion of the object in the video capture comprises predefined commands for processing the video capture.

18. The method of claim 14 , wherein the hands-free and non-visually occluding video capture device is controlled by instructions spoken into the hands-free audio capture device.

19. A non-transitory computer-readable medium having computer-executable instructions stored thereon which, when executed by a computer, cause the computer to:

receive a video capture of a portion of an object through a hands-free and non-visually occluding video capture device attached to a user operating the hands-free and non-visually occluding video capture device at the same location as the object, the hands-free and non-visually occluding video capture device being configured to directly capture video of the portion of the object as actually viewed by the user;

receive an audio capture from the user through a hands-free and non-visually occluding audio capture device attached to the user, the audio capture comprising a request for instructions for performing a maintenance task on the portion of the object in the video capture;

convert the audio capture into text comprising the request for instructions;

retrieve the instructions according to the request for instructions in the text;

convert the instructions into a spoken dialog through a speech synthesizer; and

transmit the instructions to the user for playback through the a hands-free and non-visually occluding audio output device attached to the user.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the object comprises an aircraft component or assembly.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 9, 2008
From: MARSH, JOSEPH C.; SMITH, ERIC M.
To: THE BOEING COMPANY
Reel/Frame 021503/0080 →
Continuity (1)
Related Publication 20100063821A1 · Mar 11, 2010