IP Library Granted Patent US 10,228,905
Granted Patent B2
US 10,228,905 · App. 15/434,439 · Granted Mar 12, 2019

Pointing support apparatus and pointing support method

Inventors: Hiroshi Nakayama (Kawasaki, JP); Junichi Odagiri (Kawasaki, JP); Satoshi Nakashima (Kawasaki, JP); Kentaro Murase (Yokohama, JP); Masakiyo Tanaka (Hachioji, JP)
Assignee: FUJITSU LIMITED
G06F3/167G06F3/013G06F3/0481G06F3/0482G06F3/0487G06F3/04812G06F3/04842
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,228,905
App. No.
15/434,439
Granted
Mar 12, 2019
Kind
B2
Abstract

A pointing support apparatus includes a memory, and a processor coupled to the memory and configured to detect a line-of-sight position of a user on a screen, extract a command included in a search range of the screen with reference to the line-of-sight position, generate a table in which the command and speech information of the command are associated with each other, and decide, when speech information from outside is accepted, a command based on comparison of the recognized speech information and the speech information of the table.

Claims (43)

1. A pointing support apparatus comprising:

a memory; and

a processor coupled to the memory and configured to:

detect an activation of a screen based on an operation of a user;

scan the screen to extract first information regarding character strings of commands on the screen;

generate first speech information corresponding to the first information;

register, in a first table, the first information and the first speech information in association with each other;

detect a line-of-sight position of the user on the screen;

set the line-of-sight position as an initial position of a pointer;

extract one or more commands included in a search range of the screen, which is defined based on the line-of-sight position and is smaller than the screen, to generate second information regarding character strings of the one or more commands;

generate a second table in which the second information and the first speech information stored in the first table are associated with each other;

compare, when first speech information from the user is recognized, the recognized first speech information and the first speech information of the second table;

specify the second information having the first speech information which matches the recognized first speech information; and

move the pointer on the screen to a command corresponding to the specified second information,

wherein the processor causes, based on route definition information that defines a route along which the pointer is moved and a movement pattern of the route, the route whose start point is given by the initial position to be displayed on the screen, and causes the pointer to move along the route.

2. The pointing support apparatus according to claim 1 , wherein the processor accepts selection of one of a first process to move the pointer to a position of the command corresponding to the specified second information and a second process to move the pointer based on the route definition information, and switches a process between the first process and the second process.

3. The pointing support apparatus according to claim 1 , wherein the processor generates the first information regarding character strings of all commands which exist on the screen.

4. The pointing support apparatus according to claim 3 , wherein the processor generates, when one command on the screen is selected and another screen is displayed, second speech information of all commands which exist on the second screen, and stores the second speech information in the memory.

5. The pointing support apparatus according to claim 3 , wherein, every time the one or more commands are extracted, the processor increases a use frequency of the first speech information corresponding to the one or more commands and stored in the first table and preferentially leaves first speech information whose use frequency is high in the first table.

6. A pointing support method comprising:

detecting, by a processor, an activation of a screen based on an operation of a user;

scanning the screen to extract first information regarding character strings of commands on the screen;

generating first speech information corresponding to the first information;

registering, in a first table, the first information and the first speech information in association with each other;

detecting a line-of-sight position of the user on the screen;

setting the line-of-sight position as an initial position of a pointer;

extracting one or more commands included in a search range of the screen, which is defined based on the line-of-sight position and is smaller than the screen, to generate second information regarding character strings of the one or more commands;

generating a second table in which the second information and the first speech information stored in the first table are associated with each other;

comparing, when first speech information from the user is accepted, the accepted first speech information and the first speech information of the second table;

specifying the second information having the first speech information which matches the accepted first speech information;

moving the pointer on the screen to a command corresponding to the specified second information;

displaying, based on route definition information that defines a route along which the pointer is moved and a movement pattern of the route, the route whose start point is given by the initial position on the screen; and

moving the pointer along the route.

7. The pointing support method according to claim 6 , further comprising:

accepting selection of one of a first process to move the pointer to a position of the command corresponding to the specified second information and a second process to move the pointer based on the route definition information; and

switching a process between the first process and the second process.

8. The pointing support method according to claim 6 , wherein the first information regarding character strings of all commands which exist on the screen is generated.

9. The pointing support method according to claim 8 , further comprising:

generating, when one command on the screen is selected and another screen is displayed, second speech information of all commands which exist on the second screen; and

storing the second speech information in a memory.

10. The pointing support method according to claim 8 , further comprising:

increasing, every time the one or more commands are extracted, a use frequency of the first speech information corresponding to the one or more commands and stored in the first table; and

preferentially leaving first speech information whose use frequency is high in the first table.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2017
From: NAKAYAMA, HIROSHI; ODAGIRI, JUNICHI; NAKASHIMA, SATOSHI; MURASE, KENTARO; TANAKA, MASAKIYO
To: FUJITSU LIMITED
Reel/Frame 041782/0153 →
Priority Claims (1)
JP 2016-037415 · Feb 29, 2016 · national
Continuity (1)
Related Publication 20170249124A1 · Aug 31, 2017
Cited By (1)
US 12,592,228