IP Library Granted Patent US 11,501,459
Granted Patent B2
US 11,501,459 · App. 16/765,804 · Granted Nov 15, 2022

Information processing apparatus, method of information processing, and information processing system

Inventors: Takuya Ikeda (Tokyo, JP); Kentaro Ida (Tokyo, JP); Yuri Kusakabe (Tokyo, JP)
Assignee: SONY CORPORATION
G06T7/70G06F3/0346G06F3/042G06T7/50H04N5/23299H04N5/247G06T2207/30196G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,501,459
App. No.
16/765,804
Granted
Nov 15, 2022
Kind
B2
Abstract

Provided is an information processing apparatus including a determination unit that determines an input state of a user based on a captured image of a first imaging device and a sensor controller that determines an imaging position at which an input image is captured by a second imaging device different from the first imaging device, based on a positional relation between an input-related object related to input by the user and each of a plurality of imaging position candidates and the input state.

Claims (44)

1. An information processing apparatus, comprising:

a determination unit configured to determine an input state of a user based on a captured image of a first imaging device, wherein the input state includes at least one of a state in which an operator indicates a pointing surface or a state in which a distance between the user and the pointing surface is shorter than a threshold distance; and

a sensor controller configured to determine an imaging position, from a plurality of imaging position candidates, at which an input image is captured by a second imaging device different from the first imaging device, wherein

the determination of the imaging position is based on the input state and a positional relation between an input-related object and each of the plurality of imaging position candidates, and

the input-related object includes at least one of the operator or the pointing surface.

2. The information processing apparatus according to claim 1 , wherein the positional relation includes at least one of a distance between the input-related object and each of the plurality of imaging position candidates or a direction of each of the plurality of imaging position candidates based on the input-related object.

3. The information processing apparatus according to claim 1 , wherein at least one of a view angle of the second imaging device is narrower than a view angle of the first imaging device or a resolution of the second imaging device is higher than a resolution of the first imaging device.

4. The information processing apparatus according to claim 1 , wherein

the first imaging device is an overhead sensor, and

the second imaging device is a homing sensor.

5. The information processing apparatus according to claim 1 , further comprising an input controller configured to execute a process related to the input image.

6. The information processing apparatus according to claim 5 , wherein the process related to the input image is started by indication of the pointing surface with the operator.

7. The information processing apparatus according to claim 5 , wherein the process related to the input image ends in a case where at least one of a first time has elapsed after the operator no longer indicates the pointing surface, the second imaging device no longer images the user, a second time has elapsed after movement of the operator is no longer detected, connection between an input device and a computer is cut, or power supply to the input device ends.

8. The information processing apparatus according to claim 5 , wherein the process related to the input image includes display of a first image based on first pointing coordinates of the operator recognized from the input image.

9. The information processing apparatus according to claim 8 , wherein the input controller is further configured to display a second image, based on second pointing coordinates of the operator recognized from the captured image of the first imaging device prior to the display of the first image.

10. The information processing apparatus according to claim 8 , wherein the input controller is further configured to display the first image by control of a projector to project the first image onto the pointing surface.

11. The information processing apparatus according to claim 5 , wherein

the sensor controller is further configured to determine the imaging position based on a plurality of evaluation values and a weight coefficient of each of the plurality of evaluation values,

the plurality of evaluation values includes an evaluation value based on the positional relation, and

the weight coefficient is based on a type of the operator.

12. The information processing apparatus according to claim 5 , wherein the sensor controller is further configured to:

select, based on the input state and a positional relation between the input-related object and an imaging device candidate provided at each of the plurality of imaging position candidates, the second imaging device from a plurality of imaging device candidates; and

determine, as the imaging position, a position at which the selected second imaging device is provided.

13. The information processing apparatus according to claim 12 , wherein the sensor controller is further configured to determine the imaging position, based on the input state and an evaluation value related to the positional relation between the input-related object and the imaging device candidate, and

the evaluation value includes one of:

a size of the input-related object appearing in a captured image by each of the plurality of imaging device candidates,

a rotational angle of each of the plurality of imaging device candidates required for the input-related object to be included in an imaging range,

a shielded area size of the pointing surface by at least one of an obstacle or the user appearing in the captured image by each of the plurality of imaging device candidates, or

an area size in a certain direction based on a position of the user of the pointing surface appearing in the captured image by each of the plurality of imaging device candidates.

14. The information processing apparatus according to claim 12 , wherein the sensor controller is further configured to determine the imaging position based on at least one of a resolution of each of the plurality of imaging device candidates or a detection result of external light by each of the plurality of imaging device candidates.

15. The information processing apparatus according to claim 1 , wherein the sensor controller is further configured to move the second imaging device to the determined imaging position.

16. The information processing apparatus according to claim 15 , wherein the sensor controller is further configured to determine the imaging position based on a movement distance until the second imaging device reaches each of the plurality of imaging position candidates.

17. A method of information processing, comprising:

determining, by a processor, an input state of a user based on a captured image of a first imaging device, wherein the input state includes at least one of a state in which an operator indicates a pointing surface or a state in which a distance between the user and the pointing surface is shorter than a threshold distance; and

determining, by the processor, an imaging position, from a plurality of imaging position candidates, at which an input image is captured by a second imaging device different from the first imaging device, wherein

the determination of the imaging position is based on the input state and a positional relation between an input-related object and each of the plurality of imaging position candidates, and

the input-related object includes at least one of the operator or the pointing surface.

18. An information processing system, comprising:

a first imaging device configured to capture an image; and

an information processing apparatus that comprises:

a determination unit configured to determine an input state of a user based on the captured image, wherein the input state includes at least one of a state in which an operator indicates a pointing surface or a state in which a distance between the user and the pointing surface is shorter than a threshold distance; and

a sensor controller configured to determine an imaging position, from a plurality of imaging position candidates, at which an input image is captured by a second imaging device different from the first imaging device, wherein

the determination of the imaging position is based on the input state and a positional relation between an input-related object and each of the plurality of imaging position candidates, and

the input-related object includes at least one of the operator or the pointing surface.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2020
From: IKEDA, TAKUYA; IDA, KENTARO; KUSAKABE, YURI
To: SONY CORPORATION
Reel/Frame 053550/0929 →
Priority Claims (1)
JP JP2017-250390 · Dec 27, 2017 · national
Continuity (1)
Related Publication 20200320729A1 · Oct 8, 2020
Cited By (1)
US 12,705,776