Information processing apparatus and information processing method
An information processing apparatus includes circuitry configured to provide a user interface to a user, determine whether a predetermined object is present in an input recognition area of a sensor, and determine a region of interest within the user interface. The circuitry is further configured to determine, while the predetermined object is determined to be present in the input recognition area, whether the region of interest within the user interface changes. The circuitry is further configured to perform an operation based on whether the region of interest is determined, while the predetermined object is determined to be present in the input recognition area, to change.
1. An information processing apparatus, comprising:
a first camera configured to capture an image of a plurality of objects in real space;
a storage unit configured to store a plurality of first feature amounts corresponding to the plurality of objects; and
circuitry configured to:
determine a plurality of second feature amounts corresponding to the plurality of objects, wherein the plurality of second feature amounts is determined based on the captured image;
recognize the plurality of objects in the real space based on the plurality of first feature amounts and the plurality of second feature amounts;
control, based on the recognition, a display screen to display a user interface, wherein
the user interface comprises a plurality of virtual objects,
a position of each virtual object of the plurality of virtual objects in the display screen corresponds to a position of a respective object of the recognized plurality of objects in the captured image, and
a posture of each virtual object of the plurality of virtual objects is changeable based on a posture of the respective object of the recognized plurality of objects;
determine, as a user gesture, a presence of a user object in an input recognition area of the first camera;
determine a region of interest within the user interface, wherein the region of interest corresponds to a virtual object of the plurality of virtual objects;
control, based on the determined region of interest, the display screen to display a specific object at a position corresponding to a position of an object of the recognized plurality of objects;
determine that the region of interest remains same during the presence of the user object in the input recognition area;
detect, as a termination of the user gesture, an absence of the user object in the input recognition area;
execute one of a tap operation, a click operation, or a select operation on the specific object based on
the determination that the region of interest remains same and
the detected absence of the user object in the input recognition area; and
control the display screen to display specific information on the specific object, wherein
the display of the specific information on the specific object is based on the detected absence of the user object in the input recognition area and the execution of the one of the tap operation, the click operation, or the selection operation, and
the specific information includes at least one of a title associated with the specific object or contents associated with the specific object.
2. The information processing apparatus according to claim 1 , further comprising a sensor configured to generate data representative of a user viewing direction, wherein
the circuitry is further configured to determine the region of interest within the user interface based on the generated data.
3. The information processing apparatus according to claim 2 , wherein the sensor is one of a second camera, an acceleration sensor, or a gyro sensor.
4. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to:
determine a change in the region of interest within the user interface based on the user gesture; and
execute a drag operation on the specific object based on the change of the region of interest.
5. The information processing apparatus according to claim 4 , wherein the circuitry is further configured to control, based on the change in the region of interest, a first device to transmit content information to a second device.
6. The information processing apparatus according to claim 5 , wherein the first device is a television.
7. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to transmit, based on the user gesture, a request to a first device to send determined data to the information processing apparatus.
8. The information processing apparatus according to claim 7 , wherein the circuitry is further configured to receive the determined data from the first device based on the request.
9. The information processing apparatus according to claim 7 , wherein the circuitry is further configured to receive a discount coupon from the first device based on the request.
10. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to:
determine a change in the region of interest within the user interface based on the user gesture; and
execute a drag operation on the specific object based on an amount of the change of the region of interest.
11. The information processing apparatus according to claim 1 , wherein the first camera is further configured to capture an image of the user object.
12. The information processing apparatus according to claim 1 , further comprising the display screen, wherein the display screen is in a head-mounted display unit.
13. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to determine, as the user gesture, a gesture of movement of a part of an operation body.
14. An information processing system, comprising:
a camera configured to capture an image of a plurality of objects in real space;
a storage unit configured to store a plurality of first feature amounts corresponding to the plurality of objects; and
circuitry configured to:
determine a plurality of second feature amounts corresponding to the plurality of objects, wherein the plurality of second feature amounts is determined based on the captured image;
recognize the plurality of objects in the real space based on the plurality of first feature amounts and the plurality of second feature amounts;
control, based on the recognition, a display screen to display a user interface, wherein
the user interface comprises a plurality of virtual objects,
a position of each virtual object of the plurality of virtual objects in the display screen corresponds to a position of a respective object of the recognized plurality of objects in the captured image, and
a posture of each virtual object of the plurality of virtual objects is changeable based on a posture of the respective object of the recognized plurality of objects;
determine, as a user gesture, a presence of a user object in an input recognition area of the camera;
determine a region of interest within the user interface, wherein the region of interest corresponds to a virtual object of the plurality of virtual objects;
control, based on the determined region of interest, the display screen to display a specific object at a position corresponding to a position of an object of the recognized plurality of objects;
determine that the region of interest remains same during the presence of the user object in the input recognition area;
detect, as a termination of the user gesture, an absence of the user object in the input recognition area;
execute one of a tap operation, a click operation, or a select operation on the specific object based on
the determination that the region of interest remains same and
the detected absence of the user object in the input recognition area; and
control the display screen to display specific information on the specific object, wherein
the display of the specific information on the specific object is based on the detected absence of the user object in the input recognition area and the execution of the one of the tap operation, the click operation, or the selection operation, and
the specific information includes at least one of a title associated with the specific object or contents associated with the specific object.
15. A method, comprising:
capturing, by a camera of an information processing apparatus, an image of a plurality of objects in real space;
storing, in a storage unit of the information processing apparatus, a plurality of first feature amounts corresponding to the plurality of objects;
determining, by circuitry of the information processing apparatus, a plurality of second feature amounts corresponding to the plurality of objects, wherein the plurality of second feature amounts is determined based on the captured image;
recognizing, by the circuitry, the plurality of objects in the real space based on the plurality of first feature amounts and the plurality of second feature amounts;
controlling, by the circuitry, based on the recognition, a display screen to display a user interface, wherein
the user interface comprises a plurality of virtual objects,
a position of each virtual object of the plurality of virtual objects in the display screen corresponds to a position of a respective object of the recognized plurality of objects in the captured image, and
a posture of each virtual object of the plurality of virtual objects is changeable based on a posture of the respective object of the recognized plurality of objects;
determining, by the circuitry, a presence of a user object in an input recognition area of the camera, as a user gesture;
determining, by the circuitry, a region of interest within the user interface, wherein the region of interest corresponds to a virtual object of the plurality of virtual objects;
controlling, by the circuitry, based on the determined region of interest, the display screen to display a specific object at a position corresponding to a position of an object of the recognized plurality of objects;
determining, by the circuitry, that the region of interest remains same during the presence of the user object in the input recognition area;
detecting, by the circuitry, an absence of the user object in the input recognition area, as a termination of the user gesture;
executing, by the circuitry, one of a tap operation, a click operation, or a select operation on the specific object based on
the determination that the region of interest remains same and
the detected absence of the user object in the input recognition area; and
controlling, by the circuitry, the display screen to display specific information on the specific object, wherein
the display of the specific information on the specific object is based on the detected absence of the user object in the input recognition area and the execution of the one of the tap operation, the click operation, or the selection operation, and
the specific information includes at least one of a title associated with the specific object or contents associated with the specific object.
16. A non-transitory computer-readable storage medium having stored thereon, computer-executable instructions which when executed by a processor of an information processing apparatus, cause the processor to execute operations, the operations comprising:
controlling a camera of the information processing apparatus to capture an image of a plurality of objects in real space;
controlling a storage unit of the information processing apparatus to store a plurality of first feature amounts corresponding to the plurality of objects;
determining a plurality of second feature amounts corresponding to the plurality of objects, wherein the plurality of second feature amounts is determined based on the captured image;
recognizing the plurality of objects in the real space based on the plurality of first feature amounts and the plurality of second feature amounts;
controlling, based on the recognition, a display screen to display a user interface, wherein
the user interface comprises a plurality of virtual objects,
a position of each virtual object of the plurality of virtual objects in the display screen corresponds to a position of a respective object of the recognized plurality of objects in the captured image, and
a posture of each virtual object of the plurality of virtual objects is changeable based on a posture of the respective object of the recognized plurality of objects;
determining, as a user gesture, a presence of a user object in an input recognition area of the camera;
determining a region of interest within the user interface, wherein the region of interest corresponds to a virtual object of the plurality of virtual objects;
controlling, based on the determined region of interest, the display screen to display a specific object at a position corresponding to a position of an object of the recognized plurality of objects;
determining that the region of interest remains same during the presence of the user object in the input recognition area;
detecting, as a termination of the user gesture, an absence of the user object in the input recognition area;
executing one of a tap operation, a click operation, or a select operation on the specific object based on
the determination that the region of interest remains same and
the detected absence of the user object in the input recognition area; and
controlling the display screen to display specific information on the specific object, wherein
the display of the specific information on the specific object is based on the detected absence of the user object in the input recognition area and the execution of the one of the tap operation, the click operation, or the selection operation, and
the specific information includes at least one of a title associated with the specific object or contents associated with the specific object.
17. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to:
determine a change in the region of interest within the user interface based on the user gesture;
select a specific region of the user interface based on the change in the region of interest;
select the plurality of virtual objects based on the selected specific region;
sort the plurality of virtual objects; and
control the display screen to display the sorted plurality of virtual objects.