Cross-reality device, storage medium, processing device, generation method, and processing method

1 . A cross-reality device, comprising:

an imaging device including a camera and configured to acquire an image;

a display device including a monitor and configured to display a virtual space; and

a processing device including processing circuitry, the processing circuitry being configured to:

detect fingertips of a hand from the image and a voice input to a microphone,

generate an object in the virtual space in response to a user operation at least including

specifying, based on the detected fingertips, at least one selected from the group consisting of a region, a movement, and a dimension, and

a voice utterance indicating a command, and

change a position and a size of the generated object in accordance with movements of the detected fingertips,

wherein the voice utterance specifies a type of operation for the change, and, when changing the position and the size of the generated object, the processing circuitry is configured to specify, based on movements of the detected fingertips, at least one selected from the group consisting of a region, a movement, and a dimension as a parameter of the type of operation and to apply the type of operation using the parameter.

2 . The cross-reality device according to claim 1 , wherein

in a case where a marker exists in a real space is imaged by the camera, the processing circuitry sets a three-dimensional coordinate system in the virtual space based on the marker, and changes the position and the size of the object in the three-dimensional coordinate system.

3 . The cross-reality device according to claim 1 , wherein

the processing circuitry is configured to accept a setting of an attribute indicating a function of the object, and

the function is a detection of contact between a prescribed physical object and the object.

4 . The cross-reality device according to claim 1 , wherein

the object is stored in association with data related to a fastening location of an article in a real space.

5 . The cross-reality device according to claim 1 , wherein

the command is inputted by a voice or a hand gesture.

6 . The cross-reality device according to claim 1 , wherein

the processing circuitry is configured to

display an object with a plurality of commands on the monitor, and

accept an input of the plurality of commands selected by a user.

7 . The cross-reality device according to claim 1 , wherein

the display device is configured to display the virtual space overlaid on a real space.

8 . A non-transitory computer-readable storage medium storing a program, the program causing a computer to:

detect fingertips of a hand from an image and a voice of a human;

generate an object in a virtual space in response to a user operation at least including

specifying, based on the fingertips, at least one selected from the group consisting of a region, a movement, and a dimension, and

a voice utterance indicating a command; and

change a position and a size of the generated object in accordance with movements of the detected fingertips,

wherein the voice utterance specifies a type of operation for the change, and, when changing the position and the size of the generated object, the computer is caused to specify, based on movements of the detected fingertips, at least one selected from the group consisting of a region, a movement, and a dimension as a parameter of the type of operation and to apply the type of operation using the parameter.

9 . A processing device, configured to:

read the program from the storage medium according to claim 8 ; and

execute the program.

10 . An object generation method performed by a first processing device, comprising:

detecting fingertips of a hand from an image and a voice of a human;

generating an object in a virtual space in response to a user operation at least including

specifying, based on the fingertips, at least one selected from the group consisting of a region, a movement, and a dimension, and

a voice utterance indicating a command; and

changing a position and a size of the object in accordance with movements of the detected fingertips,

wherein the voice utterance specifies a type of operation for the change, and, when changing the position and the size of the generated object, the first processing device is configured to specify, based on movements of the detected fingertips, at least one selected from the group consisting of a region, a movement, and a dimension as a parameter of the type of operation and to apply the type of operation using the parameter.

11 . The generation method according to claim 10 , wherein

the first processing device

detects a marker from the image,

sets a three-dimensional coordinate system based on the marker, and

changes the position and the size of the object in the three-dimensional coordinate system.

12 . A processing method performed by a second processing device, comprising:

displaying the object generated by the generation method according to claim 10 on a display device; and

detecting contact between a prescribed physical object and the object from the image in a case where the prescribed physical object is imaged.

13 . The cross-reality device according to claim 3 , wherein

the attribute indicates a jig region in the virtual space corresponding to a region in a real space into which a tool is not allowed to enter, and

the processing circuitry is configured to determine that the tool enters the jig region based on the image and to output a notification in response to the determination.

14 . The cross-reality device according to claim 3 , wherein

the object includes a guide having a spherical shape in the virtual space, and

the processing circuitry is configured to

detect contact between the guide and a prescribed real object based on the image and

update a work state of fastening in response to the detected contact.

15 . The cross-reality device according to claim 2 , wherein

the marker is disposed on a real article and is captured both in a preparation phase in which the object is generated and in an actual work phase in which the object is used, and

the three-dimensional coordinate system is set based on the marker so that a position and a size of the object defined in the preparation phase are displayed in the actual work phase.

16 . The cross-reality device according to claim 1 , wherein

the type of operation includes a move operation, and

when the type of operation is the move operation, the processing circuitry is configured to apply the type of operation so that the movement selected from the group is used as a translation amount of the object.

17 . The cross-reality device according to claim 1 , wherein

the type of operation includes a resize operation, and

when the type of operation is the resize operation, the processing circuitry is configured to apply the type of operation so that the dimension selected from the group is used as a scaling amount of the object.