Information processing apparatus, information processing method, and program
An information processing apparatus includes a detection unit that detects a three-dimensional position and a posture of an object in an instruction three-dimensional region having an enlarged or reduced relationship with an observation three-dimensional region in which a virtual viewpoint and a virtual visual line are defined, a derivation unit that derives the viewpoint and the visual line corresponding to detection results of the detection unit depending on positional relationship information indicating a relative positional relationship between the observation three-dimensional region and the instruction three-dimensional region, and an acquisition unit that acquires a virtual viewpoint image showing a subject in a case in which the subject is observed with the viewpoint and the visual line derived by the derivation unit, the virtual viewpoint image being based on a plurality of images obtained by imaging an imaging region included in the observation three-dimensional region by a plurality of imaging apparatuses.
1. An information processing apparatus comprising:
a processor; and
a memory built in or connected to the processor,
wherein the processor
detects a three-dimensional position and a posture of an object in an instruction three-dimensional region having an enlarged or reduced relationship with an observation three-dimensional region in which a virtual viewpoint and a virtual visual line are defined,
derives the viewpoint and the visual line corresponding to detection results of the three-dimensional position and the posture of the object depending on positional relationship information indicating a relative positional relationship between the observation three-dimensional region and the instruction three-dimensional region, and
acquires, from among virtual viewpoint images based on a plurality of images obtained by imaging an imaging region included in the observation three-dimensional region by a plurality of imaging apparatuses, a virtual viewpoint image showing a subject present in the imaging region in a case in which the imaging region is observed with the derived viewpoint and the derived visual line.
2. The information processing apparatus according to claim 1 ,
wherein the relative positional relationship is a relationship between a position of the observation three-dimensional region with respect to a reference point and a position of the instruction three-dimensional region with respect to the reference point.
3. The information processing apparatus according to claim 2 ,
wherein the positional relationship information is information in which a coordinate indicating a position in the observation three-dimensional region with the reference point as an origin and a coordinate indicating a position in the instruction three-dimensional region with the reference point as an origin are associated with each other.
4. The information processing apparatus according to claim 2 ,
wherein the positional relationship information is information including a degree of difference between a distance between a position in the observation three-dimensional region and the reference point and a distance between a position in the instruction three-dimensional region and the reference point.
5. The information processing apparatus according to claim 2 ,
wherein the reference point is classified into an observation reference point applied to the observation three-dimensional region and an instruction reference point applied to the instruction three-dimensional region, and
the instruction reference point is located at a position different from the observation reference point and has a correspondence with the observation reference point.
6. The information processing apparatus according to claim 5 ,
wherein the instruction reference point is decided by detecting the three-dimensional position by the processor.
7. The information processing apparatus according to claim 1 ,
wherein the processor
detects at least three points of the three-dimensional positions, and
generates the instruction three-dimensional region based on an instruction reference plane specified by using the detected at least three points of the three-dimensional positions.
8. The information processing apparatus according to claim 1 ,
wherein the processor generates the instruction three-dimensional region based on an instruction reference plane specified by using an image obtained by imaging a reference subject.
9. The information processing apparatus according to claim 1 ,
wherein, in a case in which a display surface of a first observation three-dimensional region image in a state in which the first observation three-dimensional region image is displayed by a first display apparatus is defined as an instruction reference plane, the first observation three-dimensional region image being obtained by imaging the observation three-dimensional region by a first imaging apparatus or obtained by imaging the observation three-dimensional region by a first virtual imaging apparatus, the processor generates the instruction three-dimensional region based on the instruction reference plane.
10. The information processing apparatus according to claim 9 ,
wherein the first observation three-dimensional region image is an image showing the observation three-dimensional region in a bird's-eye view.
11. The information processing apparatus according to claim 9 ,
wherein the processor enlarges or reduces the first observation three-dimensional region image in response to a given instruction.
12. The information processing apparatus according to claim 7 ,
wherein the instruction reference plane is a plane for defining one outer plane of the instruction three-dimensional region, one outer plane of the instruction three-dimensional region, or an inner plane of the instruction three-dimensional region.
13. The information processing apparatus according to claim 1 ,
wherein the processor
detects the posture by measuring a pitch angle, a yaw angle, and a roll angle of the object, and
acquires the virtual viewpoint image in a direction corresponding to the posture detected by measuring the roll angle.
14. The information processing apparatus according to claim 1 ,
wherein the observation three-dimensional region is a three-dimensional region defined based on a second observation three-dimensional region image, which is obtained by imaging a specific reference plane by a second imaging apparatus or obtained by imaging the specific reference plane by a second virtual imaging apparatus, and has a similarity relationship with the instruction three-dimensional region,
the processor is configured to:
acquire a size change instruction for changing a size of the specific reference plane, and
change the size of the observation three-dimensional region by changing the size of the specific reference plane in response to the acquired size change instruction, and
the similarity relationship is maintained regardless of a change in the size of the observation three-dimensional region.
15. The information processing apparatus according to claim 1 , wherein:
the processor is configured to acquire an image size change instruction for enlarging or reducing a specific reference plane image showing a specific reference plane in a state in which an image including the specific reference plane image is displayed by a second display device, and
the positional relationship information is information including information indicating a relative positional relationship between a three-dimensional region in real space corresponding to the specific reference plane image enlarged or reduced in response to the acquired image size change instruction, and the instruction three-dimensional region.
16. The information processing apparatus according to claim 14 ,
wherein the processor changes the size of the observation three-dimensional region by enlarging or reducing the second observation three-dimensional region image in response to the size change instruction.
17. The information processing apparatus according to claim 14 ,
wherein the processor further acquires an imaging direction change instruction for changing an imaging direction of the second imaging apparatus or the second virtual imaging apparatus, and
changes the imaging direction in response to the acquired imaging direction change instruction.
18. The information processing apparatus according to claim 1 ,
wherein the processor outputs the acquired virtual viewpoint image.
19. An information processing method comprising:
detecting a three-dimensional position and a posture of an object in an instruction three-dimensional region having an enlarged or reduced relationship with an observation three-dimensional region in which a virtual viewpoint and a virtual visual line are defined;
deriving the viewpoint and the visual line corresponding to detection results of the three-dimensional position and the posture of the object depending on positional relationship information indicating a relative positional relationship between the observation three-dimensional region and the instruction three-dimensional region; and
acquiring, from among virtual viewpoint images based on a plurality of images obtained by imaging an imaging region included in the observation three-dimensional region by a plurality of imaging apparatuses, a virtual viewpoint image showing a subject present in the imaging region in a case in which the imaging region is observed with the derived viewpoint and the derived visual line.
20. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a process comprising:
detecting a three-dimensional position and a posture of an object in an instruction three-dimensional region having an enlarged or reduced relationship with an observation three-dimensional region in which a virtual viewpoint and a virtual visual line are defined;
deriving the viewpoint and the visual line corresponding to detection results of the three-dimensional position and the posture of the object depending on positional relationship information indicating a relative positional relationship between the observation three-dimensional region and the instruction three-dimensional region; and
acquiring, from among virtual viewpoint images based on a plurality of images obtained by imaging an imaging region included in the observation three-dimensional region by a plurality of imaging apparatuses, a virtual viewpoint image showing a subject present in the imaging region in a case in which the imaging region is observed with the derived viewpoint and the derived visual line.
21. The information processing apparatus according to claim 1 ,
wherein the visual line is a visual line of a user in a case in which the user observes the imaging region in the observation three-dimensional region.