Image processing device and image processing method

1. An image processing device comprising:

a processor;

a projector configured to be controlled by the processor; and

a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute:

acquiring an image, including a first object having a character or a figure and a second object, captured by an image sensor;

computing, from the image, flatness information on flatness of a projection plane of the first object on which a superimposed image is to be displayed by the projector, wherein the superimposed image is relevant to the character or the figure and includes additional information associated with the first object, wherein the flatness information is defined from three-dimensional rectangular coordinates of the first object computed from the image;

extracting display candidate regions based on a predetermined feature quantity with respect to the first object and the three-dimensional rectangular coordinates of the first object computed from the image;

defining plane rectangular coordinates serving as a display position, which is a partial part of the first object for displaying the superimposed image by the projector on the basis of the flatness information and a dispersion of the plurality of the display candidate regions relative to a reference point; and

projecting the superimposed image on the partial part of the first object.

2. The device according to claim 1 ,

wherein there are a plurality of the images, and, in the computing, three-dimensional rectangular coordinates of a user's fingertip serving as the second object are computed on the basis of coordinates of the fingertip included in each of the plurality of images and a reference point of the three-dimensional rectangular coordinates.

3. The device according to claim 1 , wherein there are a plurality of the images, and, in the computing, three-dimensional rectangular coordinates of the projection plane including the character or the figure of the first object are computed on the basis of an amount of change in each of a plurality of pixels of a superimposed image included in each of the plurality of the images.

4. The device according to claim 1 ,

wherein, in the defining, the display candidate region in which dispersion is equal to or less than a first threshold is selected among the plurality of the display candidate regions, and the plane rectangular coordinates are defined.

5. The device according to claim 1 ,

wherein, in the defining, plane rectangular coordinates of the superimposed image are defined on the basis of relative positions of the projecting and the first object.

6. An image processing method comprising:

acquiring an image including a first object having a character or a figure and a second object, captured by an image sensor;

computing, from the image, flatness information on flatness of a projection plane of the first object on which a superimposed image is to be displayed by a projector, wherein the superimposed image is relevant to the character or the figure and includes additional information associated with the first object, wherein the flatness information is defined from three-dimensional rectangular coordinates of the first object computed from the image;

extracting display candidate regions based on a predetermined feature quantity with respect to the first object and the three-dimensional rectangular coordinates of the first object computed from the image;

defining, by a computer processor, plane rectangular coordinates serving as a display position which is a partial part of the first object displaying the superimposed image by the projector on the basis of the flatness information and a dispersion of the plurality of the display candidate reqions relative to a reference point; and

projecting the superimposed image on the partial part of the first object.

7. The method according to claim 6 ,

wherein there are a plurality of the images, and, in the computing, three-dimensional rectangular coordinates of a user's fingertip serving as the second object are computed on the basis of coordinates of the fingertip included in each of the plurality of images and a reference point of the three-dimensional rectangular coordinates.

8. The method according to claim 6 , wherein there are a plurality of the images, and, in the computing, three-dimensional rectangular coordinates of the projection plane including the character or the figure of the first object are computed on the basis of an amount of change in each of a plurality of pixels of a superimposed image included in each of the plurality of the images.

9. The method according to claim 6 ,

wherein, in the defining, the display candidate region in which dispersion is equal to or less than a first threshold is selected among the plurality of the display candidate regions, and the plane rectangular coordinates are defined.

10. The method according to claim 6 ,

wherein, in the defining, plane rectangular coordinates of the superimposed image are defined on the basis of relative positions of the projecting and the first object.

11. A non-transitory computer-readable storage medium storing an image processing program that causes a computer to execute a process comprising:

acquiring an image, including a first object having a character or a figure and a second object, captured by an image sensor;

computing, from the image, flatness information on flatness of a projection plane of the first object on which a superimposed image is to be displayed by a projector, wherein the superimposed image is relevant to the character or the figure and includes additional information associated with the first object, wherein the flatness information is defined from three-dimensional rectangular coordinates of the first object computed from the image;

extracting display candidate regions based on a predetermined feature quantity with respect to the first object and the three-dimensional rectangular coordinates of the first object computed from the image;

defining plane rectangular coordinates serving as a display position which is a partial part of the first object, and displaying the superimposed image by a projector on the basis of the flatness information and a dispersion of the plurality of the display candidate regions relative to a reference point; and

projecting the superimposed image on the partial part of the first object.