Image processing apparatus, image processing method, and image processing program
In an aspect, an image processing apparatus includes a processor and a memory in which a plurality of images obtained by capturing images of a building and a three-dimensional model of the building are stored in association with each other. The processor is configured to perform a development process to perform development of the three-dimensional model into a two-dimensional image, an extraction process to extract defect information of the building on the basis of the plurality of images, a mapping process to perform mapping of the defect information to the two-dimensional image, and an output process to output the two-dimensional image to which the mapping is performed.
1 . An image processing apparatus comprising:
a processor; and
a memory in which a plurality of images obtained by capturing images of a building and a three-dimensional model of the building are stored in association with each other, wherein
the processor is configured to:
perform development of the three-dimensional model into a two-dimensional image;
extract at least one of type of a defect, number of defects, size of a defect, degree of a defect, and change in degree of a defect over time as defect information of the building on the basis of the plurality of images;
perform mapping of the defect information to the two-dimensional image; and
output the two-dimensional image to which the mapping is performed,
wherein the three-dimensional model is a three-dimensional surface model or a three-dimensional solid model,
the processor is configured to perform the development by connecting or separating designated surfaces among surfaces constituting the three-dimensional surface model or the three-dimensional solid model along designated edges for a designated range of the three-dimensional surface model or the three-dimensional solid model, for a designated range of the three-dimensional surface model or the three-dimensional solid model,
the processor is configured to receive designation of the edge for the development,
the processor is configured to specify a member of the building using a learning machine configured by machine learning in which members constituting the three-dimensional model are given as correct labels,
the processor is configured to extract the defect information using a learning machine configured by machine learning, and
the processor is configured to associate the three-dimensional model with an actual scale by performing a comparison with a member having known dimensions or analyzing an image of the building which shows a scaler having a known dimension installed on the building.
2 . The image processing apparatus according to claim 1 , wherein the processor is configured to:
generate the three-dimensional surface model or the three-dimensional solid model on the basis of the plurality of images; and
store the generated three-dimensional surface model or the three-dimensional solid model in the memory in association with the plurality of images.
3 . The image processing apparatus according to claim 2 , wherein the processor is configured to reconstruct the generated three-dimensional surface model or the three-dimensional solid model such that designated regions of the generated three-dimensional surface model or the three-dimensional solid model are included in the same surface.
4 . The image processing apparatus according to claim 1 , wherein the processor is configured to:
perform the development by dividing a designated region of the three-dimensional model into planes having a designated size.
5 . The image processing apparatus according to claim 1 , wherein the processor is configured to align normal directions of regions constituting the two-dimensional image during the development.
6 . The image processing apparatus according to claim 1 , wherein the processor is configured to perform the development on a designated portion of the three-dimensional model.
7 . The image processing apparatus according to claim 1 , wherein the processor is configured to generate a three-dimensional point cloud model of the building and generate the three-dimensional surface model or the three-dimensional solid model on the basis of the three-dimensional point cloud model.
8 . The image processing apparatus according to claim 1 , wherein the processor is configured to generate a composite image based on the plurality of images and perform mapping of the composite image to the two-dimensional image.
9 . The image processing apparatus according to claim 8 , wherein the processor is configured to generate the composite image such that feature points in the plurality of images match or the plurality of images are projected onto the same plane.
10 . The image processing apparatus according to claim 1 , wherein the processor is configured to perform:
an acquisition process to acquire a plurality of images obtained by capturing images of the building, the plurality of images having different dates and times of capture from the plurality of images stored in the memory; and
an association process to associate the plurality of acquired images with the three-dimensional model stored in the memory.
11 . The image processing apparatus according to claim 10 , wherein the processor is configured to perform the association process on the basis of a correlation between the plurality of acquired images and the plurality of images stored in the memory.
12 . An image processing method performed by an image processing apparatus comprising
a processor, and
a memory in which a plurality of images obtained by capturing images of a building and a three-dimensional model of the building are stored in association with each other,
the image processing method causing the processor to perform a process including:
a development step of developing the three-dimensional model into a two-dimensional image;
an extraction step of extracting at least one of type of defect, number of defects, size of a defect, degree of a defect, and change of a defect over times as defect information of the building on the basis of the plurality of images;
a mapping step of performing mapping of the defect information to the two-dimensional image; and
an output step of outputting the two-dimensional image to which the mapping is performed,
wherein the three-dimensional model is a three-dimensional surface model or a three-dimensional solid model,
the processor is configured to perform the development by connecting or separating designated surfaces among surfaces constituting the three-dimensional surface model or the three-dimensional solid model along designated edges for a designated range of the three-dimensional surface model or the three-dimensional solid model, for a designated range of the three-dimensional surface model or the three-dimensional solid model,
the processor is configured to receive designation of the edge for the development,
the processor is configured to specify a member of the building using a learning machine configured by machine learning in which members constituting the three-dimensional model are given as correct labels,
the processor is configured to extract the defect information using a learning machine configured by machine learning, and
the processor is configured to associate the three-dimensional model with an actual scale by performing a comparison with a member having known dimensions or analyzing an image of the building which shows a scaler having a known dimension installed on the building.
13 . A non-transitory computer readable recording medium storing an image processing program for causing a computer to execute the image processing method according to claim 12 .
14 . The image processing apparatus according to claim 1 , wherein the processor is configured to generate a composite image from an image group constituting a part of the plurality of images or an image group selected by a user's operation, and further configured to perform mapping of the composite image to the three-dimensional image or a two-dimensional image obtained by developing the three-dimensional image.