Image processing device, information storage device, and image processing method
An image summarization device includes a first image summarization section that performs a first image summarization process based on a similarity between a plurality of images to acquire a first summary image sequence, a second image summarization section that performs a second image summarization process based on a target object/scene recognition process on each image among the plurality of images to acquire a second summary image sequence, and an integration processing section that performs an integration process on the first summary image sequence and the second summary image sequence, or performs an integration process on the first image summarization process and the second image summarization process to acquire an output summary image sequence.
1. An image processing device comprising:
a first image summarization section that performs a first image summarization process based on a similarity between a plurality of images to acquire a first summary image sequence;
a second image summarization section that performs a second image summarization process based on a target object/scene recognition process on each image among the plurality of images to acquire a second summary image sequence; and
an integration processing section that performs an integration process on the first summary image sequence and the second summary image sequence, or performs an integration process on the first image summarization process and the second image summarization process to acquire an output summary image sequence,
wherein the first image summarization section selects a reference image and a determination target image from the plurality of images, calculates a coverage ratio of the determination target image by the reference image as the similarity based on deformation information about the reference image and the determination target image, and performs the first image summarization process that determines whether or not the determination target image can be deleted based on the coverage ratio.
2. The image processing device as defined in claim 1 , wherein:
first to Nth (N is an integer equal to or larger than 2) images are input as an input image sequence,
the first image summarization section selects a pth (p is an integer that satisfies 1≦p≦N) image as a first reference image, selects a qth (q is an integer equal to or larger than p+2) image as a second reference image, and selects an rth (r is an integer that satisfies p+1≦r≦q−1) image as the determination target image, and
the first image summarization section calculates the coverage ratio based on the deformation information about the first reference image and the determination target image and the deformation information about the second reference image and the determination target image, and determines whether or not the determination target image can be deleted based on the coverage ratio.
3. The image processing device as defined in claim 2 , wherein the first image summarization section selects a (q+1)th image as the second reference image when it has been determined that (p+1)th to (q−1)th images can be deleted.
4. The image processing device as defined in claim 3 , wherein the first image summarization section allows the image selected as the first reference image to remain in the first summary image sequence when it has been determined that at least one image among the (p+1)th to (q−1)th images cannot be deleted, selects the (q−1)th image as a next first reference image, and performs the process again.
5. The image processing device as defined in claim 1 , wherein the first image summarization section calculates a coverage area based on the deformation information about the reference image and the determination target image, and calculates a ratio of the coverage area to the determination target image as the coverage ratio, the coverage area being an area in which the determination target image is covered by the reference image.
6. The image processing device as defined in claim 1 , wherein the second image summarization section sets consecutive images among the plurality of images that include an identical target object, or consecutive images among the plurality of images that have been recognized to be an identical scene, to be a consecutive image sequence from which a summary image is extracted, based on the recognition process, and performs the second image summarization process that extracts at least one image from the consecutive image sequence as the summary image.
7. The image processing device as defined in claim 6 , wherein the second image summarization section selects the summary image extracted from the consecutive image sequence based on an area of the target object.
8. The image processing device as defined in claim 6 , wherein the integration processing section integrates a plurality of the consecutive image sequences that have been set during the second image summarization process into one integrated consecutive image sequence based on the first summary image sequence, and extracts at least one image from the integrated consecutive image sequence as the summary image to update the second summary image sequence.
9. The image processing device as defined in claim 8 , wherein the integration processing section performs the integration process on the first summary image sequence and the second summary image sequence by selecting an image that is included in at least one of the first summary image sequence and the updated second summary image sequence as the summary image of the output summary image sequence.
10. The image processing device as defined in claim 1 , wherein the integration processing section performs the integration process on the first summary image sequence and the second summary image sequence by selecting an image that is included in at least one of the first summary image sequence and the second summary image sequence as a summary image of the output summary image sequence.
11. The image processing device as defined in claim 1 , wherein the integration processing section performs the integration process on the first image summarization process and the second image summarization process by causing the first image summarization section to perform the first image summarization process based on results of the second image summarization process.
12. The image processing device as defined in claim 11 , wherein the first image summarization section selects an image included in the second summary image sequence from the plurality of images as a reference image based on the integration process, selects a determination target image from the plurality of images, calculates a coverage ratio of the determination target image by the reference image as the similarity based on deformation information about the reference image and the determination target image, and performs the first image summarization process that determines whether or not the determination target image can be deleted based on the coverage ratio.
13. The image processing device as defined in claim 11 , wherein the integration processing section performs a second summary image sequence update process that reduces a number of images included in the second summary image sequence based on the first summary image sequence generated by the first image summarization section via the integration process.
14. The image processing device as defined in claim 13 , wherein the integration processing section performs the integration process on the first image summarization process and the second image summarization process by causing the first image summarization section to perform the first image summarization process based on results of the second image summarization process after the second summary image sequence update process.
15. The image processing device as defined in claim 11 , wherein the integration processing section acquires the first summary image sequence generated by the first image summarization section as the output summary image sequence via the integration process.
16. The image processing device as defined in claim 1 , wherein the first image summarization section detects a scene change based on the similarity between the plurality of images, and performs the first image summarization process.
17. The image processing device as defined in claim 1 , wherein:
the plurality of images are a capsule endoscopic image, and
the second image summarization section performs the recognition process on an in vivo attention area captured within the capsule endoscopic image as the target object.
18. The image processing device as defined in claim 17 , wherein the second image summarization section performs the recognition process on the in vivo attention area based on a special light image acquired by applying light within a specific wavelength band.
19. A computer-readable storage device having stored thereon, a program that is executable by a computer to cause the computer to function as:
a first image summarization section that performs a first image summarization process based on a similarity between a plurality of images to acquire a first summary image sequence;
a second image summarization section that performs a second image summarization process based on a target object/scene recognition process on each image among the plurality of images to acquire a second summary image sequence; and
an integration processing section that performs an integration process on the first summary image sequence and the second summary image sequence, or performs an integration process on the first image summarization process and the second image summarization process to acquire an output summary image sequence,
wherein the first image summarization section selects a reference image and a determination target image from the plurality of images, calculates a coverage ratio of the determination target image by the reference image as the similarity based on deformation information about the reference image and the determination target image, and performs the first image summarization process that determines whether or not the determination target image can be deleted based on the coverage ratio.
20. An image processing method comprising:
performing a first image summarization process based on a similarity between a plurality of images to acquire a first summary image sequence;
performing a second image summarization process based on a target object/scene recognition process on each image among the plurality of images to acquire a second summary image sequence; and
performing an integration process on the first summary image sequence and the second summary image sequence, or performing an integration process on the first image summarization process and the second image summarization process to acquire an output summary image sequence;
wherein the first image summarization process comprises selecting a reference image and a determination target image from the plurality of images, calculating a coverage ratio of the determination target image by the reference image as the similarity based on deformation information about the reference image and the determination target image, and performing the first image summarization process that determines whether or not the determination target image can be deleted based on the coverage ratio.