Character recognition device, image display device, image retrieval device, character recognition method, and computer program product

1. A character recognition device comprising:

processing circuitry configured to function as:

a detector configured to detect a visible text area that represents an area in which a character is visible, from an input image;

a first recognizer configured to perform character pattern recognition with respect to the visible text area, and calculate a character pattern recognition cost according to a likelihood of a character pattern;

a first estimator configured to estimate a partially-hidden text area into which a hidden text area and one or more visible text areas are integrated, the hidden text area representing an area estimated to have a hidden character;

a second recognizer configured to calculate an integrated cost into which the character pattern recognition cost calculated with respect to the visible text area included in the partially-hidden text area and a linguistic cost corresponding to a linguistic likelihood of a text that fits in the entire of the partially-hidden text area are integrated; and

an output unit configured to output a text that is selected or ranked based on the integrated cost, as a result of character recognition performed with respect to the partially-hidden text area; wherein the processing circuitry further functions as:

a second estimator configured to estimate a multiline text area into which a hidden text line area and the one or more visible text areas or the partially-hidden text area are integrated, the hidden text line area having a text direction consistent with a text direction of the one or more visible text areas or the partially-hidden text area and being in a direction perpendicular to the text direction, wherein

when the multiline text area is estimated, the second recognizer calculates, as the integrated cost with respect to the multiline text area, an integrated cost into which either the character pattern recognition cost calculated with respect to the visible text area included in the multiline text area or the integrated cost calculated with respect to the partially-hidden text area included in the multiline text area and the linguistic cost corresponding to a linguistic likelihood of text fitting in entire of the multiline text area are integrated.

2. The device according to claim 1 , wherein

the first estimator

detects a pair of visible text areas linearly arranged in line and having a text direction consistent with a direction of arrangement of the visible text areas,

estimates, as the hidden text area, an area sandwiched between two visible text areas included in the pair, and

estimates, as the partially-hidden text area, an area into which the hidden text area and the pair are integrated.

3. The device according to claim 1 , wherein

the first estimator

estimates, as the hidden text area, an additional area by which the visible text area adjacent to a rim of the input image is extended toward outside of the input image from the rim in a direction consistent with a text direction of the visible text area, and

estimates, as the partially-hidden text area, an area into which the hidden text area and the visible text area are integrated.

4. The device according to claim 1 , wherein

the first estimator

estimates, as the hidden text area, an additional area by which the visible text area is extended in a direction consistent with a text direction of the visible text area and without causing overlapping with another visible text area, and

estimates, as the partially-hidden text area, an area into which the hidden text area and the visible text area are integrated.

5. The device according to claim 1 , wherein the second recognizer determines a text candidate that fits in the hidden text area based on a size of the hidden text area.

6. The device according to claim 1 , wherein the second recognizer calculates the integrated cost into which a space cost corresponding to a size of the hidden text area is further integrated.

7. The device according to claim 1 , wherein the second recognizer calculates the integrated cost into which a space cost corresponding to a reason behind an occurrence of the hidden text area is further integrated.

8. The device according to claim 1 , wherein the output unit outputs a result of character recognition with respect to the partially-hidden text area selected using the integrated cost when there is a plurality of estimated candidates for the partially-hidden text area.

9. The device according to claim 1 , wherein

the second estimator

detects a pair of visible text areas distantly arranged in the direction perpendicular to the text direction, or a pair of partially-hidden text areas distantly arranged in the direction perpendicular to the text direction, or a pair of the visible text area and the partially-hidden text area distantly arranged in the direction perpendicular to the text direction,

estimates, as the hidden text line area, an area sandwiched between two visible text areas in the pair, or between two partially-hidden text areas in the pair, or between the visible text area and the partially-hidden text area in the pair, and

estimates, as the multiline text area, an area into which the hidden text line area and the pair are integrated.

10. The device according to claim 1 , wherein

the second estimator

estimates, as the hidden text line area, an additional area by which the visible text area or the partially-hidden text area adjacent to a rim of the input image is extended toward outside of the input image from the rim in the direction perpendicular to the text direction of the visible text area or the partially-hidden text area, and

estimates, as the multiline text area, an area into which the hidden text area and the visible text area or the partially-hidden text area are integrated.

11. The device according to claim 1 , wherein

the second estimator

estimates, as the hidden text line area, an additional area by which the visible text area or the partially-hidden text area is extended in the direction perpendicular to the text direction of the visible text area or the partially-hidden text area without causing overlapping with another visible text area, and

estimates, as the multiline text area, an area into which the hidden text area and the visible text area or the partially-hidden text area are integrated.

12. An image display device comprising:

the character recognition device according to claim 1 ; and

a display controller comprising processing circuitry configured to:

generate a display screen in which the result of character recognition that is output from the character recognition device is superimposed on the input image so that text corresponding to the visible text area and text corresponding to the hidden text area are distinguishable from each other, and

display the display screen on a display.

13. The device according to claim 12 , wherein

the display controller generates the display screen in which text corresponding to a candidate of the hidden text area that is not incorporated in the partially-hidden text area is further superimposed on the input image so that the text corresponding to the candidate of the hidden text area which is not incorporated in the partially-hidden text area and text corresponding to a candidate of the hidden text area which is incorporated in the partially-hidden text area are distinguishable from each other, and displays the display screen on the display, and

when an operation is performed on the display screen to specify text corresponding to a candidate of the hidden text area which is not incorporated in the partially-hidden text area, the display controller updates the display screen according to the operation.

14. An image retrieval device comprising:

the character recognition device according to claim 1 ; and

processing circuitry configured to function as:

an index generator configured to input an image stored in an image storage to the character recognition device, and generate an index for the input image based on the result of character recognition that is output from the character recognition device; and

a searcher configured to retrieve an image matching a search query using the index from the image storage, and output the retrieved image.

15. The device according to claim 14 , wherein

the index generator generates the index having a different weight assigned thereto depending on the result of character recognition that is output from the character recognition device, and

the searcher ranks the image matching with the search query according to the weight assigned to the index of the image, and outputs the ranked image.

16. A character recognition method implemented in a character recognition device, the method comprising:

detecting a visible text area that represents an area in which a character is visible, from an input image;

performing character pattern recognition with respect to the visible text area;

calculating a character pattern recognition cost according to a likelihood of a character pattern;

estimating a partially-hidden text area into which a hidden text area and one or more visible text areas are integrated, the hidden text area representing an area estimated to have a hidden character;

calculating an integrated cost into which the character pattern recognition cost calculated with respect to the visible text area included in the partially-hidden text area and a linguistic cost corresponding to a linguistic likelihood of a text that fits in the entire of the partially-hidden text area are integrated; and

outputting a text that is selected or ranked based on the integrated cost, as a result of character recognition performed with respect to the partially-hidden text area; wherein the method further comprises:

estimating a multiline text area into which a hidden text line area and the one or more visible text areas or the partially-hidden text area are integrated, the hidden text line area having a text direction consistent with a text direction of the one or more visible text areas or the partially-hidden text area and being in a direction perpendicular to the text direction, wherein

when the multiline text area is estimated, as the integrated cost with respect to the multiline text area, an integrated cost is calculated into which either the character pattern recognition cost calculated with respect to the visible text area included in the multiline text area or the integrated cost calculated with respect to the partially-hidden text area included in the multiline text area, and the linguistic cost corresponding to a linguistic likelihood of text fitting in entire of the multiline text area are integrated.

17. A computer program product comprising a non-transitory computer-readable medium containing a program for recognizing a character, the program causing a computer to execute:

detecting a visible text area that represents an area in which a character is visible, from an input image;

performing character pattern recognition with respect to the visible text area;

calculating a character pattern recognition cost according to a likelihood of a character pattern;

estimating a partially-hidden text area into which a hidden text area and one or more visible text areas are integrated, the hidden text area representing an area estimated to have a hidden character;

calculating an integrated cost into which the character pattern recognition cost calculated with respect to the visible text area included in the partially-hidden text area and a linguistic cost corresponding to a linguistic likelihood of a text that fits in the entire of the partially-hidden text area are integrated; and

outputting a text that is selected or ranked based on the integrated cost, as a result of character recognition performed with respect to the partially-hidden text area; wherein the program further causes the computer to execute:

estimating a multiline text area into which a hidden text line area and the one or more visible text areas or the partially-hidden text area are integrated, the hidden text line area having a text direction consistent with a text direction of the one or more visible text areas or the partially-hidden text area and being in a direction perpendicular to the text direction, wherein

when the multiline text area is estimated, as the integrated cost with respect to the multiline text area, an integrated cost is calculated into which either the character pattern recognition cost calculated with respect to the visible text area included in the multiline text area or the integrated cost calculated with respect to the partially-hidden text area included in the multiline text area, and the linguistic cost corresponding to a linguistic likelihood of text fitting in entire of the multiline text area are integrated.