Analysis system, analysis method, and recording medium
An analysis system is configured to include an extraction unit, an output unit, an acquisition unit, and a dictionary generation unit. The extraction unit extracts, as a token pair, two tokens that consecutively occur in a sentence separated using the basic dictionary. Regarding a token in which two tokens included in a token pair are connected, the output unit displays a selection screen for selecting any of registering with a dictionary, registering with an on-hold list for holding registration with the dictionary, and not registering with the dictionary. The acquisition unit acquires a selection result for the selection screen. The dictionary generation unit generates an on-hold list with which the token pair whose selection result indicates on-hold is registered as a token, and a dictionary with which the token pair whose selection result indicates registering is registered as a connected token.
1. An analysis system comprising:
at least one memory storing instructions; and
at least one processor configured to access the at least one memory and execute the instructions to:
acquire, from a terminal device operated by an operator, text data for generating a dictionary;
extract, as a token pair, two tokens that consecutively occur in a sentence by using natural language processing based on a simple rule and a basic dictionary;
output, to the terminal device, a selection screen for selecting, regarding a token to which the two tokens included in the token pair are connected, any of:
registering with the dictionary,
registering with an on-hold list for holding registration with the dictionary, and
not registering with the dictionary,
wherein the selection screen includes a button for registering with the dictionary, a button for registering with the on-hold list for holding the registration with the dictionary, and a button for not registering with the dictionary;
acquire, from the terminal device, a selection result selected by the operator on the selection screen that is displayed on the terminal device;
when the button for registering with the dictionary is selected by the operator on the selection screen, register the token in the dictionary with the two tokens in the token pair;
when the button for registering with the on-hold list is selected by the operator on the selection screen, generate the on-hold list and register the token in the on-hold list with the two tokens in the token pair; and
when the button for not registering with the dictionary is selected by the operator on the selection screen, do not register the token in the dictionary with the two tokens in the token pair.
2. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
extract, as the token pair, two tokens that consecutively occur in the sentence based on the tokens registered with the basic dictionary, the dictionary, and the on-hold list;
acquire the selection result for the token pair extracted using the basic dictionary, the dictionary, and the on-hold list; and
update the dictionary and the on-hold list based on the selection result.
3. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
narrow down the token pair that is a target of processing based on a predetermined criterion.
4. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
display the selection screen that displays token pairs in a predetermined order; and
display the selection screen that displays, in the predetermined order, the token pairs further extracted from the sentence based on the token registered with the basic dictionary, the dictionary, and the on-hold list after the selection screen for all the token pairs extracted from the sentence is output.
5. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
display the selection screen that displays information on a number of occurrences in the sentence for each token included in the token pair.
6. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
display the selection screen that displays, as an example, a part including the token pair in the sentence.
7. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
extract a token from a sentence using the dictionary and analyze the sentence; and
output an image that displays at least one of a size and a position of a character of the token extracted from the sentence according to a number of occurrences of the token in the sentence based on a result of the analysis.
8. The analysis system according to claim 7 , wherein
the at least one processor is further configured to execute the instructions to:
extract a token from the sentence by using a token having a larger number of connected tokens than other tokens in a case where, regarding a part including a same place of the sentence, there are a plurality of ways of separation using different tokens included with the dictionary.
9. The analysis system according to claim 7 , wherein
the at least one processor is further configured to execute the instructions to:
remove a token included in a token list from tokens extracted from the sentence, wherein the token list includes an unnecessary token as an analysis target; and
generate a result of the analysis.
10. The analysis system according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to:
extract the token pair based on the token registered with the basic dictionary, the dictionary, and the on-hold list by excluding a token pair for which a selection result indicates that registration is unnecessary.
11. An analysis method performed by a computer and comprising:
acquiring, from a terminal device operated by an operator, text data for generating a dictionary;
extracting, as a token pair, two tokens that consecutively occur in a sentence by using natural language processing based on a simple rule and a basic dictionary;
outputting, to the terminal device, a selection screen for selecting, regarding a token to which the two tokens included in the token pair are connected, any of:
registering with the dictionary,
registering with an on-hold list for holding registration with the dictionary, and
not registering with the dictionary,
wherein the selection screen includes a button for registering with the dictionary, a button for registering with the on-hold list for holding the registration with the dictionary, and a button for not registering with the dictionary;
acquiring, from the terminal device, a selection result selected by the operator on the selection screen that is displayed on the terminal device;
when the button for registering with the dictionary is selected by the operator on the selection screen, registering the token in the dictionary with the two tokens in the token pair;
when the button for registering with the on-hold list is selected by the operator on the selection screen, generating the on-hold list and register the token in the on-hold list with the two tokens in the token pair; and
when the button for not registering with the dictionary is selected by the operator on the selection screen, not registering the token in the dictionary with the two tokens in the token pair.
12. A non-transitory recording medium storing an analysis program executable a computer to perform processing comprising:
acquiring, from a terminal device operated by an operator, text data for generating a dictionary;
extracting, as a token pair, two tokens that consecutively occur in a sentence by using natural language processing based on a simple rule and a basic dictionary;
outputting, to the terminal device, a selection screen for selecting, regarding a token to which the two tokens included in the token pair are connected, any of:
registering with the dictionary,
registering with an on-hold list for holding registration with the dictionary, and
not registering with the dictionary,
wherein the selection screen includes a button for registering with the dictionary, a button for registering with the on-hold list for holding the registration with the dictionary, and a button for not registering with the dictionary;
acquiring, from the terminal device, a selection result selected by the operator on the selection screen that is displayed on the terminal device;
when the button for registering with the dictionary is selected by the operator on the selection screen, registering the token in the dictionary with the two tokens in the token pair;
when the button for registering with the on-hold list is selected by the operator on the selection screen, generating the on-hold list and register the token in the on-hold list with the two tokens in the token pair; and
when the button for not registering with the dictionary is selected by the operator on the selection screen, not registering the token in the dictionary with the two tokens in the token pair.