Unit group generation and relationship establishment
Described are systems and methods for establishing a unit group dictionary based on user provided annotations. The unit group dictionary may be used to identify relationships between multiple items in a corpus. Those relationships may facilitate the display of object identifiers and/or other aspects used and/or provided by the object management service.
1. A computing system, comprising:
one or more processors; and
a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least:
identify a corpus including a plurality of items, wherein each of the plurality of items include associated annotations;
process the associated annotations to determine a plurality of unit groups, wherein each unit group includes at least two units included in the associated annotations;
determine a non-random unit group from the plurality of unit groups at least in part by determining that a probability that a plurality of units of the plurality of unit groups are more likely to appear together in the corpus of items is greater than a probability of the plurality of units appearing separately in the corpus of items; and
update a dictionary to include the non-random unit group.
2. The computing system of claim 1 , wherein the probability is determined based at least in part on a number of times at least two units of the plurality of unit groups appear together in the corpus.
3. The computing system of claim 1 , wherein the probability is determined based at least in part on a number of times at least two units of the plurality of unit groups appear independently in the corpus.
4. The computing system of claim 1 , the program instructions that when executed by the one or more processors further cause the one or more processors to at least:
for the non-random unit group, generate a discount factor, wherein the discount factor is based at least in part on a frequency of the non-random unit group in the corpus.
5. The computing system of claim 4 , the program instructions that when executed by the one or more processors further cause the one or more processors to at least:
include the non-random unit group and the discount factor in the dictionary.
6. The computing system of claim 4 , the program instructions that when executed by the one or more processors to generate a discount factor, further cause the one or more processors to at least:
determine a quotient of a number of times the non-random unit group appears in the corpus of items and a number of items in the corpus of items.
7. The computing system of claim 1 , wherein the non-random unit group includes at least two units.
8. A computer-implemented method, comprising:
identifying a corpus including a plurality of items, wherein each of the plurality of items include associated annotations;
processing the associated annotations to determine a plurality of unit groups, wherein each unit group includes at least two units included in the associated annotations;
determining a non-random unit group from the plurality of unit groups at least in part by determining that a probability that a plurality of units of the plurality of unit groups are more likely to appear together in the corpus of items is greater than a probability of the plurality of units appearing separately in the corpus of items; and
generating a dictionary that includes the non-random unit group.
9. The computer-implemented method of claim 8 , wherein the probability is determined based at least in part on a number of times at least two units of the plurality of unit groups appear together in the corpus.
10. The computer-implemented method of claim 8 , wherein the probability is determined based at least in part on a number of times at least two units of the plurality of unit groups appear independently in the corpus.
11. The computer-implemented method of claim 8 , further comprising:
for the non-random unit group, generating a discount factor, wherein the discount factor is based at least in part on a frequency of the non-random unit group in the corpus.
12. The computer-implemented method of claim 11 , further comprising:
including the non-random unit group and the discount factor in the dictionary.
13. The computer-implemented method of claim 11 , wherein generating the discount factor includes:
determining a quotient of a number of times the non-random unit group appears in the corpus of items and a number of items in the corpus of items.
14. The computer-implemented method of claim 8 , wherein the non-random unit group includes at least two units.
15. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to at least:
identify a corpus including a plurality of items, wherein each of the plurality of items include associated annotations;
process the associated annotations to determine a plurality of unit groups, wherein each unit group includes at least two units included in the associated annotations;
determine a plurality of non-random unit groups from the plurality of unit groups at least in part by determining that a probability that a plurality of units of the plurality of unit groups are more likely to appear together in the corpus of items is greater than a probability of the plurality of units appearing separately in the corpus of items; and
update a dictionary to include each of the plurality of non-random unit groups.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the probability is determined based at least in part on a number of times at least two units of the plurality of unit groups appear together in the corpus.
17. The non-transitory computer-readable storage medium of claim 15 , wherein the probability is determined based at least in part on a number of times at least two units of the plurality of unit groups appear independently in the corpus.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, that when executed by the one or more processors further cause the one or more processors to at least:
for each non-random unit group of the plurality of non-random unit groups, generate a discount factor, wherein the discount factor is based at least in part on a frequency of the non-random unit group in the corpus.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, that when executed by the one or more processors further cause the one or more processors to at least:
include a non-random unit group of the plurality of non-random unit groups and a corresponding discount factor in the dictionary.
20. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, that when executed by the one or more processors further cause the one or more processors to at least:
for each non-random unit group of the plurality of non-random unit groups, determine a quotient of a number of times the non-random unit group appears in the corpus of items and a number of items in the corpus of items.