Techniques for customized topic determination for high-volume document collections
A topic mapping system generates customized mapping schemas for multiple topic sets. The topic mapping system generates document clusters that represent groups of digital documents. The topic mapping system also generates, for each topic set, a document-topic mapping data object (“DTM data object”) that describes a customized mapping schema of the document clusters to labels in the topic set. The topic mapping system identifies customized groups of documents for responding to multiple requests that have a particular keyword. For each request, the topic mapping system identifies a particular topic set and DTM data object associated with a computing system that provided the request. Based on the keyword, the topic mapping system identifies documents that are categorized according to the customized mapping schema in the DTM data object. The topic mapping system can provide customized groups of documents to respective computing systems that provided the multiple requests.
1 . A system for generating a customized data mapping for a digital document collection, the system comprising:
a processing device; and
a memory device in which instructions executable by the processing device are stored for causing the processing device to perform operations implementing:
a document cluster identification module configured for:
receiving text data extracted from multiple documents in a digital document collection; and
identifying a group of clusters that includes the multiple documents, in which each cluster is identified based on (i) a first set of text data extracted from a first document of the multiple documents and (ii) a second set of text data extracted from a second document of the multiple documents,
a cluster-topic mapping module configured for:
receiving multiple topic sets, each topic set of the multiple topic sets including a respective group of topic labels;
generating a first document-topic mapping data object (“DTM data object”) for a first topic set from the multiple topic sets, wherein the first DTM data object represents a first mapping schema that associates a particular cluster from the group of clusters with a first topic label from the first topic set; and
generating a second DTM data object for a second topic set from the multiple topic sets, wherein the second DTM data object represents a second mapping schema that associates the particular cluster from the group of clusters with a second topic label from the second topic set, and
a document mapping curation module configured for:
receiving a request from a requesting computing system, the request indicating a keyword that corresponds to the first topic set and the second topic set;
determining that the requesting computing system is associated with the first topic set;
responsive to determining that the requesting computing system is associated with the first topic set:
i) selecting the first DTM data object,
ii) identifying that the first mapping schema of the first DTM data object associates a group of documents with the keyword corresponding to the first topic set, and
iii) generating a response document set that includes the group of documents associated with the keyword via the first mapping schema of the first DTM data object and omits an additional group of documents associated with the keyword via the second mapping schema of the second DTM data object; and
providing, to the requesting computing system, the response document set including the group of documents and omitting the additional group of documents.
2 . The system of claim 1 , wherein the group of documents included in the response document set are included in the particular cluster from the group of clusters, and the additional group of documents omitted from the response document set are included in an additional cluster from the group of clusters,
wherein the particular cluster is associated with the keyword via the first mapping schema and the additional cluster is associated with the keyword via the second mapping schema.
3 . The system of claim 1 , the document mapping curation module further configured for:
receiving an additional request from an additional requesting computing system, the additional request indicating the keyword;
determining that the additional requesting computing system is associated with the second topic set;
responsive to determining that the additional requesting computing system is associated with the second topic set:
iv) selecting the second DTM data object, and
v) identifying that the second mapping schema of the second DTM data object associates the additional group of documents with the keyword corresponding to the second topic set; and
providing, to the additional requesting computing system, an additional response document set.
4 . The system of claim 3 , wherein the additional response document set omits the group of documents associated with the keyword via the first mapping schema of the first DTM data object and includes the additional group of documents associated with the keyword via the second mapping schema of the second DTM data object.
5 . The system of claim 1 , in which the instructions executable by the processing device are further for causing the processing device to perform additional operations implementing:
a content vectorization module configured for:
generating, for each particular document in the multiple documents, a respective vector data object describing respective text data extracted from the particular document,
wherein the first set of text data extracted from the first document is a first vector data object describing the text data of the first document.
6 . The system of claim 5 , the document cluster identification module further configured for:
calculating respective vector similarity values between (i) the first vector data object describing the text data of the first document and (ii) each respective vector data object describing the respective text data of the each particular document,
wherein identifying the group of clusters that includes the multiple documents is further based on the respective vector similarity values.
7 . The system of claim 1 , the cluster-topic mapping module further configured for:
calculating, for each cluster, a cluster similarity value between (i) a content term from the first set of text data and (ii) the first topic label from the first topic set,
wherein generating the first DTM data object is based on the cluster similarity value.
8 . A non-transitory computer-readable medium embodying program code for generating a customized data mapping for a digital document collection, the program code comprising instructions which, when executed by a processor, cause the processor to perform operations comprising:
identifying a group of clusters that includes multiple documents in a digital document collection, in which each cluster is identified based on (i) a first set of data extracted from a first document of the multiple documents and (ii) a second set of data extracted from a second document of the multiple documents;
receiving multiple topic sets, each topic set of the multiple topic sets including a respective group of topic labels;
generating, for each respective topic set of the multiple topic sets, a respective document-topic mapping data object (“DTM data object”), wherein each respective DTM data object represents a respective mapping schema that associates a particular cluster from the group of clusters with a particular topic label from the respective topic set;
receiving a request from a requesting computing system, the request indicating a keyword that corresponds to a first topic set and a second topic set;
responsive to determining that the requesting computing system is associated with the first topic set:
i) selecting a first DTM data object that is associated with the first topic set, wherein the first DTM data object represents a first mapping schema that associates a particular cluster from the group of clusters with a first topic label from the first topic set,
ii) identifying that the first mapping schema of the first DTM data object associates a group of documents with the keyword corresponding to the first topic set, and
iii) generating a response document set that includes the group of documents associated with the keyword via the first mapping schema of the first DTM data object and omits an additional group of documents associated with the keyword via a second mapping schema of represented by a second DTM data object, wherein the second mapping schema associates the particular cluster from the group of clusters with a second topic label from the second topic set; and
providing, to the requesting computing system, the response document set including the group of documents and omitting the additional group of documents.
9 . The non-transitory computer-readable medium of claim 8 , wherein the group of documents included in the response document set are included in the particular cluster from the group of clusters, and the additional group of documents omitted from the response document set are included in an additional cluster of the group of clusters,
wherein the particular cluster is associated with the keyword via the first mapping schema and the additional cluster is associated with the keyword via the second mapping schema.
10 . The non-transitory computer-readable medium of claim 8 , the instructions causing the processor to perform further operations comprising:
receiving an additional request from an additional requesting computing system, the additional request indicating the keyword;
determining that the additional requesting computing system is associated with the second topic set;
responsive to determining that the additional requesting computing system is associated with the second topic set:
iv) selecting the second DTM data object that is associated with the second topic set, and
v) identifying that the second mapping schema of the second DTM data object associates the additional group of documents with the keyword corresponding to the second topic set; and
providing, to the additional requesting computing system, an additional response document set, wherein the additional response document set omits the group of documents associated with the keyword via the first mapping schema of the first DTM data object and includes the additional group of documents associated with the keyword via the second mapping schema of the second DTM data object.
11 . The non-transitory computer-readable medium of claim 8 , the instructions causing the processor to perform further operations comprising:
generating, for each particular document in the multiple documents, a respective vector data object describing respective data extracted from the particular document,
wherein the first set of data extracted from the first document is a first vector data object describing the data of the first document.
12 . The non-transitory computer-readable medium of claim 11 , the instructions causing the processor to perform further operations comprising:
calculating respective vector similarity values between (i) the first vector data object describing the data of the first document and (ii) each respective vector data object describing the respective data extracted from the each particular document,
wherein identifying the group of clusters that includes the multiple documents is further based on the respective vector similarity values.
13 . The non-transitory computer-readable medium of claim 8 , the instructions causing the processor to perform further operations comprising:
calculating, for each cluster, a cluster similarity value between (i) a content term from the first set of data and (ii) the first topic label from the first topic set,
wherein generating the first DTM data object is based on the cluster similarity value.
14 . A method of generating a customized data mapping for a document digital collection, the method including instructions that, when implemented via one or more processing devices, perform operations comprising:
identifying a group of clusters that includes multiple documents in a digital document collection, in which each cluster is identified based on (i) a first set of data extracted from a first document of the multiple documents and (ii) a second set of data extracted from a second document of the multiple documents;
receiving multiple topic sets, each topic set of the multiple topic sets including a respective group of topic labels;
generating, for each respective topic set of the multiple topic sets, a respective document-topic mapping data object (“DTM data object”), wherein each respective DTM data object represents a respective mapping schema that associates a particular cluster from the group of clusters with a particular topic from the respective topic set;
receiving a request from a requesting computing system, the request indicating a keyword that corresponds to a first topic set and a second topic set;
responsive to determining that the requesting computing system is associated with the first topic set:
i) selecting a first DTM data object that is associated with the first topic set, wherein the first DTM data object represents a first mapping schema that associates a particular cluster from the group of clusters with a first topic label from the first topic set,
ii) identifying that the first mapping schema of the first DTM data object associates a group of documents with the keyword corresponding to the first topic set, and
iii) generating a response document set that includes the group of documents associated with the keyword via the first mapping schema of the first DTM data object and omits an additional group of documents associated with the keyword via a second mapping schema of represented by a second DTM data object, wherein the second mapping schema associates the particular cluster from the group of clusters with a second topic label from the second topic set; and
providing, to the requesting computing system, the response document set including the group of documents and omitting the additional group of documents.
15 . The method of claim 14 , wherein the group of documents included in the response document set are included in the particular cluster from the group of clusters, and the additional group of documents omitted from the response document set are included in an additional cluster of the group of clusters,
wherein the particular cluster is associated with the keyword via the first mapping schema and the additional cluster is associated with the keyword via the second mapping schema.
16 . The method of claim 14 , further comprising:
receiving an additional request from an additional requesting computing system, the additional request indicating the keyword;
determining that the additional requesting computing system is associated with the second topic set;
responsive to determining that the additional requesting computing system is associated with the second topic set;
iv) selecting the second DTM data object that is associated with the second topic set, and
v) identifying that the second mapping schema of the second DTM data object associates the additional group of documents with the keyword corresponding to the second topic set; and
providing, to the additional requesting computing system, an additional response document set.
17 . The method of claim 16 , wherein the response document set omits the additional group of documents associated with the keyword via the first mapping schema of the first DTM data object and includes the additional group of documents associated with the keyword via the second mapping schema of the second DTM data object.
18 . The method of claim 14 , further comprising:
generating, for each particular document in the multiple documents, a respective vector data object describing respective data extracted from the particular document,
wherein the first set of data extracted from the first document is a first vector data object describing the data of the first document.
19 . The method of claim 18 , further comprising:
calculating respective vector similarity values between (i) the first vector data object describing the data of the first document and (ii) each respective vector data object describing the respective data extracted from the each particular document,
wherein identifying the group of clusters that includes the multiple documents is further based on the respective vector similarity values.
20 . The method of claim 14 , further comprising:
calculating, for each cluster, a cluster similarity value between (i) a content term from the first set of data and (ii) the first topic label from the first topic set,
wherein generating the first DTM data object is based on the cluster similarity value.