Systems and computer implemented methods for semantic data compression

1. A computer-implemented method for semantic compression and transmission of data, comprising:

receiving, at a first computer, a query from a second computer to transmit a plurality of artifacts to the second computer over a network, wherein the plurality of artifacts are from a corpus of documents;

queuing, at the first computer, the plurality of artifacts in response to the received query;

performing semantic compression, at the first computer, on the plurality of queued artifacts, wherein the semantic compression comprises:

determining, at the first computer, for each of the plurality of queued artifacts, to send one of: an artifact, and only a hyperlink to the artifact based on semantic relatedness of queued artifacts to other queued artifacts, the determining resulting in a first set of artifacts to send, and a second set of artifacts to only send links, wherein a similarity score of queued artifacts to other queued artifacts is calculated, wherein queued artifacts having a similarity score greater than a predetermined value are assigned to the first set of artifacts, and wherein queued artifacts having a similarity score less than the predetermined value are assigned to the second set of artifacts, and

calculating, at the first computer, an optimum batch size of the semantically compressed queued artifacts;

batching, at the first computer, the compressed queued artifacts into one or more batches based on the calculating; and

transmitting, by the first computer, the one or more batches over the network to the second computer.

2. The computer-implemented method of claim 1 , further comprising:

ranking the queued artifacts by a semantic similarity score, wherein the semantic similarity score determines a strength of a relationship between artifacts; and

prioritizing transmission based on the semantic similarity score.

3. The computer-implemented method of claim 1 , further comprising:

performing network analytics on the plurality of queued artifacts; and

prioritizing the queued artifacts based on the network analytics.

4. The computer-implemented method of claim 3 , wherein network analytics comprises one or more of relationship driven optimization, GIS coordinate based tiling, GIS place name index, and GIS shape file optimization.

5. The computer-implemented method of claim 1 , wherein determining whether to send the queued artifact or only a link to the queued artifact further comprises:

clustering the queued artifacts based on semantic similarity;

determining for each cluster, using natural language processing, artifacts that contain unique information and artifacts that contain cumulative information; and

sending hyperlinks to the queued artifacts that contain cumulative information.

6. The computer-implemented method of claim 5 , wherein sending further comprises:

identifying an artifact of the plurality of artifacts that comprises an image; and

sending a segment of the image determined to be of relevance to the query.

7. The computer-implemented method of claim 5 , wherein sending further comprises:

identifying an artifact that comprises a video image; and

sending one or more frames of the video image determined to be of relevance to the query.

8. The computer-implemented method of claim 5 , wherein sending further comprises:

monitoring the hyperlink by a machine learning system to determine whether users open the hyperlink to the artifact; and

when the hyperlink is opened by a majority of the users to access the artifact, replacing the hyperlink with the artifact in a subsequent transmission.

9. The computer-implemented method of claim 8 , wherein a user requests transmission of the corresponding artifact for the hyperlink in a subsequent transmission.

10. A cloud transfer service system for semantic compression and transmission of data, the system comprising:

one or more processors;

a network interface coupled to the one or more processors, wherein the network interface is communicatively coupled to a network;

one or more computer readable storage media; and

computer readable program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the computer readable program instructions comprising instructions to:

receive, at a first computer, a query from a second computer to transmit a plurality of artifacts to the second computer over a network, wherein the plurality of artifacts are from a corpus of documents;

queue, at the first computer, the plurality of artifacts in response to the received query;

perform semantic compression, at the first computer, on the plurality of queued artifacts, wherein the computer readable program instructions further comprises instructions to:

determine, at the first computer, for each of the plurality of queued artifacts, to send one of: an artifact, and only a hyperlink to the artifact based on semantic relatedness of queued artifacts to other queued artifacts, the determining resulting in a first set of artifacts to send, and a second set of artifacts to only send links, wherein a similarity score of queued artifacts to other queued artifacts is calculated, wherein queued artifacts having a similarity score greater than a predetermined value are assigned to the first set of artifacts, and wherein queued artifacts having a similarity score less than the predetermined value are assigned to the second set of artifacts, and

calculate, at the first computer, an optimum batch size of the semantically compressed queued artifacts;

batch, at the first computer, the compressed queued artifacts into one or more batches based on the calculating; and

transmit, by the first computer, the one or more batches over the network to the second computer.

11. The cloud transfer service system of claim 10 , wherein the computer readable program instructions further comprise instructions to:

rank the queued artifacts by a semantic similarity score, wherein the semantic similarity score determines a strength of a relationship between artifacts; and

prioritize transmission based on the semantic similarity score.

12. The cloud transfer service system of claim 10 , wherein the computer readable program instructions further comprise instructions to:

perform network analytics on the plurality of queued artifacts; and

prioritize the queued artifacts based on the network analytics.

13. The cloud transfer service system of claim 12 , wherein network analytics comprises one or more of relationship driven optimization, GIS coordinate based tiling, GIS place name index, and GIS shape file optimization.

14. The cloud transfer service system of claim 10 , wherein the computer readable program instructions, configured to determine whether to send the queued artifact or only a link to the queued artifact, further comprise instructions to:

cluster the queued artifacts based on semantic similarity;

determine for each cluster, using natural language processing, artifacts that contain unique information and artifacts that contain cumulative information; and

send hyperlinks to the queued artifacts that contain cumulative information.

15. The cloud transfer service system of claim 10 , wherein the computer readable program instructions further comprise instructions to:

identify an artifact of the plurality of artifacts that comprises an image; and

send a segment of the image determined to be of relevance to the query.

16. The cloud transfer service system of claim 10 , wherein the computer readable program instructions further comprise instructions to:

identify an artifact that comprises a video image; and

send one or more frames of the video image determined to be of relevance to the query.

17. The cloud transfer service system of claim 10 , wherein the computer readable program instructions further comprise instructions to:

monitor the hyperlink by a machine learning system to determine whether users open the hyperlink to the artifact; and

when the hyperlink is opened by a majority of the users to access the artifact, replace the hyperlink with the artifact in a subsequent transmission.

18. The cloud transfer service system of claim 17 , wherein a user requests transmission of the corresponding artifact for the hyperlink in a subsequent transmission.

19. A computer-implemented method for semantic compression and transmission of data, comprising:

receiving, at a first computer, a query from a second computer to transmit a plurality of artifacts to the second computer over a network, wherein the plurality of artifacts are from a corpus of documents;

queuing, at the first computer, the plurality of artifacts in response to the received query;

performing semantic compression, at the first computer, on the plurality of queued artifacts, wherein the semantic compression comprises:

determining, at the first computer, for each of the plurality of queued artifacts, to send one of: an artifact, and only a hyperlink to the artifact based on semantic relatedness of queued artifacts to other queued artifacts, the determining resulting in a first set of artifacts to send, and a second set of artifacts to only send links, and

calculating, at the first computer, an optimum batch size of the semantically compressed queued artifacts, wherein calculating the optimal batch size further comprises:

obtaining a sorted list of the queued artifacts and corresponding scores;

processing the scores into a set of <index, value> pairs to generate a matrix of results;

determining, from the matrix of results, cutoff points; and

assigning the queued artifacts into batches based on the cutoff points,

batching, at the first computer, the compressed queued artifacts into one or more batches based on the calculating; and

transmitting, by the first computer, the one or more batches over the network to the second computer.

20. A cloud transfer service system for semantic compression and transmission of data, the system comprising:

one or more processors;

a network interface coupled to the one or more processors, wherein the network interface is communicatively coupled to a network;

one or more computer readable storage media; and

computer readable program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the computer readable program instructions comprising instructions to:

receive, at a first computer, a query from a second computer to transmit a plurality of artifacts to the second computer over a network, wherein the plurality of artifacts are from a corpus of documents;

queue, at the first computer, the plurality of artifacts in response to the received query;

perform semantic compression, at the first computer, on the plurality of queued artifacts, wherein the computer readable program instructions further comprises instructions to:

determine, at the first computer, for each of the plurality of queued artifacts, to send one of: an artifact, and only a hyperlink to the artifact based on semantic relatedness of queued artifacts to other queued artifacts, the determining resulting in a first set of artifacts to send, and a second set of artifacts to only send links, and

calculate, at the first computer, an optimum batch size of the semantically compressed queued artifacts, wherein the instructions to calculate the optimal batch size further comprise instructions to:

obtain a sorted list of the queued artifacts and corresponding scores;

process the scores into a set of <index, value> pairs to generate a matrix of results;

determine, from the matrix of results, cutoff points; and

assign the queued artifacts into batches based on the cutoff point;

batch, at the first computer, the compressed queued artifacts into one or more batches based on the calculating; and

transmit, by the first computer, the one or more batches over the network to the second computer.