Distributed data processing

1. A computer-implemented method, comprising:

selecting a first number of candidate key-value pairs from input key-value pairs stored in a distributed file system, wherein selecting the first number of candidate key-value pairs comprises at least one of: random selection, or selection based on service types;

identifying an invocation frequency for each of the first number of candidate key-value pairs in a predetermined period of time, wherein the invocation frequency is a number of times the corresponding candidate key-value pair was called by a service system in the predetermined period of time;

selecting a second number of hot key-value pairs from the candidate key-value pairs, wherein the hot key-value pairs comprise at least one of: candidate key-value pairs that have invocation frequencies higher than a predetermined threshold, or candidate key-value pairs with key values within a key value interval;

mapping, as intermediate key-value pairs, the second number of hot key-value pairs; and

reducing, as resultant key-value pairs, the intermediate key-value pairs, wherein the resultant key-value pairs are to be called by the service system.

2. The computer-implemented method of claim 1 , wherein the first number of candidate key-value pairs is randomly selected from the input key-value pairs.

3. The computer-implemented method of claim 1 , wherein selecting a first number of candidate key-value pairs further comprises:

determining a service type of a service provided by the service system;

determining one or more key types based on the service type; and

selecting key-value pairs with the determined one or more key types as the first number of candidate key-value pairs.

4. The computer-implemented method of claim 1 , wherein selecting a second number of hot key-value pairs further comprises:

sorting the candidate key-value pairs based on their corresponding invocation frequencies; and

selecting the second number of hot key-value pairs based on the sorted candidate key-value pairs that have invocation frequencies higher than a predetermined threshold.

5. The computer-implemented method of claim 1 , wherein selecting a second number of hot key-value pairs further comprises:

determining a distribution of the invocation frequencies of the candidate key-value pairs;

determining one or more key value intervals as filtering conditions for selecting the hot key-value pairs based on a predetermined frequency threshold; and

selecting candidate key-value pairs with key values within the key value interval as hot key-value pairs.

6. The computer-implemented method of claim 5 , wherein the one or more key value intervals are related to key values of the same key type and the corresponding key-value pairs have a total invocation frequency greater than the predetermined frequency threshold.

7. The computer-implemented method of claim 5 , wherein the one or more key value intervals are related to key values of one or more key types and the corresponding key-value pairs have a total invocation frequency greater than the predetermined frequency threshold.

8. The computer-implemented method of claim 1 , further comprising:

mapping non-hot key-value pairs to intermediate key-value pairs when the non-hot key-value pairs are called by the service system; and

reducing the intermediate key-value pairs to resultant key-value pairs.

9. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

selecting a first number of candidate key-value pairs from input key-value pairs stored in a distributed file system, wherein selecting the first number of candidate key-value pairs comprises at least one of: random selection, or selection based on service types;

identifying an invocation frequency for each of the first number of candidate key-value pairs in a predetermined period of time, wherein the invocation frequency is a number of times the corresponding candidate key-value pair was called by a service system in the predetermined period of time;

selecting a second number of hot key-value pairs from the candidate key-value pairs, wherein the hot key-value pairs comprise at least one of: candidate key-value pairs that have invocation frequencies higher than a predetermined threshold, or candidate key-value pairs with key values within a key value interval;

mapping, as intermediate key-value pairs, the second number of hot key-value pairs; and

reducing, as resultant key-value pairs, the intermediate key-value pairs, wherein the resultant key-value pairs are to be called by the service system.

10. The non-transitory, computer-readable medium of claim 9 , wherein the first number of candidate key-value pairs is randomly selected from the input key-value pairs.

11. The non-transitory, computer-readable medium of claim 9 , wherein selecting a first number of candidate key-value pairs further comprises:

determining a service type of a service provided by the service system;

determining one or more key types based on the service type; and

selecting key-value pairs with the determined one or more key types as the first number of candidate key-value pairs.

12. The non-transitory, computer-readable medium of claim 9 , wherein selecting a second number of hot key-value pairs further comprises:

sorting the candidate key-value pairs based on their corresponding invocation frequencies; and

selecting the second number of hot key-value pairs based on the sorted candidate key-value pairs that have invocation frequencies higher than a predetermined threshold.

13. The non-transitory, computer-readable medium of claim 9 , wherein selecting a second number of hot key-value pairs further comprises:

determining a distribution of the invocation frequencies of the candidate key-value pairs;

determining one or more key value intervals as filtering conditions for selecting the hot key-value pairs based on a predetermined frequency threshold; and

selecting candidate key-value pairs with key values within the key value interval as hot key-value pairs.

14. The non-transitory, computer-readable medium of claim 13 , wherein the one or more key value intervals are related to key values of the same key type and the corresponding key-value pairs have a total invocation frequency greater than the predetermined frequency threshold.

15. The non-transitory, computer-readable medium of claim 13 , wherein the one or more key value intervals are related to key values of one or more key types and the corresponding key-value pairs have a total invocation frequency greater than the predetermined frequency threshold.

16. The non-transitory, computer-readable medium of claim 9 , further comprising:

mapping non-hot key-value pairs to intermediate key-value pairs when the non-hot key-value pairs are called by the service system; and

reducing the intermediate key-value pairs to resultant key-value pairs.

17. A computer-implemented system, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing instructions, that when executed by the one or more computers, perform operations comprising:

selecting a first number of candidate key-value pairs from input key-value pairs stored in a distributed file system, wherein selecting the first number of candidate key-value pairs comprises at least one of: random selection, or selection based on service types;

identifying an invocation frequency for each of the first number of candidate key-value pairs in a predetermined period of time, wherein the invocation frequency is a number of times the corresponding candidate key-value pair was called by a service system in the predetermined period of time;

selecting a second number of hot key-value pairs from the candidate key-value pairs, wherein the hot key-value pairs comprise at least one of: candidate key-value pairs that have invocation frequencies higher than a predetermined threshold, or candidate key-value pairs with key values within a key value interval;

mapping, as intermediate key-value pairs, the second number of hot key-value pairs; and

reducing, as resultant key-value pairs, the intermediate key-value pairs, wherein the resultant key-value pairs are to be called by the service system.

18. The computer-implemented system of claim 17 , wherein the first number of candidate key-value pairs is randomly selected from the input key-value pairs.

19. The computer-implemented system of claim 17 , wherein selecting a first number of candidate key-value pairs further comprises:

determining a service type of a service provided by the service system;

determining one or more key types based on the service type; and

selecting key-value pairs with the determined one or more key types as the first number of candidate key-value pairs.

20. The computer-implemented system of claim 17 , wherein selecting a second number of hot key-value pairs further comprises:

sorting the candidate key-value pairs based on their corresponding invocation frequencies; and

selecting the second number of hot key-value pairs based on the sorted candidate key-value pairs that have invocation frequencies higher than a predetermined threshold.