System and method for data placement in multiple tier storage systems

1. A method for data placement in a storage system comprising a first tier and a second tier, the method comprising:

storing access statistics per object;

obtaining a request to perform a write operation;

calculating a recency factor related to timing and frequency of accessing an object to a first object based on the access statistics;

writing the first object to the first tier if the recency factor is above a threshold, and to the second tier otherwise; and

performing garbage collection process on the second tier by:

reading metadata of objects stored in the second tier;

for each of the objects stored in the second tier, determining whether the object stored in the second tier is valid or invalid based on the metadata;

discarding invalid objects; and

for each valid object:

calculating a recency factor related to timing and frequency of accessing an object for the valid object based on the access statistics of the valid object; and

moving the valid object to the first tier if the recency factor is above a threshold or leaving the valid object in the second tier otherwise,

wherein latency of the first tier is lower compared with latency of the second tier, and

wherein the recency factor is calculated by;

scanning entries in an access statistics list;

for each entry in the access statistics list

if a timestamp of the entry is older than a maximum recency value than discarding the entry;

else, calculating a recency element for the entry such that the recency element is higher as the access is more recent; and

summing the recency elements of the entries in the access statistics list.

2. The method of claim 1 , wherein the access statistics per object comprises timing of last N write and read operations to the object, wherein N is an integer larger than 1.

3. The method of claim 1 , comprising writing the first object to the first tier if the recency factor of the first object is above a recency threshold and to the second tier otherwise.

4. The method of claim 3 , comprising:

increasing the recency threshold if utilization of the first tier is above a first utilization threshold; and

decreasing the recency threshold if the utilization of the first tier is below a second utilization threshold.

5. The method of claim 1 , comprising performing evacuation process on the first tier, wherein performing the evacuation process comprises:

calculating a recency factor for a second object stored in the first tier based on the access statistics of the second object; and

leaving the second object in the first tier or moving the second object to the second tier, depending on the recency factor of the second object.

6. The method of claim 1 , wherein the recency factor depends on the timing and number of access operations to the first object.

7. The method of claim 1 , wherein the first tier comprises a storage device with access latencies of below 30 microseconds and the second tier comprises a NAND storage device.

8. The method of claim 1 , wherein endurance of the first tier is higher compared with endurance of the second tier.

9. A system for monitoring of movement of data in a computer network, the system comprising:

a first tier;

a second tier;

a memory; and

a processor configured to:

store access statistics per object;

obtain a request to perform a write operation:

calculate a recency factor related to timing and frequency of accessing an object to a first object based on the access statistics;

write the first object to the first tier if the recency factor is above a threshold, and the second tier; and

perform garbage collection process on the second tier by:

reading metadata of objects stored in the second tier;

for each of the objects stored in the second tier, determining whether the object stored in the second tier is valid or invalid based on the metadata;

discarding invalid objects; and

for each valid object:

calculating a recency factor related to timing and frequency of accessing an object for the valid object based on the access statistics of the valid object; and

moving the valid object to the first tier if the recency factor is above a threshold or leaving the valid object in the second tier otherwise,

wherein latency of the first tier is lower compared with latency of the second tier, and wherein the recency factor is calculated by;

scanning entries in an access statistics list;

for each entry in the access statistics list

if a timestamp of the entry is older than a maximum recency value than discarding the entry;

else, calculating a recency element for the entry such that the recency element is higher as the access is more recent; and

summing the recency elements of the entries in the access statistics list.

10. The system of claim 9 , wherein the access statistics per object comprises timing of last N write and read operations to the object, wherein N is an integer larger than 1.

11. The system of claim 9 , wherein the processor is configured to write the first object to the first tier if the recency factor of the first object is above a recency threshold and to the second tier otherwise.

12. The system of claim 11 , wherein the processor is configured to:

increase the recency threshold if utilization of the first tier is above a first utilization threshold; and

decrease the recency threshold if the utilization of the first tier is below a second utilization threshold.

13. The system of claim 9 , wherein the processor is configured to perform evacuation process on the first tier by:

calculating a recency factor for a second object stored in the first tier based on the access statistics of the second object; and

leaving the second object in the first tier or moving the second object to the second tier, depending on the recency factor of the second object.

14. The system of claim 9 , wherein the recency factor depends on the timing and number of access operations to the first object.

15. The system of claim 9 , wherein the first tier comprises a storage device with access latencies of below 30 microseconds and the second tier comprises a NAND storage device.

16. The system of claim 9 , wherein endurance of the first tier is higher compared with endurance of the second tier.