Efficiently storing low priority data in high priority storage devices
Efficiently storing low priority data in high priority storage devices is described. A system receives a request from an application to store a data chunk received from the application. The system determines a priority of the application. The system executes a write command to store the data chunk to a high priority storage device. If the application is low priority and the identifier of the received data chunk is a duplicate of any of the identifiers of the data chunks that were previously stored in the high priority storage device, the system stores, into the high priority storage device, the received data chunk as a low priority reference to a data chunk that was previously stored in the high priority storage device.
1. A system for efficiently storing low priority data in high priority storage devices, the system comprising:
a processor-based application, which when executed on a computer, will cause the processor to:
identify a priority level associated with an application in response to receiving a request from the application to store a data chunk received from the application, the data chunk received from the application being high priority data or low priority data based on the priority level associated with the application;
execute a write command to store the data chunk to a high priority storage device, the data chunk being high priority data or low priority data based on the priority level associated with the application;
determine whether an identifier of the received data chunk is a duplicate of any of a plurality of identifiers corresponding to a plurality of stored data chunks that were previously stored in the high priority storage device in response to executing the write command to store the data chunk to the high priority storage device; and
store, into the high priority storage device, the received data chunk as a low priority reference to a stored data chunk that was previously stored in the high priority storage device, in response to a determination that the application is low priority and a determination that the identifier of the received data chunk is the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
2. The system of claim 1 , wherein the processor-based application further causes the processor to store the received data chunk into a low priority storage device in response to a determination that the application is low priority and a determination that the identifier of the received data chunk is not the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
3. The system of claim 1 , wherein the processor-based application further causes the processor to store the received data chunk into the high priority storage device as a high priority reference to the stored data chunk, in response to a determination that the application is high priority and the determination that the identifier of the received data chunk is the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
4. The system of claim 1 , wherein the processor-based application further causes the processor to store the received data chunk into the high priority storage device in response to a determination that the application is high priority and a determination that the identifier of the received data chunk is not the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority.
5. The system of claim 3 , wherein the processor-based application further causes the processor to:
determine whether a high priority reference count that is associated with the high priority reference to the stored data chunk is reduced to zero;
store the stored data chunk into a low priority storage device in response to the determination that the high priority reference count is reduced to zero; and
delete the stored data chunk from the high priority storage device.
6. The system of claim 1 , wherein the identifier of the received data chunk comprises a hash value based on applying a hash function to the received data chunk.
7. The system of claim 2 , wherein the high priority storage device comprises a flash-based storage device and the low priority storage device comprises a rotating disk drive.
8. A computer-implemented method for efficiently storing low priority data in high priority storage devices, the method comprising:
identifying a priority level associated with an application in response to receiving a request from the application to store a data chunk received from the application, the data chunk received from the application being high priority data or low priority data based on the priority level associated with the application;
executing a write command to store the data chunk to a high priority storage device, the data chunk being high priority data or low priority data based on the priority level associated with the application;
determining whether an identifier of the received data chunk is a duplicate of any of a plurality of identifiers corresponding to a plurality of stored data chunks that were previously stored in the high priority storage device in response to executing the write command to store the data chunk to the high priority storage device; and
storing, into the high priority storage device, the received data chunk as a low priority reference to a stored data chunk that was previously stored in the high priority storage device, in response to a determination that the application is low priority and a determination that the identifier of the received data chunk is the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
9. The method of claim 8 , wherein the method further comprises storing the received data chunk into a low priority storage device in response to a determination that the application is low priority and a determination that the identifier of the received data chunk is not the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
10. The method of claim 8 , wherein the method further comprises storing the received data chunk into the high priority storage device as a high priority reference to the stored data chunk, in response to a determination that the application is high priority and the determination that the identifier of the received data chunk is the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
11. The method of claim 8 , wherein the method further comprises storing the received data chunk into the high priority storage device in response to a determination that the application is high priority and a determination that the identifier of the received data chunk is not the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority.
12. The method of claim 10 , wherein the method further comprises:
determining whether a high priority reference count that is associated with the high priority reference to the stored data chunk is reduced to zero;
storing the stored data chunk into a low priority storage device in response to the determination that the high priority reference count is reduced to zero; and
deleting the stored data chunk from the high priority storage device.
13. The method of claim 8 , wherein the identifier of the received data chunk comprises a hash value based on applying a hash function to the received data chunk.
14. The method of claim 9 , wherein the high priority storage device comprises a flash-based storage device and the low priority storage device comprises a rotating disk drive.
15. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
identify a priority level associated with an application in response to receiving a request from the application to store a data chunk received from the application, the data chunk received from the application being high priority data or low priority data based on the priority level associated with the application;
execute a write command to store the data chunk to a high priority storage device, the data chunk being high priority data or low priority data based on the priority level associated with the application;
determine whether an identifier of the received data chunk is a duplicate of any of a plurality of identifiers corresponding to a plurality of stored data chunks that were previously stored in the high priority storage device in response to executing the write command to store the data chunk to the high priority storage device; and
store into the high priority storage device, the received data chunk as a low priority reference to a stored data chunk that was previously stored in the high priority storage device, in response to a determination that the application is low priority and a determination that the identifier of the received data chunk is the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
16. The computer program product of claim 15 , wherein the program code includes further instructions to store the received data chunk into a low priority storage device in response to a determination that the application is low priority and a determination that the identifier of the received data chunk is not the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
17. The computer program product of claim 15 , wherein the program code includes further instructions to store the received data chunk into the high priority storage device as a high priority reference to the stored data chunk, in response to a determination that the application is high priority and the determination that the identifier of the received data chunk is the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority storage device.
18. The computer program product of claim 15 , wherein the program code includes further instructions to store the received data chunk into the high priority storage device in response to a determination that the application is high priority and a determination that the identifier of the received data chunk is not the duplicate of any of the plurality of identifiers corresponding to the plurality of stored data chunks that were previously stored in the high priority.
19. The computer program product of claim 17 , wherein the program code includes further instructions to:
determine whether a high priority reference count that is associated with the high priority reference to the stored data chunk is reduced to zero;
store the stored data chunk into a low priority storage device in response to the determination that the high priority reference count is reduced to zero; and
delete the stored data chunk from the high priority storage device.
20. The computer program product of claim 16 , wherein the identifier of the received data chunk comprises a hash value based on applying a hash function to the received data chunk, the high priority storage device comprises a flash-based storage device, and the low priority storage device comprises a rotating disk drive.