Data storage methods and apparatuses for reducing the number of writes to flash-based storage
Methods and apparatuses are provided for reducing the number of write operations to a flash-based storage system that stores and replaces data. The storage system includes a first storage implemented using non-flash storage and a second storage implemented using flash memory. Missed data is first stored in the first storage, which can be less sensitive than flash to write operations. The missed data is stored in the flash-based second storage only after the missed data satisfies a storage management algorithm.
1. A computer-implemented method of reducing a number of write operations to storage, the method comprising:
receiving a request for data;
determining whether the requested data is stored in either a primary storage or a secondary storage, wherein the primary storage comprises flash-based storage, and wherein the secondary storage is non-flash storage;
retrieving the requested data from an external storage and storing the retrieved data in the secondary storage if the requested data is not stored in either the primary storage or the secondary storage, wherein the retrieved data from the external storage is not simultaneously stored to both the primary storage and secondary storage; and
retrieving the requested data from the secondary storage and storing the requested data in the primary storage based on a storage management algorithm if the requested data is not stored in the primary storage but stored in the secondary storage, wherein, according to the storage management algorithm, the requested data is stored in the primary storage after the data has been requested a predetermined number of times.
2. The computer-implemented method of claim 1 wherein the primary storage includes flash and a second storage media.
3. The computer-implemented method of claim 1 further comprising replacing stored data in the secondary storage with the retrieved data based on a first replacement algorithm.
4. The computer-implemented method of claim 1 further comprising removing from the secondary storage the retrieved data stored in the secondary storage after it is stored in the primary storage.
5. The computer-implemented method of claim 1 wherein retrieving the requested data from the secondary storage and storing the requested data in the primary storage based on a storage management algorithm if the requested data is not stored in the primary storage but stored in the secondary storage comprises storing the retrieved data from the secondary storage in the primary storage after the retrieved data has been stored in the secondary storage and requested a predetermined number of times.
6. The computer-implemented method of claim 1 further comprising replacing stored data in the primary storage with the retrieved data stored in the secondary storage based on a second replacement algorithm.
7. The computer-implemented method of claim 1 wherein the requested data is video.
8. A storage system comprising:
primary storage, wherein the primary storage comprises flash-based storage;
secondary storage, wherein the secondary storage is non-flash storage; and
a non-transitory computer readable medium having instructions stored thereon for causing a computer to execute a method for reducing the number of write operations to the primary storage, the method comprising:
determining whether requested data is stored in either the primary storage or the secondary storage;
retrieving the requested data from an external storage and storing the retrieved data in the secondary storage if the requested data is not stored in either the primary storage or the secondary storage, wherein the retrieved data from the external storage is not simultaneously stored to both the primary storage and secondary storage; and
retrieving the requested data from the secondary storage and storing the requested data in the primary storage based on a storage management algorithm if the requested data is not stored in the primary storage but stored in the secondary storage, wherein, according to the storage management algorithm, the requested data is stored in the primary storage after the data has been requested a predetermined number of times.
9. The storage system of claim 8 wherein the primary storage includes flash and a second storage media.
10. The storage system of claim 8 , the method further comprising replacing stored data in the secondary storage with the retrieved data based on a first replacement algorithm.
11. The storage system of claim 8 , the method further comprising removing from the secondary storage the retrieved data stored in the secondary storage after it is stored in the primary storage.
12. The storage system of claim 8 wherein retrieving the requested data from the secondary storage and storing the requested data in the primary storage based on a storage management algorithm if the requested data is not stored in the primary storage but stored in the secondary storage comprises storing the retrieved data from the secondary storage in the primary storage after the retrieved data has been stored in the secondary storage and requested a predetermined number of times.
13. The storage system of claim 8 , the method further comprising replacing stored data in the primary storage with the retrieved data stored in the secondary storage based on a second replacement algorithm.
14. The storage system of claim 8 wherein the requested data is video.
15. A computer-implemented method of reducing a number of write operations to storage, the method comprising:
receiving a request for data;
determining whether the requested data is stored in a primary storage, wherein the primary storage comprises flash-based storage;
retrieving the requested data from an external storage if the requested data is not stored in the primary storage, wherein the external storage is non-flash storage;
determining whether a reference to the requested data is stored in a secondary storage, wherein the external storage is non-flash storage;
storing in the secondary storage a reference to the requested data if the requested data is not stored in the primary storage and a reference to the requested data is not stored in the secondary storage, wherein the reference to the requested data comprises a reference to the requested data at the external storage; and
storing in the primary storage the retrieved data based on a storage management algorithm, wherein, according to the storage management algorithm, the retrieved data is stored in the primary storage after the data has been requested a predetermined number of times;
wherein the retrieved data from the external storage is not simultaneously stored to both the primary storage and secondary storage.
16. The computer-implemented method of claim 15 wherein the primary storage includes flash and a second storage media.
17. The computer-implemented method of claim 15 wherein storing in the secondary storage a reference to the requested data if the requested data is not stored in the primary storage and a reference to the requested data is not stored in the secondary storage comprising replacing in the secondary storage a stored reference to data in the external storage with the reference to the requested data based on a first replacement algorithm.
18. The computer-implemented method of claim 15 further comprising incrementing a counter for the requested data each time the requested data is retrieved from the external storage.
19. The computer-implemented method of claim 18 wherein storing in the primary storage the retrieved data based on a storage management algorithm comprises storing the retrieved data from the external storage in the primary storage after the retrieved data has been requested a predetermined number of times based on the counter.
20. The computer-implemented method of claim 15 further comprising replacing stored data in the primary storage with the retrieved data based on a second replacement algorithm.
21. The computer-implemented method of claim 15 wherein the requested data is video.
22. A storage system comprising:
primary storage, wherein the primary storage comprises flash-based storage;
secondary storage, wherein the secondary storage is non-flash storage; and
a non-transitory computer readable medium having instructions stored thereon for causing a computer to execute a method for reducing the number of write operations to the primary storage, the method comprising:
determining whether requested data is stored in the primary storage;
retrieving the requested data from an external storage if the requested data is not stored in the primary storage, wherein the external storage is non-flash storage;
determining whether a reference to the requested data is stored in the secondary storage;
storing in the secondary storage a reference to the requested data if the requested data is not stored in the primary storage and a reference to the requested data is not stored in the secondary storage, wherein the reference to the requested data comprises a reference to the requested data at the external storage; and
storing in the primary storage the retrieved data based on a storage management algorithm, wherein, according to the storage management algorithm, the retrieved data is stored in the primary storage after the data has been requested a predetermined number of times;
wherein the retrieved data from the external storage is not simultaneously stored to both the primary storage and secondary storage.
23. The storage system of claim 22 wherein the primary storage includes flash and a second storage media.
24. The storage system of claim 22 , wherein storing in the secondary storage a reference to the requested data if the requested data is not stored in the primary storage and a reference to the requested data is not stored in the secondary storage comprising replacing in the secondary storage a stored reference to data in the external storage with the reference to the requested data based on a first replacement algorithm.
25. The storage system of claim 22 , the method further comprising incrementing a counter for the requested data each time the requested data is retrieved from the external storage.
26. The storage system of claim 25 , wherein storing in the primary storage the retrieved data based on a storage management algorithm comprises storing the retrieved data from the external storage in the primary storage after the retrieved data has been requested a predetermined number of times based on the counter.
27. The storage system of claim 22 , the method further comprising replacing stored data in the primary storage with the retrieved data based on a second replacement algorithm.
28. The storage system of claim 22 wherein the requested data is video.