Morphic storage device

1. A method for allocating portions of a flash memory in a solid state drive connected to a host, the flash memory comprising a plurality of physical flash memory blocks, the method comprising:

performing a plurality of read operations on the flash memory, one or more read operations comprising an error correcting code decoding operation, the error correcting code decoding operation producing a bit error count;

calculating, from a plurality of bit error counts, a raw bit error rate for one or more of the plurality of physical flash memory blocks;

counting programming and erase cycles performed for each of the plurality of physical flash memory blocks;

calculating, for one or more physical flash memory blocks, from the raw bit error rate and the number of programming and erase cycles, for each of a plurality of programming methods:

an average response time, and

an estimated allowable retention time;

receiving, from a first application executing on the host, a first set of requirements;

determining, for one or more physical flash memory blocks, whether it is suitable for meeting the first set of requirements;

forming a first list of physical flash memory blocks, the physical flash memory blocks in the first list being suitable for meeting the first set of requirements;

receiving a request for an allocation of storage space from the first application; and

allocating, to the first application, a first physical flash memory block from the first list.

2. The method of claim 1 , wherein the plurality of programming methods comprises:

an incremental step pulse programming method utilizing a first step size; and

an incremental step pulse programming method utilizing a second step size,

the first step size being greater than the second step size.

3. The method of claim 1 , wherein the first set of requirements includes a required allowable retention time and a required average response time.

4. The method of claim 3 , wherein the determining, for one or more physical flash memory blocks, whether it is suitable for meeting the first set of requirements comprises determining whether for any programming method of the plurality of programming methods:

the calculated average response time is less than the required average response time; and

the calculated estimated allowable retention time is greater than the required allowable retention time.

5. The method of claim 1 , further comprising erasing the first physical flash memory block.

6. The method of claim 5 , wherein the erasing of the first physical flash memory block comprises:

employing a shallow erase process when the predicted effect of a shallow erase process is that the first physical flash memory block remains suitable for meeting the first set of requirements; and

employing a normal erase process when the predicted effect of a shallow erase process is that the first physical flash memory block does not remain suitable for meeting the first set of requirements.

7. The method of claim 1 , wherein the calculating, for one or more physical flash memory blocks, of an estimated allowable retention time comprises performing a linear fit to bit error counts as a function of retention time and calculating the retention time for which the linear fit exceeds a maximum acceptable raw bit error rate.

8. The method of claim 7 , wherein the maximum acceptable raw bit error rate is one bit error per million bits.

9. The method of claim 1 , further comprising:

receiving, from a second application executing on the host, a second set of requirements different from the first set of requirements;

determining, for one or more physical flash memory blocks, whether it is suitable for meeting the second set of requirements;

forming a second list of physical flash memory blocks, the physical flash memory blocks in the second list being suitable for meeting the second set of requirements;

receiving a request for an allocation of storage space from the second application; and

allocating, to the second application, a second physical flash memory block from the second list.

10. The method of claim 9 , further comprising erasing the second physical flash memory block.

11. The method of claim 10 , wherein the erasing of the second physical flash memory block comprises:

employing a shallow erase process when the predicted effect of a shallow erase process is that the first physical flash memory block remains suitable for meeting the first set of requirements; and

employing a normal erase process when the predicted effect of a shallow erase process is that the first physical flash memory block does not remain suitable for meeting the first set of requirements.

12. A solid state drive, comprising:

a storage controller, the storage controller comprising a hardware processing circuit; and

a flash memory, including a plurality of physical flash memory blocks, each physical flash memory block comprising a plurality of physical pages,

the storage controller comprising:

a morphic engine comprising:

a NAND controller;

a retention predictor;

a wear-out tracker; and

a raw bit error rate tracker; and

a storage virtualizer comprising:

a virtual storage table;

an address translation block; and

a clustering engine;

the NAND controller being configured to perform a plurality of read operations on the flash memory, one or more read operations comprising an error correcting code decoding operation, the error correcting code decoding operation producing a bit error count;

the raw bit error rate tracker being configured to calculate, from a plurality of bit error counts, a raw bit error rate for one or more of the plurality of physical flash memory blocks;

the wear-out tracker being configured to count programming and erase cycles performed for one or more of the plurality of physical flash memory blocks;

the clustering engine being configured to calculate, for one or more physical flash memory blocks, from the raw bit error rate and the number of programming and erase cycles for each of a plurality of programming methods:

an average response time, and

an estimated allowable retention time;

the virtual storage table being configured to receive, from a first application executing on a host, a first set of requirements;

the clustering engine being configured to receive the first set of requirements from the virtual storage table, and to determine, for one or more physical flash memory blocks, whether it is suitable for meeting the first set of requirements;

the clustering engine being configured to form a first list of physical flash memory blocks, the physical flash memory blocks in the first list being suitable for meeting the first set of requirements;

the clustering engine being configured to receive a request for an allocation of storage space from the first application; and

the clustering engine being configured to allocate, to the first application, a first physical flash memory block from the first list.

13. The solid state drive of claim 12 , wherein the plurality of programming methods comprises:

an incremental step pulse programming method utilizing a first step size; and

an incremental step pulse programming method utilizing a second step size,

the first step size being greater than the second step size.

14. The solid state drive of claim 12 , wherein the first set of requirements includes a required allowable retention time and a required average response time.

15. The solid state drive of claim 12 , wherein the determining, for one or more physical flash memory blocks, whether it is suitable for meeting the first set of requirements comprises determining whether for any programming method of the plurality of programming methods:

the calculated average response time is less than the required average response time; and

the calculated estimated allowable retention time is greater than the required allowable retention time.

16. The solid state drive of claim 12 , wherein the NAND controller is further configured to erase the first physical flash memory block.

17. The solid state drive of claim 12 , wherein the erasing of the first physical flash memory block comprises:

employing a shallow erase process when the predicted effect of a shallow erase process is that the first physical flash memory block remains suitable for meeting the first set of requirements; and

employing a normal erase process when the predicted effect of a shallow erase process is that the first physical flash memory block does not remain suitable for meeting the first set of requirements.

18. The solid state drive of claim 12 , wherein the calculating, for one or more physical flash memory blocks, of an estimated allowable retention time as a function of a performance metric comprises performing a linear fit to bit error counts as a function of retention time and calculating the retention time for which the linear fit exceeds a maximum acceptable raw bit error rate.

19. The solid state drive of claim 18 , wherein the maximum acceptable raw bit error rate is one bit error per million bits.

20. The solid state drive of claim 12 , wherein:

the virtual storage table is further configured to receive, from a second application executing on the host, a second set of requirements different from the first set of requirements;

the clustering engine is further configured to:

receive the second set of requirements from the virtual storage table, and to determine, for one or more physical flash memory blocks, whether it is suitable for meeting the second set of requirements;

form a second list of physical flash memory blocks, the physical flash memory blocks in the second list being suitable for meeting the second set of requirements;

receive a request for an allocation of storage space from the second application; and

allocate, to the second application, a second physical flash memory block from the second list.