IP Library Granted Patent US 12,449,997
Granted Patent B2
US 12,449,997 · App. 18/608,221 · Granted Oct 21, 2025

Memory system with multiple striping of raid groups and method for performing the same

Inventor: Jon C. R. Bennett (Sudbury, MA)
Assignee: Innovations In Memory LLC
G06F3/064G06F3/0619G06F3/0689G06F11/1068G06F11/108G11B20/1833G11C29/52G06F3/0611G06F3/0659G06F12/0246G11B2220/60G11C7/1072
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,449,997
App. No.
18/608,221
Granted
Oct 21, 2025
Kind
B2
Abstract

A data memory system is described, where there may be an asymmetry in the time needed to write or erase data and the time needed to read data. The data may be stored using a RAID data storage arrangement and the reading, writing and erasing operations on the modules arranged such that the erasing and writing operations may be performed without significant latency for performing a read operation. Where a failure of a memory module in the memory system occurs, methods for recovering the data of the failed module are disclosed which may selected in accordance with policies that may relate to the minimizing the possibility of irretrievable data loss, or degradation of latency performance.

Claims (36)

1. A method comprising:

receiving data;

determining redundancy data for the received data;

storing the received data and the redundancy data in one or more memory modules of a group of memory modules;

receiving a read data request; and

reconstructing the requested data, based on an erase data operation preventing a read data operation on the one or more memory modules where the requested data is stored, wherein a time delay to reconstruct the requested data is less than a time period where a memory module of the group of memory modules is in an erase state.

2. The method of claim 1 , further comprising:

reconstructing the requested data without a time delay due to the erase data operation.

3. The method of claim 1 , further comprising:

reading sufficient stored data and redundancy data from other memory modules of the group of memory modules; and

reconstructing the requested data based on the reading sufficient stored data and redundancy data.

4. The method of claim 1 , further comprising:

providing the requested data, based on the read data request, prior to completion of the erase data operation.

5. The method of claim 1 , wherein when the redundancy data is single parity data, and the method further comprising: reconstructing the requested data using an exclusive-OR operation.

6. The method of claim 1 , wherein the redundancy data is dual parity data.

7. The method of claim 1 , where memory modules of the group of memory modules are Solid State Disks (SSDs).

8. The method of claim 1 , wherein the memory modules are comprised of NAND FLASH memory circuits.

9. The method of claim 1 , wherein the group of memory modules are configured as a Redundant Array of Independent Disks (RAID) group.

10. A method comprising:

receiving data;

determining redundancy data for the received data;

storing the received data and the redundancy data in one or more memory modules of a group of memory modules;

receiving a read data request; and

reconstructing the requested data, based on a write data operation preventing a read data operation on the one or more memory modules where the requested data is stored, wherein a time delay to reconstruct the requested data is less than a time period where a memory module of the group of memory modules is in a write state.

11. The method of claim 10 , further comprising:

reconstructing the requested data without a time delay due to the write data operation.

12. The method of claim 10 , further comprising:

reading sufficient stored data and redundancy data from other memory modules of the group of memory modules; and

reconstructing the requested data based on the reading sufficient stored data and redundancy data.

13. The method of claim 10 , further comprising:

providing the requested data, based on the read data request, prior to completion of the write data operation.

14. The method of claim 10 , wherein when the redundancy data is single parity data, and the method further comprising: reconstructing the requested data using an exclusive-OR operation.

15. The method of claim 10 , wherein the redundancy data is dual parity data.

16. The method of claim 10 , where memory modules of the group of memory modules are Solid State Disks (SSDs).

17. The method of claim 10 , wherein the memory modules are comprised of NAND FLASH memory circuits.

18. The method of claim 10 , wherein the group of memory modules are configured as a Redundant Array of Independent Disks (RAID) group.

Continuity (9)
Continuation 18178629 · Mar 6, 2023
Continuation 17321189 · May 14, 2021
Continuation 16444176 · Jun 18, 2019
Continuation 15467515 · Mar 23, 2017
Continuation 12901224 · Oct 8, 2010
Continuation In Part 12079364 · Mar 26, 2008
Provisional Application 61250216 · Oct 9, 2009
Provisional Application 60920737 · Mar 29, 2007
Related Publication 20240302983A1 · Sep 12, 2024
References Cited (161)
US 4363094A · Kaul · 1982 [cited by applicant]
US 4843288A · Volz · 1989 [cited by applicant]
US 5128810A · Halford · 1992 [cited by applicant]
US 5379262A · Okamoto · 1995 [cited by applicant]
US 5522031A · Ellis et al. · 1996 [cited by applicant]
US 5623595A · Bailey · 1997 [cited by applicant]
US 5680579A · Young · 1997 [cited by applicant]
US 5687183A · Chesley · 1997 [cited by applicant]
US 5889795A · Niijima et al. · 1999 [cited by applicant]
US 5974503A · Venkatesh · 1999 [cited by applicant]
US 6009547A · Jaquette · 1999 [cited by applicant]
US 6038570A · Hitz · 2000 [cited by applicant]
US 6118612A · Judd · 2000 [cited by applicant]
US 6151289A · Rope · 2000 [cited by applicant]
US 6185134B1 · Tanaka · 2001 [cited by applicant]
US 6205571B1 · Comporese et al. · 2001 [cited by applicant]
US 6226708B1 · McGoldrick · 2001 [cited by applicant]
US 6237052B1 · Stolowitz · 2001 [cited by applicant]
US 6321345B1 · Mann · 2001 [cited by applicant]
US 6353895B1 · Stephenson · 2002 [cited by applicant]
US 6374337B1 · Estakhri · 2002 [cited by applicant]
US 6425052B1 · Hashemi · 2002 [cited by applicant]
US 6539463B1 · Kuwata · 2003 [cited by applicant]
US 6567311B2 · Ishii · 2003 [cited by applicant]
US 6581185B1 · Hughes · 2003 [cited by applicant]
US 6591339B1 · Horst · 2003 [cited by applicant]
US 7111140B2 · Estakhri · 2006 [cited by applicant]
US 7136964B2 · Yamamoto · 2006 [cited by applicant]
US 7143305B2 · Hajji · 2006 [cited by applicant]
US 7149950B2 · Spencer · 2006 [cited by applicant]
US 7191304B1 · Cameron · 2007 [cited by applicant]
US 7203889B2 · Oza · 2007 [cited by applicant]
US 7213103B2 · Eng · 2007 [cited by applicant]
US 7310703B2 · Frolund · 2007 [cited by applicant]
US 7356730B2 · Ulrich · 2008 [cited by applicant]
US 7363455B2 · Shimada · 2008 [cited by applicant]
US 7366852B2 · Hung · 2008 [cited by applicant]
US 7451265B2 · Traister · 2008 [cited by applicant]
US RE40877E · Singhal · 2009 [cited by applicant]
US 7734867B1 · Keeton · 2010 [cited by applicant]
US 7818525B1 · Frost · 2010 [cited by applicant]
US 7984084B2 · Sinclair · 2011 [cited by applicant]
US 10157016B2 · Bennett · 2018 [cited by applicant]
US 10204042B2 · Bennett · 2019 [cited by applicant]
US 10754769B2 · Bennett · 2020 [cited by applicant]
US 11010076B2 · Bennett · 2021 [cited by applicant]
US 20020046322A1 · Butterworth et al. · 2002 [cited by applicant]
US 20020073285A1 · Butterworth · 2002 [cited by applicant]
US 20020083379A1 · Nishikawa · 2002 [cited by applicant]
US 20020099904A1 · Conley · 2002 [cited by applicant]
US 20020103966A1 · Wu · 2002 [cited by applicant]
US 20020120901A1 · Poirier · 2002 [cited by applicant]
US 20030046489A1 · Yagi · 2003 [cited by applicant]
US 20030056142A1 · Hashemi · 2003 [cited by applicant]
US 20030084397A1 · Peleg · 2003 [cited by applicant]
US 20030097524A1 · Brant · 2003 [cited by applicant]
US 20030105928A1 · Ash · 2003 [cited by applicant]
US 20030110343A1 · Hagiwara et al. · 2003 [cited by applicant]
US 20030208511A1 · Earl · 2003 [cited by applicant]
US 20030212858A1 · Apperley · 2003 [cited by applicant]
US 20030227800A1 · Saito · 2003 [cited by applicant]
US 20040073767A1 · Johnson · 2004 [cited by applicant]
US 20040153817A1 · Norman · 2004 [cited by applicant]
US 20040153902A1 · Machado · 2004 [cited by applicant]
US 20050015522A1 · Elboim · 2005 [cited by applicant]
US 20050044313A1 · Amemiya · 2005 [cited by applicant]
US 20050111249A1 · Yagisawa · 2005 [cited by applicant]
US 20050146939A1 · Conley · 2005 [cited by applicant]
US 20050193166A1 · Johnson · 2005 [cited by applicant]
US 20050223269A1 · Stolowitz · 2005 [cited by applicant]
US 20050240743A1 · Eng · 2005 [cited by applicant]
US 20050246362A1 · Borland · 2005 [cited by applicant]
US 20050279837A1 · Haiji · 2005 [cited by applicant]
US 20060064545A1 · Wintergerst · 2006 [cited by applicant]
US 20060123312A1 · Forhan · 2006 [cited by applicant]
US 20060129876A1 · Uemura · 2006 [cited by applicant]
US 20060161678A1 · Bopardikar · 2006 [cited by applicant]
US 20060187715A1 · Narvaez · 2006 [cited by applicant]
US 20060195657A1 · Tien et al. · 2006 [cited by applicant]
US 20060271725A1 · Wong · 2006 [cited by applicant]
US 20070061513A1 · Tsumagari · 2007 [cited by applicant]
US 20070067563A1 · Smith · 2007 [cited by applicant]
US 20070079219A1 · Nagai · 2007 [cited by applicant]
US 20070088754A1 · Brannon · 2007 [cited by applicant]
US 20070089045A1 · Corbett et al. · 2007 [cited by applicant]
US 20070101075A1 · Jeddeloh · 2007 [cited by applicant]
US 20070113029A1 · Bennett · 2007 [cited by applicant]
US 20070124532A1 · Bennett · 2007 [cited by applicant]
US 20070162516A1 · Thiel · 2007 [cited by applicant]
US 20070279993A1 · Hemink · 2007 [cited by applicant]
US 20080052446A1 · Lasser · 2008 [cited by applicant]
US 20080059869A1 · Brewer · 2008 [cited by applicant]
US 20080089135A1 · Ito · 2008 [cited by applicant]
US 20080098054A1 · Carmody · 2008 [cited by applicant]
US 20080109616A1 · Taylor · 2008 [cited by applicant]
US 20080126673A1 · Kaneda · 2008 [cited by applicant]
US 20080155160A1 · McDaniel · 2008 [cited by applicant]
US 20080168304A1 · Flynn · 2008 [cited by applicant]
US 20080183953A1 · Flynn · 2008 [cited by applicant]
US 20080250270A1 · Bennett · 2008 [cited by applicant]
US 20080282025A1 · Biswas · 2008 [cited by applicant]
US 20080316822A1 · Ahn · 2008 [cited by applicant]
US 20080320366A1 · Lin · 2008 [cited by applicant]
US 20090006886A1 · O'Connor · 2009 [cited by applicant]
US 20090010067A1 · Lee · 2009 [cited by applicant]
US 20090089045A1 · Lenat · 2009 [cited by applicant]
US 20090106491A1 · Piszczek · 2009 [cited by applicant]
US 20090147573A1 · Hemink · 2009 [cited by applicant]
US 20090150599A1 · Bennett · 2009 [cited by applicant]
US 20090172258A1 · Olbrich · 2009 [cited by applicant]
US 20090172335A1 · Kulkarni · 2009 [cited by applicant]
US 20090198947A1 · Khmelnitsky · 2009 [cited by applicant]
US 20090204872A1 · Yu · 2009 [cited by applicant]
US 20100023672A1 · Gorobets · 2010 [cited by applicant]
US 20100115189A1 · Lin · 2010 [cited by applicant]
US 20110099320A1 · Lucas · 2011 [cited by applicant]
US 20130024642A1 · Flynn et al. · 2013 [cited by applicant]
US 20180082745A1 · Hisamoto · 2018 [cited by applicant]
EP 977121 · 2000 [cited by applicant]
JP 09265357 · 1997 [cited by applicant]
WO 2010071655 · 2010 [cited by applicant]
Argument and Amendment in Response to the Notice of Preliminary Rejection for Korean Application No. 10-2012-7000724, dated Jul. 13, 2012, 3 pgs (with translation). [cited by applicant]
Amendment from counterpart Korean Application No. 10-2012-7009026, dated Aug. 9, 2016, 14 pgs (with partial translation). [cited by applicant]
Argument and Amendment from counterpart South Korean Application No. 10-2013-7005934, dated Dec. 18, 2013, 24 pages. [cited by applicant]
Chang, L.P., et al., “Real-Time Garbage Collection for Flash Memory Storage Systems of Real-Time Embedded Systems”, ACM Transactions on Embedded Computing Systems, vol. V, No. N, Jun. 2004, pp. 1-26. [cited by applicant]
Chen, P.M., et al., “Maximizing Performance in a Striped Disk Array,” Computer Science Division, University of California, Berkeley, 10 pgs. (Published prior to Jan. 1, 1990). [cited by applicant]
Choudhuri, S., et al., “Deterministic Service Guarantees for NAND Flash using Partial Block Cleaning”, CODES+ISSS/08, Oct. 19-24, 2008, Atlanta, Georgia, USA, Copyright 2008 ACM 978-1-60558-470—Jun. 8, 2010, pp. 19-24. [cited by applicant]
Choudhuri, S., et al., “Real-Time Access Guarantees for NAND Flash Using Partial Block Cleaning”, SEUS 2008, LNCS 5287, pp. 138-149, 2008, .COPYRGT. IFIP International Federation for Information Processing 2008. [cited by applicant]
English translation of Argument and Amendment from counterpart South Korean Application No. 10-2012-7009026, dated Aug. 9, 2013, 8 pages. [cited by applicant]
European Patent Office Communication for European Application No. 08742320.8, dated Feb. 28, 2012, 5 pgs. [cited by applicant]
European Patent Office Communication from EP Application N. 11168499.9, dated Oct. 22, 2012, 4 pgs. [cited by applicant]
Extended European Search Report for European Application No. 10786789.7, dated Nov. 22, 2012, 6 pgs. [cited by applicant]
Extended European Search Report for European Application No. 10822795.0, dated Apr. 29, 2013, 8 pgs. [cited by applicant]
Graefe, G., “Write-Optimized B-Trees”, Proceedings of the 30.sup.th VLDB Conference, Toronto, Canada, 2004, pp. 672-683. [cited by applicant]
International Preliminary Report on Patentability for International Application No. PCT/2010/052074, 4 pgs. [cited by applicant]
International Search Report and the Written Opinion of the International Searching Authority for International Application No. PCT/US2010/037987, dated Dec. 30, 2010, 9 pgs. [cited by applicant]
International Search Report for International Application No. PCT/2010/052074 (3 pgs). dated Mar. 11, 2012. [cited by applicant]
International Search Report for International Application No. PCT/US2008/083969, dated Apr. 30, 2009, 3 pgs. [cited by applicant]
Jeffrey, C.A., “Performance Analysis of Dynamic Sparing and Error Correction Techniques for Fault Tolerance in Nanoscale Memory Structures”, Thesis Presentation to the University of Florida, 2004, 104 pgs. [cited by applicant]
Juonolainen, M.K., “Cand Scient Thesis”, University of Oslo, Jan. 2, 1999, 129 pgs. [cited by applicant]
Norvag, K., et al., “Log-Only Temporal Object Storage”, Copyright 1997 IEEE, published in the Proceedings of DEXA'97, Sep. 1-2, 1997, Toulouse, France, 6 pp. [cited by applicant]
Notice of Preliminary Rejection for Korean Application No. 10-2012-7000724, dated Feb. 20, 2013, 12 pgs. [cited by applicant]
Notice of Preliminary Rejection for Korean Application No. 10-2012-7000724, dated Jun. 20, 2012, 3 pgs (with translation). [cited by applicant]
Notice of Preliminary Rejection for South Korean Application No. 10-2012-7009026, dated Nov. 9, 2012, 10 pgs. [cited by applicant]
Office Action for related U.S. Appl. No. 14/642,910 dated Apr. 28, 2015. [cited by applicant]
Office Action from counterpart Chinese Application No. 200880007937.4, dated Jun. 29, 2012, and Response dated Oct. 8, 2012, 30 pgs. [cited by applicant]
Office Action from U.S. Appl. No. 12/901,224, dated Oct. 18, 2012, 23 pgs. [cited by applicant]
Patterson, D.A., et al., “A Case for Redundant Arrays of Inexpensive Disks (RAID),”, Reprint from Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 3-14, Jun. 1988. [cited by applicant]
Response to Communication from the European Patent Office for European Application No. 08742320.8, dated Sep. 4, 2012, 14 pgs. [cited by applicant]
Sanders, D.A., et al., “Terabyte IDE RAID-5 Disk Arrays”, 2003 Conference for Computing in High Energy and Nuclear Physics, LaJolla, California, Mar. 24-28, 2003, pp. 1-8. [cited by applicant]
Shirriff, K.W., “Sawmill: A Logging File System for a High-Performance RAID Disk Array”, Dissertation Presentation, University of California, 1995, pp. 164. [cited by applicant]
Unknown author, Intel.RTM., “Intelligent RAID 6 Theory Overview and Implementation”, www.intel.com/design/storage/intelligent_raid.htm, Copyright 2005 Intel Corporation, 13 pgs. [cited by applicant]
Unknown author, Intel.RTM., “Understanding the Flash Translation Layer (FTL) Specification”, Dec. 1998, Order No. 2997816-002, .COPYRGT. 1997, Intel Corporation, pp. 1-20. [cited by applicant]
Unknown author, NXP Corp “LPC122x”, datasheet, in U.S. Appl. No. 12/079,364, filed Mar. 26, 2008, updated, p. 44 of 60. [cited by applicant]
Unknown author, Samsung Electronics Flash Memory, “1G.times.8 Bit/2G.times.8 Bit/4G.times.Bit NAND Flash Memory”, K9XXGO8UXA, Date Unknown, 50 pgs (published prior to Jan. 10, 2006). [cited by applicant]
Unknown author, Violin Memory, Inc., “Spike Free Low Latency”, obtained from Internet Address www.vmern.com, .COPYRGT. 2011, Violin Memory, Inc., 8 pgs. [cited by applicant]
Wan, Jiguang et al., “S2-RAID: Parallel RAID Architecture for Fast Data Recovery”; Journal of Latex Class Files, vol. 6, No. 1, Jan. 2007, 16 pgs. [cited by applicant]
Written Opinion of the International Searching Authority for International Application No. PCT/US2008/083969 dated Apr. 30, 2009, 5 pgs. [cited by applicant]
Written Opinion of the International Searching Authority International Application No. PCT/2010/052074 dated Jun. 30, 2011, 5 pgs. [cited by applicant]
Written Opinion of the International Searching Authority International Application No. PCT/2010/052074, 8 pgs. [cited by applicant]
Extended European Search Report dated May 22, 2024 for European Patent Application No. 23214266.1. [cited by applicant]