System and method for deletion compactor for large static data in NoSQL database
System and method to compact a NoSQL database, the method including: receiving, by a receiver coupled to a processor, an indication of a record to delete in the NoSQL database; for each file in the NoSQL database, perform the steps of: if said file does not contain the record to delete, placing said file in a first memory; if said file contains the record to delete: placing said file in a second memory; searching whether the record to delete from said file in the second memory matches a record in one or more files in the first memory; and if a searched files in the first memory contain the record to delete from said file in the second memory, compacting said file in the second memory with the files in the first memory that contain the record to delete.
1. A method to compact a NoSQL database, comprising:
receiving, by a receiver coupled to a processor, an indication of a record to delete in the NoSQL database wherein the database includes one or more files containing the record to delete and one or more files not containing the record to delete;
for each file in the NoSQL database, perform the steps of:
if said file does not contain the indication of the record to delete, placing said file in a first memory;
if said file contains the indication of the record to delete:
placing said file in a second memory;
searching whether the record to delete from said file in the second memory matches a record in any files in the first memory; and
if any searched files in the first memory contain the record to delete from said file in the second memory, compacting said file in the second memory with the any files in the first memory that contain the record to delete.
2. The method of claim 1 , wherein the step of searching said file comprises using a key field to find matching records.
3. The method of claim 1 , wherein the step of searching comprises the step of searching any files in the first memory for the record to delete from said file in the second memory.
4. The method of claim 1 , further comprising recording a range of key field values in said file.
5. The method of claim 1 , wherein the step of compacting said file in second memory is performed each time a searched file in the first memory is found to include the record to delete from the file in the second memory.
6. The method of claim 1 , wherein the step of compacting said file in the second memory is performed for a plurality of files in the first memory, which include a record to delete from the file in the first memory.
7. The method of claim 1 , wherein the step of compacting is performed by a Cassandra process.
8. A system to compact a NoSQL database, comprising:
a receiver coupled to a processor, the receiver configured to receive an indication of a record to delete in the NoSQL database wherein the database includes one or more files containing the record to delete and one or more files not containing the record to delete;
a processor coupled to a first memory and a second memory, the processor configured to perform for each file in the NoSQL database, the steps of:
if said file does not contain the indication of a record to delete, placing said file in the first memory;
if said file contains the indication of a record to delete:
placing said file in the second memory;
searching whether the record to delete from said file in the second memory matches a record in any files in the first memory; and
if any searched files in the first memory contain the record to delete from said file in the second memory, compacting said file in the second memory with the any files in the first memory that contain the record to delete.
9. The system of claim 8 , wherein the step of searching said file comprises using a key field to find matching records.
10. The system of claim 8 , wherein the step of searching comprises the step of searching any files in the first memory for the record to delete from said file in the second memory.
11. The system of claim 8 , further comprising a module configured to record a range of key field values in said file.
12. The system of claim 8 , wherein the step of compacting said file in second memory is performed each time a searched file in the first memory is found to include the record to delete from the file in the second memory.
13. The system of claim 8 , wherein the step of compacting said file in the second memory is performed for a plurality of files in the first memory, which include a record to delete from the file in the first memory.
14. The system of claim 8 , wherein the step of compacting is performed by a Cassandra process.