Dedicating disks to reading or writing
A method of increasing the speed of disk reading and writing operations in a database by using nodes of memory and processors where the memory stores programs to be executed by the processors. The data in the database is separated into one or more buckets and at least two replicas of each bucket of data are stored in different storage devices of the nodes. Determining a subset of storage devices that will be dedicated to writing, while keeping at least one replica of each bucket dedicated to reading and dedicating at least a substantial portion of the remaining storage devices to reading.
1. A method of increasing the speed of disk reading and writing operations in a database system comprising:
at the database system having one or more nodes comprising one or more processors and memory, the memory of the one or more nodes storing one or more programs to be executed by the one or more processors,
wherein the data in the database is partitioned into one or more buckets and at least two replicas of each bucket of data are stored in different storage devices of the one or more nodes;
determining a subset of storage devices that will be dedicated to writing, while keeping at least one replica of each bucket dedicated to reading;
dedicating at least a substantial portion of the remaining storage devices to reading, wherein designating at least one bucket for writing comprises:
determining a loop ranging from which one or more nodes have been written to least recently, to which one or more nodes have been written to most recently;
determining which buckets of each of the one or more nodes in the loop need read operations; and
if none of the buckets on the respective one or more nodes need read operations, designating the node to operate a set of write operations.
2. The method of claim 1 , further comprising,
generating a map that identifies which of the one or more buckets of the one or more nodes are available for reading and which of the one or more buckets of the one or more nodes are available for writing;
distributing the map to the one or more nodes; and
using this map locally within each node to inform each bucket whether it should be reading or writing.
3. The method of claim 2 , further comprising, generating a replacement map after every predetermined time period and distributing the replacement map to the one or more nodes.
4. The method of claim 3 , wherein the one or more nodes are synchronized by a respective clock, and the replacement map is adopted when the respective clocks of the one or more nodes reaches an N secondary boundary.
5. The method of claim 1 , wherein designating at least one bucket for writing includes determining buckets for writing based on the elapsed time since the last write event on the respective bucket.
6. A method of increasing the speed of disk reading and writing operations in a database system comprising:
at the database system having one or more nodes comprising one or more processors and memory, the memory of the one or more nodes storing one or more programs to be executed by the one or more processors,
wherein the data in the database is partitioned into one or more buckets and at least two replicas of each bucket of data are stored in different storage devices of the one or more nodes;
determining a subset of storage devices that will be dedicated to writing, while keeping at least one replica of each bucket dedicated to reading;
dedicating at least a substantial portion of the remaining storage devices to reading, wherein designating at least one bucket for writing comprises:
organizing the one or more buckets into a plurality of zones; and
rotating write only operations through each of the plurality of zones,
wherein at least one of the plurality of zones containing one replication of the at least two replications of data is designated read only operations.
7. The method of claim 6 , further comprising,
generating a map that identifies which of the one or more buckets of the one or more nodes are available for reading and which of the one or more buckets of the one or more nodes are available for writing;
distributing the map to the one or more nodes; and
using this map locally within each node to inform each bucket whether it should be reading or writing.
8. The method of claim 7 , further comprising, generating a replacement map after every predetermined time period and distributing the replacement map to the one or more nodes.
9. The method of claim 8 , wherein the one or more nodes are synchronized by a respective clock, and the replacement map is adopted when the respective clocks of the one or more nodes reaches an N secondary boundary.
10. The method of claim 6 , wherein designating at least one bucket for writing includes determining buckets for writing based on the elapsed time since the last write event on the respective bucket.