Systems and methods of generating and using a bitmap index
Systems and methods of generating and using a bitmap index are disclosed. The bitmap index stores bit strings. In a particular implementation, each bit string in the bitmap index corresponds to an attribute of an audience. Each location in each bit string corresponds to the same audience member. The bitmap index supports parallelized and distributed execution of queries with respect to data indexed by the bitmap index. For example, the described bitmap index can be used to index advertising data, healthcare data, financial data, etc.
1. A method comprising:
receiving a first signal at a computing device comprising a processor, wherein the first signal includes information corresponding to a first user identifier associated with a first user;
modifying a value of a first bit of a bitmap index responsive to the first signal, wherein the bitmap index includes a plurality of bit strings, wherein a value stored in a particular location in each of the bit strings indicates whether a corresponding signal associated with a corresponding user identifier has been received, and wherein the first bit is stored in a first location of the bitmap index that corresponds to the first user identifier and to the first signal;
receiving a second signal that includes second information corresponding to a second user identifier associated with a second user;
modifying a value of a second bit of the bitmap index responsive to the second signal, wherein the second bit is stored in a second location of the bitmap index that corresponds to the second user identifier and to the second signal;
receiving a query directed to the bitmap index; and
outputting a result of the query based on executing a query execution plan for the query, wherein executing the query execution plan comprises performing one or more set operations with respect to the first bit and the second bit.
2. The method of claim 1 , wherein the first location is in a first column of the bitmap index that corresponds to the first user identifier and in a first row of the bitmap index that corresponds to the first signal.
3. The method of claim 1 , wherein the query execution plan identifies:
the one or more set operations;
that one or more first nodes are to send stored portions of one or more bit strings to a second node;
that the second node is to perform the one or more set operations with respect to:
the portions of the one or more bit strings received from the one or more first nodes; and
portions of one or more bit strings stored at the second node; and
that the second node is to concatenate results of performing the one or more set operations to generate a result bit string that indicates a result of the query.
4. The method of claim 1 , wherein the first user identifier is assigned to the first user based on a registration event.
5. The method of claim 4 , wherein the registration event indicates demographic information of the first user based on social network data.
6. The method of claim 1 , wherein one or more of the plurality of bit strings corresponds to a demographic attribute, a behavior, a brand affinity, or a combination thereof.
7. The method of claim 1 , further comprising determining a top Z signals for a subset of users based on the bitmap index, wherein Z is a positive integer.
8. The method of claim 7 , wherein determining the top Z signals comprises:
generating a filter string corresponding to the subset of users;
performing intersection operations of the filter string to generate a plurality of result strings; and
identifying the top Z signals for the subset of users based on the result strings.
9. The method of claim 8 , wherein generating the filter string includes performing one or more logical operations.
10. The method of claim 1 , wherein a first portion of a first bit string is stored at a first particular node and a second portion of the first bit string is stored at a second particular node that is different from the first particular node.
11. The method of claim 1 , wherein each of the plurality of bit strings is stored as one or more distributed slices.
12. The method of claim 11 , wherein:
each of the distributed slices comprises one or more chunks; and
a particular chunk is stored in a compressed form if each bit of the particular chunk has a particular value.
13. The method of claim 1 , wherein data corresponding to each row of the bitmap index is stored in a data store.
14. The method of claim 1 , wherein data corresponding to at least one signal is stored in a data store but not in the bitmap index.
15. The method of claim 1 , wherein the bitmap index is associated with at least one of advertising data, customer data, inventory data, sensor data, healthcare data, or financial data.
16. A system comprising:
a processor; and
a memory storing instructions executable by the processor to perform operations comprising:
receiving a first signal, wherein the first signal includes information corresponding to a first user identifier associated with a first user;
modifying a value of a first bit of a bitmap index responsive to the first signal, wherein the bitmap index includes a plurality of bit strings, wherein a value stored in a particular location in each of the bit strings indicates whether a corresponding signal associated with a corresponding user identifier has been received, and wherein the first bit is stored in a first location of the bitmap index that corresponds to the first user identifier and to the first signal;
receiving a second signal that includes second information corresponding to a second user identifier associated with a second user; and
modifying a value of a second bit of the bitmap index responsive to the second signal, wherein the second bit is stored in a second location of the bitmap index that corresponds to the second user identifier and to the second signal.
17. The system of claim 16 , wherein the bitmap index is associated with at least one of advertising data, customer data, inventory data, sensor data, healthcare data, or financial data.
18. The system of claim 16 , wherein one or more of the plurality of bit strings corresponds to a demographic attribute, a behavior, a brand affinity, or a combination thereof.
19. A non-transitory computer readable storage device stores instructions that, when executed, cause a computer to perform operations comprising:
receiving a first signal at a computing device comprising a processor, wherein the first signal includes information corresponding to a first user identifier associated with a first user;
modifying a value of a first bit of a bitmap index responsive to the first signal, wherein the bitmap index includes a plurality of bit strings, wherein a value stored in a particular location in each of the bit strings indicates whether a corresponding signal associated with a corresponding user identifier has been received, and wherein the first bit is stored in a first location of the bitmap index that corresponds to the first user identifier and to the first signal;
receiving a second signal that includes second information corresponding to a second user identifier associated with a second user; and
modifying a value of a second bit of the bitmap index responsive to the second signal, wherein the second bit is stored in a second location of the bitmap index that corresponds to the second user identifier and to the second signal.
20. The non-transitory computer readable storage device of claim 19 , wherein the bitmap index is associated with at least one of advertising data, customer data, inventory data, sensor data, healthcare data, or financial data.