Formatting semi-structured data in a database

1. A method for formatting semi-structured data in a column store database, comprising:

receiving data from a data source to be stored in one of a plurality of columns of the column store database, wherein the plurality of columns of the column store database include explicit columns for storing data that fits into key-and-value formats of the explicit columns and a semi-structured column for storing data that does not fit into the key-and-value formats of the explicit columns;

prior to loading the received data to any of the plurality of columns of the column store database, determining, by a processor, whether a format of the received data is compatible with the key-and-value formats of the explicit columns of the column store database;

in response to a determination that the format of the received data is compatible with the key-and-value formats of the explicit columns of the column store database, loading, by the processor, the received data to one explicit column of the explicit columns of the column store database, wherein loading the received data to the explicit column comprises:

determining a key-and-value format of the received data,

matching a key of the key-and-value format of the received data to a key of the explicit column, and

loading the received data to the explicit column whose key matches the key of the key-and-value format of the received data;

in response to a determination that the format of the received data is not compatible with the key-and-value formats of the explicit columns of the column store database, loading, by the processor, the received data to the semi-structured column of the column store database; and

formatting, by the processor, the received data in the semi-structured column with an optimal search format, including adding a keyword that describes the received data.

2. The method of claim 1 , further comprising searching said semi-structured column in response to search queries.

3. The method of claim 1 , wherein formatting the received data with said optimal search format further includes extracting a portion of the received data and placing the portion of the received data in a newly created explicit column having a new key-and-value format.

4. The method of claim 2 , wherein searching said semi-structured column in response to said search queries includes searching said semi-structured column when said search queries fail to match the explicit columns in said column store database.

5. The method of claim 1 , wherein said semi-structured column further stores data having formats that are unknown to said column store database.

6. The method of claim 3 , wherein formatting the received data with said optimal search format further includes reformatting the received data with indexing information in addition to information in the new key-and-value format.

7. The method of claim 1 , further comprising:

loading the received data to both the semi-structured column and an existing explicit column of the explicit columns of the column store database in response to a determination that the received data belongs to information already stored in the existing explicit column of the column store database.

8. A system for formatting semi-structured data in a column store database, comprising:

a processor; and

a memory storing instructions that when executed cause the processor to:

receive data from a data source to be stored in one of a plurality of columns of the column store database, wherein the plurality of columns of the column store database include explicit columns for storing data that fits into key-and-value formats of the explicit columns and a semi-structured column for storing data that does not fit into the key-and-value formats of the explicit columns;

prior to loading the received data to any of the plurality of columns of the column store database, determine whether a format of the received data is compatible with the key-and-value formats of the explicit columns of the column store database;

in response to a determination that the format of the received data is compatible with the key-and-value formats of the explicit columns of the column store database, load the received data to one explicit column of the explicit columns of the column store database, wherein, to load the received data to the explicit column, the processor is to:

determine a key-and-value format of the received data,

match a key of the key-and-value format of the received data to a key of the explicit column, and

load the received data to the explicit column whose key matches the key of the key-and-value format of the received data;

in response to a determination that the format of the received data is not compatible with the key-and-value formats of the explicit columns of the column store database, load the received data to the semi-structured column of the column store database; and

format the received data in the semi-structured column with an optimal search format, including adding a keyword that describes the received data.

9. The system of claim 8 , wherein the instructions are executable to cause the processor to:

search the semi-structured column in response to a search query;

in response to finding semi-structured data in the semi-structured column in response to the search query, analyze the search query for metadata; and

reformat the semi-structured data in the semi-structured column for optimal search characteristics, including adding the metadata to the semi-structured data in the semi-structured column.

10. The system of claim 8 , wherein, to format the received data in the semi-structured column with the optimal search format, the instructions are executable to cause the processor to add indexing information to the keyword.

11. The system of claim 8 , wherein the instructions are executable to cause the processor to:

load the received data into both the semi-structured column and an existing explicit column of the explicit columns of the column store database in response to a determination that the received data belongs to information already stored in the existing explicit column of the column store database.

12. A non-transitory computer readable storage medium storing program instructions that, when executed, cause a processor to:

receive data from a data source to be stored in one of a plurality of columns of a column store database, wherein the plurality of columns of the column store database include explicit columns for storing data that fits into key-and-value formats of the explicit columns and a semi-structured column for storing data that does not fit into the key-and-value formats of the explicit columns;

prior to loading the received data to any of the plurality of columns of the column store database, determine whether a format of the received data is compatible with the key-and-value formats of the explicit columns of the column store database;

in response to a determination that the format of the received data is compatible with the key-and-value formats of the explicit columns of the column store database, load the received data to one explicit column of the explicit columns of the column store database, wherein, to load the received data to the explicit column, the processor is to:

determine a key-and-value format of the received data,

match a key of the key-and-value format of the received data to a key of the explicit column, and

load the received data to the explicit column whose key matches the key of the key-and-value format of the received data;

in response to a determination that the format of the received data is not compatible with the key-and-value formats of the explicit columns of the column store database, load the received data to the semi-structured column of the column store database; and

format the received data in the semi-structured column with an optimal search format, including adding a keyword that describes the received data.

13. The non-transitory computer readable storage medium of claim 12 , wherein, to format the received data in the semi-structured column with the optimal search format, the program instructions are executable to cause the processor to:

extract a portion of the received data; and

place the portion of the received data in a newly created explicit column having a new key-and-value format.

14. The non-transitory computer readable storage medium of claim 12 , wherein the program instructions are executable to cause the processor to:

search the semi-structured column in response to a search query;

in response to finding semi-structured data in the semi-structured column in response to the search query, analyze the search query for metadata; and

reformat the semi-structured data in the semi-structured column for optimal search characteristics by adding the metadata to the semi-structured data in the semi-structured column.

15. The non-transitory computer readable storage medium of claim 12 , wherein the program instructions are executable to cause the processor to:

load the received data to both the semi-structured column and an existing explicit column of the explicit columns of the column store database in response to a determination that the received data belongs to information already stored in the existing explicit column of the column store database.

16. The non-transitory computer readable storage medium of claim 12 , wherein the semi-structured column further stores data having formats that are unknown to the column store database.

17. The system of claim 8 , wherein the semi-structured column further stores data having formats that are unknown to the column store database.