Dynamic modification of schemas in streaming databases

1. A method for dynamically modifying a database schema in a streaming database management system, the method comprising:

receiving a new database schema, wherein upon receipt the new database schema is determined to be valid;

comparing the new database schema to an existing database schema of a database in the streaming database management system, the database having a plurality of data streams as inputs;

identifying differences between the new database schema and the existing database schema based on the comparing;

determining that a particular data stream fits in the existing database schema and does not fit into the new database schema;

deleting the particular data stream;

providing an end-of-data indication to one or more applications using the deleted stream;

re-creating the particular data stream using a new identifier upon a determination that the particular data stream is modified;

instantiating a conversion model of the new database schema;

copying data of the database, with the existing database schema, into the conversion model;

converting the copied data into a format compatible with the new database schema;

modifying the database in accordance with the new database schema in a single transaction, wherein a plurality of modifications to the existing database schema are applied as a single transaction upon successful compilation of the each of the plurality of modifications, and none of the plurality of modifications of the dynamic modification are applied upon unsuccessful compilation of any of the plurality of modifications to the existing database schema; and

copying the converted data from the conversion model to the modified database, wherein data corresponding to the particular data stream is not stored in the modified database.

2. The method of claim 1 , wherein inputs to the database are suspended during the modifying.

3. The method of claim 2 , wherein the suspended inputs are limited to those inputs being affected by the changes, wherein one or more inputs affected by the changes are suspended and one or more inputs not affected by the changes are not suspended.

4. The method of claim 1 , wherein the converting comprises automatically converting a selected data stream formatted for the existing database schema into a format compatible with the new database schema.

5. The method of claim 1 , wherein the instantiating comprises instantiating the conversion model based on the identified schema differences, the conversion model organized according to the new database schema and including at least one data stream selected from the plurality of data streams of the database to convert data formatted for the existing database schema into a format compatible with the new database schema.

6. The method of claim 5 , further comprising converting data in database records into a format compatible with the new database schema.

7. The method of claim 5 , further comprising converting data in each of the plurality of data streams into a format compatible with the new database schema.

8. The method of claim 1 , further comprising backing up the existing database schema prior to the modifying.

9. The method of claim 8 , further comprising backing up data in each of the plurality of data streams.

10. The method of claim 1 , further comprising, upon detection of an error during application of the identified schema differences, rolling back the copying of the converted data.

11. The method of claim 1 , wherein one of the identified differences includes the removal of data from the existing database schema, and further comprising removing the data.

12. The method of claim 1 , wherein the receiving comprises:

receiving a configuration file including the new database schema; and

determining whether the configuration file is invalid, wherein if the configuration file is invalid: halting of the dynamic comparison of the database, and returning of an error message.

13. The method of claim 12 , wherein the determining whether the configuration file is invalid comprises:

parsing the configuration file for self-consistency; and

determining that the configuration file is inconsistent based on the parsing.

14. The method of claim 1 , further comprising;

deleting the conversion model after the copying has been performed.

15. The method of claim 1 , wherein the determining the particular data stream comprises:

determining data of the database that fits in the existing database schema that does not fit in the new database schema; and

wherein the copying comprises copying the data of the database excluding the data that does not fit in the new database schema into the modified database.

16. A computer-implemented streaming database management system, the system comprising:

a processor; and

a memory having processor readable instructions that when executed by the processor implement:

an interface for receiving user input; and

a database transaction module operable to perform operations, including:

receiving a new database schema, wherein upon receipt the new database schema is determined to be valid upon receipt;

comparing the new database schema to an existing database schema of a database in the streaming database management system,

the database having a plurality of data streams as inputs;

identifying differences between the new database schema and the existing database schema based on the comparing;

determining that a particular data stream fits in the existing database schema and does not fit into the new database schema;

deleting the particular data stream;

providing an end-of-data indication to one or more applications using the deleted stream;

re-creating the particular data stream using a new identifier upon a determination that the particular data stream is modified;

instantiating a conversion model of the new database schema;

copying data of the database, with the existing database schema, into the conversion model;

converting the copied data into a format compatible with the new database schema;

modifying the database in accordance with the new database schema in a single transaction, wherein a plurality of modifications to the existing database schema are applied as a single transaction upon successful compilation of the each of the plurality of modifications, and none of the plurality of modifications of the dynamic modification are applied upon unsuccessful compilation of any of the plurality of modifications to the existing database schema; and

copying the converted data from the conversion model to the modified database, wherein data corresponding to the particular data stream is not stored in the modified database.

17. The system of claim 16 , further comprising a database stream module for providing data to one or more data streams in the plurality of data streams.

18. The system of claim 16 , further comprising a data store for backing up the transaction.

19. The system of claim 16 , further comprising a database client authorized for configuring the database.

20. The system of claim 16 , wherein the database transaction module configured to perform the determining is configured to perform the operations comprising:

receiving a configuration file including the new database schema; and

determining whether the configuration file is invalid, wherein if the configuration file is invalid: halting of the dynamic comparison of the database, and returning of an error message.

21. The system of claim 20 , wherein the database transaction module configured to determine whether the configuration file is invalid is configured to perform the operations comprising:

parsing the configuration file for self-consistency; and

determining that the configuration file is inconsistent based on the parsing.

22. A non-transitory tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

receiving a new database schema, wherein upon receipt the new database schema is determined to be valid;

comparing the new database schema to an existing database schema of a database in the streaming database management system, the database having a plurality of data streams as inputs;

identifying differences between the new database schema and the existing database schema based on the comparing;

determining that a particular data stream fits in the existing database schema and does not fit into the new database schema;

deleting the particular data stream;

providing an end-of-data indication to one or more applications using the deleted stream;

re-creating the particular data stream using a new identifier upon a determination that the particular data stream is modified;

instantiating a conversion model of the new database schema;

copying data of the database, with the existing database schema, into the conversion model;

converting the copied data into a format compatible with the new database schema;

modifying the database in accordance with the new database schema in a single transaction, wherein a plurality of modifications to the existing database schema are applied as a single transaction upon successful compilation of the each of the plurality of modifications, and none of the plurality of modifications of the dynamic modification are applied upon unsuccessful compilation of any of the plurality of modifications to the existing database schema; and

copying the converted data from the conversion model to the modified database, wherein data corresponding to the particular data stream is not stored in the modified database.