System and method for gradually bringing rolled in data online with incremental deferred integrity processing

1. A data processing system for providing general user availability while integrity processing of rolled-in data is deferred and performed incrementally, the data processing system comprising:

a data warehouse administration module for administering a data warehouse to comprise:

a table dividable into portions for containing rows of rolled-in data;

a first and a second delimiter delimiting the start and the end respectively of each portion;

a metadata element having an entry corresponding to the start and end delimiters delimiting each portion;

a third delimiter for delimiting, between the first delimiter and the third delimiter, a sub-portion of the portion; and

an operations management module having operation mechanisms for performing operations on the data warehouse responsive to the delimiters, wherein the operation mechanisms include an integrity processing mechanism that uses the first and the third delimiters to identify the sub-portion and thereby determine which data requires integrity processing and further performs integrity processing on the rolled-in data contained in the sub-portion.

2. The data processing system of claim 1 , wherein the operation mechanisms further include any one of:

a table scan mechanism;

an index scan mechanism; and

an update mechanism.

3. The data processing system of claim 2 , wherein the table scan mechanism is arranged to inhibit general users from accessing rows in a portion delimited by the first and second delimiters.

4. The data processing system of claim 2 , wherein the data warehouse administration module administers the data warehouse to further comprise an index mechanism having indices that reference rows and wherein the index scan mechanism is arranged to inhibit general users from accessing any index that refers to a row in a portion delimited by the first and second delimiters.

5. The data processing system of claim 2 , wherein the update mechanism provides for modify, insert and delete operations on data in the table,

for the modify and the delete operations, the update mechanism treats as invisible any rows in a portion delimited by the first and second delimiters and therefore does not permit the operation to occur, and

for the insert operations, the update mechanism treats as invisible all portions delimited by the first and second delimiters.

6. The data processing system of claim 2 , wherein the integrity processing of the sub-portion includes constraint checking, extraction of index key and insertion into the index mechanism, and incremental refresh of dependent materialized views for each said row in the sub-portion.

7. The data processing system of claim 1 , wherein the data contained in the sub-portion is made accessible online to a user or made accessible to a user in general after said integrity processing on the data has been completed.

8. The data processing system of claim 7 , wherein the data is made accessible online or made accessible to a user in general by making the first delimiter equal to the third delimiter.

9. A data processing system-implemented method of directing a data processing system

to provide general user availability while integrity processing of rolled-in data is

deferred and performed incrementally, the data processing system-implemented method comprising:

administering a data warehouse to comprise:

a table dividable into portions for containing rows of rolled-in data;

a first and a second delimiter delimiting the start and the end respectively of each portion;

a metadata element having an entry corresponding to the start and end delimiters delimiting each portion;

a third delimiter for delimiting, between the first delimiter and the third delimiter, a sub-portion of the portion; and

performing operations on the data warehouse responsive to the delimiters, wherein the performing includes an integrity processing that uses the first and the third delimiters to identify the sub-portion and thereby determine which data requires integrity processing and further performs integrity processing on the rolled-in data contained in the sub-portion.

10. The data processing system-implemented method of claim 9 , wherein the performing further includes any one of:

a table scan;

an index scan; and

an update.

11. The data processing system-implemented method of claim 10 , wherein the table scan is arranged to inhibit general users from accessing rows in a portion delimited by the first and second delimiters.

12. The data processing system-implemented method of claim 10 , wherein the administering administers the data warehouse to further comprise an index mechanism having indices that reference rows and wherein the index scan mechanism is arranged to inhibit general users of from accessing any index that refers to a row in a portion delimited by the first and second delimiters.

13. The data processing system-implemented method of claim 10 , wherein the update provides for modify, insert and delete operations on data in the table,

for the modify and the delete operations, the update mechanism treats as invisible any rows in a portion delimited by the first and second delimiters and therefore does not permit the operation to occur, and

for the insert operations, the update mechanism treats as invisible all portions delimited by the first and second delimiters.

14. An article of manufacture for directing a data processing system to provide general user availability while integrity processing of rolled-in data is deferred and performed incrementally, the article of manufacture comprising:

a program usable medium embodying one or more instructions executable by the data processing system, the one or more instructions comprising:

data processing system executable instructions for administering a data warehouse to comprise:

a table dividable into portions for containing rows of rolled-in data;

a first and a second delimiter delimiting the start and the end respectively of each portion;

a metadata element having an entry corresponding to the start and end delimiters delimiting each portion;

a third delimiter for delimiting, between the first delimiter and the third delimiter, a sub-portion of the portion; and

data processing system executable instructions for performing operations on the data warehouse responsive to the delimiters, wherein the operations include integrity processing mechanism that uses the first and the third delimiters to identify the sub-portion and thereby determine which data requires integrity processing and further performs integrity processing on the rolled-in data contained in the sub-portion.

15. The article of manufacture of claim 14 , wherein the data processing system executable instructions for performing further includes performing any one of:

a table scan;

an index scan; and

an update.

16. The article of manufacture of claim 15 , wherein the table scan is arranged to inhibit general users from accessing rows in a portion delimited by the first and second delimiters.

17. The article of manufacture of claim 15 , wherein the data processing system executable instructions for administering administers the data warehouse to further comprise an index mechanism having indices that reference rows and wherein the index scan mechanism is arranged to inhibit general users from accessing an index that refers to a row in a portion delimited by the first and second delimiters.

18. The article of manufacture of claim 15 , wherein the update provides for modify, insert and delete operations on data in the table,

for the modify and the delete operations, the update mechanism treats as invisible

any rows in a portion delimited by the first and second delimiters and therefore does not permit the operation to occur, and

for the insert operations, the update mechanism treats as invisible all portions delimited by the first and second delimiters.