Patent Application

App. No. 14/693,792

Abstract

Claims (68)

1 . A computer implemented method, performed using one or more processors, comprising:

retrieving a data file;

parsing the data file into a plurality of object fragments;

differencing the plurality of object fragments against one or more objects in an object database to determine differences;

storing the differences in an object fragment table;

retrieving the differences from the object fragment table; and

applying the differences to the object database.

2 . The computer implemented method of claim 1 , further comprising:

generating object pointers corresponding to the differences in the object fragment table, the object pointers identifying a location of the differences within the object fragment table;

storing the object pointers within a pointer table;

accessing the pointer table to retrieve the object pointers; and

retrieving the differences based on the location identified by the object pointers.

3 . The computer implemented method of claim 1 , further comprising:

identifying object types of each of the plurality of object fragments;

defining one or more import phases determining an order to retrieve the plurality of object fragments, the one or more import phases based on the object types; and

retrieving the plurality of object fragments based on the one or more import phases.

4 . The computer implemented method of claim 1 , further comprising:

defining a sequence of phases to retrieve the differences based on optimization factors, wherein the optimization factors include a time of day, a capacity of a server, and a task priority; and

retrieving the differences based on the sequence of phases.

5 . The computer implemented method of claim 1 , further comprising:

defining a sequence of phases to retrieve the differences based on an object type of the object fragments;

retrieving the object fragments in the defined sequence of phases; and

adding an additional phase at the occurrence of a lock contention.

6 . The computer implemented method of claim 1 , wherein the differences are a first set of differences, and wherein applying the first set of differences to the object database further comprises:

differencing the first set of differences against the one or more objects in the object database, to determine a second set of difference; and

storing the second set of difference in the object database.

7 . The computer implemented method of claim 1 , wherein combining the plurality of object fragments and the differences results in the one or more objects within the object database.

8 . The computer implemented method of claim 1 , wherein the one or more object fragments include a corresponding object definition, an access control list, a media value, and a link set fragment.

9 . The computer implemented method of claim 1 , wherein the data file includes a raw JSON file, a GZIP JZON file, and a ZIP file.

10 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

retrieving a data file;

parsing the data file into a plurality of object fragments;

differencing the plurality of object fragments against one or more objects in an object database to determine a difference;

storing the difference in an object fragment table;

retrieving the differences; and

applying the differences to the object database.

11 . The non-transitory machine-readable medium of claim 10 , wherein the operations further comprise:

generating object pointers for the differences, the object pointers identifying a location of the differences within the object fragment table;

storing the object pointers within a pointer table;

accessing the pointer table to retrieve the object pointers; and

retrieving the differences based on the location identified by the object pointers.

12 . The non-transitory machine-readable medium of claim 10 , wherein the operations further comprise:

identifying object types of each of the plurality of object fragments;

defining one or more import phases determining an order to retrieve the plurality of object fragments, the one or more import phases based on the object types; and

retrieving the plurality of object fragments based on the one or more import phases.

13 . The non-transitory machine-readable medium of claim 10 , wherein the operations further comprise:

defining an order to retrieve the differences based on optimization factors, wherein the optimization factors include a time of day, a capacity of a server, and a task priority; and

retrieving the differences based on the order.

14 . The non-transitory machine-readable medium of claim 10 , wherein the differences are a first set of differences, and wherein applying the first set of differences to the object database further comprises:

differencing the first set of differences against the one or more objects in the object database, to determine a second set of difference; and

storing the second set of difference in the object database.

15 . The non-transitory machine-readable medium of claim 10 , wherein combining the plurality of object fragments and the differences results in the one or more objects within the object database.

16 . The non-transitory machine-readable medium of claim 10 , wherein the data file includes a raw JSON file, a GZIP JZON file, and a ZIP file.

17 . The non-transitory machine-readable medium of claim 10 , wherein the operations further comprise:

defining a sequence of phases to import the object fragments, the sequence of phases based at least on an object type of the object fragments; and

retrieving the object fragments based on the defined sequence of phases.

18 . A system comprising:

a file retrieval module comprising one or more processors and configured to retrieve data;

an ingest module comprising one or more processors and configured to parse the data into a plurality of object fragments;

the ingest module further configured to store the one or more object fragments in an object fragment table;

an import module comprising one or more processors and configured to retrieve the one or more object fragments; and

the import module further configured to apply the one or more object fragments to a local object database.

19 . The system of claim 18 , wherein the ingest module is further configured to:

generate object pointers corresponding to each of the object fragments among the plurality of object fragments in the object fragment table, the object pointers defining locations of each of the object fragments within the object fragment table; and

store the object pointers in a pointer table.

20 . The system of claim 19 , wherein the import module is further configured to:

access the pointer table to retrieve the object pointers; and

retrieve the object fragments based on the locations defined by the object pointers.

Assignments (8)