Big data parser
Computer-readable media include computer-readable instructions. The computer readable instructions include a class definition for a first object and a class definition for a second object. The first object includes a buffer for storing first information that identifies fields; a first function for storing the first information in the buffer; and a second function for extracting values of the fields, identified by the first information stored in the buffer, from a portion of a log. The second object includes a third function for obtaining configuration information from a configuration file, wherein the configuration information includes the first information; storing the configuration information at a first memory location; and performing a process.
1. A system comprising:
a map module, implemented in hardware, comprising:
a buffer to store first information that identifies fields;
a first function to store the first information in the buffer;
a second function to extract values of the fields, identified by the first information stored in the buffer, from a portion of a log;
a driver module, implemented in hardware, comprising a third function to:
obtain configuration information from a configuration file, wherein the configuration information includes the first information;
store the configuration information at a first memory location;
create a job object;
store, in the job object, an address of a memory location at which the map module is stored; and
perform a process to:
obtain the first information from the configuration information;
store the obtained first information in the buffer;
divide a log parsing job into two or more tasks, wherein each of the tasks includes executing the second function at a different slave device; and
cause the slave devices to execute the tasks,
wherein the configuration information includes information that is different from the first information.
2. A first method comprising:
a second method for storing first information that identifies fields in a buffer;
a third method for extracting values of the fields, identified by the first information stored in the buffer, from a portion of a log; and
a fourth method for:
obtaining configuration information from a configuration file, wherein the configuration information includes the first information;
storing the configuration information at a first memory location;
creating a job object;
storing, in the job object, an address of a memory location at which a class definition of an object that includes the fourth method is stored; and
performing a process for:
obtaining the first information from the configuration information;
storing the obtained first information in the buffer;
dividing a log parsing job into two or more tasks, wherein each of the tasks includes executing the third method at a different slave device; and
causing the slave devices to execute the tasks,
wherein the configuration information includes information that is different from the first information.