Method and system to process unstructured data

1. A computer-implemented system comprising:

at least one processor coupled to a memory;

a receiver to access, using the at least one processor, a source of unstructured data;

an entity extractor to:

utilize a template to extract, using the at least one processor, from the source of unstructured data, entity instances, an entity instance from the entity instances being an alpha-numeric string, the entity instance corresponding to an entity type from the template, the template comprising two or more entity types, and

organize, using the at least one processor, the extracted entity instances into an entity instance table;

a pattern generator to, using the at least one processor:

select one or more entity types from the entity instance table, the one or more entity types to be included as pattern entities in a pattern, and

based on the one or more entity types selected from the entity instance table, generate the pattern comprising the one or more entity types as the pattern entities, the pattern entities comprising a key entity and one or more non-key entities associated with the key entity, the pattern comprising a subset of entity types present in the template; and

a dataset generator to generate, using the at least one processor, a two-dimensional table based on the pattern and the entity instance table, in the two-dimensional table each row comprising data related to an instance of the key entity of the pattern.

2. The system of claim 1 , wherein the source of unstructured data is a web page.

3. The system of claim 1 , wherein the source of unstructured data is an email.

4. The system of claim 1 , wherein the pattern generator is to:

determine an entity type characterized by the highest frequency of occurrence in the entity instance table; and

identify the entity type characterized by the highest frequency of occurrence in the entity instance table as the key entity.

5. The system of claim 1 , wherein the pattern generator is to:

determine that the frequency of occurrence of an entity type from the entity instance table is above a predetermined threshold; and

include the entity type in the pattern.

6. The system of claim 1 , wherein the pattern generator is to:

determine that the frequency of occurrence of an entity type from the entity instance table is below a predetermined threshold; and

exclude the entity type from the pattern.

7. The system of claim 1 , wherein the dataset generator is to:

detect a first instance of the key entity in the entity instance table;

add the first instance of the key entity into a new row of the two-dimensional table;

determine that the next entity instance in the entity instance table is not an instance of the key entity; and

add the next entity instance into the new row of the two-dimensional table.

8. The system of claim 1 , comprising a data quality module to:

determine that a first row in the two-dimensional table and a second row in the two-dimensional table are both characterized by duplicate instances of the key entity; and

consolidate the first row with the second row in the two-dimensional table.

9. The system of claim 1 , comprising a data quality module to:

determine that a row in the two-dimensional table is associated with a default value for an instance of an entity type from the pattern; and

add the default value into a field in the row, the field associated with the entity type.

10. The system of claim 1 , wherein the pattern generator is to save the pattern for future use.

11. A computer-implemented method comprising:

using one or more processors to perform operations of:

accessing a source of unstructured data;

utilizing a template to extract, from the source of unstructured data, entity instances, an entity instance from the entity instances being an alpha-numeric string, the entity instance corresponding to an entity type from the template, the template comprising two or more entity types;

organizing the extracted entity instances into an entity instance table;

selecting one or more entity types from the entity instance table, the one or more entity types to be included as pattern entities in a pattern;

based on the one or more entity types selected from the entity instance table, generating the pattern comprising the one or more entity types as the pattern entities, the pattern entities comprising a key entity and one or more non-key entities associated with the key entity, the pattern comprising a subset of entity types present in the template; and

generating a two-dimensional table based on the pattern and the entity instance table, in the two-dimensional table each row comprising data related to an instance of the key entity of the pattern.

12. The method of claim 11 , wherein the source of unstructured data is a web page.

13. The method of claim 11 , wherein the source of unstructured data is an email.

14. The method of claim 11 , comprising:

determining an entity type characterized by the highest frequency of occurrence in the entity instance table; and

identifying the entity type characterized by the highest frequency of occurrence in the entity instance table as the key entity.

15. The method of claim 11 , comprising:

determining that the frequency of occurrence of an entity type from the entity instance table is above a predetermined threshold; and

adding the entity type in the pattern.

16. The method of claim 11 , comprising:

determining that the frequency of occurrence of an entity type from the entity instance table is below a predetermined threshold; and

excluding the entity type from the pattern.

17. The method of claim 11 , comprising:

detecting a first instance of the key entity in the entity instance table;

adding the first instance of the key entity into a new row of the two-dimensional table;

determining that the next entity instance in the entity instance table is not an instance of the key entity; and

adding the next entity instance into the new row of the two-dimensional table.

18. The method of claim 11 , comprising:

determining that a first row in the two-dimensional table and a second row in the two-dimensional table are both characterized by duplicate instances of the key entity; and

consolidating the first row with the second row in the two-dimensional table.

19. The method of claim 11 , comprising:

determining that a row in the two-dimensional table is associated with a default value for an instance of an entity type from the pattern; and

adding the default value into a field in the row, the field associated with the entity type.

20. A machine-readable non-transitory medium having instruction data to cause a machine to:

access a source of unstructured data;

utilize a template to extract, from the source of unstructured data, entity instances, an entity instance from the entity instances being an alpha-numeric string, the entity instance corresponding to an entity type from the template, the template comprising two or more entity types;

organize the extracted entity instances into an entity instance table;

select one or more entity types from the entity instance table, the one or more entity types to be included as pattern entities in a pattern;

based on the one or more entity types selected from the entity instance table, generate the pattern comprising the one or more entity types as the pattern entities, the pattern entities comprising a key entity and one or more non-key entities associated with the key entity, the pattern comprising a subset of entity types present in the template; and

generate a two-dimensional table based on the pattern and the entity instance table, in the two-dimensional table each row comprising data related to an instance of the key entity of the pattern.