Converting tabular demographic information into an export entity file
The present disclosure is directed to methods and non-transitory program storage devices for identifying demographic information in an input data file and converting that data into autonomous data of an export entity file. In an embodiment, the received tabular data is sorted based on a lowest number of unique entities of a certain type in the revised data file, common data describing the lowest number of unique entities is extracted, labeled, and stored as an autonomous export entity file. In an embodiment a plurality of autonomous export entities are linked to create a master export entity comprising the plurality of autonomous export entities.
1. A method for processing personal data, the method comprising:
(a) receiving a data file containing a plurality of fields of demographic information from a third-party;
(b) analyzing the data file to distinguish between each of the plurality of fields of demographic information;
(c) detecting a data type for one of the plurality of fields of demographic information;
(d) standardizing data in the data file, wherein the standardizing comprises;
(I) sorting the data based on a lowest number of unique entities of a certain type in the data file;
(II) extracting common data describing the lowest number of unique entities of (I),
(III) labeling the common data as describing a single entity,
(IV) storing the extracted data of (II) as an autonomous entity under the label of (III), and
(V) linking a plurality of single entities;
(e) generating one or more export entities based on the data standardized at (d);
(f) combining the one or more export entities of (e) into a first combined file; and
(g) outputting the first combined file.
2. The method of claim 1 , wherein the third-party is a healthcare provider and the data file containing the plurality of fields of demographic information is a medical roster.
3. The method of claim 1 , wherein the data type is detected based on semantic content, a data shape, style, name, or phrase.
4. The method of claim 3 , wherein the data type is determined based on a column type, a column title, a neighboring data type, an active label stored in memory, or a combination thereof.
5. The method of claim 1 , wherein the lowest number of unique entities comprises data describing the same group, person, practice group, specialty, practice, street address, mailing address, zipcode, or license number.
6. The method of claim 1 , wherein the extracting at (II) further comprises converting the extracted common data into a language-independent format.
7. The method of claim 1 , wherein the data stored at (IV) is safe to process without any knowledge of any other data and without corrupting any of the received data.
8. The method of claim 1 , wherein the linking of (V) is a pointer, a logical connection between entities, or data describing a first entity that is embedded within data describing a second entity.
9. The method of claim 1 , wherein a transaction record of each of the standardizing, sorting, extracting, labeling, storing, and linking is stored in an auditable persistent file.
10. The method of claim 1 , further comprising comparing the combined entities of (f) with an existing second combined file to determine differences between the first and second combined files.
11. The method of claim 10 , wherein the generating export entities of (e) comprises adding, removing or updating data based on the determined differences between the first and second combined files.
12. The method of claim 11 , wherein the first combined file is a diff file comprising the added, removed, or updated data.
13. The method of claim 10 , wherein the second combined file comprises a previously stored output file.
14. The method of claim 10 , wherein the second combined file comprises information collected from a plurality of publicly available sources.
15. A non-transitory program storage device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform a method, the method comprising:
(a) receiving a data file containing a plurality of fields of demographic information from a third-party;
(b) analyzing the data file to distinguish between each of the plurality of fields of demographic information;
(c) detecting a data type for one of the plurality of fields of demographic information;
(d) standardizing data in the data file, wherein the standardizing comprises;
(I) sorting the data based on a lowest number of unique entities of a certain type in the data file;
(II) extracting common data describing the lowest number of unique entities of (I),
(III) labeling the common data as describing a single entity,
(IV) storing the extracted data of (II) as an autonomous entity under the label of (III), and
(V) linking a plurality of single entities;
(e) generating one or more export entities based on the data standardized at (d);
(f) combining the one or more export entities of (e) into a first combined file; and
(g) outputting the first combined file.
16. The non-transitory program storage device of claim 15 , wherein the extracting at (II) further comprises converting the extracted common data into a language-independent format.
17. The non-transitory program storage device of claim 15 , wherein the data stored at (IV) is safe to process without any knowledge of any other data and without corrupting any of the received data.
18. The non-transitory program storage device of claim 15 , further comprising comparing the combined entities of (f) with an existing second combined file to determine differences between the first and second combined files.
19. The non-transitory program storage device of claim 18 , wherein the generating export entities of (e) comprises adding, removing or updating data based on the determined differences between the first and second combined files.
20. The non-transitory program storage device of claim 18 , wherein the second combined file comprises a previously stored output file or information collected from a plurality of publicly available sources.