Ensuring data quality by filtering network address observations
In one embodiment, a filtering technique is provided for ensuring data quality of network address observations. A network address observation is obtained of a network address associated with a source device, the network address observation associating the network address with one or more directly observed attributes. The network address observation is filtered based on a comparison of a selected one of the one or more directly observed attributes to a predetermined criteria, and using a result of the comparison as indicative of whether the network address observation should be used for association of the network address with one or more directly observed attributes. The filtering either associates one or more indicators with the network address observation, or removes the network address observation. A network address to attribute association system executed on one or more electronic devices stores a record that maintains any network address observation that has not been removed and any indicator.
1. A method for ensuring data quality by filtering network address observations used to produce a database, comprising:
obtaining a plurality of network address observations of a network address associated with a source device in communication with a network, each network address observation of the plurality of network address observations captured by the source device or a network node of the network, each network address observation of the plurality of network address observations associating the network address with one or more directly observed attributes that each describe at least one of a location, time, intent, or identity of the source device observed by the source device or the network node;
filtering the plurality of network address observations based on a determination that the plurality of network address observations belong to a same probability distribution for at least one of the directly observed attributes as a reference set of network address observations that are known to be unsuitable for producing attribute associations, wherein the filtering either associates one or more indicators with the plurality of network address observations when the plurality of the network address observations are not to be used for association of the network address with the one or more directly observed attributes, or removes the plurality of network address observations when the plurality of network address observations are not to be used for association of the network address with the one or more directly observed attributes, in response to a result of the determination;
storing, by a network address to attribute association system executed on one or more electronic devices, a record that maintains any network address observations that have not been removed and any indicators, in a storage device of the one or more electronic devices; and
producing the database, by the network address to attribute association system executed on one or more electronic devices, to include one or more attributes derived from the directly observed attributes in the record, the database usable to provide the one or more attributes in response to a network address.
2. The method of claim 1 , wherein the network address is an Internet Protocol (IP) address associated with the source device.
3. The method of claim 1 , wherein the determination that the plurality of network address observations belong to the same probability distribution for the at least one of the directly observed attributes as the reference set of network address observations is based on a distribution of locations of the plurality of network address observations and the reference set.
4. The method of claim 1 , wherein the reference set of network address observations is a plurality of known false, fraudulent, or misrepresented observations, and the determination comprises:
determining whether the plurality of network address observations belongs to the set of known false, fraudulent, or misrepresented observations.
5. The method of claim 1 , wherein the network node is a WiFi access point, and at least one of the one or more attributes is a location determined by WiFi-positioning.
6. A non-transitory electronic-device readable medium having executable instruction stored thereon, the executable instructions when executed by one or more processors of one or more electronic devices operable to:
obtain a network address observation of a network address associated with a source device in communication with a network, the network address observation captured by the source device or a network node of the network, the network address observation associating the network address with a plurality of directly observed attributes that each describe at least one of a location, time, intent, or identity of the source device observed by the source device or the network node;
filter the network address observation based on a determination that the network address observation belongs to a same probability distribution for at least one of the directly observed attributes as a reference set of network address observations that are known to be unsuitable for producing attribute associations, wherein the filtering either associates an indicator with the network address observation when the network address observation is not to be used for association of the network address with the plurality of directly observed attributes, or removes the network address observation when the network address observation is not to be used for association of the network address with the plurality of directly observed attributes;
store a record that maintains any network address observation that has not been removed and any indicator; and
produce the database, by the network address to attribute association system executed on one or more electronic devices, to include one or more attributes derived from the directly observed attributes in the record, the database usable to provide the one or more attributes in response to a network address.
7. The non-transitory electronic-device readable medium of claim 6 , wherein the network address is an Internet Protocol (IP) address associated with the source device.