SCALABILITY IMPROVEMENTS OF PEOPLE-COUNTING SENSOR NETWORKS
Disclosed herein is a technique to improve the processing of people counting systems. The technique involves ingesting raw people counting data into a time-series database sorted by point of origin and recorded by end visit time. That raw data is periodically summarized and then written to a relational database. Queries of the relational database for total visitor count over a period of time are modified by approximations of the raw data for particular classes of visitors. Examples of visitor classes include recurring visitors and pedestrians vs. people in cars.
1 . A method of improving scalability of people-counting systems comprising:
collecting raw data by a people-counter sensor wherein the raw data includes a timestamp for each detected person and any of:
a point of origin;
a region of origin;
a dwell time;
a location of person;
a device ID; or
an exit time;
storing the raw data in a time-series columnar distributed database keyed to the point of origin;
batching a portion of the raw data into a package, the portion of the raw data included in the package is data pertaining to people visits from a given origin over a period;
aggregating data of a plurality of packages into a relational database of visits to the people-counter sensor keyed to time; and
determining, a number of visitors for a queried group of people-counter sensors wherein the number of visitors is based on query of the relational database modified by approximations of visitor classes, the approximations of visitor classes based on a subset of the raw data.
2 . The method of claim 1 , further comprising:
purging the raw data from the distributed database on a periodic basis.
3 . The method of claim 1 , wherein user queries for the number of visitors only access the relational database.
4 . The method of claim 3 , wherein queries of the relational database specify only a first time period and a first group of people-counter sensors.
5 . A system of improving scalability of people-counting systems comprising:
a people-counter sensor positioned at a field location and configured to collect raw data, wherein the raw data includes a timestamp for each detected person and any of:
a point of origin;
a region of origin;
a location of person;
a dwell time;
a device ID; or
an exit time;
a plurality of distributed database servers configured to intake and store the raw data in a time-series database wherein the raw data in the distributed database servers is discarded after a first period of time;
generating approximations of statistics of a class of visitor represented in the raw data based on the raw data;
a batching module in communication with the distributed database and configured to batch a portion of the raw data into a package, the portion of the raw data included in the package is data pertaining to people visits from a given origin over a period; and
a relational database server in communication with the batching module and configured to write data of a plurality of packages into a relational database supporting queries of a number of visits to the people-counter sensor keyed to time.
6 . The system of claim 5 , wherein time is sortable by hours or days.
7 . The system of claim 5 , wherein the plurality of distributed database servers are configured to purge the raw data on a periodic basis.
8 . The system of claim 5 , wherein user queries for the number of visitors only access the relational database.
9 . The system of claim 8 , wherein queries of the relational database include specify only a first time period and a first group of people-counter sensors.
10 . A method of improving scalability of people-counting systems comprising:
collecting raw data by a people-counter sensor wherein the raw data includes a timestamp for each detected person and any of:
a point of origin;
a region of origin;
a dwell time;
a location of person;
a device ID; or
an exit time;
storing the raw data in a time-series column database;
summarizing the raw data in the time-series column database to generate summarized data; and
writing, the summarized data to a relational database server wherein the relational database is queryable by an end user.
11 . The method of claim 10 , wherein the summarized data includes less fields than the raw data and has a smaller data size than raw data corresponding to a same time period.
12 . The method of claim 11 , wherein the fields included in the summarized data are timestamp and point of origin.
13 . The method of claim 10 , wherein the belongs time-series column database is supported on a distributed group of servers, wherein the distributed group of servers are a buffering mechanism of the raw data for the relational database.
14 . The method of claim 10 , wherein the time-series column database uses the point of origin field as a primary key, and the region of origin field as a secondary key.
15 . The method of claim 10 , further comprising:
purging raw data that is 24 hours old from the time-series column database.
16 . The method of claim 10 , wherein said summarizing occurs hourly and includes a portion of the raw data collected and stored within the last previous hour.
17 . A method of improving scalability of people-counting systems comprising:
collecting raw data by a people-counter sensor wherein the raw data includes a timestamp for each detected person and fields including any of:
a point of origin;
a region of origin;
a dwell time;
a location of person;
a device ID; or
an exit time; and
generating approximations of statistics of a class of visitor represented in the raw data based on the raw data;
determining, a count of visitors for a first group of people-counter sensors over a first time period based on a query of a database including a summarized version of the raw data; and
modifying the count of visitors based on said approximations.
18 . The method of claim 17 , wherein the class of visitor represented in the raw data is a percentage of counted people whom are pedestrians, and wherein said generating approximations is performed by:
evaluating the location of person field of the raw data wherein the raw data includes each person detected from a second time period and a second group of people-counter sensors, wherein each person detected at less than a threshold distance from the people-counter sensor is evaluated as a pedestrian; and
determining a ratio of pedestrians to non-pedestrians based on said evaluation of the location of person field wherein the ratio is used to approximate the class of visitor over a greater time period than the second time period for the second group of people-counter sensors.
19 . The method of claim 18 , wherein the first time period and the second time period are different, and wherein the second group of people-counter sensors is representative of the first group of people-counter sensors.
20 . The method of claim 17 , wherein the class of visitor represented in the raw data is a percentage visitor whom are recurring visitors, and wherein said generating approximations is performed by:
evaluating the device ID field of the raw data wherein the raw data includes each person detected from a second time period and a second group of people-counter sensors, wherein each person detected at less than a threshold distance from the people-counter sensor is evaluated as a pedestrian; and
determining a ratio of visit recurrence based on said evaluation of the device ID field wherein the ratio is used to approximate the class of visitor over a greater time period than the second time period for the second group of people-counter sensors.
21 . The method of claim 20 , wherein the first time period and the second time period are different, and wherein the second group of people-counter sensors is representative of the first group of people-counter sensors.
22 . The method of claim 20 , further comprising:
estimating a number of unique visitors based on the number of visitors modified by the approximated visitor class.
23 . The method of claim 17 , wherein the approximations of visitor class are periodically updated based on a recent subset of the raw data.