Data processing system and method

1. A data pipeline system comprising:

a binary data extractor configured to:

receive a data portion identifier;

extract a portion of a binary data item based on the data portion identifier; and

output the portion of the binary data item;

a data iterator configured to:

provide a first data portion identifier to the binary data extractor;

receive, from the binary data extractor, a first portion of the binary data item;

determine a second data portion identifier;

provide the second data portion identifier to the binary data extractor;

receive, from the binary data extractor, a second portion of the binary data item; and

output the second portion of the binary data item; and

a data converter configured to:

receive, from the data iterator, the first portion of the binary data item;

receive, from the data iterator, the second portion of the binary data item; and

transform, based on a data format specification, at least the second portion of the binary data item for processing by one or more other components of the data pipeline system.

2. The data pipeline system of claim 1 , wherein the data portion identifier comprises at least one of a data superframe identifier, a data frame identifier, a data subframe identifier, a data word identifier or a data line identifier.

3. The data pipeline system of claim 1 , wherein determining the second data portion identifier comprises extracting, from the first portion of the binary data item, a reference to a second portion of the binary data item, and wherein the second data portion identifier is based on the reference to the second portion of the binary data item.

4. The data pipeline system of claim 1 , wherein one or more components of the data pipeline system are configured to store the transformation of at least the second portion of the binary data item to a database, wherein access to the database is restricted to one or more users.

5. The data pipeline system of claim 1 , wherein transforming at least the second portion of the binary data item comprises:

determining a scaling factor based on the data format specification; and

transforming at least part of the second portion of the binary data item to a numerical value based on the scaling factor.

6. The data pipeline system of claim 5 , wherein the numerical value is in a floating point format.

7. A method, performed by one or processors, comprising:

providing, by a data iterator, a first data portion identifier to a binary data extractor;

receiving, by the binary data extractor, the first data portion identifier;

extracting, by the binary data extractor, a first portion of a binary data item based on the first data portion identifier;

outputting, by the binary data extractor, the first portion of the binary data item;

receiving, by the data iterator, the first portion of the binary data item from the binary data extractor;

determining, by the data iterator, a second data portion identifier;

providing, by the data iterator, the second data portion identifier to the binary data extractor;

receiving, by the binary data extractor, the second data portion identifier;

extracting, by the binary data extractor, a second portion of the binary data item based on the second data portion identifier;

outputting, by the binary data extractor, the second portion of the binary data item;

receiving, by the data iterator, the second portion of the binary data item from the binary data extractor;

outputting, by the data iterator, the second portion of the binary data item;

receiving, by a data converter, the second portion of the binary data item, from the data iterator; and

transforming, by the data converter, based on a data format specification, the at least the second portion of the binary data item for processing by one or more components of a data pipeline system.

8. The method of claim 7 , wherein the first data portion identifier and the second data portion identifier each comprise at least one of a data frame identifier, a data subframe identifier or a data word identifier.

9. The method of claim 7 , wherein determining the second data portion identifier comprises extracting, from the first portion of the binary data item, a reference to a second portion of the binary data item, and wherein the second data portion identifier is based on the reference to the second portion of the binary data item.

10. The method of claim 7 , further comprising storing the transformation of at least the second portion of the binary data item to a database, wherein access to the database is restricted to one or more users.

11. The method of claim 7 , wherein transforming at least the second portion of the binary data item comprises:

determining a scaling factor based on the data format specification; and

transforming at least part of the second portion of the binary data item to a numerical value based on the scaling factor.

12. The method of claim 11 , wherein the numerical value is in a floating point format.

13. Apparatus comprising one or more processors or special-purpose computing hardware configured to:

provide, by a data iterator, a first data portion identifier to a binary data extractor;

receive, by the binary data extractor, the first data portion identifier;

extract, by the binary data extractor, a first portion of a binary data item based on the first data portion identifier;

output, by the binary data extractor, the first portion of the binary data item;

receive, by the data iterator, the first portion of the binary data item from the binary data extractor;

determine, by the data iterator, a second data portion identifier;

provide, by the data iterator, the second data portion identifier to the binary data extractor;

receive, by the binary data extractor, the second data portion identifier;

extract, by the binary data extractor, a second portion of the binary data item based on the second data portion identifier;

output, by the binary data extractor, the second portion of the binary data item;

receive, by the data iterator, the second portion of the binary data item from the binary data extractor;

output, by the data iterator, the second portion of the binary data item;

receive, by a data converter, the second portion of the binary data item, from the data iterator; and

transform by the data converter, based on a data format specification, the at least the second portion of the binary data item for processing by one or more components of a data pipeline system.

14. A method, performed by one or more processors, comprising:

receiving a plurality of landmark location indicators;

receiving a first dataset comprising a time series of location indicators;

determining, for each of the time series of location indicators, a respective closest landmark location indicator of the plurality of landmark location indicators wherein determining, for each of the time series of location indicators, a respective closest landmark location indicator of the plurality of landmark location indicators comprises:

determining a first closest landmark location indicator for a first location indicator in the time series of location indicators;

determining a second closest landmark location indicator for a second location indicator in the time series of location indicators by:

determining a distance based on the second location indicator and the first closest landmark location indicator;

calculating a bounding area based on the distance; and

determining the second closest landmark location indicator based on the bounding area; and

obtaining, based on the first dataset, a second dataset comprising the respective closest landmark location indicator for each of the time series of location indicators.

15. The method of claim 14 , wherein determining the second closest landmark location indicator based on the bounding area comprises selecting a subset of the plurality of landmark location indicators based on the bounding area.

16. The method of claim 15 , wherein determining the second closest landmark location indicator based on the bounding area further comprises:

in response to the subset consisting of one landmark location indicator, determining the second closest landmark location indicator to be the same as the first closest landmark location indicator.

17. The method of claim 15 , wherein determining the second closest landmark location indicator based on the bounding area further comprises, in response to the subset comprising multiple landmark location indicators:

for each of the multiple landmark location indicators, determining a respective distance based on the respective landmark location indicator and the second location indicator; and

determining the second closest landmark location indicator to be the landmark location indicator for which the respective distance is smallest.

18. The method of claim 14 , wherein access to the first dataset is restricted to a first one or more users.

19. A computer program, optionally stored on a non-transitory computer readable medium, which, when executed by one or more processors of a data processing apparatus cause the data processing apparatus to:

provide, by a data iterator, a first data portion identifier to a binary data extractor;

receive, by the binary data extractor, the first data portion identifier;

extract, by the binary data extractor, a first portion of a binary data item based on the first data portion identifier;

output, by the binary data extractor, the first portion of the binary data item;

receive, by the data iterator, the first portion of the binary data item from the binary data extractor;

determine, by the data iterator, a second data portion identifier;

provide, by the data iterator, the second data portion identifier to the binary data extractor;

receive, by the binary data extractor, the second data portion identifier;

extract, by the binary data extractor, a second portion of the binary data item based on the second data portion identifier;

output, by the binary data extractor, the second portion of the binary data item;

receive, by the data iterator, the second portion of the binary data item from the binary data extractor;

output, by the data iterator, the second portion of the binary data item;

receive, by a data converter, the second portion of the binary data item, from the data iterator; and

transform by the data converter, based on a data format specification, the at least the second portion of the binary data item for processing by one or more components of a data pipeline system.