Data communication
There is described a method for communicating data, the method comprising: receiving an incomplete data stream, wherein the incomplete data stream comprises a plurality of sequences of data points having respective values and a plurality of sequences of missing data points; receiving a missing data model; determining values for each of the plurality of sequences of missing data points, comprising: selecting a sequence of missing data points that has not previously been processed, wherein the sequence of missing data points to be processed is selected as a smallest sequence of missing data points of the plurality of sequences of missing data points that have not previously been processed; processing the incomplete data stream to determine values for the selected sequence of missing data points based upon the missing data model; updating the incomplete data stream to include the determined values for the selected sequence of missing data points; and wherein values for subsequent sequences of missing data points are generated based upon the updated data stream; and outputting a corrected data stream comprising the determined values for each of the plurality of sequences of missing data points.
1. A method of communicating data comprising:
receiving a first data stream, the first data stream being an incomplete data stream, wherein the incomplete data stream comprises a plurality of sequences of data points having respective values and a plurality of sequences of missing data points;
receiving a missing data model generated based on one or more second data streams;
determining values for each of the plurality of sequences of missing data points, comprising:
selecting a sequence of missing data points from the plurality of sequences of missing data points, wherein the selected sequence of missing data points is a smallest sequence of missing data points of the plurality of sequences of missing data points that has not previously been processed by the missing data model;
determining estimated values for the selected sequence of missing data points using the missing data model;
updating the incomplete data stream to include the determined estimated values for the selected sequence of missing data points; and
wherein values for subsequent sequences of missing data points from the plurality of sequences of missing data points are generated based upon the updated incomplete data stream; and
outputting a corrected data stream comprising the determined estimated values for each of the plurality of sequences of missing data points.
2. The method of claim 1 , wherein determining the estimated values for the selected sequence of missing data points using the missing data model further comprises:
selecting a subset of the incomplete data stream to input to the missing data model based upon the selected sequence of missing data points.
3. The method of claim 2 , wherein the subset is centered on the selected sequence of missing data points.
4. The method of claim 2 , wherein data points of the subset of the incomplete data stream are weighted based upon a distance between a respective data point and the selected sequence of missing data points.
5. The method of claim 1 , wherein the one or more second data streams comprise a plurality of sequences of data points having respective values and a plurality of sequences of missing data points.
6. The method of claim 1 , wherein the missing data model is generated based upon optimising an objective function comprising a relationship between the plurality of sequences of data points having respective values in the incomplete data stream and the one or more second data streams.
7. The method of claim 6 , wherein the relationship is based upon predicting the plurality of sequences of data points having respective values in the incomplete data stream based upon the one or more second data streams.
8. The method of claim 1 , wherein the missing data model is based upon a linear model.
9. The method of claim 1 , where in the missing data model is based upon a regularized model.
10. The method of claim 1 , wherein processing the data stream to determine estimated values for the selected sequence of missing data points further comprises:
verifying the determined estimated values for the selected sequence of missing data points.
11. The method of claim 10 , wherein the verification is based upon a comparison of a magnitude of differences in the determined estimated values to a magnitude of differences in non-missing data points in the incomplete data stream.
12. The method of claim 10 , wherein the verification is based upon a comparison of a frequency of change in values in the determined estimated values to a frequency of change in values of non-missing data points in the incomplete data stream.
13. A computer apparatus for communicating data comprising:
a memory storing processor readable instructions;
a processor arranged to read and execute instructions in said memory; wherein said processor readable instructions comprise instructions arranged to control the computer to carry out a method according to claim 1 .
14. A non-transitory computer readable medium carrying computer readable instructions configured to cause a computer to carry out a method according to claim 1 .