Server-side remediation for incoming sensor data
View Patent ↗Incoming sensor data from a data collection device may be received at the data processing platform that includes the plurality of data processing microservices. A data processing microservice of the data processing platform may detect that the incoming sensor data from the data collection device caused an error. As a result, the incoming sensor data may be queued in a faulty data cache of the data processing platform. Subsequently, at least one of the data processing microservice or the incoming sensor data stored in the faulty data cache may be modified such that the incoming sensor data is processed by the data processing microservice without the error. Following the processing, the incoming sensor data may be deleted from the faulty data queue of the data processing platform.
1 . One or more non-transitory computer-readable media of a data processing platform that includes a plurality of data processing microservices, the one or more non-transitory computer-readable media storing computer-executable instructions that upon execution cause one or more processors to perform acts comprising:
receiving a data file from a data collection device at the data processing platform, wherein the data processing platform includes the plurality of data processing microservices;
detecting that the data file is associated with an error at a data processing microservice of the data processing platform;
queueing the data file in a faulty data cache of the data processing platform;
modifying the data file stored in the faulty data cache; and
reprocessing the modified data file in the faulty data cache of the data processing platform.
2 . The one or more non-transitory computer-readable media of claim 1 , wherein the data file comprises incoming sensor data.
3 . The one or more non-transitory computer-readable media of claim 2 , wherein the acts further comprise modifying the data processing microservice.
4 . The one or more non-transitory computer-readable media of claim 1 , wherein the detecting that the data file is associated with the error at the data processing microservice of the data processing platform comprises detecting that the data file cannot be processed.
5 . The one or more non-transitory computer-readable media of claim 1 , wherein the detecting that the data file is associated with the error at the data processing microservice of the data processing platform comprises detecting the error when the data processing microservice fails to generate an output within a predetermined amount of time following a receipt of the data file.
6 . The one or more non-transitory computer-readable media of claim 1 , wherein the acts further comprise:
detecting whether the error caused a lock up of the data collection device; and
if the error caused a lock up of the data collection device, sending a command to the data collection device to resume a working operation state.
7 . The one or more non-transitory computer-readable media of claim 6 , wherein detecting whether the error caused the lock up of the data collection device comprises determining if the data collection device is retrying sending incoming sensor data to the data processing platform.
8 . The one or more non-transitory computer-readable media of claim 1 , wherein the acts further comprise reviewing information in the data file via one or more logic algorithms to determine modifications for the data file.
9 . The one or more non-transitory computer-readable media of claim 8 , wherein the one or more logic algorithms comprises at least one of a decision tree algorithm or a pattern matching algorithm.
10 . The one or more non-transitory computer-readable media of claim 1 , wherein the error at the data processing microservice of the data processing platform comprises information missing from the data file.
11 . The one or more non-transitory computer-readable media of claim 10 , wherein the missing information comprises missing time stamp information.
12 . The one or more non-transitory computer-readable media of claim 10 , wherein the missing information comprises missing geolocation information.
13 . The one or more non-transitory computer-readable media of claim 10 , wherein modifying the data file stored in the faulty data cache comprises obtaining additional information corresponding to the missing information from other data collection devices that are determined to be proximate the data collection device.
14 . The one or more non-transitory computer-readable media of claim 13 , wherein modifying the data file stored in the faulty data cache comprises integrating the additional information into the data file in the faulty data cache.
15 . The one or more non-transitory computer-readable media of claim 1 , wherein detecting that the data file is associated with the error at the data processing microservice of the data processing platform comprises detecting that one or more file parameters of the data file do not meet file parameter requirements of the data processing microservice.
16 . The one or more non-transitory computer-readable media of claim 15 , wherein modifying the data file stored in the faulty data cache comprises converting the data file stored in the faulty data cache into a converted data file that has compatible file parameters with the data processing microservice.
17 . The one or more non-transitory computer-readable media of claim 16 , wherein converting the data file stored in the faulty data cache into the converted data file that has the compatible file parameters with the data processing microservice comprises a least one of:
converting a file format of the data file stored in the faulty data cache into a compatible file format with the data processing microservice; or
splitting the data file stored in the faulty data cache into a plurality of data files.
18 . The one or more non-transitory computer-readable media of claim 1 , wherein modifying the data file stored in the faulty data cache comprises applying a data modification input by an administrator.
19 . A system, comprising:
one or more processors; and
memory including a plurality of computer-executable components that are executable by the one or more processors to perform a plurality of actions, the plurality of actions comprising:
receiving a data file from a data collection device at a data processing platform that includes a plurality of data processing microservices;
detecting that the data file is associated with an error at a data processing microservice of the data processing platform;
queueing the data file in a faulty data cache of the data processing platform;
modifying the data file stored in the faulty data cache; and
reprocessing the modified data file in the faulty data cache of the data processing platform.
20 . A computer-implemented method, comprising:
receiving a data file from a data collection device at a data processing platform that includes a plurality of data processing microservices;
detecting that the data file is associated with an error at a data processing microservice of the data processing platform;
queueing the data file in a faulty data cache of the data processing platform;
modifying the data file stored in the faulty data cache; and
reprocessing the modified data file in the faulty data cache of the data processing platform.