Data pipeline error detection

1. A method comprising:

accessing from a data pipeline, first data offset information for a first data origin and second data offset information for a second data origin;

bisecting the first data origin using the first data offset information, the bisecting including generating bisected first data offset information;

processing the data pipeline with the bisected first data offset information and the second data offset information; and

based on the processing of the data pipeline, determining a data pipeline status.

2. The method of claim 1 , further comprising:

receiving an identification from a data pipeline, wherein the identification of the data pipeline indicates a data pipeline failure.

3. The method of claim 1 , wherein the first data offset information represents data from the first data origin that was most recently processed by the data pipeline, the method further comprising:

storing the first offset information in a permanent storage system.

4. The method of claim 1 , wherein the bisected first data offset information comprises a first set of bisected first data offset information and a second set of bisected first data offset information.

5. The method of claim 4 , wherein the data pipeline status is a partial failure, the method further comprising:

bisecting the first set of bisected first data information, the bisecting including generated updated bisected first data offset information; and

performing a second processing of the data pipeline using the updated bisected first data offset information and the second data offset information.

6. The method of claim 5 , further comprising:

in response to the second processing the data pipeline, receiving a second notification indicating the data pipeline status; and

based on the data pipeline status, causing presentation of the second notification on a graphical user interface of a computing device.

7. The method of claim 4 , wherein the data pipeline status is a complete failure, the method further comprising:

bisecting the second data origin using the second data offset information, the bisecting including generating bisected second data offset information; and

performing a second processing of the data pipeline using the first data offset information and the bisected second data offset information.

8. The method of claim 7 , further comprising:

in response to the second processing the data pipeline, receiving a second notification indicating the data pipeline status; and

based on the data pipeline status, causing presentation of the second notification on a graphical user interface of a computing device.

9. The method of claim 7 further comprising:

generating an indication of data from the first data origin that caused the failure; and

causing presentation of the indication on a graphical user interface of a computing device.

10. A computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to perform operations comprising:

accessing from a data pipeline, first data offset information for a first data origin and second data offset information for a second data origin;

bisecting the first data origin using the first data offset information, the bisecting including generating bisected first data offset information;

processing the data pipeline with the bisected first data offset information and the second data offset information; and

based on the processing of the data pipeline, determining a data pipeline status.

11. The computing apparatus of claim 10 , further comprising:

causing presentation of a notification indicating the data pipeline status on a graphical user interface of a computing device.

12. The computing apparatus of claim 10 , wherein the first data offset information represents data from the first data origin that was most recently processed by the data pipeline, the operations further comprising:

storing the first offset information in a permanent storage system.

13. The computing apparatus of claim 10 , wherein the bisected first data offset information comprises a first set of bisected first data offset information and a second set of bisected first data offset information.

14. The computing apparatus of claim 13 , wherein the data pipeline status is a partial failure, the operations further comprising:

bisecting the first set of bisected first data information, the bisecting including generated updated bisected first data offset information; and

performing a second processing of the data pipeline using the updated bisected first data offset information and the second data offset information.

15. The computing apparatus of claim 14 , the operations further comprising:

in response to the second processing the data pipeline, receiving a second notification indicating the data pipeline status; and

based on the data pipeline status, causing presentation of the second notification on a graphical user interface of a computing device.

16. The computing apparatus of claim 13 , wherein the data pipeline status is a complete failure, the operations further comprising:

bisecting the second data origin using the second data offset information, the bisecting including generating bisected second data offset information; and

performing a second processing of the data pipeline using the first data offset information and the bisected second data offset information.

17. The computing apparatus of claim 16 , the operations further comprising:

generating an indication of data from the first data origin that caused the failure; and

causing presentation of the indication on a graphical user interface of a computing device.

18. The computing apparatus of claim 10 , wherein the first data origin is a first data source and the second data origin is a second data source.

19. A non-transitory computer-storage medium storing instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

accessing from a data pipeline, first data offset information for a first data origin and second data offset information for a second data origin;

bisecting the first data origin using the first data offset information, the bisecting including generating bisected first data offset information;

processing the data pipeline with the bisected first data offset information and the second data offset information; and

based on the processing of the data pipeline, determining a data pipeline status.

20. The non-transitory computer-storage medium of claim 19 , further comprising:

receiving an identification from a data pipeline, wherein the identification of the data pipeline indicates a data pipeline failure.