Correlation of video stream frame timestamps based on a system clock

1. A computerized display system for correlating timestamps of video frames from a capture device comprising:

at least one processor;

at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the at least one processor to:

receive, by a correlator application, a frame of a video stream from a capture device via a bus interface, the frame including a first start frame timestamp and a first end frame timestamp based on a device clock associated with the capture device;

predict, by the correlator application, based on a plurality of previously received frames of the video stream, a second start frame timestamp and a second end frame timestamp associated with the received frame, the second start frame timestamp and second end frame timestamp being based on a system clock associated with the display system;

calculate, by the correlator application, a skew value of the received frame based on a difference between the first start frame timestamp and the second start frame timestamp, and a difference between the first end frame timestamp and the second end frame timestamp;

determine that the calculated skew value exceeds a skew threshold;

based on the determining, correct, by the correlator application, the first start frame timestamp and the first end frame timestamp of the received frame to correlate to the second start frame timestamp and the second end frame timestamp; and

provide the corrected frame for display.

2. The computerized system of claim 1 , wherein predicting the second start frame timestamp includes maintaining a mapping of start frame timestamps of the plurality of previously received frames of the video stream.

3. The computerized system of claim 1 , wherein predicting the second start frame timestamp and the second end frame timestamp includes determining an average frame length of the plurality of previously received frames of the video stream, wherein the predicted second start frame timestamp and second end frame timestamp are separated by the determined average frame length.

4. The computerized system of claim 1 , wherein correcting the received frame includes clipping the received frame to fit the predicted second start frame timestamp and second end frame timestamp and re-timestamping the received frame with the predicted second start frame timestamp and second end frame timestamp.

5. The computerized system of claim 1 , wherein the at least one memory comprises computer program code, the at least one memory and the computer program code configured to, with the at least one processor, further cause the at least one processor to combine the received frame with the plurality of received frames of the video stream.

6. The computerized system of claim 1 , wherein the video stream is synchronized with another video stream; and

the corrected frame is synchronized with a matching frame of the other video stream based on at least one of the second start frame timestamp and the second end frame timestamp.

7. The computerized system of claim 6 , wherein the video stream is a visible light-based video stream and the other video stream is an infrared-based video stream.

8. The computerized system of claim 6 , wherein the video stream is a visible-light based video stream capturing a scene from a first angle and the other video stream is a visible-light based stream capturing the scene from a second angle.

9. The computerized system of claim 1 , wherein the video stream includes live video.

10. The computerized system of claim 1 , wherein the bus interface is a universal serial bus (USB) interface.

11. A computerized method for correlating timestamps of video frames from a capture device, the computerized method comprising:

receiving, by a correlator application, a frame of a video stream from a capture device via a bus interface, the frame including a first start frame timestamp and a first end frame timestamp based on a device clock associated with the capture device;

predicting, by the correlator application, based on a plurality of previously received frames of the video stream, a second start frame timestamp and a second end frame timestamp associated with the received frame, the second start frame timestamp and second end frame timestamp being based on a system clock associated with a computing device of the correlator application;

calculating, by the correlator application, a skew value of the received frame based on a difference between the first start frame timestamp and the second start frame timestamp, and a difference between the first end frame timestamp and the second end frame timestamp;

determining that the calculated skew value exceeds a skew threshold;

based on the determining, correcting, by the correlator application, the first start frame timestamp and the first end frame timestamp of the received frame to correlate to the second start frame timestamp and the second end frame timestamp;

combining the received frame with the plurality of received frames of the video stream; and

providing the video stream for display.

12. The computerized method of claim 11 , wherein predicting the second start frame timestamp includes maintaining a mapping of start frame timestamps of the plurality of previously received frames of the video stream.

13. The computerized method of claim 11 , wherein predicting the second start frame timestamp and the second end frame timestamp includes determining an average frame length of the plurality of previously received frames of the video stream, wherein the predicted second start frame timestamp and second end frame timestamp are separated by the determined average frame length.

14. The computerized method of claim 11 , wherein correcting the received frame includes clipping the received frame to fit the predicted second start frame timestamp and second end frame timestamp and re-timestamping the received frame with the predicted second start frame timestamp and second end frame timestamp.

15. The computerized method of claim 11 , wherein the video stream is synchronized with another video stream; and

the corrected frame is synchronized with a matching frame of the other video stream based on at least one of the second start frame timestamp and the second end frame timestamp.

16. The computerized method of claim 15 , wherein the video stream is a visible light-based video stream and the other video stream is a depth-based video stream.

17. The computerized method of claim 15 , wherein the video stream is a visible-light based video stream capturing a scene from a first angle and the other video stream is a visible-light based stream capturing the scene from a second angle.

18. One or more computer storage hardware devices having computer-executable instructions for correlating timestamps of video frames from capture devices that, upon execution by a processor, cause the processor to at least:

receive a frame of a video stream from a capture device via a bus interface, the frame including a first start frame timestamp and a first end frame timestamp based on a device clock associated with the capture device;

predict based on a plurality of previously received frames of the video stream, a second start frame timestamp and a second end frame timestamp associated with the received frame, the second start frame timestamp and second end frame timestamp being based on a system clock associated with the processor;

calculate a skew value of the received frame based on a difference between the first start frame timestamp and the second start frame timestamp, and a difference between the first end frame timestamp and the second end frame timestamp;

determine that the calculated skew value exceeds a skew threshold;

based on the determining, correct the first start frame timestamp and the first end frame timestamp of the received frame to correlate to the second start frame timestamp and the second end frame timestamp; and

provide the corrected frame for display.

19. The one or more computer storage hardware devices of claim 18 , wherein the computer-executable instructions, upon execution by a processor, further cause the processor to at least:

receive frames of a second video stream from a second capture device;

timestamp the received frames of the second video stream;

combine the video stream with the second video stream; and

wherein the corrected frame is synchronized with a matching frame of the second video stream.

20. The one or more computer storage hardware devices of claim 19 , wherein the video stream is a visible light-based video stream and the second video stream is a depth camera-based video stream; and

wherein combining the video stream with the second video stream creates a three-dimensional correlated video stream.