Event correlation using data having different time resolutions

1. A method comprising:

storing a timing of a reference event, the reference event occurring during a frame time of an image frame from a plurality of imaging events, the frame time being an exposure time and a readout time for an imaging source, each imaging event having at least one set of image frames, each set of image frames being recorded simultaneously, each set of image frames having a plurality of image frames;

receiving at least one channel of image data for each imaging event from the imaging source, where one channel of image data corresponds to one set of image frames, respectively;

calculating, for each imaging event, image values for a representative frame set, the representative frame set being determined based on the at least one set of image frames, where one image value is calculated for each image frame of the representative frame set, each image value being based on a characteristic of a plurality of pixels within a respective image frame of the representative frame set, the representative frame set having the same number of image frames as the at least one set of image frames, the characteristic being determined from the at least one channel of image data;

assigning, for each imaging event, the calculated image value of a respective image frame as a trace value for each trace point during the exposure time of the respective image frame and a portion of the readout time before the exposure time of the respective image frame and a portion of the readout time following the exposure time of the respective image frame to generate an intensity trace for a respective imaging event, the assigning being performed for each image frame in the representative frame set for the respective imaging event;

shifting in time, the intensity trace for each respective imaging event to align the timing of the reference event for each imaging event of the plurality of imaging events;

averaging, trace values of trace points aligned in time from each shifted intensity trace to generate an average continuous intensity trace over time; and

determining, a timing of an event based on the average continuous intensity trace.

2. The method of claim 1 , further comprising:

comparing the timing of the event with the timing of the reference event.

3. The method of claim 1 , wherein the timing of the event is determined based on an intensity change in the average continuous intensity trace.

4. The method of claim 3 , wherein the timing of the intensity change is a time where an intensity of the average continuous intensity trace is half maximum intensity.

5. The method of claim 1 , wherein the portion of the readout time before exposure time of the respective image frame and the portion of the readout time following the exposure time of the respective image frame is half the readout time.

6. The method of claim 1 , wherein the reference event is a single vesicle release event.

7. The method of claim 1 , wherein a number of pixels and a location of pixels included in the plurality of pixels is determined based on a location of the single vesicle release event.

8. The method of claim 3 , wherein the intensity change indicates a fluorescence resonance energy transfer (FRET).

9. The method of claim 8 , wherein the at least one channel of image data includes two channels of image data, and wherein the at least one set of image frames includes two sets of image frames, and wherein the representative frame set is determined based on a ratio of the two sets of image frames.

10. The method of claim 9 , wherein one of the two channels of image data has a wavelength band of 465-495 nm and one of the two channels of image data has a wavelength band of 520-550 nm.

11. The method of claim 1 , further comprising: fitting a step response function to the average continuous intensity trace, and determining the timing of the event based on trace value for the step response function.

12. The method of claim 1 , further comprising: determining the timing of the reference event.

13. A device comprising:

a storage device configured to store at least one channel of image data for a plurality of imaging events received from an imaging source, each imaging event having at least one set of image frames, each set of image frames being recorded simultaneously, each set of image frames having plurality of image frames, and a timing of a reference event, the reference event occurring during a frame time of an image frame from the plurality of imaging events, the frame time being an exposure time and a readout time for the imaging source;

a processor comprising hardware configured to:

calculate, for each imaging event, image values for a representative frame set, the representative frame set being determined based on the at least one set of image frames, where one image value is calculated for each image frame of the representative frame set, each image value being based on a characteristic of a plurality of pixels within a respective image frame of the representative frame set, the representative frame set having the same number of image frames as the at least one set of image frames, the characteristic being determined from the at least one channel of image data;

assign, for each imaging event, the calculated image value of a respective image frame as a trace value for each trace point during the exposure time of the respective image frame and a portion of the readout time before exposure time of the respective image frame and a portion of the readout time following the exposure time of the respective image frame to generate an intensity trace for a respective imaging event, the assigning being performed for each image frame in the representative frame set for the respective imaging event,

shift, in time, the intensity trace for each respective imaging event to align the timing of the reference event for each imaging event of the plurality of imaging events,

average, trace values of trace points aligned in time from each shifted intensity trace to generate an average continuous intensity trace over time; and

determine, a timing of an event based on the average continuous intensity trace;

a display configured to display the average continuous intensity trace.

14. The device of claim 13 , wherein the timing of the event is based on a timing of an intensity change in the average continuous intensity trace.

15. The device of claim 14 , wherein the timing of the intensity change is a time where an intensity of the average continuous intensity trace is half maximum intensity.

16. The device of claim 13 , wherein the processor is further configured to compare a timing of the event with the timing of the reference event.

17. The device of claim 13 , wherein the imaging source is an electron multiplying charge coupled device camera.

18. The device of claim 13 , wherein the reference event is a single vesicle release event.

19. The device of claim 18 , further comprising an interface configured to receive current values from an electrode array, the electrode array comprising at least three electrodes, wherein the storage device is further configured to store the received current values, the current values being simultaneously recorded during each imaging event.

20. The device of claim 19 , wherein the processor is further configured to determine a timing of a single vesicle release event from a magnitude of the current values received from each of the at least three electrodes in the electrode array.

21. The device of claim 20 , wherein the processor is further configured to compare the timing of the single vesicle release event with the timing of the event.

22. The device of claim 15 , wherein the intensity change indicates a fluorescence resonance energy transfer (FRET).

23. The device of claim 22 , wherein the at least one channel of image data includes two channels of image data, and wherein the at least one set of image frames includes two sets of image frames, and wherein the representative frame set is determined based on a ratio of the two sets of image frames.

24. The device of claim 23 , wherein one of the two channels of image data has a wavelength band of 465-495 nm and one of the two channels of image data has a wavelength band of 520-550 nm.

25. A computer readable storage device having a program, which when executed by a computer, causes the computer to execute a method of:

calculating, for each imaging event of a plurality of imaging events, where each imaging event has at least one set of image frames, each set of image frames being recorded simultaneously and each set of image frames having plurality of image frames, image values for a representative frame set, the representative frame set being determined based on the at least one set of image frames, where one image value is calculated for each image frame of the representative frame set, each image value being based on a characteristic of a plurality of pixels within a respective image frame of the representative frame set, the representative frame set having the same number of frames as the at least one set of image frames, the characteristic being determined from the at least one channel of image data;

assigning, for each imaging event of the plurality of imaging events, the calculated image value of a respective image frame as a trace value for each trace point during the exposure time of the respective image frame and a portion of the readout time before exposure time of the respective image frame and a portion of the readout time following the exposure time of the respective image frame to generate an intensity trace for a respective imaging event, the assigning being performed for each image frame in the representative frame set for the respective imaging event;

shifting in time, the intensity trace for each respective imaging event to align the timing of a reference event for each imaging event of the plurality of imaging events, the reference event occurring during a frame time of an image frame in each of the plurality of imaging events, the frame time being an exposure time and a readout time for an imaging source;

averaging, trace values of trace points aligned in time from each shifted intensity trace to generate an average continuous intensity trace over time; and

determining, a timing of an event based on the average continuous intensity trace.