Method, system, and storage medium for collecting SNMP bandwidth data

1. A method for collecting SNMP bandwidth data from a network device via a data collection system, comprising:

simultaneously collecting a plurality of data samples from a port on said network device by a plurality of collecting servers, said plurality of collecting servers comprising a master server and at least one slave server;

producing a master text file by said master server and producing a slave text file from said at least one slave server; and

performing data computation activities including:

generating a clean data file utilizing said master text file and said slave text file; and

computing delta values for data samples in said clean text file;

wherein said data collection system transfers control of said data computation activities to said slave server upon a failure detected at said master server.

2. The method of claim 1 , wherein said master text file and said slave text file each comprise:

a time on a collecting server wherein each of said plurality of data samples was gathered;

an identification of said network device;

a number of bytes received on said port for each of said plurality of data samples;

a time on said network device at which said number of bytes was received;

a number of bytes transmitted on said port for each of said plurality of data samples; and

a time on said network device at which said number of bytes was transmitted.

3. The method of claim 1 , wherein said clean data file is generated by:

comparing said master text file and said slave text file; and

filling in data missing from said master text file using data in said slave text file.

4. The method of claim 3 , wherein said filling in data missing from said master text file comprises:

sorting data in said master text file and said slave text file by port;

sorting data samples for said port by time; and

for each of said plurality of data samples:

adding a designated interval of time to a time on said network device resulting in a target network device time, said time on said network device associated with a data sample;

examining corresponding data samples in said master text file and said slave text file;

selecting one of said corresponding data samples from said master text file and said slave text file that most closely matches said target network device time; and

storing closely matching data sample in a clean data file.

5. The method of claim 1 , wherein said producing a master text file by said master server and producing a slave text file by said at least one slave server further comprises:

scanning a recently completed master text file for a final sampled value and recording said final sampled value in a subsequent text file for said master server; and

discarding a final sampled value if its time of collection exceeds a designated time limit.

6. The method of claim 1 , wherein said producing a slave text file by said at least one slave server further comprises:

scanning a recently completed slave server text file for a final sampled value and recording said final sampled value in a subsequent text file for said at least one slave server; and

discarding a final sampled value if its time of collection exceeds a designated time limit.

7. The method of claim 1 , wherein said computing delta values for data samples in said clean text file comprises:

for each data sample in said clean text file:

subtracting a number of bytes for a data sample from a number of bytes for a preceding data sample resulting in a delta value; and

storing delta values in a database table.

8. The method of claim 1 , wherein said transferring control of data computation activities includes performing for each hourly run:

querying a control table by said master server and said at least one slave server operable for determining which server assumes control over said data computation activities;

querying said control table by said at least one slave server operable for determining whether said master server updated a current hourly run for a given text file;

if said current hourly run has not been updated upon reaching a predetermined time limit, updating said control table with an identification of a slave server and a current hourly run by said slave server; and

performing said data computation activities by said slave server;

wherein said updating said control table is operable for notifying said master server that control is transferred to said slave server.

9. A storage medium encoded with machine-readable computer program code for collecting SNMP bandwidth data from a network device via a data collection system, the storage medium including instructions for causing a computer to implement a method, comprising:

simultaneously collecting a plurality of data samples from a port on said network device by a plurality of collecting servers, said plurality of collecting servers comprising a master server and at least one slave server;

producing a master text file by said master server and producing a slave text file from said at least one slave server; and

performing data computation activities including:

generating a clean data file utilizing said master text file and said slave text file; and

computing delta values for data samples in said clean text file;

wherein said data collection system transfers control of said data computation activities to said slave server upon a failure detected at said master server.

10. The storage medium of claim 9 , wherein said master text file and said slave text file each comprise:

a time on a collecting server wherein each of said plurality of data samples was gathered;

an identification of said network device;

a number of bytes received on said port for each of said plurality of data samples;

a time on said network device at which said number of bytes was received;

a number of bytes transmitted on said port for each of said plurality of data samples; and

a time on said network device at which said number of bytes was transmitted.

11. The storage medium of claim 9 , wherein said clean data file is generated by:

comparing said master text file and said slave text file; and

filling in data missing from said master text file using data in said slave text file.

12. The storage medium of claim 11 , wherein said filling in data missing from said master text file comprises:

sorting data in said master text file and said slave text file by port;

sorting data samples for said port by time; and

for each of said plurality of data samples:

adding a designated interval of time to a time on said network device resulting in a target network device time, said time on said network device associated with a data sample;

examining corresponding data samples in said master text file and said slave text file;

selecting one of said corresponding data samples from said master text file and said slave text file that most closely matches said target network device time; and

storing closely matching data sample in a file.

13. The storage medium of claim 9 , wherein said producing a master text file by said master server and producing a slave text file by said at least one slave server further comprises:

scanning a recently completed master text file for a final sampled value and recording said final sampled value in a subsequent text file for said master server; and

discarding a final sampled value if its time of collection exceeds a designated time limit.

14. The storage medium of claim 9 , wherein said producing a slave text file by said at least one slave server further comprises:

scanning a recently completed slave server text file for a final sampled value and recording said final sampled value in a subsequent text file for said at least one slave server; and

discarding a final sampled value if its time of collection exceeds a designated time limit.

15. The storage medium of claim 9 , wherein said computing delta values for data samples in said clean text file comprises:

for each data sample in said clean text file:

subtracting a number of bytes for a data sample from a number of bytes for a preceding data sample resulting in a delta value; and

storing delta values in a database table.

16. The storage medium of claim 9 , wherein said transferring control of data computation activities includes performing for each hourly run:

querying a control table by said master server and said at least one slave server operable for determining which server assumes control over said data computation activities;

querying said control table by said at least one slave server operable for determining whether said master server updated a current hourly run for a given text file;

if said current hourly run has not been updated upon reaching a predetermined time limit, updating said control table with an identification of a slave server and a current hourly run by said slave server; and

performing duties of said master server by said slave server;

wherein said updating said control table is operable for notifying said master server that control is transferred to said slave server.

17. A system for collecting SNMP bandwidth data via a data collection system, said SNMP bandwidth data collected from a network device, the data collection system comprising:

a plurality of collecting servers comprising:

a master server in communication with said network device, said master server storing a master text file, a clean data file, and a lock file; and

at least one slave server in communication with said network device, said at least one slave server storing a slave text file and a lock file;

a database in communication with said master server and said at least one slave server, said database storing:

a control table storing an identifier of said master server and an update of a most recent hourly run; and

computed delta values for said clean data file;

wherein said master server and said at least one slave server perform redundant periodic data sampling of at least one port for said network device resulting in said master text file and said slave text file and wherein further said master server stores a copy of said slave text file and said at least one slave server stores a copy of said master text file.

18. The system of claim 17 , wherein said master text file and said slave text file each store:

a time on a respective collecting server wherein each of said plurality of data samples was gathered;

an identification of said network device;

a number of bytes received on said port for each of said plurality of data samples;

a time on said network device at which said number of bytes was received;

a number of bytes transmitted on said port for each of said plurality of data samples; and

a time on said network device at which said number of bytes was transmitted;

wherein said master server generates said clean data file utilizing said master text file and said slave text file.

19. The system of claim 17 , further comprising a last raw value table stored in said database, said last raw value table storing a final sampled data point for a text file operable for concatenating successive text files.

20. The system of claim 17 , wherein said data collection system transfers control of data computation activities to said slave server upon a failure detected at said master server, said data computation activities comprising:

sorting data in said master text file and said slave text file by port;

sorting data samples for said port by time; and

for each of said plurality of data samples:

adding a designated interval of time to a time on said network device resulting in a target network device time, said time on said network device associated with a data sample;

examining corresponding data samples in said master text file and said slave text file;

selecting one of said corresponding data samples from said master text file and said slave text file that most closely matches said target network device time; and

storing closely matching data sample in a clean data file.

21. The system of claim 20 , wherein said data computation activities further comprises:

for each data sample in said clean text file:

subtracting a number of bytes for a data sample from a number of bytes for a preceding data sample resulting in a delta value; and

storing delta values in said database.

22. The system of claim 21 , wherein said control of said data computation activities remains with said slave server until a failure is detected at said slave server.

23. The system of claim 22 , wherein said control of data computation activities is managed by said lock file and said control table via said data collection system;

wherein said lock file serializes hourly runs for text files and prevents a subsequent hourly run from executing before a current hourly run has completed.

24. The system of claim 17 , further comprising an alert system operable for monitoring data sampling and said data computation activities, said alert system issues error messages to a network administrator client system in communication with said data collection system upon detecting a problem.