INROUTE STREAM ERROR RATE SHEDDING
Techniques are described herein for intelligent shedding of inroutes determined to have high burst error rates. Embodiments operate in context of a multi-spot beam satellite communication system that uses time-division multiple access (TDMA) communication protocols and inroute groups. Burst error rates of inroutes can be monitored over time to identify bad inroutes. Each bad inroute can be shut down and quarantined. After a predetermined amount of time, one or more selected terminals can be redistributed to the bad inroutes, and the burst error rates of those inroutes can be monitored again to determine whether those inroutes should remain quarantined. If an inroute is no longer bad, it can be removed from quarantine and returned to an inroute group list. If the inroute is still bad, it can be periodically re-checked until it ultimately improves, is permanently quarantined, etc.
1 . A method for automatic inroute shedding in a satellite communication system, the method comprising:
monitoring monitor-time burst error rates of a plurality of inroutes of the satellite communication system;
identifying, based on the monitoring, a set of bad inroutes as those of the plurality of inroutes that have burst error rates exceeding a predetermined threshold; and
for each bad inroute of the set of bad inroutes:
shutting down the bad inroute by migrating an impacted set of user terminals from the bad inroute to one or more others of the plurality of inroutes that are not part of the set of bad inroutes, and quarantining the bad inroute, wherein the impacted set of user terminals is a subset of a plurality of user terminals of the satellite communication system that is serviced by the bad inroute;
testing the bad inroute, subsequent to elapsing of a predefined shutdown time, by redistributing a selected user terminal to the bad inroute and monitoring a test-time burst error rate of the bad inroute;
determining, based on the testing, that the test-time burst error rate of the bad inroute no longer exceeds the predetermined threshold; and
restoring the bad inroute, based on the determining, by removing the bad inroute from quarantine.
2 . The method of claim 1 , further comprising:
generating and outputting one or more operator notifications responsive to and based on the identifying.
3 . The method of claim 1 , wherein the determining comprises:
determining, based on the testing, whether the test-time burst error rate of the bad inroute no longer exceeds the predetermined threshold; and
responsive to determining that the test-time burst error rate of the bad inroute still exceeds the predetermined threshold, migrating the selected user terminal away from the bad inroute and repeating the testing step and the determining step,
wherein the restoring the bad inroute is performed only in response to the determining that the test-time burst error rate of the bad inroute no longer exceeds the predetermined threshold.
4 . The method of claim 3 , wherein the repeating comprises increasing the predefined shutdown time so that repeating of the testing step is subsequent to elapsing of a larger amount of time.
5 . The method of claim 3 , wherein the repeating further comprises:
detecting, subsequent to the increasing the predefined shutdown time and prior to repeating the testing step and the determining step, whether a predefined iteration threshold has been reached; and
adding the bad inroute to a persistent quarantine list responsive to detecting that the predefined iteration threshold has been reached,
wherein the testing step and the determining step are repeated only responsive to detecting that the predefined iteration threshold has not been reached.
6 . The method of claim 1 , wherein the testing comprises selecting the selected user terminal to redistribute to the bad inroute based on determining that traffic from the selected user terminal presently exhibits a low data error rate and/or that the selected user terminal is presently carrying a low traffic level.
7 . The method of claim 1 , wherein:
the monitoring the monitor-time burst error rates comprises:
monitoring one or more long-term burst error rates over a timeframe associated with the monitor-time, each long-term burst error rate representing a prevalence of burst errors detected during elapsing of a long timer; and
monitoring a plurality of short-term burst error rates over the timeframe associated with the monitor-time, each short-term burst error rate representing a prevalence of burst errors detected during elapsing of a short timer, wherein each iteration of the long timer encompasses a plurality of iterations of the short timer; and
the identifying comprises identifying the set of bad inroutes as those of the plurality of inroutes that have burst error rates exceeding a predetermined threshold according to both the long-term burst error rates and the short-term burst error rates.
8 . The method of claim 1 , wherein:
each of the plurality of inroutes is part of one of a plurality of inroute groups, such that each bad inroute is part of a corresponding inroute group; and
the migrating the impacted set of user terminals is from the bad inroute to one or more others of the plurality of inroutes that are not part of the set of bad inroutes and are part of the corresponding inroute group.
9 . The method of claim 8 , wherein the identifying comprises:
comparing the burst error rate of the bad inroute to burst error rates of other inroutes in the corresponding inroute group; and
identifying the set of bad inroutes as those of the plurality of inroutes that have burst error rates exceeding a predetermined threshold relative to the burst error rates of other inroutes in the corresponding inroute group.
10 . The method of claim 1 , wherein:
the quarantining the bad inroute comprises adding the bad inroute to an inroute quarantine list and removing the bad inroute from an inroute group list; and
the restoring the bad inroute comprises removing the bad inroute from the inroute quarantine list and adding the bad inroute back to the inroute group list.
11 . The method of claim 1 , wherein the monitoring the monitor-time burst error rates of the plurality of inroutes comprises determining a prevalence of cyclic redundancy check (CRC) code errors and/or missed transmission errors.
12 . An inroute group manager (IGM) of a ground network of a satellite communication system, the IGM comprising:
a set of processors; and
a non-transitory memory having processor-readable instructions stored thereon, which, when executed, cause the set of processors to perform steps comprising:
monitoring monitor-time burst error rates of a plurality of inroutes;
identifying, based on the monitoring, a set of bad inroutes as those of the plurality of inroutes that have burst error rates exceeding a predetermined threshold; and
for each bad inroute of the set of bad inroutes:
shutting down the bad inroute by migrating an impacted set of user terminals from the bad inroute to one or more others of the plurality of inroutes that are not part of the set of bad inroutes, and quarantining the bad inroute, wherein the impacted set of user terminals is a subset of a plurality of user terminals of the satellite communication system that is serviced by the bad inroute;
testing the bad inroute, subsequent to elapsing of a predefined shutdown time, by redistributing a selected user terminal to the bad inroute and monitoring a test-time burst error rate of the bad inroute;
determining, based on the testing, that the test-time burst error rate of the bad inroute no longer exceeds the predetermined threshold; and
restoring the bad inroute, based on the determining, by removing the bad inroute from quarantine.
13 . The IGM of claim 12 , wherein the steps further comprise:
generating and outputting one or more operator notifications responsive to and based on the identifying.
14 . The IGM of claim 12 , wherein the determining step further comprises:
determining, based on the testing, whether the test-time burst error rate of the bad inroute no longer exceeds the predetermined threshold; and
responsive to determining that the test-time burst error rate of the bad inroute still exceeds the predetermined threshold, migrating the selected user terminal away from the bad inroute and repeating the testing step and the determining step,
wherein the restoring the bad inroute is performed only in response to the determining that the test-time burst error rate of the bad inroute no longer exceeds the predetermined threshold.
15 . The IGM of claim 14 , wherein the repeating step comprises increasing the predefined shutdown time so that repeating of the testing step is subsequent to elapsing of a larger amount of time.
16 . The IGM of claim 14 , wherein the repeating step further comprises:
detecting, subsequent to the increasing the predefined shutdown time and prior to repeating the testing step and the determining step, whether a predefined iteration threshold has been reached; and
adding the bad inroute to a persistent quarantine list responsive to detecting that the predefined iteration threshold has been reached,
wherein the testing step and the determining step are repeated only responsive to detecting that the predefined iteration threshold has not been reached.
17 . The IGM of claim 12 , wherein the testing step comprises selecting the selected user terminal to redistribute to the bad inroute based on determining that traffic from the selected user terminal presently exhibits a low data error rate and/or that the selected user terminal is presently carrying a low traffic level.
18 . The IGM of claim 12 , wherein:
the monitoring step comprises:
monitoring one or more long-term burst error rates over a timeframe associated with the monitor-time, each long-term burst error rate representing a prevalence of burst errors detected during elapsing of a long timer; and
monitoring a plurality of short-term burst error rates over the timeframe associated with the monitor-time, each short-term burst error rate representing a prevalence of burst errors detected during elapsing of a short timer, wherein each iteration of the long timer encompasses a plurality of iterations of the short timer; and
the identifying step comprises identifying the set of bad inroutes as those of the plurality of inroutes that have burst error rates exceeding a predetermined threshold according to both the long-term burst error rates and the short-term burst error rates.
19 . The IGM of claim 12 , wherein:
each of the plurality of inroutes is part of one of a plurality of inroute groups, such that each bad inroute is part of a corresponding inroute group; and
the migrating the impacted set of user terminals is from the bad inroute to one or more others of the plurality of inroutes that are not part of the set of bad inroutes and are part of the corresponding inroute group.
20 . The IGM of claim 19 , wherein the identifying comprises:
comparing the burst error rate of the bad inroute to burst error rates of other inroutes in the corresponding inroute group; and
identifying the set of bad inroutes as those of the plurality of inroutes that have burst error rates exceeding a predetermined threshold relative to the burst error rates of other inroutes in the corresponding inroute group.