Fano-based information theoretic method (FBIT) for design and optimization of nonlinear systems
Methods are provided for identifying and quantifying information loss in a system due to uncertainty and analyzing the impact on the reliability of system performance. Models and methods join Fano's equality with the Data Processing Inequality in a Markovian channel construct in order to characterize information flow within a multi-component nonlinear system and allow the determination of risk and characterization of system performance upper bounds based on the information loss attributed to each component. The present disclosure additionally includes methods for estimating the sampling requirements and for relating sampling uncertainty to sensing uncertainty. The present disclosure further includes methods for determining the optimal design of components of a nonlinear system in order to minimize information loss, while maximizing information flow and mutual information.
1. A radar design and tuning system comprising:
a) a computer comprising an input/output controller, a random access memory unit, a hard drive memory unit, a unifying computer bus system and a central processing unit comprising an entropy computation processer, said input/output controller being configured to receive a digital signal from said instrumentation interface unit and transmit said digital signal to said central processing unit comprising said entropy computation processer; said central processing unit programmed to identify and characterize a component-level information loss in a nonlinear system configured to observe a target, the nonlinear system comprising a plurality of components with each component of the plurality having a control parameter, and the plurality of components of the nonlinear system being subject to at least one random input variable, via:
determining, for the nonlinear system, a true target state and a decision state, the true target and decision states being characterized in a Markovian channel model;
modeling settings of the at least one random input variable to create a plurality of distributions, wherein each distribution comprises values ranging from a theoretical maximum entropy to a theoretical minimum entropy;
calculating an entropy at each component of the plurality that is attributable to the respective control parameter wherein the entropy for each component of the plurality is directly related to an amount of uncertainty at each respective component of the plurality;
computing a mutual information between the true state and the decision state;
calculating the mutual information for each component of the plurality between the true target state and the respective component;
deriving, using Fano's inequality, a bound for a component probability of error for each component of the plurality using the mutual information associated with the respective component of the plurality;
determining the optimal design of components of the nonlinear system that minimize information loss, while maximizing information flow and mutual information; and
generating a design model for the nonlinear system to guide production;
b) an instrumentation interface unit, said instrumentation interface unit placing said computer in communication with a radar system; and
c) a communication device for communicating an output from said central processing unit to a human and/or an information storage device.
2. The radar design and tuning system of claim 1 , wherein said central processing unit is further programmed to:
compute a reliability of the probability of error of each component of the plurality and attributing a contribution of each random input variable to the reliability of the probability of error using real world samples of system uncertainty parameters based on information from a selected one or more of: (i) actual events; and (ii) experiments, the method comprising:
determine a real world statistical variation of the at least one random input variable based on the real world samples;
perform a Monte-Carlo simulation of the statistical variations to calculate a statistical distribution of the probability of error for each component of the plurality;
determine the reliability based on a standard deviation of the statistical distribution of the probability of error for each component of the plurality; and
correlating the contribution of each random input variable to the reliability.
3. The radar design and tuning system of claim 2 , wherein said instrumentation interface unit comprises a hardware-in-the-loop facility.
4. The radar design and tuning system of claim 2 , wherein said central processing unit is further programmed to determine a proper ensemble sample size while performing the Monte-Carlo simulation.
5. The radar design and tuning system of claim 2 , wherein said central processing unit is further programmed to:
determine an ensemble sampling requirement for the method of claim 1 , the method comprising:
determine a set of test criteria for a maximum allowable sampling uncertainty of the mutual information relative to the standard deviation of the reliability probability of error;
determine a sample ensemble size for the mutual information using a phase transition method; and
compute the reliability using a numerical simulation method on the sample ensemble size.
6. The radar design and tuning system of claim 5 , wherein the numerical simulation method comprises Monte Carlo modeling.
7. The radar design and tuning system of claim 2 , wherein performing the Monte-Carlo simulation further comprises:
introducing a plurality of random input variables.
8. The radar design and tuning system of claim 7 , wherein said central processing unit is further programmed to:
correlate the each random input variable of the plurality to the its respective contribution to the reliability, the standard deviation of the distributions of the probabilities of error at each component, or both.
9. A radar design and tuning system comprising:
a) a computer comprising an input/output controller, a random access memory unit, a hard drive memory unit, a unifying computer bus system and a central processing unit comprising an entropy computation processer, said input/output controller being configured to receive a digital signal from said instrumentation interface unit and transmit said digital signal to said central processing unit comprising said entropy computation processer; said central processing unit programmed to: characterize information in a nonlinear system comprising a component characterized by a design control parameter, and the nonlinear system having a decision state and a true state, said programming comprising:
computing a mutual information between the decision state and the true state of the nonlinear system;
modeling a range for the design control parameter;
calculating an entropy at the component;
calculating an information loss at the component that is attributable to the control parameter;
determining the optimal design of components of the nonlinear system that optimize the design control parameter that minimizes information loss while maximizing information flow and mutual information;
computing a reliability of the system by:
defining a random input variable;
iteratively calculating a statistical distribution of the probability of error by iteratively modeling random input variable settings;
determining a standard deviation for the component-level statistical distribution of the reliability probability of error; and
correlating the random input variable setting to the standard deviation;
b) an instrumentation interface unit, said instrumentation interface unit placing said computer in communication with a radar system; and
c) a communication device for communicating an output from said central processing unit to a human and/or an information storage device.
10. The radar design and tuning system of claim 9 , wherein said programming comprises iteratively calculating a statistical distribution of the probability of error performing the Monte-Carlo simulation further comprises:
introducing a plurality of random input variables.
11. The radar design and tuning system of claim 10 , wherein said programming further comprising:
correlating the each random input variable of the plurality to the its respective contribution to the reliability, the standard deviation of the distributions of the probabilities of error at each component, or both.