Nonlinear function approximation over high-dimensional domains
View Patent ↗An algorithm is disclosed for constructing nonlinear models from high-dimensional scattered data. The algorithm progresses iteratively adding a new basis function at each step to refine the model. The placement of the basis functions is driven by a statistical hypothesis test that reveals geometric structure when it fails. At each step the added function is fit to data contained in a spatio-temporally defined local region to determine the parameters, in particular, the scale of the local model. The proposed method requires no ad hoc parameters. Thus, the number of basis functions required for an accurate fit is determined automatically by the algorithm. The approach may be applied to problems including modeling data on manifolds and the prediction of financial time-series. The algorithm is presented in the context of radial basis functions but in principle can be employed with other methods for function approximation such as multi-layer perceptrons.
1. A method for modeling a relationship between first and second data collections, wherein for each member of the first data collection, there is a corresponding member of the second collection, comprising:
determining residuals between the first and second collections;
determining, using the residuals, a position of one of the members of the first data collection;
determining proximity data relative to a value (V) for the one member of the first data collection at the position;
determining a subcollection of the first collection, wherein each member of the subcollection has a value that is in proximity to the value V according to the proximity data; and
generating a basis function from the subcollection for obtaining a model of the relationship;
outputting model information for presentation to a user or a predetermined process for affecting or identifying a physical event, wherein the model information includes at least one of: (a) data indicative of a correspondence between the model and the relationship, (b) data indicative of a variance between the model and the relationship, (c) an extrapolation of the relationship, (d) an interpolation of the relationship, and (e) notification of the physical event and
wherein the basis function is modulated, or skewed, by a shape function.
2. The method of claim 1 , wherein the steps of claim 1 are iteratively performed for generating a model of the relationship, and with at least most iterations, a step of determining structure or information content in the residuals.
3. The method of claim 2 , wherein for each of at least most of the iterations, the corresponding instance of the subcollection is determined accordingly to an instance of the position determined as a function of a plurality of autocorrelation components.
4. The method of claim 2 , wherein the second collection has a dimensionality greater than one, and the step of determining the position includes, for each of at least most of the iterations, performing a test on the residuals for determining whether or not the steps of claim 1 are to be performed again.
5. The method of claim 4 , wherein the test includes using the residuals to determine an error that is a function of more than one of the dimensions of the second collection.
6. The method of claim 1 , wherein the step of determining the position includes performing an autocorrelation for independent identically and distributed in the residuals.
7. The method of claim 1 , wherein the step of generating includes determining at least one parameter value for the basis function by iteratively adjusting the parameter value for reducing a result of a function dependent upon a difference between an instance of the basis function, and members of the second data collection that correspond to the members of the subcollection.
8. The method of claim 1 , wherein the basis function generated using a mollifier, circle or bump function.
9. The method of claim 1 further including a step of testing the model on addition data indicative of the relationship, wherein the additional data includes at least one a first member related to a second member according to the relationship, and wherein at least one of: the first member is not a member of the first data collection, and the second member is not a member of the second data collection.
10. The method of claim 1 , wherein the physical event includes one of: an airflow around a physical object, a water flow over an object, a radio signal, and recognition of an object, maneuvering of an object.