|
Abstract : |
It is known that superpositions of ridge functions (single hidden-layer feedforward neural networks) may give good approximations to certain kinds of multivariate functions. It remains unclear, however, how to eectively obtain such approximations. In this paper, we use ideas from harmonic analysis to attack this question. We introduce a special admissibility condition for neural activation functions. The new condition is not satis-ed by the sigmoid activation in current use by the neural networks community; instead, our condition requires that the neural activation function be oscillatory. Using an admissible neuron we construct linear transforms which represent quite general functions f as a superposition of ridge functions. We develop a continuous transform which satises a Parseval-like relation a discrete transform which satises frame bounds Both transforms represent f in a stable and eective way. The discrete transform is more challenging to construct and involves an interesting new discretization of time-frequency-direction space in order to obtain frame bounds for functions in L 2, |
