Smoothing spline

Smoothing splines are function estimates, , obtained from a set of noisy observations of the target , in order to balance a measure of goodness of fit of to with a derivative based measure of the smoothness of . They provide a means for smoothing noisy data. The most familiar example is the cubic smoothing spline, but there are many other possibilities, including for the case where is a vector quantity. Let be a set of observations, modeled by the relation where the are independent, zero mean random variables (usually assumed to have constant variance). The cubic smoothing spline estimate of the function is defined to be the minimizer (over the class of twice differentiable functions) of Remarks: is a smoothing parameter, controlling the trade-off between fidelity to the data and roughness of the function estimate. This is often estimated by generalized cross-validation, or by restricted marginal likelihood (REML) which exploits the link between spline smoothing and Bayesian estimation (the smoothing penalty can be viewed as being induced by a prior on the ). The integral is often evaluated over the whole real line although it is also possible to restrict the range to that of . As (no smoothing), the smoothing spline converges to the interpolating spline. As (infinite smoothing), the roughness penalty becomes paramount and the estimate converges to a linear least squares estimate. The roughness penalty based on the second derivative is the most common in modern statistics literature, although the method can easily be adapted to penalties based on other derivatives. In early literature, with equally-spaced ordered , second or third-order differences were used in the penalty, rather than derivatives. The penalized sum of squares smoothing objective can be replaced by a penalized likelihood objective in which the sum of squares terms is replaced by another log-likelihood based measure of fidelity to the data. The sum of squares term corresponds to penalized likelihood with a Gaussian assumption on the .