Goodness of fit | EPFL Graph Search

The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question. Such measures can be used in statistical hypothesis testing, e.g. to test for normality of residuals, to test whether two samples are drawn from identical distributions (see Kolmogorov–Smirnov test), or whether outcome frequencies follow a specified distribution (see Pearson's chi-square test). In the analysis of variance, one of the components into which the variance is partitioned may be a lack-of-fit sum of squares. Fit of distributions In assessing whether a given distribution is suited to a data-set, the following tests and their underlying measures of fit can be used: *Bayesian information criterion *Kolmogorov–Smirnov test *Cramér–von Mises criterion *Anderson–Darling test *Berk-Jones tests *Shapiro–Wilk test *Chi-squared test *Akaike

Stratégies de compromis pour l'estimation des paramètres de régression et pour la classification floue

Nicolas Fournier

Model specification is an integral part of any statistical inference problem. Several model selection techniques have been developed in order to determine which model is the best one among a list of possible candidates. Another way to deal with this question is the so-called model averaging, and in particular the frequentist approach. An estimation of the parameters of interest is obtained by constructing a weighted average of the estimates of these quantities under each candidate model. We develop compromise frequentist strategies for the estimation of regression parameters, as well as for the probabilistic clustering problem. In the regression context, we construct compromise strategies based on the Pitman estimators associated with various underlying errors distributions. The weight given to each model is equal to its profile likelihood, which gives a measure of the goodness-of-fit. Asymptotic properties of both Pitman estimators and profile likelihood allow us to define a minimax strategy for choosing the distributions of the compromise, involving a notion of distance between distributions. Performances of such estimators are then compared to other usual and robust procedures. In the second part of the thesis, we develop compromise strategies in the probabilistic clustering context. Although this clustering method is based on mixtures of distributions, our compromise strategies are not applied directly to the estimates of the parameters, but on the posterior probabilities of membership. Two types of compromise are presented, and the performances of resulting classification rules are investigated.

EPFL2011

Applying the Multivariate Time-Rescaling Theorem to Neural Population Models

Joao Emanuel Felipe Gerhard

Statistical models of neural activity are integral to modern neuroscience. Recently interest has grown in modeling the spiking activity of populations of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing. However, any statistical model must be validated by an appropriate goodness-of-fit test. Kolmogorov-Smirnov tests based on the time-rescaling theorem have proven to be useful for evaluating point-process-based statistical models of single-neuron spike trains. Here we discuss the extension of the time-rescaling theorem to the multivariate (neural population) case. We show that even in the presence of strong correlations between spike trains, models that neglect couplings between neurons can be erroneously passed by the univariate time-rescaling test. We present the multivariate version of the time-rescaling theorem and provide a practical step-by-step procedure for applying it to testing the sufficiency of neural population models. Using several simple analytically tractable models and more complex simulated and real data sets, we demonstrate that important features of the population activity can be detected only using the multivariate extension of the test.

Massachusetts Institute of Technology Press2011

Goodness-of-fit tests for neural population models: the multivariate time-rescaling theorem

Joao Emanuel Felipe Gerhard

Statistical models of neural activity are at the core of the field of modern computational neuroscience. The activity of single neurons has been modeled to successfully explain dependencies of neural dynamics to its own spiking history, to external stimuli or other covariates. Recently, there has been a growing interest in modeling spiking activity of a population of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing (existing models include generalized linear models or maximum-entropy approaches). For point-process-based models of single neurons, the time-rescaling theorem has proven to be a useful toolbox to assess goodness-of-fit. In its univariate form, the time-rescaling theorem states that if the conditional intensity function of a point process is known, then its inter-spike intervals can be transformed or “rescaled” so that they are independent and exponentially distributed. However, the theorem in its original form lacks sensitivity to detect even strong dependencies between neurons. Here, we present how the theorem can be extended to be applied to neural population models and we provide a step-by-step procedure to perform the statistical tests. We then apply both the univariate and multivariate tests to simplified toy models, but also to more complicated many-neuron models and to neuronal populations recorded in V1 of awake monkey during natural scenes stimulation. We demonstrate that important features of the population activity can only be detected using the multivariate extension of the test. The time-rescaling theorem has been used extensively to assess goodness-of-fit and to compare different single-neuron models. Multivariate population models became popular only recently. Some of the approaches did not attempt any goodness-of-fit analysis at all or used the time-rescaling theorem separately for each modeled spike train. The proposed multivariate time-rescaling theorem fills the missing gap. Our studies using experimental data show that the use of the univariate theorem may erroneously indicate a good fit for independent encoding models. The lack of fit is detected by the multivariate extension and can be partly corrected for by including additional cross-interaction terms in the model. Overall, the proposed procedure is a simple-to-implement analysis tool for any population model that is based on the conditional intensity formalism.

2010

Goodness-of-fit tests for neural population models: the multivariate time-rescaling theorem

Joao Emanuel Felipe Gerhard

2010