Rescaling, thinning or complementing? On goodness-of-fit procedures for point process models and Generalized Linear Models

Generalized Linear Models (GLMs) are an increasingly popular framework for modeling neural spike trains. They have been linked to the theory of stochastic point processes and researchers have used this relation to assess goodness-of-fit using methods from point-process theory, e.g. the time-rescaling theorem. However, high neural firing rates or coarse discretization lead to a breakdown of the assumptions necessary for this connection. Here, we show how goodness-of-fit tests from point-process theory can still be applied to GLMs by constructing equivalent surrogate point processes out of time-series observations. Furthermore, two additional tests based on thinning and complementing point processes are introduced. They augment the instruments available for checking model adequacy of point processes as well as discretized models.

Statistical tests for neural population models - The multivariate time rescaling theorem

Joao Emanuel Felipe Gerhard

The coordinated, collective spiking activity of neuronal populations encodes and processes information. One approach towards understanding such population based computation is to fit statistical models to simultaneously recorded spike trains and use these models to make inferences about correlations, functional connectivity, and so forth. Any statistical model must, prior to making inferences from it, be validated by an appropriate goodness of fit measure, but as yet there is no commonly agreed upon statistical sufficiency test for neuronal population models. For single neuron spike trains, the time rescaling theorem provides a goodness of fit test consistent with their point process nature. Interspike intervals (ISIs) are rescaled, as a function of the model's conditional intensity function (time and history dependent spike probability) to be independent and exponentially distributed if the model is accurate. A Kolmogorov Smirnov (KS) test between the rescaled ISIs and the exponential distribution is then used to check goodness of fit. The time rescaling theorem applies to models of single spike trains, however some researchers have used it to validate population models, by applying it separately to each neuron in the population. Here we demonstrate that this approach is insufficient and that tests based upon single neuron statistics can not validate a population model. If interactions between neurons are ignored univariate tests may be erroneously passed, even when the interactions are strong. To remedy this we present the multivariate version of the time rescaling theorem and develop a practical statistical test for implementing it. The test relies on the fact that if the conditional intensity functions for all neurons are modeled correctly then the rescaled ISIs should be independent not only within a single spike train, but also across spike trains. Testing for such independence across spike trains determines if the collective activity of the neuronal population is captured by the model. Using Generalized Linear Models (GLMs) fitted to both simulated spike trains and spike trains recorded experimentally in monkey V1 cortex we demonstrate that population models are much more reliably validated by the multivariate test. This suggests that neuronal populations must be considered collectively as important features of their dynamics can only be detected from a multivariate viewpoint.

2010

Goodness-of-fit tests for neural population models: the multivariate time-rescaling theorem

Joao Emanuel Felipe Gerhard

Statistical models of neural activity are at the core of the field of modern computational neuroscience. The activity of single neurons has been modeled to successfully explain dependencies of neural dynamics to its own spiking history, to external stimuli or other covariates. Recently, there has been a growing interest in modeling spiking activity of a population of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing (existing models include generalized linear models or maximum-entropy approaches). For point-process-based models of single neurons, the time-rescaling theorem has proven to be a useful toolbox to assess goodness-of-fit. In its univariate form, the time-rescaling theorem states that if the conditional intensity function of a point process is known, then its inter-spike intervals can be transformed or “rescaled” so that they are independent and exponentially distributed. However, the theorem in its original form lacks sensitivity to detect even strong dependencies between neurons. Here, we present how the theorem can be extended to be applied to neural population models and we provide a step-by-step procedure to perform the statistical tests. We then apply both the univariate and multivariate tests to simplified toy models, but also to more complicated many-neuron models and to neuronal populations recorded in V1 of awake monkey during natural scenes stimulation. We demonstrate that important features of the population activity can only be detected using the multivariate extension of the test. The time-rescaling theorem has been used extensively to assess goodness-of-fit and to compare different single-neuron models. Multivariate population models became popular only recently. Some of the approaches did not attempt any goodness-of-fit analysis at all or used the time-rescaling theorem separately for each modeled spike train. The proposed multivariate time-rescaling theorem fills the missing gap. Our studies using experimental data show that the use of the univariate theorem may erroneously indicate a good fit for independent encoding models. The lack of fit is detected by the multivariate extension and can be partly corrected for by including additional cross-interaction terms in the model. Overall, the proposed procedure is a simple-to-implement analysis tool for any population model that is based on the conditional intensity formalism.

2010

Applying the Multivariate Time-Rescaling Theorem to Neural Population Models

Joao Emanuel Felipe Gerhard

Statistical models of neural activity are integral to modern neuroscience. Recently interest has grown in modeling the spiking activity of populations of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing. However, any statistical model must be validated by an appropriate goodness-of-fit test. Kolmogorov-Smirnov tests based on the time-rescaling theorem have proven to be useful for evaluating point-process-based statistical models of single-neuron spike trains. Here we discuss the extension of the time-rescaling theorem to the multivariate (neural population) case. We show that even in the presence of strong correlations between spike trains, models that neglect couplings between neurons can be erroneously passed by the univariate time-rescaling test. We present the multivariate version of the time-rescaling theorem and provide a practical step-by-step procedure for applying it to testing the sufficiency of neural population models. Using several simple analytically tractable models and more complex simulated and real data sets, we demonstrate that important features of the population activity can be detected only using the multivariate extension of the test.

Massachusetts Institute of Technology Press2011

Goodness-of-fit tests for neural population models: the multivariate time-rescaling theorem

Joao Emanuel Felipe Gerhard

2010

Statistical tests for neural population models - The multivariate time rescaling theorem

Joao Emanuel Felipe Gerhard

2010