Modèle additif | EPFL Graph Search

En statistiques, le modèle additif (MA, ou AM pour Additive Model) est une méthode de régression non paramétrique. Il a été suggéré par Jerome H. Friedman et Werner Stuetzle et est utilisé par l'algorithme ACE. Le modèle additif utilise un lissage unidimensionnel pour construire une classe restreinte de modèles de régression non paramétriques. De ce fait, il est moins affecté par le fléau de la dimension que, par exemple, un lissage p-dimensionnel. De plus, le modèle additif est plus flexible qu'un modèle linéaire standard, tout en étant plus interprétable qu'une surface de régression générale. Les difficultés liées aux modèles additifs, comme de nombreuses autres méthodes d'apprentissage automatique, incluent la sélection de modèles, le surapprentissage et la multicolinéarité. Description Sachant un ensemble de données {y_i,, x_{i1}, \ldots, x_{ip}}{i=1}^n de n unités statistiques, où {x{i1}, \ldots, x_{ip}}_{i=1}^n représenten

Automatic L2 Regularization for Multiple Generalized Additive Models

Yousra El Bachir

Multiple generalized additive models are a class of statistical regression models wherein parameters of probability distributions incorporate information through additive smooth functions of predictors. The functions are represented by basis function expansions, whose coefficients are the regression parameters. The smoothness is induced by a quadratic roughness penalty on the functionsâ curvature, which is equivalent to a weighted

L_2

regularization controlled by smoothing parameters. Regression fitting relies on maximum penalized likelihood estimation for the regression coefficients, and smoothness selection relies on maximum marginal likelihood estimation for the smoothing parameters. Owing to their nonlinearity, flexibility and interpretability, generalized additive models are widely used in statistical modeling, but despite recent advances, reliable and fast methods for automatic smoothing in massive datasets are unavailable. Existing approaches are either reliable, complex and slow, or unreliable, simpler and fast, so a compromise must be made. A bridge between these categories is needed to extend use of multiple generalized additive models to settings beyond those possible in existing software. This thesis is one step in this direction. We adopt the marginal likelihood approach to develop approximate expectation-maximization methods for automatic smoothing, which avoid evaluation of expensive and unstable terms. This results in simpler algorithms that do not sacrifice reliability and achieve state-of-the-art accuracy and computational efficiency. We extend the proposed approach to big-data settings and produce the first reliable, high-performance and distributed-memory algorithm for fitting massive multiple generalized additive models. Furthermore, we develop the underlying generic software libraries and make them accessible to the open-source community.

EPFL2019

Global sensitivity analysis of computer models with functional inputs

Global sensitivity analysis is used to quantify the influence of uncertain model inputs on the response variability of a numerical model. The common quantitative methods are appropriate with computer codes having scalar model inputs. This paper aims at illustrating different variance-based sensitivity analysis techniques, based on the so-called Sobol's indices, when some model inputs are functional, such as stochastic processes or random spatial fields. In this work, we focus on large cpu time computer codes which need a preliminary metamodeling step before performing the sensitivity analysis. We propose the use of the joint modeling approach, i.e., modeling simultaneously the mean and the dispersion of the code outputs using two interlinked generalized linear models (GLMs) or generalized additive models (GAMs). The "mean model" allows to estimate the sensitivity indices of each scalar model inputs, while the "dispersion model" allows to derive the total sensitivity index of the functional model inputs. The proposed approach is compared to some classical sensitivity analysis methodologies on an analytical function. Lastly, the new methodology is applied to an industrial computer code that simulates the nuclear fuel irradiation. (C) 2008 Elsevier Ltd. All rights reserved.

2009

Multi-task additive models with shared transfer functions based on dictionary learning

Alhussein Fawzi, Pascal Frossard

Additive models form a widely popular class of regression models which represent the relation between covariates and response variables as the sum of low-dimensional transfer functions. Besides flexibility and accuracy, a key benefit of these models is their interpretability: the transfer functions provide visual means for inspecting the models and identifying domain-specific relations between inputs and outputs. However, in large-scale problems involving the prediction of many related tasks, learning independently additive models results in a loss of model interpretability, and can cause overfitting when training data is scarce. We introduce a novel multi-task learning approach which provides a corpus of accurate and interpretable additive models for a large number of related forecasting tasks. Our key idea is to share transfer functions across models in order to reduce the model complexity and ease the exploration of the corpus. We establish a connection with sparse dictionary learning and propose a new efficient fitting algorithm which alternates between sparse coding and transfer function updates. The former step is solved via an extension of Orthogonal Matching Pursuit, whose properties are analyzed using a novel recovery condition which extends existing results in the literature. The latter step is addressed using a traditional dictionary update rule. Experiments on real-world data demonstrate that our approach compares favorably to baseline methods while yielding an interpretable corpus of models, revealing structure among the individual tasks and being more robust when training data is scarce. Our framework therefore extends the well-known benefits of additive models to common regression settings possibly involving thousands of tasks.

Institute of Electrical and Electronics Engineers2017

Automatic L2 Regularization for Multiple Generalized Additive Models

Yousra El Bachir

L_2

EPFL2019