Linear map | EPFL Graph Search

In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that preserves the operations of vector addition and scalar multiplication. The same names and the same definition are also used for the more general case of modules over a ring; see Module homomorphism. If a linear map is a bijection then it is called a . In the case where V = W, a linear map is called a linear endomorphism. Sometimes the term refers to this case, but the term "linear operator" can have different meanings for different conventions: for example, it can be used to emphasize that V and W are real vector spaces (not necessarily with V = W), or it can be used to emphasize that V is a function space, which is a common convention in functional analysis. Sometimes

Sharp estimates on the first eigenvalue of the p-Laplacian with negative Ricci lower bound

Daniele Valtorta

We complete the picture of sharp eigenvalue estimates for the -Laplacian on a compact manifold by providing sharp estimates on the first nonzero eigenvalue of the nonlinear operator when the Ricci curvature is bounded from below by a negative constant. We assume that the boundary of the manifold is convex, and put Neumann boundary conditions on it. The proof is based on a refined gradient comparison technique and a careful analysis of the underlying model spaces.

Springer Verlag2014

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, our understanding of these models, in particular of what happens during their training, remains very limited. This PhD started with the introduction of the Neural Tangent Kernel (NTK) to describe the evolution of the function represented by the network during training. In the infinite-width limit, i.e. when the number of neurons in the layers of the network grows to infinity, the NTK converges to a deterministic and time-independent limit, leading to a simple yet complete description of the dynamics of infinitely-wide DNNs. This allowed one to give the first general proof of convergence of DNNs to a global minimum, and yielded the first description of the limiting spectrum of the Hessian of the loss surface of DNNs throughout training.More importantly, the NTK plays a crucial role in describing the generalization abilities of DNNs, i.e. the performance of the trained network on unseen data. The NTK analysis uncovered a direct link between the function learned by infinitely wide DNNs and Kernel Ridge Regression predictors, whose generalization properties are studied in this thesis using tools of random matrix theory. Our analysis of KRR reveals the importance of the eigendecomposition of the NTK, which is affected by a number of architectural choices. In very deep networks, an ordered regime and a chaotic regime appear, determined by the choice of non-linearity and the balance between the weights and bias parameters; these two phases are characterized by different speeds of decay of the eigenvalues of the NTK, leading to a tradeoff between convergence speed and generalization. In practical contexts such as Generative Adversarial Networks or Topology Optimization, the network architecture can be chosen to guarantee certain properties of the NTK and its spectrum.These results give an almost complete description DNNs in this infinite-width limit. It is then natural to wonder how it extends to finite-width networks used in practice. In the so-called NTK regime, the discrepancy between finite- and infinite-widths DNNs is mainly a result of the variance w.r.t. to the sampling of the parameters, as shown empirically and mathematically relying on the similarity between DNNs and random feature models.In contrast to the NTK regime, where the NTK remains constant during training, there exist so-called active regimes, where the evolution of the NTK is significant, which appear in a number of settings. One such regime appears in Deep Linear Networks with a very small initialization, where the training dynamics approach a sequence of saddle-points, representing linear maps of increasing rank, leading to a low-rank bias which is absent in the NTK regime.

EPFL2022

Extrapolation et synthèse

Christian Fiorillo

Let G a locally compact group, H a closed subgroup and 1 < p < ∞. It's well-known that the restriction of the functions from G to H is a surjective linear contraction from Ap(G) onto Ap(H). We prove, when H is amenable, that every element in Ap(H) with compact support can be extended to an element in Ap(G) of which we can check norm and support. This result is already known in the case of normal subgroups and also for compact subgroups. We obtain the existence of a quasi-coretract in the BAN category, as a substitute of a morphism ΓH such that ResH ◦ ΓH = idAp(H). Indeed, for an amenable subsgroup, the morphism ΓH, a priori, doesn't exist. So, we construct a net of morphismes in BAN from Ap(H) into Ap(G), that converge to idAp(H) for the strong operator's topology on Ap(H) (that's for us the notion of a quasi-coretract in BAN). Furthermore, if H is metrizable and σ-compact we obtain, more precisely, a sequence. Moreover, our approach allows us to extend to the non-abelian case some works of H. Reiter and C. Herz concerning the spectral synthesis of bounded uniformly continuous functions. My results are new even for the Fourier algebra.

EPFL2007

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

EPFL2022