Publications related to Hessian matrix

Efficient second-order methods for model compression

Second-order information, in the form of Hessian- or Inverse-Hessian-vector products, is a fundamental tool for solving optimization problems. Recently, there has been a tremendous amount of work on utilizing this information for the current compute and me ...

2020

hm-toolbox: MATLAB SOFTWARE FOR HODLR AND HSS MATRICES

Daniel Kressner, Stefano Massei

Matrices with hierarchical low-rank structure, including HODLR and HSS matrices, constitute a versatile tool to develop fast algorithms for addressing large-scale problems. While existing software packages for such matrices often focus on linear systems, t ...

SIAM PUBLICATIONS2020

Instanton theory of ground-state tunneling splittings with general paths

Christophe Louis Joseph Jean-Luc Vaillant, Mihael Erakovic

We derive a multidimensional instanton theory for calculating ground-state tunneling splittings in Cartesian coordinates for general paths. It is an extension of the method by Mil'nikov and Nakamura [J. Chem. Phys. 115, 6881 (2001)] to include asymmetric p ...

2020

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Volkan Cevher, Ali Kavis, Shaul Nadav Hallak

This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm’s convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with prob ...

2020

Principal Component Analysis By Optimization Of Symmetric Functions Has No Spurious Local Optima

Armin Eftekhari

Principal component analysis (PCA) finds the best linear representation of data and is an indispensable tool in many learning and inference tasks. Classically, principal components of a dataset are interpreted as the directions that preserve most of its "e ...

SIAM PUBLICATIONS2020

Rational Krylov for Stieltjes matrix functions: convergence and pole selection

Stefano Massei

Evaluating the action of a matrix function on a vector, that is

x=f(\mathcal M)v

, is an ubiquitous task in applications. When

\mathcal M

is large, one usually relies on Krylov projection methods. In this paper, we provide effective choices for the pole ...

2019

hm-toolbox: Matlab software for HODLR and HSS matrices

Daniel Kressner, Stefano Massei

Matrices with hierarchical low-rank structure, including HODLR and HSS matrices, constitute a versatile tool to develop fast algorithms for addressing large-scale problems. While existing software packages for such matrices often focus on linear systems, t ...

2019

MATHICSE Technical Report: Block Krylov subspace methods for functions of matrices II: Modified block FOM

Kathryn Dianne Lund

We analyze an expansion of the generalized block Krylov subspace framework of [Electron.\ Trans.\ Numer.\ Anal., 47 (2017), pp. 100-126]. This expansion allows the use of low-rank modifications of the matrix projected onto the block Krylov subspace and con ...

MATHICSE2019

A Distributed Second-Order Algorithm You Can Trust

Martin Jaggi, Aurélien Lucchi, Thomas Hofmann, Matilde Gargiani

Due to the rapid growth of data and computational resources, distributed optimization has become an active research area in recent years. While first-order methods seem to dominate the field, second-order methods are nevertheless attractive as they potenti ...

2018

MATHICSE Technical Report : Quasi-Toeplitz matrix arithmetic : a Matlab toolbox

Stefano Massei

A Quasi Toeplitz (QT) matrix is a semi-infinite matrix of the kind

A = T(a) + E

where

T(a) = (aj-i)_{i,j\in \mathbb{Z}_{+}}

,

E = (e_{i,j})_{i,j\in\mathbb{Z}_{+}}

is compact and the norms

\| a\|_{\mathcal{W}} =\sum_{i\in\mathbb{Z}} |a|_j

and $|E|_{ ...

MATHICSE2018

Hessian matrix

Graph Chatbot

Chat with Graph Search

Efficient second-order methods for model compression

hm-toolbox: MATLAB SOFTWARE FOR HODLR AND HSS MATRICES

Instanton theory of ground-state tunneling splittings with general paths

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Principal Component Analysis By Optimization Of Symmetric Functions Has No Spurious Local Optima

Rational Krylov for Stieltjes matrix functions: convergence and pole selection

hm-toolbox: Matlab software for HODLR and HSS matrices

MATHICSE Technical Report: Block Krylov subspace methods for functions of matrices II: Modified block FOM

A Distributed Second-Order Algorithm You Can Trust

MATHICSE Technical Report : Quasi-Toeplitz matrix arithmetic : a Matlab toolbox

hm-toolbox: Matlab software for HODLR and HSS matrices

MATHICSE Technical Report: Block Krylov subspace methods for functions of matrices II: Modified block FOM

Principal Component Analysis By Optimization Of Symmetric Functions Has No Spurious Local Optima

hm-toolbox: MATLAB SOFTWARE FOR HODLR AND HSS MATRICES

A Distributed Second-Order Algorithm You Can Trust

MATHICSE Technical Report : Quasi-Toeplitz matrix arithmetic : a Matlab toolbox

Instanton theory of ground-state tunneling splittings with general paths

Rational Krylov for Stieltjes matrix functions: convergence and pole selection

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Efficient second-order methods for model compression