Logarithmic derivative | EPFL Graph Search

In mathematics, specifically in calculus and complex analysis, the logarithmic derivative of a function f is defined by the formula where is the derivative of f. Intuitively, this is the infinitesimal relative change in f; that is, the infinitesimal absolute change in f, namely scaled by the current value of f. When f is a function f(x) of a real variable x, and takes real, strictly positive values, this is equal to the derivative of ln(f), or the natural logarithm of f. This follows directly from the chain rule: Many properties of the real logarithm also apply to the logarithmic derivative, even when the function does not take values in the positive reals. For example, since the logarithm of a product is the sum of the logarithms of the factors, we have So for positive-real-valued functions, the logarithmic derivative of a product is the sum of the logarithmic derivatives of the factors. But we can also use the Leibniz law for the derivative of a product to get Thus, it is true for any function that the logarithmic derivative of a product is the sum of the logarithmic derivatives of the factors (when they are defined). A corollary to this is that the logarithmic derivative of the reciprocal of a function is the negation of the logarithmic derivative of the function: just as the logarithm of the reciprocal of a positive real number is the negation of the logarithm of the number. More generally, the logarithmic derivative of a quotient is the difference of the logarithmic derivatives of the dividend and the divisor: just as the logarithm of a quotient is the difference of the logarithms of the dividend and the divisor. Generalising in another direction, the logarithmic derivative of a power (with constant real exponent) is the product of the exponent and the logarithmic derivative of the base: just as the logarithm of a power is the product of the exponent and the logarithm of the base. In summary, both derivatives and logarithms have a product rule, a reciprocal rule, a quotient rule, and a power rule (compare the list of logarithmic identities); each pair of rules is related through the logarithmic derivative.

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022