Mathematics of Data: Computation Role

This lecture by the instructor covers the role of computation in the mathematics of data, focusing on iterative descent methods, unconstrained minimization, maximum-likelihood estimators, approximate vs. exact optimality, and the basic principles of descent methods. The lecture delves into the challenges faced by iterative optimization algorithms, the conditions for local descent directions, and the basic iterative strategy for optimization algorithms. It also discusses the concepts of local steepest descent direction, stationarity, and the geometric interpretation of descent methods.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (30)

Related concepts (106)

EE-556: Mathematics of data: from theory to computation

This course provides an overview of key advances in continuous optimization and statistical analysis for machine learning. We review recent learning formulations and models as well as their guarantees

Gradient Descent for Linear MSE CS-433: Machine learning

Explores Gradient Descent for Linear MSE in machine learning, covering computation, complexity, variants, Stochastic Gradient Descent, penalty functions, implementation issues, and non-convex optimization.

Nonlinear Programming: Part I EE-715: Optimal control

Covers the fundamentals of Nonlinear Programming and its applications in Optimal Control, exploring techniques, examples, optimality definitions, and necessary conditions.

Optimization Methods: Theory Discussion ME-454: Modelling and optimization of energy systems

Explores optimization methods, including unconstrained problems, linear programming, and heuristic approaches.

Optimization Methods in Machine Learning CS-433: Machine learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Convex Functions MATH-329: Continuous optimization

Covers the properties and operations of convex functions.

Convex optimization

Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization problems admit polynomial-time algorithms, whereas mathematical optimization is in general NP-hard.

Computational mathematics

Computational mathematics is an area of mathematics devoted to the interaction between mathematics and computer computation. A large part of computational mathematics consists roughly of using mathematics for allowing and improving computer computation in areas of science and engineering where mathematics are useful. This involves in particular algorithm design, computational complexity, numerical methods and computer algebra. Computational mathematics refers also to the use of computers for mathematics itself.

Gradient descent

In mathematics, gradient descent (also often called steepest descent) is a iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent.

Computational science

Computational science, also known as scientific computing, technical computing or scientific computation (SC), is a division of science that uses advanced computing capabilities to understand and solve complex physical problems. This includes Algorithms (numerical and non-numerical): mathematical models, computational models, and computer simulations developed to solve sciences (e.

Discrete mathematics

Discrete mathematics is the study of mathematical structures that can be considered "discrete" (in a way analogous to discrete variables, having a bijection with the set of natural numbers) rather than "continuous" (analogously to continuous functions). Objects studied in discrete mathematics include integers, graphs, and statements in logic. By contrast, discrete mathematics excludes topics in "continuous mathematics" such as real numbers, calculus or Euclidean geometry.