Logistic Regression: Probability Modeling and Optimization

This lecture on logistic regression covers the modeling of binary classification problems using the logistic function to predict class probabilities. It explains the motivation behind logistic regression, the logistic function properties, label prediction, and the comparison with linear regression. The lecture delves into maximum likelihood estimation, the gradient of the negative log likelihood, convexity of the loss function, and optimization methods like gradient descent and Newton's method. It also addresses issues with linearly separable data and introduces regularization techniques to prevent overfitting.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (49)

Related concepts (43)

CS-433: Machine learning

Machine learning methods are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed and pr

Probability distribution

In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space). For instance, if X is used to denote the outcome of a coin toss ("the experiment"), then the probability distribution of X would take the value 0.5 (1 in 2 or 1/2) for X = heads, and 0.

Probability

Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speaking, 0 indicates impossibility of the event and 1 indicates certainty. The higher the probability of an event, the more likely it is that the event will occur. A simple example is the tossing of a fair (unbiased) coin.

Joint probability distribution

Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered for any given number of random variables. The joint distribution encodes the marginal distributions, i.e. the distributions of each of the individual random variables. It also encodes the conditional probability distributions, which deal with how the outputs of one random variable are distributed when given information on the outputs of the other random variable(s).

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Conditional probability distribution

In probability theory and statistics, given two jointly distributed random variables and , the conditional probability distribution of given is the probability distribution of when is known to be a particular value; in some cases the conditional probabilities may be expressed as functions containing the unspecified value of as a parameter. When both and are categorical variables, a conditional probability table is typically used to represent the conditional probability.

Probabilistic Linear Regression PHYS-467: Machine learning for physicists

Explores probabilistic linear regression, covering joint and conditional probability, ridge regression, and overfitting mitigation.

Generalized Linear Regression: Classification BIO-322: Introduction to machine learning for bioengineers

Explores Generalized Linear Regression, Classification, confusion matrices, ROC curves, and noise in data.

Logistic Regression: Probabilistic Interpretation ME-390: Foundations of artificial intelligence

Covers logistic regression's probabilistic interpretation, multinomial regression, KNN, hyperparameters, and curse of dimensionality.

Logistic Regression: Statistical Inference and Machine Learning MGT-448: Statistical inference and machine learning

Covers logistic regression, likelihood function, Newton's method, and classification error estimation.

Logistic Regression: Probability Modeling CS-433: Machine learning

Covers logistic regression for binary classification using probability modeling and optimization methods.