Better Guarantees for k-Means and Euclidean k-Median by Primal-Dual Algorithms

Clustering is a classic topic in optimization with k-means being one of the most fundamental such problems. In the absence of any restrictions on the input, the best-known algorithm for k-means in Euclidean space with a provable guarantee is a simple local search heuristic yielding an approximation guarantee of 9+epsilon, a ratio that is known to be tight with respect to such methods. We overcome this barrier by presenting a new primal-dual approach that allows us to (1) exploit the geometric structure of k-means and (2) satisfy the hard constraint that at most k clusters are selected without deteriorating the approximation guarantee. Our main result is a 6.357-approximation algorithm with respect to the standard linear programming (LP) relaxation. Our techniques are quite general, and we also show improved guarantees for k-median in Euclidean metrics and for a generalization of k-means in which the underlying metric is not required to be Euclidean.

Algorithms For Clustering Problems

Ashkan Norouzi Fard

Clustering is a classic topic in combinatorial optimization and plays a central role in many areas, including data science and machine learning. In this thesis, we first focus on the dynamic facility location problem (i.e., the facility location problem in evolving metrics). We present a new LP-rounding algorithm for facility location problems, which yields the first constant factor approximation algorithm for the dynamic facility location problem. Our algorithm installs competing exponential clocks on clients and facilities, and connects every client by the path that repeatedly follows the smallest clock in the neighborhood. The use of exponential clocks gives rise to several properties that distinguish our approach from previous LP-roundings for facility location problems. In particular, we use \emph{no clustering} and we enable clients to connect through paths of \emph{arbitrary lengths}. In fact, the clustering-free nature of our algorithm is crucial for applying our LP-rounding approach to the dynamic problem. Furthermore, we present both empirical and theoretical aspects of the

k

-means problem. The best known algorithm for

k

-means with a provable guarantee is a simple local-search heuristic that yields an approximation guarantee of

9+\epsilon

, a ratio that is known to be tight with respect to such methods. We overcome this barrier by presenting a new primal-dual approach that enables us (1) to exploit the geometric structure of

k

-means and (2) to satisfy the hard constraint that at most

k

clusters are selected without deteriorating the approximation guarantee. Our main result is a

6.357

-approximation algorithm with respect to the standard LP relaxation. Our techniques are quite general and we also show improved guarantees for the general version of

k

-means where the underlying metric is not required to be Euclidean and for

k

-median in Euclidean metrics. We also improve the running time of our algorithm to almost linear running time and still maintain a provable guarantee. We compare our algorithm with {\sc K-Means++} (a widely studied algorithm) and show that we obtain better accuracy with comparable and even better running time.

EPFL2018

LP-Based Algorithms for Capacitated Facility Location

Hyung Chan An, Ola Nils Anders Svensson

Linear programming has played a key role in the study of algorithms for combinatorial optimization problems. In the field of approximation algorithms, this is well illustrated by the uncapacitated facility location problem. A variety of algorithmic methodologies, such as LP-rounding and primal-dual method, have been applied to and evolved from algorithms for this problem. Unfortunately, this collection of powerful algorithmic techniques had not yet been applicable to the more general capacitated facility location problem. In fact, all of the known algorithms with good performance guarantees were based on a single technique, local search, and no linear programming relaxation was known to efficiently approximate the problem. In this paper, we present a linear programming relaxation with constant integrality gap for capacitated facility location. We demonstrate that the fundamental theories of multi-commodity flows and matchings provide key insights that lead to the strong relaxation. Our algorithmic proof of integrality gap is obtained by finally accessing the rich toolbox of LP-based methodologies: we present a constant factor approximation algorithm based on LP-rounding. © 2014 IEEE.

IEEE2014

Shapes from Pixels

Mitra Fatemi

In today's digital world, sampling is at the heart of any signal acquisition device. Imaging devices are ubiquitous examples that capture two-dimensional visual signals and store them as the pixels of discrete images. The main concern is whether and how the pixels provide an exact or at least a fair representation of the original visual signal in the continuous domain. This motivates the design of exact reconstruction or approximation techniques for a target class of images. Such techniques benefit different imaging tasks such as super-resolution, deblurring and compression. This thesis focuses on the reconstruction of visual signals representing a shape over a background, from their samples. Shape images have only two intensity values. However, the filtering effect caused by the sampling kernel of imaging devices smooths out the sharp transitions in the image and results in samples with varied intensity levels. To trace back the shape boundaries, we need strategies to reconstruct the original bilevel image. But, abrupt intensity changes along the shape boundaries as well as diverse shape geometries make reconstruction of this class of signals very challenging. Curvelets and contourlets have been proved as efficient multiresolution representations for the class of shape images. This motivates the approximation of shape images in the aforementioned domains. In the first part of this thesis, we study generalized sampling and infinite-dimensional compressed sensing to approximate a signal in a domain that is known to provide a sparse or efficient representation for the signal, given its samples in a different domain. We show that the generalized sampling, due to its linearity, is incapable of generating good approximation of shape images from a limited number of samples. The infinite-dimensional compressed sensing is a more promising approach. However, the concept of random sampling in this scheme does not apply to the shape reconstruction problem. Next, we propose a sampling scheme for shape images with finite rate of innovation (FRI). More specifically, we model the shape boundaries as a subset of an algebraic curve with an implicit bivariate polynomial. We show that the image parameters are solutions of a set of linear equations with the coefficients being the image moments. We then replace conventional moments with more stable generalized moments that are adjusted to the given sampling kernel. This leads to successful reconstruction of shapes with moderate complexities from samples generated with realistic sampling kernels and in the presence of moderate noise levels. Our next contribution is a scheme for recovering shapes with smooth boundaries from a set of samples. The reconstructed image is constrained to regenerate the same samples (consistency) as well as forming a bilevel image. We initially formulate the problem by minimizing the shape perimeter over the set of consistent shapes. Next, we relax the non-convex shape constraint to transform the problem into minimizing the total variation over consistent non-negative-valued images. We introduce a requirement -called reducibility- that guarantees equivalence between the two problems. We illustrate that the reducibility effectively sets a requirement on the minimum sampling density. Finally, we study a relevant problem in the Boolean algebra: the Boolean compressed sensing. The problem is about recovering a sparse Boolean vector from a few collective binary tests. We study a formulation of this problem as a binary linear program, which is NP hard. To overcome the computational burden, we can relax the binary constraint on the variables and apply a rounding to the solution. We replace the rounding procedure with a randomized algorithm. We show that the proposed algorithm considerably improves the success rate with only a slight increase in the computational cost.

EPFL2016