Selection algorithm | EPFL Graph Search

In computer science, a selection algorithm is an algorithm for finding the kth smallest value in a collection of ordered values, such as numbers. The value that it finds is called the kth order statistic. Selection includes as special cases the problems of finding the minimum, median, and maximum element in the collection. Selection algorithms include quickselect, and the median of medians algorithm. When applied to a collection of n values, these algorithms take linear time, O(n) as expressed using big O notation. For data that is already structured, faster algorithms may be possible; as an extreme case, selection in an already-sorted array takes time O(1). Problem statement An algorithm for the selection problem takes as input a collection of values, and a number k. It outputs the k

Sublinear Algorithms for Spectral Graph Clustering

Aidasadat Mousavifar

This thesis focuses on designing spectral tools for graph clustering in sublinear time. With the emergence of big data, many traditional polynomial time, and even linear time algorithms have become prohibitively expensive. Processing modern datasets requires a new set of algorithms for computing with extremely constrained resources, i.e., \emph{sublinear algorithms}. Clustering is one of the well-known techniques for solving large-scale optimization problems in a wide variety of domains, including machine learning, data science and graph analysis~\cite{aydin2016distributed, rolnick2016geocuts, gargi2011large}. Efficient sublinear solutions for fundamental graph clustering problems require going well beyond classic techniques. In this thesis, we present an \emph{optimal} sublinear-time algorithm for \textit{testing

k

-clusterability problem}, i.e., quickly determining whether the graph can be partitioned into at most

k

expanders, or is far from any such graph. This is a generalization of a well-studied problem of testing graph expansion. The classic results on testing

k

-clusterability either consider the testing expansion problem (i.e,

k=1

k\geq 2

) \cite{KaleS_SIAMJC11,NachmiasS10}, or address the problem for larger values of

k

under the assumption that the gap between conductances of accepted and rejected graphs is at least logarithmic in the size of the graph \cite{CzumajPS_STOC15}. We overcome these barriers by developing novel spectral techniques based on analyzing the spectrum of the Gram matrix ofrandom walk transition probabilities. We complement our algorithm with a matching lower bound on the query complexity of testing

k

-clusterability, which improves upon the long-standing previous lower bound for testing graph expansion.Furthermore, we extend our previous result from the \textit{property testing} framework to an efficient clustering algorithm in the \textit{local computation algorithm} (LCA) model. We focus on a popular variant of graph clustering where the input graph can be partitioned into

k

expanders with outer conductance bounded by

\epsilon

. We construct a small space data structure that allows quickly classifying vertices of

G

according to the cluster they belong to in sublinear time. Our spectral clustering oracle provides

O(\epsilon \log k)

error per cluster for any

\epsilon \ll 1/\log k

. Our main contribution is a sublinear time oracle that provides dot product access to the spectral embedding of the graph. We estimate dot products with high precision using an appropriate linear transformation of the Gram matrix of random walk transition probabilities. Finally, using dot product access to the spectral embedding we design a spectral clustering oracle. At a high level, our approach amounts to hyperplane partitioning in the spectral embedding of the graph but crucially operates on a nested sequence of carefully defined subspaces in the spectral embedding to achieve per cluster recovery guarantees.

EPFL2021

Scalable greedy algorithms for transfer learning

Barbara Caputo, Ilja Kuzborskij

In this paper we consider the binary transfer learning problem, focusing on how to select and combine sources from a large pool to yield a good performance on a target task. Constraining our scenario to real world, we do not assume the direct access to the source data, but rather we employ the source hypotheses trained from them. We propose an efficient algorithm that selects relevant source hypotheses and feature dimensions simultaneously, building on the literature on the best subset selection problem. Our algorithm achieves state-of-the-art results on three computer vision datasets, substantially outperforming both transfer learning and popular feature selection baselines in a small-sample setting. We also present a randomized variant that achieves the same results with the computational cost independent from the number of source hypotheses and feature dimensions. Also, we theoretically prove that, under reasonable assumptions on the source hypotheses, our algorithm can learn effectively from few examples.

2017

Transfer Learning Through Greedy Subset Selection

Barbara Caputo, Ilja Kuzborskij

We study the binary transfer learning problem, focusing on how to select sources from a large pool and how to combine them to yield a good performance on a target task. In particular, we consider the transfer learning setting where one does not have direct access to the source data, but rather employs the source hypotheses trained from them. Building on the literature on the best subset selection problem, we propose an efficient algorithm that selects relevant source hypotheses and feature dimensions simultaneously. On three computer vision datasets we achieve state-of-the-art results, substantially outperforming transfer learning and popular feature selection baselines in a small-sample setting. Also, we theoretically prove that, under reasonable assumptions on the source hypotheses, our algorithm can learn effectively from few examples.

Springer Int Publishing Ag2015

Sublinear Algorithms for Spectral Graph Clustering

Aidasadat Mousavifar

k

-clusterability problem}, i.e., quickly determining whether the graph can be partitioned into at most

k

expanders, or is far from any such graph. This is a generalization of a well-studied problem of testing graph expansion. The classic results on testing

k

-clusterability either consider the testing expansion problem (i.e,

k=1

k\geq 2

) \cite{KaleS_SIAMJC11,NachmiasS10}, or address the problem for larger values of

k

k

k

expanders with outer conductance bounded by

\epsilon

. We construct a small space data structure that allows quickly classifying vertices of

G

according to the cluster they belong to in sublinear time. Our spectral clustering oracle provides

O(\epsilon \log k)

error per cluster for any

\epsilon \ll 1/\log k

EPFL2021