Publications by Freya Behrens | EPFL Graph Search

A phase transition between positional and semantic learning in a solvable model of dot-product attention

Florent Gérard Krzakala, Lenka Zdeborová, Freya Behrens, Hugo Chao Cui

Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements in the model capabilities. Yet, a theoretical characterization of how such mecha ...

2025

Dynamical cavity method for hypergraphs and its application to quenches in the k-XOR-SAT problem

Lenka Zdeborová, Freya Behrens

The dynamical cavity method and its backtracking version provide a powerful approach to studying the properties of dynamical processes on large random graphs. This work extends these methods to hypergraphs, enabling the analysis of interactions involving m ...

2025

Dynamical phase transitions in graph cellular automata

Lenka Zdeborová, Freya Behrens

Discrete dynamical systems can exhibit complex behavior from the iterative application of straightforward local rules. A famous class of examples comes from cellular automata whose global dynamics are notoriously challenging to analyze. To address this, we ...

American Physical Society (APS)2024

Statistical mechanics of deep learning

Freya Behrens

2024

Counting and hardness-of-finding fixed points in cellular automata on random graphs

Lenka Zdeborová, Freya Behrens

We study the fixed points of outer-totalistic cellular automata on sparse random regular graphs. These can be seen as constraint satisfaction problems, where each variable must adhere to the same local constraint, which depends solely on its state and the ...

IOP Publishing2024

Quenches in the Sherrington-Kirkpatrick model

Florent Gérard Krzakala, Lenka Zdeborová, Freya Behrens, Vittorio Erba

The Sherrington-Kirkpatrick model is a prototype of a complex non-convex energy landscape. Dynamical processes evolving on such landscapes and locally aiming to reach minima are generally poorly understood. Here, we study quenches, i.e. dynamics that local ...

2024

Backtracking Dynamical Cavity Method

Lenka Zdeborová, Freya Behrens, Barbora Hudcová

The cavity method is one of the cornerstones of the statistical physics of disordered systems such as spin glasses and other complex systems. It is able to analytically and asymptotically exactly describe the equilibrium properties of a broad range of mode ...

American Physical Society2023

(Dis)assortative partitions on random regular graphs

Lenka Zdeborová, Freya Behrens, Yaroslav Kivva

We study the problem of assortative and disassortative partitions on random dregular graphs. Nodes in the graph are partitioned into two non-empty groups. In the assortative partition every node requires at least H of their neighbors to be in their own gro ...

IOP Publishing Ltd2022