Principe d'entropie maximale | EPFL Graph Search

Le principe d'entropie maximale consiste, lorsqu'on veut représenter une connaissance imparfaite d'un phénomène par une loi de probabilité, à :

identifier les contraintes auxquelles cette distribution doit répondre (moyenne, etc) ;
choisir de toutes les distributions répondant à ces contraintes celle ayant la plus grande entropie au sens de Shannon.

De toutes ces distributions, c'est en effet celle d'entropie maximale qui contient le moins d'information, et elle est donc pour cette raison la moins arbitraire de toutes celles que l'on pourrait utiliser. La distribution de probabilité obtenue sert ensuite de probabilité a priori dans un processus classique d'inférence bayésienne. Le principe d'entropie maximale considère un principe d'équidistribution (principe d'indifférence de Laplace) et d'indépendance entre événements élémentaires ayant donné lieu à la distribution de probabilité. Il s'agit donc d'un a priori extrêmement « neutre », si toutefois l'espace d'hypothèses est b

Unleashing the power of semantic text analysis: a complex systems approach

Andrea Martini

In the present information era, a huge amount of machine-readable data is available regarding scientific publications. Such unprecedented wealth of data offers the opportunity to investigate science itself as a complex interacting system by means of quantitative approaches. These kind of studies have the potential to provide new insights on the large-scale organization of science and the driving mechanisms underlying its evolution. A particularly important aspect of these data is the semantic information present within publications as it grants access to the concepts used by scientists to describe their findings. Nevertheless, the presence of the so-called buzzwords, \ie terms that are not specific and are used indistinctly in many contexts, hinders the emerging of the thematic organization of scientific articles. In this Thesis, I resume my original contribution to the problem of leveraging the semantic information contained in a corpus of documents. Specifically, I have developed an information-theoretic measure, based on the maximum entropy principle, to quantify the information content of scientific concepts. This measure provides an objective and powerful way to identify generic concepts acting as buzzwords, which increase the noise present in the semantic similarity between articles. I prove that the removal of generic concepts is beneficial in terms of the sparsity of the similarity network, thus allowing the detection of communities of articles that are related to more specific themes. The same effect is observed when describing the corpus of articles in terms of topics, namely clusters of concepts that compose the papers as a mixture. Moreover, I applied the method to a collection of web documents obtaining a similar effect despite their differences with scientific articles. Regarding the scientific knowledge, another important aspect I examine is the temporal evolution of the concept generality, as it may potentially describe typical patterns in the evolution of concepts that can highlight the way in which they are consumed over time.

EPFL2018

Entropy in the Coil-to-Globule Transition of Macromolecules

Carlo Maffi

The coil to globule transition is a fundamental phenomenon in the physics of macro-molecules by reason of the multiplicity of arrangements of their conformation. Such conformational freedom is the main source of entropy in the molecule and is the main opponent to the transition towards the compact state, since a system tends to the state of maximum entropy. This phenomenon is captured by very simple models, such as the ensemble of Interacting Self avoiding Walks on the lattice. This model shows that the coil to globule transition belongs to the universality class of continuous transition called Θ point. Starting from a critical inspection of the definition of the interacting walks model, we introduce a refinement aiming to represent more precisely the entropy sourced from the local fluctuations of the molecule around its equilibrium conformations; this contribution is absent in the standard model which includes only the entropy generated by the multiplicity of the global conformations. Through the study of the vibrational properties of the model and an exhaustive numerical simulation of the phase transition we show that the new model presents a transition which is different in character and belongs to a distinct universality class. In particular, in 3 dimensions the model shows a discontinuous transition; thanks to this, the model presents a common framework underlying the physics of both homo-polymers and proteins. Considering the results obtained in 2 and 3 dimensions we identify the convexity of the vibrational entropy as the responsible of the new class of transitions. These results and a revision of experimental measures of viscosity bring into question the standard description of the homo-polymer transition.

EPFL2012

Irreducible network backbones: unbiased graph filtering via maximum entropy

Alessio Vincenzo Cardillo

Networks provide an informative, yet non-redundant description of complex systems only if links represent truly dyadic relationships that cannot be directly traced back to node-specific properties such as size, importance, or coordinates in some embedding space. In any real-world network, some links may be reducible, and others irreducible, to such local properties. This dichotomy persists despite the steady increase in data availability and resolution, which actually determines an even stronger need for filtering techniques aimed at discerning essential links from non-essential ones. Here we introduce a rigorous method that, for any desired level of statistical significance, outputs the network backbone that is irreducible to the local properties of nodes, i.e. their degrees and strengths. Unlike previous approaches, our method employs an exact maximum-entropy formulation guaranteeing that the filtered network encodes only the links that cannot be inferred from local information. Extensive empirical analysis confirms that this approach uncovers essential backbones that are otherwise hidden amidst many redundant relationships and inaccessible to other methods. For instance, we retrieve the hub-and-spoke skeleton of the US airport network and many specialised patterns of international trade. Being irreducible to local transportation and economic constraints of supply and demand, these backbones single out genuinely higher-order wiring principles.

2017

Unleashing the power of semantic text analysis: a complex systems approach

Andrea Martini

EPFL2018