Publication

Direction is what you need: Improving Word Embedding Compression in Large Language Models

Related publications (30)

Text-to-Microstructure Generation Using Generative Deep Learning

Designing novel materials is greatly dependent on understanding the design principles, physical mechanisms, and modeling methods of material microstructures, requiring experienced designers with expertise and several rounds of trial and error. Although rec ...

Wiley-V C H Verlag Gmbh2024

Advancing Self-Supervised Deep Learning for 3D Scene Understanding

Seyed Mohammad Mahdi Johari

Recent advancements in deep learning have revolutionized 3D computer vision, enabling the extraction of intricate 3D information from 2D images and video sequences. This thesis explores the application of deep learning in three crucial challenges of 3D com ...

EPFL2024

Robust machine learning for neuroscientific inference

Steffen Schneider

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

EPFL2024

Augmenting large language models with chemistry tools

Philippe Schwaller, Oliver Tobias Schilter, Andres Camilo Marulanda Bran, Carlo Baldassari

Large language models (LLMs) have shown strong performance in tasks across domains but struggle with chemistry-related problems. These models also lack access to external knowledge sources, limiting their usefulness in scientific applications. We introduce ...

Nature Portfolio2024

Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange

Sabine Süsstrunk, Mathieu Salzmann, Tong Zhang, Yi Wu

In the realm of point cloud scene understanding, particularly in indoor scenes, objects are arranged following human habits, resulting in objects of certain semantics being closely positioned and displaying notable inter-object correlations. This can creat ...

2024

Aggregating Spatial and Photometric Context for Photometric Stereo

David Honzátko

Photometric stereo, a computer vision technique for estimating the 3D shape of objects through images captured under varying illumination conditions, has been a topic of research for nearly four decades. In its general formulation, photometric stereo is an ...

EPFL2024

Sparse autoregressive neural networks for classical spin systems

Giuseppe Carleo, Dian Wu, Indaco Biazzo

Efficient sampling and approximation of Boltzmann distributions involving large sets of binary variables, or spins, are pivotal in diverse scientific fields even beyond physics. Recent advances in generative neural networks have significantly impacted this ...

Iop Publishing Ltd2024

Network time series forecasting in photovoltaics power production

Jelena Simeunovic

Accurate forecasting of photovoltaic (PV) power production is crucial for the integration of more renewable energy sources into the power grid. PV power production is highly intermittent, due to the stochastic cloud behaviour and cloud dynamics. Previous w ...

EPFL2024

Performing and Detecting Backdoor Attacks on Face Recognition Algorithms

Alexander Carl Unnervik

The field of biometrics, and especially face recognition, has seen a wide-spread adoption the last few years, from access control on personal devices such as phones and laptops, to automated border controls such as in airports. The stakes are increasingly ...

EPFL2024

MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning

Alexandre Massoud Alahi, Mohamed Ossama Ahmed Abdelfattah, Mariam Ahmed Mahmoud Hegazy Hassan

Current transformer-based skeletal action recognition models tend to focus on a limited set of joints and low-level motion patterns to predict action classes. This results in significant performance degradation under small skeleton perturbations or changin ...

2024