Publication

Direction is what you need: Improving Word Embedding Compression in Large Language Models

Related publications (30)

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

EPFL2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

Driving and suppressing the human language network using large language models

Martin Schrimpf

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict t ...

Nature Portfolio2024

Social-Transmotion: Promptable Human Trajectory Prediction

Alexandre Massoud Alahi, Saeed Saadatnejad, Yang Gao, Kaouther Messaoud Ben Amor

Accurate human trajectory prediction is crucial for applications such as autonomous vehicles, robotics, and surveillance systems. Yet, existing models often fail to fully leverage the non-verbal social cues human subconsciously communicate when navigating ...

2024

The multimodality cell segmentation challenge: toward universal solutions

Sahand Jamal Rahi, Vojislav Gligorovski, Marco Labagnara, Jun Ma, Maxime Emmanuel Scheder, Xin Yang, Yao Zhang, Bo Wang, Yixin Wang, Lin Han

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimen ...

Nature Portfolio2024

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Jan Sickmann Hesthaven, Federico Pichi

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real -time and many-query evaluations of parametric ...

Academic Press Inc Elsevier Science2024

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Pakshal Narendra Bohra

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Subjective performance evaluation of bitrate allocation strategies for MPEG and JPEG Pleno point cloud compression

Touradj Ebrahimi, Michela Testolina, Davi Nachtigall Lazzarotto

The recent rise in interest in point clouds as an imaging modality has motivated standardization groups such as JPEG and MPEG to launch activities aiming at developing compression standards for point clouds. Lossy compression usually introduces visual arti ...

Springer2024

Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling

Robert West, Maxime Jean Julien Peyrard, Marija Sakota

Generative language models (LMs) have become omnipresent across data science. For a wide variety of tasks, inputs can be phrased as natural language prompts for an LM, from whose output the solution can then be extracted. LM performance has consistently be ...

Assoc Computing Machinery2024

Topics in statistical physics of high-dimensional machine learning

Hugo Chao Cui

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

EPFL2024