Publications related to Beyond fine-tuning: LoRA modules boost near-OOD detection and LLM security

Beyond fine-tuning: LoRA modules boost near-OOD detection and LLM security

Pierre Vandergheynst, Milos Vasic, Francesco Craighero, Renata Khasanova

Under resource constraints, LLMs are usually fine- tuned with additional knowledge using Parameter Efficient Fine-Tuning (PEFT), using Low-Rank Adaptation (LoRA) modules. In fact, LoRA injects a new set of small trainable matrices to adapt an LLM to a new ...

2024

Augmenting large language models with chemistry tools

Philippe Schwaller, Oliver Tobias Schilter, Andres Camilo Marulanda Bran, Carlo Baldassari

Large language models (LLMs) have shown strong performance in tasks across domains but struggle with chemistry-related problems. These models also lack access to external knowledge sources, limiting their usefulness in scientific applications. We introduce ...

Nature Portfolio2024

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Regressive Tomography and Direct Regressive Reconstruction: A Research Note

Henri Weisen

Two related methods for inverting line-integrated measurements are presented in this research paper in the context of the recent deuterium-tritium experiments in the JET tokamak. Unlike traditional methods of tomography, these methods rely on making use of ...

TAYLOR & FRANCIS INC2023

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi, Fedor Moiseev

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

End-to-End Task-Oriented Dialog Modeling With Semi-Structured Knowledge Management

Antoine Bosselut, Silin Gao

Current task-oriented dialog (TOD) systems mostly manage structured knowledge (e.g. databases and tables) to guide the goal-oriented conversations. However, they fall short of handling dialogs which also involve unstructured knowledge (e.g. reviews and doc ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

Jean-Philippe Thiran, Guillaume Marc Georges Vray, Devavrat Tomar

As distribution shifts are inescapable in realistic clinical scenarios due to inconsistencies in imaging protocols, scanner vendors, and across different centers, well-trained deep models incur a domain generalization problem in unseen environments. Despit ...

PMLR2022

Horizontal Healing

Anja Fröhlich

A Concideration of Typologies for Housing the Sick as a Spatial Manifestation of knowledge. ...

EPFL Press2022

Interpreting Language Models Through Knowledge Graph Extraction

Martin Jaggi, Vinitra Swamy, Angeliki Romanou

Transformer-based language models trained on large text corpora have enjoyed immense popularity in the natural language processing community and are commonly used as a starting point for downstream tasks. While these models are undeniably useful, it is a c ...

2021

Sublinear Bounds on the Distinguishing Advantage for Multiple Samples

Serge Vaudenay

The maximal achievable advantage of a (computationally unbounded) distinguisher to determine whether a source

Z

is distributed according to distribution

P_0

or

P_1

, when given access to one sample of

Z

, is characterized by the statistical distance ...

Springer, Cham2020

Beyond fine-tuning: LoRA modules boost near-OOD detection and LLM security

Graph Chatbot

Chat with Graph Search

Beyond fine-tuning: LoRA modules boost near-OOD detection and LLM security

Augmenting large language models with chemistry tools

Improving Generalization of Pretrained Language Models

Regressive Tomography and Direct Regressive Reconstruction: A Research Note

SKILL: Structured Knowledge Infusion for Large Language Models

End-to-End Task-Oriented Dialog Modeling With Semi-Structured Knowledge Management

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

Horizontal Healing

Interpreting Language Models Through Knowledge Graph Extraction

Sublinear Bounds on the Distinguishing Advantage for Multiple Samples

Horizontal Healing

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

Augmenting large language models with chemistry tools

Sublinear Bounds on the Distinguishing Advantage for Multiple Samples

Improving Generalization of Pretrained Language Models

SKILL: Structured Knowledge Infusion for Large Language Models

Regressive Tomography and Direct Regressive Reconstruction: A Research Note

End-to-End Task-Oriented Dialog Modeling With Semi-Structured Knowledge Management

Interpreting Language Models Through Knowledge Graph Extraction

Beyond fine-tuning: LoRA modules boost near-OOD detection and LLM security