Gabriele Merlin

Room 516, E1 5

MPI-SWS

Saarbrücken, 66123

I’m a Ph.D. student at the Max Planck Institute for Software Systems (MPI-SWS) under the CS@max planck program. I am advised by Krishna Gummadi and I am working with Mariya Toneva in the Bridging AI and Neuroscience Group (BrAIN). My research interests lie at the intersection between Machine Learning and Neuroscience. The goal is to investigate and develop brain-inspired machine learning systems as well as gain a better understanding of how the brain processes information using machine learning methods.

I received my Bachelor’s Degree in Computer Science from the University of Cagliari and my Master’s Degree in Computer Science from the University of Pisa.

Publications

EMNLP24
Language models and brains align due to more than next-word prediction and word-level information

Gabriele Merlin, and Mariya Toneva

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing 2024

Abs Bib HTML

Pretrained language models have been shown to significantly predict brain recordings of people comprehending language. Recent work suggests that the prediction of the next word is a key mechanism that contributes to this alignment. What is not yet understood is whether prediction of the next word is necessary for this observed alignment or simply sufficient, and whether there are other shared mechanisms or information that are similarly important. In this work, we take a step towards understanding the reasons for brain alignment via two simple perturbations in popular pretrained language models. These perturbations help us design contrasts that can control for different types of information. By contrasting the brain alignment of these differently perturbed models, we show that improvements in alignment with brain recordings are due to more than improvements in next-word prediction and word-level information.
@inproceedings{merlin-toneva-2024-language, title = {Language models and brains align due to more than next-word prediction and word-level information}, author = {Merlin, Gabriele and Toneva, Mariya}, editor = {Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung}, booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing}, year = {2024}, address = {Miami, Florida, USA}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.emnlp-main.1024}, pages = {18431--18454}, }
CoLLAs23
What happens during finetuning of vision transformers: An invariance based investigation

Gabriele Merlin, Vedant Nanda, Ruchit Rawal, and Mariya Toneva

In Conference on Lifelong Learning Agents 2023

Abs Bib HTML

The pretrain-finetune paradigm usually improves downstream performance over training a model from scratch on the same task, becoming commonplace across many areas of machine learning. While pretraining is empirically observed to be beneficial for a range of tasks, there is not a clear understanding yet of the reasons for this effect. In this work, we examine the relationship between pretrained vision transformers and the corresponding finetuned versions on several benchmark datasets and tasks. We present new metrics that specifically investigate the degree to which invariances learned by a pretrained model are retained or forgotten during finetuning. Using these metrics, we present a suite of empirical findings, including that pretraining induces transferable invariances in shallow layers and that invariances from deeper pretrained layers are compressed towards shallower layers during finetuning. Together, these findings contribute to understanding some of the reasons for the successes of pretrained models and the changes that a pretrained model undergoes when finetuned on a downstream task.
@inproceedings{merlin2023happens, title = {What happens during finetuning of vision transformers: An invariance based investigation}, author = {Merlin, Gabriele and Nanda, Vedant and Rawal, Ruchit and Toneva, Mariya}, booktitle = {Conference on Lifelong Learning Agents}, pages = {601--619}, year = {2023}, organization = {PMLR} }
ICIAP22
Practical Recommendations for Replay-Based Continual Learning Methods

Gabriele Merlin, Vincenzo Lomonaco, Andrea Cossu, Antonio Carta, and Davide Bacciu

In Image Analysis and Processing. ICIAP 2022 Workshops 2022

Abs Bib HTML

Continual Learning requires the model to learn from a stream of dynamic, non-stationary data without forgetting previous knowledge. Several approaches have been developed in the literature to tackle the Continual Learning challenge. Among them, Replay approaches have empirically proved to be the most effective ones [16]. Replay operates by saving some samples in memory which are then used to rehearse knowledge during training in subsequent tasks. However, an extensive comparison and deeper understanding of different replay implementation subtleties is still missing in the literature. The aim of this work is to compare and analyze existing replay-based strategies and provide practical recommendations on developing efficient, effective and generally applicable replay-based strategies. In particular, we investigate the role of the memory size value, different weighting policies and discuss about the impact of data augmentation, which allows reaching better performance with lower memory sizes.
@inproceedings{merlinpractical, author = {Merlin, Gabriele and Lomonaco, Vincenzo and Cossu, Andrea and Carta, Antonio and Bacciu, Davide}, editor = {Mazzeo, Pier Luigi and Frontoni, Emanuele and Sclaroff, Stan and Distante, Cosimo}, title = {Practical Recommendations for Replay-Based Continual Learning Methods}, booktitle = {Image Analysis and Processing. ICIAP 2022 Workshops}, year = {2022}, publisher = {Springer International Publishing}, address = {Cham}, pages = {548--559}, isbn = {978-3-031-13324-4}, }

STAG19

Design and Implementation of a Visualization Tool for the in-depth Analysis of the Domestic Electricity Consumption

Gabriele Merlin, Daniele Ortu, Gianmarco Cherchi, and Riccardo Scateni

In Smart Tools and Apps for Graphics - Eurographics Italian Chapter Conference 2019

Abs Bib HTML

@inproceedings{merlindesign,
  booktitle = {Smart Tools and Apps for Graphics - Eurographics Italian Chapter Conference},
  editor = {Agus, Marco and Corsini, Massimiliano and Pintus, Ruggero},
  title = {{Design and Implementation of a Visualization Tool for the in-depth Analysis of the Domestic Electricity Consumption}},
  author = {Merlin, Gabriele and Ortu, Daniele and Cherchi, Gianmarco and Scateni, Riccardo},
  year = {2019},
  publisher = {The Eurographics Association},
  issn = {2617-4855},
  isbn = {978-3-03868-100-7},
  doi = {10.2312/stag.20191368},
}