Publications

2022

1. Preprint

   Language models and brain alignment: beyond word-level semantics and prediction

   Gabriele Merlin, and Mariya Toneva

   2022

   Abs Bib HTML

   Pretrained language models that have been trained to predict the next word over billions of text documents have been shown to also significantly predict brain recordings of people comprehending language. Understanding the reasons behind the observed similarities between language in machines and language in the brain can lead to more insight into both systems. Recent works suggest that the prediction of the next word is a key mechanism that contributes to the alignment between the two. What is not yet understood is whether prediction of the next word is necessary for this observed alignment or simply sufficient, and whether there are other shared mechanisms or information that is similarly important. In this work, we take a first step towards a better understanding via two simple perturbations in a popular pretrained language model. The first perturbation is to improve the model’s ability to predict the next word in the specific naturalistic stimulus text that the brain recordings correspond to. We show that this indeed improves the alignment with the brain recordings. However, this improved alignment may also be due to any improved word-level or multi-word level semantics for the specific world that is described by the stimulus narrative. We aim to disentangle the contribution of next word prediction and semantic knowledge via our second perturbation: scrambling the word order at inference time, which reduces the ability to predict the next word, but maintains any newly learned word-level semantics. By comparing the alignment with brain recordings of these differently perturbed models, we show that improvements in alignment with brain recordings are due to more than improvements in next word prediction and word-level semantics.

   @misc{merlinlanguage,
     doi = {10.48550/ARXIV.2212.00596},
     author = {Merlin, Gabriele and Toneva, Mariya},
     keywords = {Computation and Language (cs.CL), Neurons and Cognition (q-bio.NC), FOS: Computer and information sciences, FOS: Computer and information sciences, FOS: Biological sciences, FOS: Biological sciences},
     title = {Language models and brain alignment: beyond word-level semantics and prediction},
     publisher = {arXiv},
     year = {2022},
     copyright = {arXiv.org perpetual, non-exclusive license}
   }

2. ICIAP22

   Practical Recommendations for Replay-Based Continual Learning Methods

   Gabriele Merlin, Vincenzo Lomonaco, Andrea Cossu, Antonio Carta, and Davide Bacciu

   In Image Analysis and Processing. ICIAP 2022 Workshops 2022

   Abs Bib HTML

   Continual Learning requires the model to learn from a stream of dynamic, non-stationary data without forgetting previous knowledge. Several approaches have been developed in the literature to tackle the Continual Learning challenge. Among them, Replay approaches have empirically proved to be the most effective ones [16]. Replay operates by saving some samples in memory which are then used to rehearse knowledge during training in subsequent tasks. However, an extensive comparison and deeper understanding of different replay implementation subtleties is still missing in the literature. The aim of this work is to compare and analyze existing replay-based strategies and provide practical recommendations on developing efficient, effective and generally applicable replay-based strategies. In particular, we investigate the role of the memory size value, different weighting policies and discuss about the impact of data augmentation, which allows reaching better performance with lower memory sizes.

   @inproceedings{merlinpractical,
     author = {Merlin, Gabriele and Lomonaco, Vincenzo and Cossu, Andrea and Carta, Antonio and Bacciu, Davide},
     editor = {Mazzeo, Pier Luigi and Frontoni, Emanuele and Sclaroff, Stan and Distante, Cosimo},
     title = {Practical Recommendations for Replay-Based Continual Learning Methods},
     booktitle = {Image Analysis and Processing. ICIAP 2022 Workshops},
     year = {2022},
     publisher = {Springer International Publishing},
     address = {Cham},
     pages = {548--559},
     isbn = {978-3-031-13324-4},
   }

2019

1. STAG19

   Design and Implementation of a Visualization Tool for the in-depth Analysis of the Domestic Electricity Consumption

   Gabriele Merlin, Daniele Ortu, Gianmarco Cherchi, and Riccardo Scateni

   In Smart Tools and Apps for Graphics - Eurographics Italian Chapter Conference 2019

   Abs Bib HTML

   In this poster, we present a visualization tool for the in-depth analysis of domestic electricity consumption. The web-interface allows users to visualize their electricity consumption, compare them with their own records or with the means of selected communities

   @inproceedings{merlindesign,
     booktitle = {Smart Tools and Apps for Graphics - Eurographics Italian Chapter Conference},
     editor = {Agus, Marco and Corsini, Massimiliano and Pintus, Ruggero},
     title = {{Design and Implementation of a Visualization Tool for the in-depth Analysis of the Domestic Electricity Consumption}},
     author = {Merlin, Gabriele and Ortu, Daniele and Cherchi, Gianmarco and Scateni, Riccardo},
     year = {2019},
     publisher = {The Eurographics Association},
     issn = {2617-4855},
     isbn = {978-3-03868-100-7},
     doi = {10.2312/stag.20191368},
   }