GODEY Nathan

PhD student


nathan.godey [at] inria.fr

Short bio

Masters of Engineering, Ecole des Ponts

Thesis topic

Cheap and expressive neural contextual representations for textual data.

Short abstract

Neural language models are pre-trained using self-supervised learning to produce contextual representations of text data like words or sentences. These representations are shaped by the pre-training procedure: its data, its task, its optimization scheme, among others. I aim at identifying ways to improve the quality of the text representations by leveraging new pre-training approaches, in order to reduce data and/or compute requirements without quality loss.