Event Date
Speaker: Isabel Papadimitriou (PhD Candidate, Stanford University)
Title: "What can language models tell us about structure in human language, and vice-versa?"
Abstract: My work applies large language models to help understand human language, and uses linguistic theory as a lens for understanding the successes of statistical learning. In this talk I present two lines of work interpreting latent representations and linguistic processing in large language models, focusing on what these experiments tell us about how people and statistical models represent complex linguistic structure. I first examine how language models represent grammatical role, or subjecthood, utilizing the internal high-dimensional space of language models as a model of functional subjecthood. I show new ways that these learned vector space representations help us reconcile continuous, prototype-based analyses of subjecthood with discrete abstract grammar. And I also show how the differences between languages that the training data is sampled from (like nominative and ergative languages) can be represented in an exemplar multilingual vector space representation. In a second line of work, I intervene on language model training to bias models towards different structural priors, analyzing how statistical modeling of raw language data interacts with different linguistic structures. My experiments on structural learning show how some structures make statistical language learning easier, opening pathways for thinking about the role of multifaceted structures in discourse, meaning, and syntax simultaneously guiding statistical learning and language representation. Large language models constitute a flexible, functional model of human language that offers exciting new empirical paradigms for linguistics and statistical learning.
Speaker webpage (links to Stanford): https://nlp.stanford.edu/~isabelvp/
Seminar date/time: Friday, February 23, 10:30am