Linear representations of grammaticality in neural language models
Jane Li, Najoung Kim
Why It Matters
What makes this one worth your time
Understanding how NLMs encode grammaticality can enhance our insights into their syntactic knowledge and improve evaluations of their linguistic competence.
This study reveals that grammaticality is distinctly represented in neural language models, independent of other sentence properties.
Summary
The paper investigates whether neural language models encode grammaticality in their internal representations, moving beyond traditional probability-based evaluations to assess representational separation of grammatical and ungrammatical sentences.
Key contributions
- Demonstration of grammaticality encoding in NLMs through representational space analysis.
- Evidence of generalization of grammaticality representations across various grammatical phenomena and languages.
- A new framework for evaluating grammatical competence that does not rely on probability comparisons.
Notable insights
- The use of mass-mean probing provides a novel approach to assess representational space in NLMs.
- The findings suggest a coherent representational dimension for grammaticality that transcends mere probability measures.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2607.15175v1 Announce Type: new Abstract: Whether neural language models (NLMs) possess the ability to distinguish strings on the basis of their grammaticality remains a debated topic in the computational linguistics literature. Existing evidence has largely relied on probability-based measures, testing whether models assign higher probabilities to grammatical than ungrammatical strings. However, probability comparisons have been criticized as a measure for grammatical knowledge based on the assumption that grammaticality is inherently entangled with likelihood. Model-assigned probability is a function of many related sentence properties, such as lexical frequency, plausibility, and world knowledge. In this work, we move beyond probability-based evaluations and investigate whether grammaticality is encoded in the internal representations of NLMs. Using mass-mean probing, we test whether grammatical and ungrammatical sentences are systematically separated in representational space. We further examine the extent to which these representations are independent of sentence properties that are correlated with grammaticality, as well as their generalization across grammatical phenomena and languages. Our results provide evidence that grammaticality is robustly encoded in sentence representations of a wide range of pretrained NLMs, yielding clear representational separation on the dimension of grammaticality that cannot be fully explained by alternative sentence-level factors. Moreover, this encoding generalizes across a broad range of grammatical phenomena and to some degree, across languages, suggesting that grammaticality constitutes a coherent representational dimension in contemporary NLMs. These findings contribute new evidence to debates about the nature of syntactic knowledge in language models and offer a complementary framework for evaluating grammatical competence that is not dependent on string probabilities alone.