Speaking of Language: Reflections on Metalanguage Research in NLP

This work aims to shine a spotlight on the topic of metalanguage. We first define metalanguage, link it to NLP and LLMs, and then discuss our two labs'metalanguage-centered efforts. Finally, we discuss four dimensions of metalanguage and metalinguistic tasks, offering a list of understudied future research directions.

Language & NLP cs.CL cs.AI

View graph Source

Paper

References (83)

01LingGym: How Far Are LLMs from Thinking Like Field Linguists?Changbing Yang, F. Ma, Freda Shi et al.2025 · Conference on Empirical Methods in Natural Language Processing · 4 citations In Library

02Prompting from the bench: Large-scale pretraining is not sufficient to prepare LLMs for ordinary meaning analysisAbhishek Purushothama, Junghyun Min, Brandon Waldon et al.2025 · Conference on Fairness, Accountability and Transparency · 5 citations In Library

03Explicit Learning and the LLM in Machine TranslationMalik Marmonier, Rachel Bawden, Benoît Sagot2025 · Conference on Empirical Methods in Natural Language Processing · 6 citations In Library

04Language Models Fail to Introspect About Their Knowledge of LanguageSiyuan Song, Jennifer Hu, Kyle Mahowald2025 · arXiv.org · 27 citations In Library

05Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on ManchuRenhao Pei, Yihong Liu, Peiqin Lin et al.2025 · Annual Meeting of the Association for Computational Linguistics · 20 citations In Library

06Back to School: Translation Using Grammar BooksJonathan Hus, Antonios Anastasopoulos2024 · Conference on Empirical Methods in Natural Language Processing · 19 citations In Library

07Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?Seth Aycock, David Stap, Di Wu et al.2024 · International Conference on Learning Representations · 36 citations In Library

08Linguini: A benchmark for language-agnostic linguistic reasoningEduardo Sánchez, Belen Alastruey, C. Ropers et al.2024 · Neural Information Processing Systems · 22 citations In Library

09Shortcomings of LLMs for Low-Resource Translation: Retrieval and Understanding are Both the ProblemSara Court, Micha Elsner2024 · Conference on Machine Translation · 24 citations In Library

10LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct LanguagesAndrew M. Bean, Simi Hellsten, Harry Mayne et al.2024 · Neural Information Processing Systems · 39 citations In Library

11Natural Language Processing RELIES on LinguisticsJuri Opitz, Shira Wein, Nathan Schneider2024 · Computational Linguistics · 16 citations In Library

12NLP Systems That Can’t Tell Use from Mention Censor Counterspeech, but Teaching the Distinction HelpsKristina Gligoric, Myra Cheng, Lucia Zheng et al.2024 · North American Chapter of the Association for Computational Linguistics · 22 citations In Library

Scroll for more · 38 remaining

Similar papers

METAL: Towards Multilingual Meta-EvaluationRishav Hada, Varun Gumma, Mohamed Ahmed et al.2024 In Library

METAL: Towards Multilingual Meta-EvaluationRishav Hada, Varun Gumma, Mohamed Ahmed et al.2024 · arXiv (Cornell University) In Library

LLM “metalinguistic” capabilities are creating a stir – here's whyAnna Demming2025 · Significance In Library

Natural Semantic MetalanguageCliff Goddard2021 In Library

Logic, Language, and CalculusFlorian Richter2020 · arXiv (Cornell University) In Library

METALINGUISTIC CONCEPTS: STRUCTURE, CLASSIFICATION AND ANALYSISM.M. Najmiddinova2025 · PHILOLOGICAL SCIENCES JOURNAL In Library

The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating ItZheng-Xin Yong, B. Ermiş, Marzieh Fadaee et al.2025 · Conference on Empirical Methods in Natural Language Processing · 39 citations In Library

Meta-LLM: Language Models as Topological Inquiry Systemsmargolin, ido2025 · Zenodo (CERN European Organization for Nuclear Research) In Library

Meta-LLM: Language Models as Topological Inquiry Systemsmargolin, ido2025 · Zenodo (CERN European Organization for Nuclear Research) In Library

Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on UnderstandingBram van Dijk, Tom Kouwenhoven, Marco Spruit et al.2023 In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal