The CMU Submission for the Shared Task on Language Identification in Code-Switched Data

We describe the CMU submission for the 2014 shared task on language identification in code-switched data. We participated in all four language pairs: Spanish‐English, Mandarin‐English, Nepali‐English, and Modern Standard Arabic‐Arabic dialects. After describing our CRF-based baseline system, we discuss three extensions for learning from unlabeled data: semi-supervised learning, word embeddings, and word lists.

Computer Science Linguistics

View graph Source

Paper

Full text

The CMU Submission for the Shared Task on Language Identification in Code-Switched Data

Semantic Scholar · Computer Science · 2014

Abstract

We describe the CMU submission for the 2014 shared task on language identification in code-switched data. We participated in all four language pairs: Spanish‐English, Mandarin‐English, Nepali‐English, and Modern Standard Arabic‐Arabic dialects. After describing our CRF-based baseline system, we discuss three extensions for learning from unlabeled data: semi-supervised learning, word embeddings, and word lists.

Source of record · Semantic Scholar

References (17)

01Conditional Random Field Autoencoders for Unsupervised Structured PredictionBridger Waleed Ammar, Chris Dyer, Noah A. Smith2014 · Neural Information Processing Systems · 79 citations In Library

02Overview for the Second Shared Task on Language Identification in Code-Switched DataGiovanni Molina, Fahad AlGhamdi, Mahmoud A. Ghoneim et al.2014 · CodeSwitch@EMNLP · 175 citations In Library

03Code switch language modeling with Functional Head ConstraintYing Li, Pascale Fung2014 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 20 citations

04Accurate Language Identification of Twitter MessagesMarco Lui, Timothy Baldwin2014 · 97 citations

05Automatic Detection and Language Identification of Multilingual DocumentsMarco Lui, Jey Han Lau, Timothy Baldwin2014 · Transactions of the Association for Computational Linguistics · 132 citations

06Word Level Language Identification in Online Multilingual CommunicationDong Nguyen, A. Seza Doğruöz2013 · Conference on Empirical Methods in Natural Language Processing · 158 citations

07Code Switch Point Detection in ArabicHeba Elfardy, Mohamed Al-Badrashiny, Mona T. Diab2013 · International Conference on Applications of Natural Language to Data Bases · 48 citations

08Improved mixed language speech recognition using asymmetric acoustic model and language model with code-switch inversion constraintsYing Li, Pascale Fung2013 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 50 citations

09Efficient Estimation of Word Representations in Vector SpaceTomas Mikolov, Kai Chen, G. Corrado et al.2013 · International Conference on Learning Representations · 35k citations In Library

10Word Representations: A Simple and General Method for Semi-Supervised LearningJoseph P. Turian, Lev-Arie Ratinov, Yoshua Bengio2010 · Annual Meeting of the Association for Computational Linguistics · 2.3k citations

11Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence DataJ. Lafferty, A. McCallum, Fernando Pereira2001 · International Conference on Machine Learning · 16k citations

12On the limited memory BFGS method for large scale optimizationDong C. Liu, J. Nocedal1989 · Mathematical programming · 8.9k citations

Scroll for more · 5 remaining

Similar papers

Overview for the First Shared Task on Language Identification in Code-Switched DataT. Solorio, Elizabeth Blair, Suraj Maharjan et al.2014 · CodeSwitch@EMNLP · 271 citations In Library

Overview for the Second Shared Task on Language Identification in Code-Switched DataGiovanni Molina, Fahad AlGhamdi, Mahmoud A. Ghoneim et al.2014 · CodeSwitch@EMNLP · 175 citations In Library

Language Identification in Code-Switching ScenarioNaman Jain, Riyaz Ahmad Bhat2014 · CodeSwitch@EMNLP · 26 citations In Library

Word-level Language Identification using CRF: Code-switching Shared Task Report of MSR India SystemGokul Chittaranjan, Yogarshi Vyas, Kalika Bali et al.2014 · CodeSwitch@EMNLP · 79 citations In Library

The Howard University System Submission for the Shared Task in Language Identification in Spanish-English CodeswitchingRouzbeh A. Shirvani, Mario Piergallini, G. S. Gautam et al.2016 · CodeSwitch@EMNLP · 21 citations In Library

CALCS 2021 Shared Task: Machine Translation for Code-Switched DataShuguang Chen, Gustavo Aguilar, Anirudh Srinivasan et al.2022 · arXiv (Cornell University) · 18 citations In Library

The IUCL+ System: Word-Level Language Identification via Extended Markov ModelsLevi King, Eric Baucom, T. Gilmanov et al.2014 · CodeSwitch@EMNLP · 25 citations In Library

Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer LearningShuyue Stella Li, Cihan Xiao, Tianjian Li et al.2023 · arXiv (Cornell University) · 4 citations In Library

Language Identification in Code-Switched Text Using Conditional Random Fields and BabelnetUtpal Kumar Sikdar, Björn Gambäck2016 · CodeSwitch@EMNLP · 10 citations In Library

Language Identification and Analysis of Code-Switched Social Media TextDeepthi Mave, Suraj Maharjan, Thamar Solorio2018 In Library

Source of record

Semantic Scholar

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal