Text generation is a highly active area of research in the computational\nlinguistic community. The evaluation of the generated text is a challenging\ntask and multiple theories and metrics have been proposed over the years.\nUnfortunately, text generation and evaluation are relatively understudied due\nto the scarcity of high-quality resources in code-mixed languages where the\nwords and phrases from multiple languages are mixed in a single utterance of\ntext and speech. To address this challenge, we present a corpus (HinGE) for a\nwidely popular code-mixed language Hinglish (code-mixing of Hindi and English\nlanguages). HinGE has Hinglish sentences generated by humans as well as two\nrule-based algorithms corresponding to the parallel Hindi-English sentences. In\naddition, we demonstrate the inefficacy of widely-used evaluation metrics on\nthe code-mixed data. The HinGE dataset will facilitate the progress of natural\nlanguage generation research in code-mixed languages.\n