Bangla -- ranked as the 6th most widely spoken language across the world\n(https://www.ethnologue.com/guides/ethnologue200), with 230 million native\nspeakers -- is still considered as a low-resource language in the natural\nlanguage processing (NLP) community. With three decades of research, Bangla NLP\n(BNLP) is still lagging behind mainly due to the scarcity of resources and the\nchallenges that come with it. There is sparse work in different areas of BNLP;\nhowever, a thorough survey reporting previous work and recent advances is yet\nto be done. In this study, we first provide a review of Bangla NLP tasks,\nresources, and tools available to the research community; we benchmark datasets\ncollected from various platforms for nine NLP tasks using current\nstate-of-the-art algorithms (i.e., transformer-based models). We provide\ncomparative results for the studied NLP tasks by comparing monolingual vs.\nmultilingual models of varying sizes. We report our results using both\nindividual and consolidated datasets and provide data splits for future\nresearch. We reviewed a total of 108 papers and conducted 175 sets of\nexperiments. Our results show promising performance using transformer-based\nmodels while highlighting the trade-off with computational costs. We hope that\nsuch a comprehensive survey will motivate the community to build on and further\nadvance the research on Bangla NLP.\n