Continual Learning (CL) aims to develop agents emulating the human ability to\nsequentially learn new tasks while being able to retain knowledge obtained from\npast experiences. In this paper, we introduce the novel problem of\nMemory-Constrained Online Continual Learning (MC-OCL) which imposes strict\nconstraints on the memory overhead that a possible algorithm can use to avoid\ncatastrophic forgetting. As most, if not all, previous CL methods violate these\nconstraints, we propose an algorithmic solution to MC-OCL: Batch-level\nDistillation (BLD), a regularization-based CL approach, which effectively\nbalances stability and plasticity in order to learn from data streams, while\npreserving the ability to solve old tasks through distillation. Our extensive\nexperimental evaluation, conducted on three publicly available benchmarks,\nempirically demonstrates that our approach successfully addresses the MC-OCL\nproblem and achieves comparable accuracy to prior distillation methods\nrequiring higher memory overhead.\n