Deep reinforcement learning has been one of the fastest growing fields of\nmachine learning over the past years and numerous libraries have been open\nsourced to support research. However, most codebases have a steep learning\ncurve or limited flexibility that do not satisfy a need for fast prototyping in\nfundamental research. This paper introduces Tonic, a Python library allowing\nresearchers to quickly implement new ideas and measure their importance by\nproviding: 1) general-purpose configurable modules 2) several baseline agents:\nA2C, TRPO, PPO, MPO, DDPG, D4PG, TD3 and SAC built with these modules 3)\nsupport for TensorFlow 2 and PyTorch 4) support for continuous-control\nenvironments from OpenAI Gym, DeepMind Control Suite and PyBullet 5) scripts to\nexperiment in a reproducible way, plot results, and play with trained agents 6)\na benchmark of the provided agents on 70 continuous-control tasks. Evaluation\nis performed in fair conditions with identical seeds, training and testing\nloops, while sharing general improvements such as non-terminal timeouts and\nobservation normalization. Finally, to demonstrate how Tonic simplifies\nexperimentation, a novel agent called TD4 is implemented and evaluated.\n