Biomedical challenges have become the de facto standard for benchmarking\nbiomedical image analysis algorithms. While the number of challenges is\nsteadily increasing, surprisingly little effort has been invested in ensuring\nhigh quality design, execution and reporting for these international\ncompetitions. Specifically, results analysis and visualization in the event of\nuncertainties have been given almost no attention in the literature. Given\nthese shortcomings, the contribution of this paper is two-fold: (1) We present\na set of methods to comprehensively analyze and visualize the results of\nsingle-task and multi-task challenges and apply them to a number of simulated\nand real-life challenges to demonstrate their specific strengths and\nweaknesses; (2) We release the open-source framework challengeR as part of this\nwork to enable fast and wide adoption of the methodology proposed in this\npaper. Our approach offers an intuitive way to gain important insights into the\nrelative and absolute performance of algorithms, which cannot be revealed by\ncommonly applied visualization techniques. This is demonstrated by the\nexperiments performed within this work. Our framework could thus become an\nimportant tool for analyzing and visualizing challenge results in the field of\nbiomedical image analysis and beyond.\n