Deep segmentation neural networks require large training datasets with\npixel-wise segmentations, which are expensive to obtain in practice. Mixed\nsupervision could mitigate this difficulty, with a small fraction of the data\ncontaining complete pixel-wise annotations, while the rest being less\nsupervised, e.g., only a handful of pixels are labeled. In this work, we\npropose a dual-branch architecture, where the upper branch (teacher) receives\nstrong annotations, while the bottom one (student) is driven by limited\nsupervision and guided by the upper branch. In conjunction with a standard\ncross-entropy over the labeled pixels, our novel formulation integrates two\nimportant terms: (i) a Shannon entropy loss defined over the less-supervised\nimages, which encourages confident student predictions at the bottom branch;\nand (ii) a Kullback-Leibler (KL) divergence, which transfers the knowledge from\nthe predictions generated by the strongly supervised branch to the\nless-supervised branch, and guides the entropy (student-confidence) term to\navoid trivial solutions. Very interestingly, we show that the synergy between\nthe entropy and KL divergence yields substantial improvements in performances.\nFurthermore, we discuss an interesting link between Shannon-entropy\nminimization and standard pseudo-mask generation and argue that the former\nshould be preferred over the latter for leveraging information from unlabeled\npixels. Through a series of quantitative and qualitative experiments, we show\nthe effectiveness of the proposed formulation in segmenting the left-ventricle\nendocardium in MRI images. We demonstrate that our method significantly\noutperforms other strategies to tackle semantic segmentation within a\nmixed-supervision framework. More interestingly, and in line with recent\nobservations in classification, we show that the branch trained with reduced\nsupervision largely outperforms the teacher.\n