Purpose To evaluate the efficacy of two self-supervised learning (SSL) methods (inpainting-based pretext tasks of context prediction and context restoration) for medical image segmentation in label-limited scenarios, and to investigate the effect of implementation design choices for SSL on downstream segmentation performance. Methods Manual segmentation labels were created for 3D knee MRI and 2D abdominal CT datasets. Multiple versions of self-supervised U-Net models were trained to segment tissues in both datasets, each using a different combination of design choices and pretext tasks to determine the effect of different design choices on segmentation performance. The combination of these design choices that resulted in the most significant improvement in Dice score over supervised learning for both datasets was used to train an optimally trained model for segmentation. This model was pretrained on different amounts of unlabeled data to determine the effect of pretraining dataset size on segmentation performance. The highest performing models from this experiment were compared with baseline supervised models for computing clinically-relevant metrics in label-limited scenarios. Results SSL pretraining with context restoration using 32x32 patches and Poission-disc sampling, transferring only the pretrained encoder weights, and fine-tuning immediately with an initial learning rate of 1e-3 provided the most benefit over supervised learning for MRI and CT tissue segmentation accuracy (p < 0.001). For both datasets and most label-limited scenarios, pretraining using the maximum amount of unlabeled images resulted in better segmentation performance than pretraining using only the training set (p < 0.05). SSL models pretrained with this amount of data also outperformed baseline supervised models in the computa-tion of clinically-relevant metrics in scenarios with very low amounts of labeled data, especially for challenging classes to segment such as intramuscular adipose tissue on CT images and patellar cartilage on MR images. Conclusion We demonstrate how SSL can overcome paucity of labels for improving tissue segmentation by using unlabeled datasets.