Reducing patient radiation exposure in Cone Beam Computed Tomography (CBCT) by acquiring fewer projections introduces severe image artefacts, limiting its clinical utility. To address this challenge, we propose MInDI-3D (Medical Inversion by Direct Iteration in 3D), a 3D conditional diffusion framework that restores volumetric data from sparse-view inputs. Our work provides two key contributions: 1) The MInDI-3D model, the first adaptation of the iterative inversion principle to fully 3D medical volumes, which offers a unique, tuneable trade-off between perceptual quality and quantitative fidelity by adjusting the number of inference steps. 2) A new, publicly available, large-scale dataset of 16,182 pseudo-CBCT volumes to facilitate robust training and future research. On an independent real-world CBCT test set, MInDI-3D achieves performance competitive with state-of-the-art methods, yielding a 0.54 SSIM gain over standard reconstructions from only 25 projections. This result enables a 16-fold reduction in radiation exposure and demonstrates robust generalisation to a new scanner geometry not seen during training. Beyond standard metrics, MInDI-3D reconstructions preserved high anatomical integrity, enabling accurate automated segmentation in task-based evaluations. In a clinical evaluation by 11 radiotherapy specialists, the reconstructions were rated as sufficient for patient positioning across all tested anatomical sites (abdomen, breast, and lung) and were noted to preserve lung tumour boundaries well.