The marginal likelihood or evidence in Bayesian statistics contains an\nintrinsic penalty for larger model sizes and is a fundamental quantity in\nBayesian model comparison. Over the past two decades, there has been steadily\nincreasing activity to understand the nature of this penalty in singular\nstatistical models, building on pioneering work by Sumio Watanabe. Unlike\nregular models where the Bayesian information criterion (BIC) encapsulates a\nfirst-order expansion of the logarithm of the marginal likelihood, parameter\ncounting gets trickier in singular models where a quantity called the real log\ncanonical threshold (RLCT) summarizes the effective model dimensionality. In\nthis article, we offer a probabilistic treatment to recover non-asymptotic\nversions of established evidence bounds as well as prove a new result based on\nthe Gibbs variational inequality. In particular, we show that mean-field\nvariational inference correctly recovers the RLCT for any singular model in its\ncanonical or normal form. We additionally exhibit sharpness of our bound by\nanalyzing the dynamics of a general purpose coordinate ascent algorithm (CAVI)\npopularly employed in variational inference.\n