Endowing robots with the capability of assessing risk and making risk-aware\ndecisions is widely considered a key step toward ensuring safety for robots\noperating under uncertainty. But, how should a robot quantify risk? A natural\nand common approach is to consider the framework whereby costs are assigned to\nstochastic outcomes - an assignment captured by a cost random variable.\nQuantifying risk then corresponds to evaluating a risk metric, i.e., a mapping\nfrom the cost random variable to a real number. Yet, the question of what\nconstitutes a "good" risk metric has received little attention within the\nrobotics community. The goal of this paper is to explore and partially address\nthis question by advocating axioms that risk metrics in robotics applications\nshould satisfy in order to be employed as rational assessments of risk. We\ndiscuss general representation theorems that precisely characterize the class\nof metrics that satisfy these axioms (referred to as distortion risk metrics),\nand provide instantiations that can be used in applications. We further discuss\npitfalls of commonly used risk metrics in robotics, and discuss additional\nproperties that one must consider in sequential decision making tasks. Our hope\nis that the ideas presented here will lead to a foundational framework for\nquantifying risk (and hence safety) in robotics applications.\n