This paper addresses learning safe output feedback control laws from partial\nobservations of expert demonstrations. We assume that a model of the system\ndynamics and a state estimator are available along with corresponding error\nbounds, e.g., estimated from data in practice. We first propose robust output\ncontrol barrier functions (ROCBFs) as a means to guarantee safety, as defined\nthrough controlled forward invariance of a safe set. We then formulate an\noptimization problem to learn ROCBFs from expert demonstrations that exhibit\nsafe system behavior, e.g., data collected from a human operator or an expert\ncontroller. When the parametrization of the ROCBF is linear, then we show that,\nunder mild assumptions, the optimization problem is convex. Along with the\noptimization problem, we provide verifiable conditions in terms of the density\nof the data, smoothness of the system model and state estimator, and the size\nof the error bounds that guarantee validity of the obtained ROCBF. Towards\nobtaining a practical control algorithm, we propose an algorithmic\nimplementation of our theoretical framework that accounts for assumptions made\nin our framework in practice. We validate our algorithm in the autonomous\ndriving simulator CARLA and demonstrate how to learn safe control laws from\nsimulated RGB camera images.\n