We present an approach for safe and object-independent human-to-robot\nhandovers using real time robotic vision and manipulation. We aim for general\napplicability with a generic object detector, a fast grasp selection algorithm\nand by using a single gripper-mounted RGB-D camera, hence not relying on\nexternal sensors. The robot is controlled via visual servoing towards the\nobject of interest. Putting a high emphasis on safety, we use two perception\nmodules: human body part segmentation and hand/finger segmentation. Pixels that\nare deemed to belong to the human are filtered out from candidate grasp poses,\nhence ensuring that the robot safely picks the object without colliding with\nthe human partner. The grasp selection and perception modules run concurrently\nin real-time, which allows monitoring of the progress. In experiments with 13\nobjects, the robot was able to successfully take the object from the human in\n81.9% of the trials.\n