Property-Based Testing in Simulation for Verifying Robot Action Execution in Tabletop Manipulation

An important prerequisite for the reliability and robustness of a service\nrobot is ensuring the robot's correct behavior when it performs various tasks\nof interest. Extensive testing is one established approach for ensuring\nbehavioural correctness; this becomes even more important with the integration\nof learning-based methods into robot software architectures, as there are often\nno theoretical guarantees about the performance of such methods in varying\nscenarios. In this paper, we aim towards evaluating the correctness of robot\nbehaviors in tabletop manipulation through automatic generation of simulated\ntest scenarios in which a robot assesses its performance using property-based\ntesting. In particular, key properties of interest for various robot actions\nare encoded in an action ontology and are then verified and validated within a\nsimulated environment. We evaluate our framework with a Toyota Human Support\nRobot (HSR) which is tested in a Gazebo simulation. We show that our framework\ncan correctly and consistently identify various failed actions in a variety of\nrandomised tabletop manipulation scenarios, in addition to providing deeper\ninsights into the type and location of failures for each designed property.\n