We present a novel application of Learning from Demonstration to realize a fully autonomous bi-manual surgical suturing task, including needle pick up, insertion, re-grasping, extraction and hand-over. Surgical action primitives are learned from a single human demonstration and encoded into an action library from which they are pulled to compose more elaborate tasks at planning/execution time. The method is demonstrated in a non-clinical setting, using unmodified surgical instruments with a custom surgical robot system. We use stereo vision to automatically detect the suture needle and entry points to close the control loop and generalize tasks to different task conditions. The suturing task is shown to generalize well to differing initial conditions with a success rate of 17 % for the full task, a mean subtask success rate of 75 % and mean needle insertion error of 3.3 mm over the course of 46 trial task executions at human speed. Failures could all be attributed to erroneous vision-based detection, pose estimation and robot calibration.
Paper
Full text
Autonomous Bi-Manual Surgical Suturing Based on Skills Learned from Demonstration
Semantic Scholar · Engineering · 2021
Abstract
We present a novel application of Learning from Demonstration to realize a fully autonomous bi-manual surgical suturing task, including needle pick up, insertion, re-grasping, extraction and hand-over. Surgical action primitives are learned from a single human demonstration and encoded into an action library from which they are pulled to compose more elaborate tasks at planning/execution time. The method is demonstrated in a non-clinical setting, using unmodified surgical instruments with a custom surgical robot system. We use stereo vision to automatically detect the suture needle and entry points to close the control loop and generalize tasks to different task conditions. The suturing task is shown to generalize well to differing initial conditions with a success rate of 17 % for the full task, a mean subtask success rate of 75 % and mean needle insertion error of 3.3 mm over the course of 46 trial task executions at human speed. Failures could all be attributed to erroneous vision-based detection, pose estimation and robot calibration.