TECHNIQUES FOR VIRTUALIZED TOOL INTERACTION

A computer implemented method is provided for a virtual training system. A virtual surgical simulation associated with a type of surgical procedure is accessed. Image data associated with a controller and a workspace is received. Controller data corresponding to a controller interaction is received. A first interaction of the controller within the workspace based on at least one of the image data and the controller data is determined. Using the set of one or more transformation rules, the first interaction of the controller is transformed to a manipulation of a virtualized surgical tool in the virtual surgical simulation. A representation is output of the manipulation of the virtualized surgical tool.