COMPUTER-IMPLEMENTED METHOD, SYSTEM AND COMPUTER PROGRAM PRODUCT FOR SIMULATING THE BEHAVIOUR OF A HAND THAT INTERACTS WITH OBJECTS IN A VIRTUAL ENVIRONMENT

A computer-implemented method for simulating the behavior of a hand interacting with objects in a virtual environment is provided, having acquiring a skeleton of a tracked hand using a hand-tracking device; computing a simulated hand interacting with objects in a virtual environment; establishing a coupling between the skeleton of a simulated hand and the tracked skeleton using coupling energies (Ucrb); establishing an internal energy of the simulated hand that accounts for joint constraints of the skeleton, skeleton-tissue coupling, and nonlinear soft-tissue deformation; establishing frictional contact between the simulated hand and virtual objects using a penalty energy considering relative interpenetration at contact points and a frictional energy considering the deviation between each contact point and its corresponding sliding anchor point; computing an updated skeleton and tissue of the simulated hand by minimizing the total energy involved.