Synthesizing Skeletal Motion and Physiological Signals as a Function of a Virtual Human's Actions and Emotions

Round-the-clock monitoring of human behavior and emotions is required in many\nhealthcare applications which is very expensive but can be automated using\nmachine learning (ML) and sensor technologies. Unfortunately, the lack of\ninfrastructure for collection and sharing of such data is a bottleneck for ML\nresearch applied to healthcare. Our goal is to circumvent this bottleneck by\nsimulating a human body in virtual environment. This will allow generation of\npotentially infinite amounts of shareable data from an individual as a function\nof his actions, interactions and emotions in a care facility or at home, with\nno risk of confidentiality breach or privacy invasion. In this paper, we\ndevelop for the first time a system consisting of computational models for\nsynchronously synthesizing skeletal motion, electrocardiogram, blood pressure,\nrespiration, and skin conductance signals as a function of an open-ended set of\nactions and emotions. Our experimental evaluations, involving user studies,\nbenchmark datasets and comparison to findings in the literature, show that our\nmodels can generate skeletal motion and physiological signals with high\nfidelity. The proposed framework is modular and allows the flexibility to\nexperiment with different models. In addition to facilitating ML research for\nround-the-clock monitoring at a reduced cost, the proposed framework will allow\nreusability of code and data, and may be used as a training tool for ML\npractitioners and healthcare professionals.\n