A Multi-Task Learning Approach for Human Activity Segmentation and Ergonomics Risk Assessment

We propose a new approach to Human Activity Evaluation (HAE) in long videos\nusing graph-based multi-task modeling. Previous works in activity evaluation\neither directly compute a metric using a detected skeleton or use the scene\ninformation to regress the activity score. These approaches are insufficient\nfor accurate activity assessment since they only compute an average score over\na clip, and do not consider the correlation between the joints and body\ndynamics. Moreover, they are highly scene-dependent which makes the\ngeneralizability of these methods questionable. We propose a novel multi-task\nframework for HAE that utilizes a Graph Convolutional Network backbone to embed\nthe interconnections between human joints in the features. In this framework,\nwe solve the Human Activity Segmentation (HAS) problem as an auxiliary task to\nimprove activity assessment. The HAS head is powered by an Encoder-Decoder\nTemporal Convolutional Network to semantically segment long videos into\ndistinct activity classes, whereas, HAE uses a Long-Short-Term-Memory-based\narchitecture. We evaluate our method on the UW-IOM and TUM Kitchen datasets and\ndiscuss the success and failure cases in these two datasets.\n