Towards Accurate Vehicle Behaviour Classification With Multi-Relational Graph Convolutional Networks

Understanding on-road vehicle behaviour from a temporal sequence of sensor\ndata is gaining in popularity. In this paper, we propose a pipeline for\nunderstanding vehicle behaviour from a monocular image sequence or video. A\nmonocular sequence along with scene semantics, optical flow and object labels\nare used to get spatial information about the object (vehicle) of interest and\nother objects (semantically contiguous set of locations) in the scene. This\nspatial information is encoded by a Multi-Relational Graph Convolutional\nNetwork (MR-GCN), and a temporal sequence of such encodings is fed to a\nrecurrent network to label vehicle behaviours. The proposed framework can\nclassify a variety of vehicle behaviours to high fidelity on datasets that are\ndiverse and include European, Chinese and Indian on-road scenes. The framework\nalso provides for seamless transfer of models across datasets without entailing\nre-annotation, retraining and even fine-tuning. We show comparative performance\ngain over baseline Spatio-temporal classifiers and detail a variety of\nablations to showcase the efficacy of the framework.\n