Extreme Sport Analysis Using Object Detection Methods and Unscented Kalman Filters

Abstract

This paper proposes a method to analyze athlete performance in extreme sport by using CNN object detection methods and unscented Kalman filters.The proposed method employs YOLOv7 pose estimation which uses the pretrained YOLOv7-x6-pose.pt weights file to cast a key-point skeleton over an athlete. This pose estimation method can operate 8 times more effectively than a prior object detection benchmark, detectron2. The resultant video is then passed twice to the YOLOv5n object detection model. On the first pass, the model is used to cast a bounding box around the athlete in each frame and the maximum box width and height value is stored. On the second pass, the model detects and tracks the athlete whilst using the two stored values to crop the video frame, centering the athlete. The use of a CNN object detector, YOLOv5, eliminated problems with tracking failure that occurred in prior research which relied on background subtraction methods. Finally, an unscented Kalman filter is run forward and backward in time on the video to track the athlete in instances where YOLOv5n cannot detect the athlete. Running the filter twice in opposite directions allows the filter to have a smoother trajectory and higher accuracy, improving upon prior research by 4.6% in accuracy.A limitation of the proposal arises when tracking multiple subjects. Future work could be aimed at improving the cropping function to recognize and analyze multiple athletes simultaneously.