Spatial Augmented Reality in Industrial Environments - An Approach for Improved Hand-Tracking Using Combined Computer Vision Technologies

Abstract

Spatial augmented reality is playing an increasingly important role in the design of digital assistance systems for industrial applications. Most of these systems use integrated computer vision technologies for supporting direct interactions in the physical workspace of the user. While these systems work well under lab conditions, field tests in real production environments revealed shortcomings of the underlying computer vision approaches with regard to their reliability. In this paper, we present a concept combing machine learning with the theories of Dempster-Schafer and fuzzy set for improved classification and reliability. For the classification, the Mediapipe-Hand machine learning framework and a segmentation-based method were combined. The fusion with the indices and the classification results is done with fuzzy inference. The subsequent results are merged with Dempster's combination rule. The results proved that the presented concept increases the reliability of the detection up to 97.8 percent.