The Importance of Being Earnest: Performance of Modulation Classification for Real RF Signals

Digital modulation classification (DMC) can be highly valuable for equipping\nradios with increased spectrum awareness in complex emerging wireless networks.\nHowever, as the existing literature is overwhelmingly based on theoretical or\nsimulation results, it is unclear how well DMC performs in practice. In this\npaper we study the performance of DMC in real-world wireless networks, using an\nextensive RF signal dataset of 250,000 over-the-air transmissions with\nheterogeneous transceiver hardware and co-channel interference. Our results\nshow that DMC can achieve a high classification accuracy even under the\nchallenging real-world conditions of modulated co-channel interference and\nlow-grade hardware. However, this only holds if the training dataset fully\ncaptures the variety of interference and hardware types in the real radio\nenvironment; otherwise, the DMC performance deteriorates significantly. Our\nwork has two important engineering implications. First, it shows that it is not\nstraightforward to exchange learned classifier models among dissimilar radio\nenvironments and devices in practice. Second, our analysis suggests that the\nkey missing link for real-world deployment of DMC is designing signal features\nthat generalize well to diverse wireless network scenarios. We are making our\nRF signal dataset publicly available as a step towards a unified framework for\nrealistic DMC evaluation.\n