The Closer You Look, The More You Learn: A Grey-box Approach to Protocol State Machine Learning

We propose a new approach to infer state machine models from protocol implementations. Our new tool, StateInspector, learns protocol states by using novel program analyses to combine observations of run-time memory and I/O. It requires no access to source code and only lightweight execution monitoring of the implementation under test. We demonstrate and evaluate StateInspector's effectiveness on numerous TLS and WPA/2 implementations. In the process, we show StateInspector enables deeper state discovery, increased learning efficiency, and more insight compared to existing approaches. Our method led us to discover several concerning deviations from the standards and vulnerabilities in IWD and WolfSSL, both of which were assigned CVEs.