Morpion Solitaire is a popular single player game, performed with paper and\npencil. Due to its large state space (on the order of the game of Go)\ntraditional search algorithms, such as MCTS, have not been able to find good\nsolutions. A later algorithm, Nested Rollout Policy Adaptation, was able to\nfind a new record of 82 steps, albeit with large computational resources. After\nachieving this record, to the best of our knowledge, there has been no further\nprogress reported, for about a decade.\n In this paper we take the recent impressive performance of deep self-learning\nreinforcement learning approaches from AlphaGo/AlphaZero as inspiration to\ndesign a searcher for Morpion Solitaire. A challenge of Morpion Solitaire is\nthat the state space is sparse, there are few win/loss signals. Instead, we use\nan approach known as ranked reward to create a reinforcement learning self-play\nframework for Morpion Solitaire. This enables us to find medium-quality\nsolutions with reasonable computational effort. Our record is a 67 steps\nsolution, which is very close to the human best (68) without any other\nadaptation to the problem than using ranked reward. We list many further\navenues for potential improvement.\n