Current state-of-the-art models for named entity recognition (NER) are neural\nmodels with a conditional random field (CRF) as the final layer. Entities are\nrepresented as per-token labels with a special structure in order to decode\nthem into spans. Current work eschews prior knowledge of how the span encoding\nscheme works and relies on the CRF learning which transitions are illegal and\nwhich are not to facilitate global coherence. We find that by constraining the\noutput to suppress illegal transitions we can train a tagger with a\ncross-entropy loss twice as fast as a CRF with differences in F1 that are\nstatistically insignificant, effectively eliminating the need for a CRF. We\nanalyze the dynamics of tag co-occurrence to explain when these constraints are\nmost effective and provide open source implementations of our tagger in both\nPyTorch and TensorFlow.\n