Large, pre-trained transformer models like BERT have achieved\nstate-of-the-art results on document understanding tasks, but most\nimplementations can only consider 512 tokens at a time. For many real-world\napplications, documents can be much longer, and the segmentation strategies\ntypically used on longer documents miss out on document structure and\ncontextual information, hurting their results on downstream tasks. In our work\non legal agreements, we find that visual cues such as layout, style, and\nplacement of text in a document are strong features that are crucial to\nachieving an acceptable level of accuracy on long documents. We measure the\nimpact of incorporating such visual cues, obtained via computer vision methods,\non the accuracy of document understanding tasks including document\nsegmentation, entity extraction, and attribute classification. Our method of\nsegmenting documents based on structural metadata out-performs existing methods\non four long-document understanding tasks as measured on the Contract\nUnderstanding Atticus Dataset.\n