Scientific document understanding is challenging as the data is highly domain\nspecific and diverse. However, datasets for tasks with scientific text require\nexpensive manual annotation and tend to be small and limited to only one or a\nfew fields. At the same time, scientific documents contain many potential\ntraining signals, such as citations, which can be used to build large labelled\ndatasets. Given this, we present an in-depth study of cite-worthiness detection\nin English, where a sentence is labelled for whether or not it cites an\nexternal source. To accomplish this, we introduce CiteWorth, a large,\ncontextualized, rigorously cleaned labelled dataset for cite-worthiness\ndetection built from a massive corpus of extracted plain-text scientific\ndocuments. We show that CiteWorth is high-quality, challenging, and suitable\nfor studying problems such as domain adaptation. Our best performing\ncite-worthiness detection model is a paragraph-level contextualized sentence\nlabelling model based on Longformer, exhibiting a 5 F1 point improvement over\nSciBERT which considers only individual sentences. Finally, we demonstrate that\nlanguage model fine-tuning with cite-worthiness as a secondary task leads to\nimproved performance on downstream scientific document understanding tasks.\n