We introduce a method to determine if a certain capability helps to achieve\nan accurate model of given data. We view labels as being generated from the\ninputs by a program composed of subroutines with different capabilities, and we\nposit that a subroutine is useful if and only if the minimal program that\ninvokes it is shorter than the one that does not. Since minimum program length\nis uncomputable, we instead estimate the labels' minimum description length\n(MDL) as a proxy, giving us a theoretically-grounded method for analyzing\ndataset characteristics. We call the method Rissanen Data Analysis (RDA) after\nthe father of MDL, and we showcase its applicability on a wide variety of\nsettings in NLP, ranging from evaluating the utility of generating subquestions\nbefore answering a question, to analyzing the value of rationales and\nexplanations, to investigating the importance of different parts of speech, and\nuncovering dataset gender bias.\n