Defeasible reasoning is the mode of reasoning where conclusions can be\noverturned by taking into account new evidence. Existing cognitive science\nliterature on defeasible reasoning suggests that a person forms a mental model\nof the problem scenario before answering questions. Our research goal asks\nwhether neural models can similarly benefit from envisioning the question\nscenario before answering a defeasible query. Our approach is, given a\nquestion, to have a model first create a graph of relevant influences, and then\nleverage that graph as an additional input when answering the question. Our\nsystem, CURIOUS, achieves a new state-of-the-art on three different defeasible\nreasoning datasets. This result is significant as it illustrates that\nperformance can be improved by guiding a system to "think about" a question and\nexplicitly model the scenario, rather than answering reflexively. Code, data,\nand pre-trained models are located at https://github.com/madaan/thinkaboutit.\n