Video Question Answering (Video QA) is a powerful testbed to develop new AI\ncapabilities. This task necessitates learning to reason about objects,\nrelations, and events across visual and linguistic domains in space-time.\nHigh-level reasoning demands lifting from associative visual pattern\nrecognition to symbol-like manipulation over objects, their behavior and\ninteractions. Toward reaching this goal we propose an object-oriented reasoning\napproach in that video is abstracted as a dynamic stream of interacting\nobjects. At each stage of the video event flow, these objects interact with\neach other, and their interactions are reasoned about with respect to the query\nand under the overall context of a video. This mechanism is materialized into a\nfamily of general-purpose neural units and their multi-level architecture\ncalled Hierarchical Object-oriented Spatio-Temporal Reasoning (HOSTR) networks.\nThis neural model maintains the objects' consistent lifelines in the form of a\nhierarchically nested spatio-temporal graph. Within this graph, the dynamic\ninteractive object-oriented representations are built up along the video\nsequence, hierarchically abstracted in a bottom-up manner, and converge toward\nthe key information for the correct answer. The method is evaluated on multiple\nmajor Video QA datasets and establishes new state-of-the-arts in these tasks.\nAnalysis into the model's behavior indicates that object-oriented reasoning is\na reliable, interpretable and efficient approach to Video QA.\n