Sequence-to-Sequence (Seq2Seq) models have witnessed a notable success in\ngenerating natural conversational exchanges. Notwithstanding the syntactically\nwell-formed responses generated by these neural network models, they are prone\nto be acontextual, short and generic. In this work, we introduce a Topical\nHierarchical Recurrent Encoder Decoder (THRED), a novel, fully data-driven,\nmulti-turn response generation system intended to produce contextual and\ntopic-aware responses. Our model is built upon the basic Seq2Seq model by\naugmenting it with a hierarchical joint attention mechanism that incorporates\ntopical concepts and previous interactions into the response generation. To\ntrain our model, we provide a clean and high-quality conversational dataset\nmined from Reddit comments. We evaluate THRED on two novel automated metrics,\ndubbed Semantic Similarity and Response Echo Index, as well as with human\nevaluation. Our experiments demonstrate that the proposed model is able to\ngenerate more diverse and contextually relevant responses compared to the\nstrong baselines.\n