Past work that improves document-level sentiment analysis by encoding user\nand product information has been limited to considering only the text of the\ncurrent review. We investigate incorporating additional review text available\nat the time of sentiment prediction that may prove meaningful for guiding\nprediction. Firstly, we incorporate all available historical review text\nbelonging to the author of the review in question. Secondly, we investigate the\ninclusion of historical reviews associated with the current product (written by\nother users). We achieve this by explicitly storing representations of reviews\nwritten by the same user and about the same product and force the model to\nmemorize all reviews for one particular user and product. Additionally, we drop\nthe hierarchical architecture used in previous work to enable words in the text\nto directly attend to each other. Experiment results on IMDB, Yelp 2013 and\nYelp 2014 datasets show improvement to state-of-the-art of more than 2\npercentage points in the best case.\n