Image Captioning using Deep Stacked LSTMs, Contextual Word Embeddings and Data Augmentation

Image Captioning, or the automatic generation of descriptions for images, is\none of the core problems in Computer Vision and has seen considerable progress\nusing Deep Learning Techniques. We propose to use Inception-ResNet\nConvolutional Neural Network as encoder to extract features from images,\nHierarchical Context based Word Embeddings for word representations and a Deep\nStacked Long Short Term Memory network as decoder, in addition to using Image\nData Augmentation to avoid over-fitting. For data Augmentation, we use\nHorizontal and Vertical Flipping in addition to Perspective Transformations on\nthe images. We evaluate our proposed methods with two image captioning\nframeworks- Encoder-Decoder and Soft Attention. Evaluation on widely used\nmetrics have shown that our approach leads to considerable improvement in model\nperformance.\n