Audio and Image Caption Generator from Image using VGG-16

This project presents a system that generates text and audio captions for images using the VGG16 convolutional neural network and Long Short-Term Memory (LSTM) recurrent neural network on the Flickr8k dataset.The proposed system comprises several stages, including image augmentation, image feature extraction, text cleaning, tokenization, and LSTM-based caption generation, to ensure accurate and diverse captions.To enhance the system's generalization ability and increase the dataset's size, image augmentation is performed, randomly applying transformations such as rotations, flips, and translations to the original images.Next, visual features are extracted from the augmented images using the pre-trained VGG16 model, which is a powerful image recognition model that extracts high-level features from images.After feature extraction, the text data undergoes text cleaning and tokenization to eliminate any unwanted characters such as punctuation marks and special symbols.The text is tokenized into a sequence of words, with each word mapped to a unique integer index.Finally, the LSTM model is trained on the pre-processed text and visual features to generate descriptive captions.The system's performance is evaluated using the BLEU score, which measures the similarity between the generated captions and the ground-truth captions.The experimental results demonstrate that the proposed system generates accurate and diverse captions for both text and audio modalities.Moreover, the system outperforms several state-of-theart caption generation systems on the Flickr8k dataset.This system can be useful in several applications, including assisting visually impaired individuals in understanding visual content or enriching multimedia content with more descriptive captions.