SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization

Novel neural architectures, training strategies, and the availability of\nlarge-scale corpora haven been the driving force behind recent progress in\nabstractive text summarization. However, due to the black-box nature of neural\nmodels, uninformative evaluation metrics, and scarce tooling for model and data\nanalysis, the true performance and failure modes of summarization models remain\nlargely unknown. To address this limitation, we introduce SummVis, an\nopen-source tool for visualizing abstractive summaries that enables\nfine-grained analysis of the models, data, and evaluation metrics associated\nwith text summarization. Through its lexical and semantic visualizations, the\ntools offers an easy entry point for in-depth model prediction exploration\nacross important dimensions such as factual consistency or abstractiveness. The\ntool together with several pre-computed model outputs is available at\nhttps://github.com/robustness-gym/summvis.\n