A Better Variant of Self-Critical Sequence Training

In this work, we present a simple yet better variant of Self-Critical Sequence Training. We make a simple change in the choice of baseline function in REINFORCE algorithm. The new baseline can bring better performance with no extra cost, compared to the greedy decoding baseline.

cs.CV cs.CL Advanced Image and Video Retrieval Techniques Multimodal Machine Learning Applications Machine Learning and Algorithms

View graph Source

Paper

Similar papers

Self-critical Sequence Training for Image CaptioningSteven J. Rennie, E. Marcheret, Youssef Mroueh et al.2016 · Computer Vision and Pattern Recognition · 2.1k citations In Library

Self-critical Sequence Training for Automatic Speech RecognitionChen Chen, Yu‐Chen Hu, Nana Hou et al.2022 · arXiv (Cornell University) · 18 citations In Library

Self-Critical Sequence Training for Automatic Speech RecognitionChen Chen, Yu‐Chen Hu, Nana Hou et al.2022 · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) In Library

A Stable and Effective Learning Strategy for Trainable Greedy DecodingYun Chen, V. Li, Kyunghyun Cho et al.2018 · Conference on Empirical Methods in Natural Language Processing · 31 citations In Library

A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence ModelsKartik Goyal, Graham Neubig, Chris Dyer et al.2017 · AAAI Conference on Artificial Intelligence · 41 citations In Library

An Actor-Critic Algorithm for Sequence PredictionDzmitry Bahdanau, Philemon Brakel, Kelvin Xu et al.2016 · International Conference on Learning Representations · 671 citations In Library

Image Captioning with Word Gate and Adaptive Self-Critical LearningXinxin Zhu, Lixiang Li, Jing Liu et al.2018 · Applied Sciences · 6 citations In Library

Rethinking the Comparison Unit in Sequence-Level Reinforcement Learning: An Equal-Length Paired Training Framework from Loss Correction to Sample ConstructionFei Ding, Yongkang Zhang, Runhao Liu et al.2026 · arXiv.org · 0 citations In Library

Self-critical n-step Training for Image CaptioningJunlong Gao, Shiqi Wang, Shanshe Wang et al.2019 · Computer Vision and Pattern Recognition · 63 citations In Library

Sequence-to-Sequence ASR Optimization via Reinforcement LearningAndros Tjandra, S. Sakti, Satoshi Nakamura2017 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 27 citations In Library

Patents near this paper

Training Ensemble Models To Improve Performance In The Presence Of Unreliable Base ClassifiersUS 11,803,780 · WESTERN DIGITAL TECHNOLOGIES, INC. · 2023 ITERATIVE DECODER EMPLOYING MULTIPLE EXTERNAL CODE ERROR CHECKS TO LOWER THE ERROR FLOORUS 6,810,502 · CONEXANT SYSTEMS, INC. · 2004 REINFORCEMENT LEARNING THROUGH A DOUBLE ACTOR CRITIC ALGORITHMUS 11,816,591 · SONY CORPORATION OF AMERICA · 2023 CONSTRAINED CORRECTIVE TRAINING FOR CONTINUOUS PARAMETER SYSTEMUS 6,044,344 · IBM CORPORATION · 2000 METHOD FOR UNSUPERVISED SEQUENCE LEARNING USING REINFORCEMENT LEARNING AND NEURAL NETWORKSUS 11,030,530 · ONU TECHNOLOGY INC. · 2021

Source of record

arXiv