Long Short-Term Memory with Smooth Adaptation

Abstract

Long Short-Term Memory (LSTM) is a type of recurrent neural network that has become important in machine learning research thanks to its high precision to solve problems such as speech recognition, handwriting recognition, natural text compression, sequential data processing among others. Although classic LSTM are powerful tools to solve such problems, their adaptation is far from showing a smooth behavior which represents a drawback to LSTM be used in applications such as real-time control of physical systems in which to fulfill restrictions of ranges of values of the control variables is important in order to preserve the physical integrity of the systems. In this paper we present a design of architecture of LSTM that overcomes the non-smooth adaptation problem by using a single forget gate for all the LSTM units and furthermore improves the accuracy of classic LSTMs in problems such as rebber grammar learning, time series forecasting and control of physical systems as it is shown in the experimental and comparison results.