SYSTEM AND METHOD FOR AVOIDING SYSTEM OVERLOAD BY MAINTAINING AN IDEAL REQUEST RATE

A computer system that provides services to clients may adaptively throttle incoming service requests in order to reach and then maintain operation at an ideal request rate. An ideal request rate may be a maximum rate at which incoming service requests can be serviced such that client-specified quality of service (QoS) expectations are met for most or all incoming service requests, or a rate within a range of rates defined by that maximum rate. Determining whether the system is operating at an ideal request rate may include determining the minimum difference between the expected and actual QoS for a group of recently serviced requests. The system may gradually modify a throttle multiplier value in order to reach or maintain an ideal request rate. Maintaining operation at an ideal request rate may allow the system to avoid entering an overloaded state, and/or oscillating between an overloaded state and a non-overloaded state.