System and Method for Compressing Video Based on Latency Measurements and Other Feedback

A system and method of playing a video game in a streaming mode includes executing a selected video game on an assigned game server, receiving execution control signals corresponding to user inputs to a client device to generate multiple video frames. Encoding the video frames in a shared hardware compression. Some of the encoded video frames depend on a preceding frame. A first and a second encoded video frame are transmitted to the client device. A first feedback signal from a decoder in the client device identifies that the second encoded video frame was not received the decoder. A third encoded video frame dependency is adjusted to produce an adjusted third encoded video frame that depends from the first encoded video frame and the adjusted third encoded video frame is transmitted to the client device.