Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models

We study prediction of future outcomes with supervised models that use\nprivileged information during learning. The privileged information comprises\nsamples of time series observed between the baseline time of prediction and the\nfuture outcome; this information is only available at training time which\ndiffers from the traditional supervised learning. Our question is when using\nthis privileged data leads to more sample-efficient learning of models that use\nonly baseline data for predictions at test time. We give an algorithm for this\nsetting and prove that when the time series are drawn from a non-stationary\nGaussian-linear dynamical system of fixed horizon, learning with privileged\ninformation is more efficient than learning without it. On synthetic data, we\ntest the limits of our algorithm and theory, both when our assumptions hold and\nwhen they are violated. On three diverse real-world datasets, we show that our\napproach is generally preferable to classical learning, particularly when data\nis scarce. Finally, we relate our estimator to a distillation approach both\ntheoretically and empirically.\n