This work introduces TrimTuner, the first system for optimizing machine\nlearning jobs in the cloud to exploit sub-sampling techniques to reduce the\ncost of the optimization process while keeping into account user-specified\nconstraints. TrimTuner jointly optimizes the cloud and application-specific\nparameters and, unlike state of the art works for cloud optimization, eschews\nthe need to train the model with the full training set every time a new\nconfiguration is sampled. Indeed, by leveraging sub-sampling techniques and\ndata-sets that are up to 60x smaller than the original one, we show that\nTrimTuner can reduce the cost of the optimization process by up to 50x.\nFurther, TrimTuner speeds-up the recommendation process by 65x with respect to\nstate of the art techniques for hyper-parameter optimization that use\nsub-sampling techniques. The reasons for this improvement are twofold: i) a\nnovel domain specific heuristic that reduces the number of configurations for\nwhich the acquisition function has to be evaluated; ii) the adoption of an\nensemble of decision trees that enables boosting the speed of the\nrecommendation process by one additional order of magnitude.\n