Recent advances in spoken language understanding benefited from\nSelf-Supervised models trained on large speech corpora. For French, the\nLeBenchmark project has made such models available and has led to impressive\nprogress on several tasks including spoken language understanding. These\nadvances have a non-negligible cost in terms of computation time and energy\nconsumption. In this paper, we compare several learning strategies aiming at\nreducing such cost while keeping competitive performances. The experiments are\nperformed on the MEDIA corpus, and show that it is possible to reduce the\nlearning cost while maintaining state-of-the-art performances.\n