Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific Tasks

Research article: Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific Tasks

Domain Adaptation and Few-Shot Learning Topic Modeling Intelligent Tutoring Systems and Adaptive Learning

Paper

The full text of this publication is not hosted on 44B due to licensing.

Read it at OpenAlex

Similar papers

Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific TasksОлег Ивченко, Iryna Ivchenko2026 · Zenodo (CERN European Organization for Nuclear Research) In Library

Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific TasksОлег Ивченко, Iryna Ivchenko2026 · Zenodo (CERN European Organization for Nuclear Research) In Library

Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific TasksОлег Ивченко, Iryna Ivchenko2026 · Zenodo (CERN European Organization for Nuclear Research) In Library

Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific TasksОлег Ивченко, Iryna Ivchenko2026 · Zenodo (CERN European Organization for Nuclear Research) In Library

Model Distillation ROI: When Smaller Models Outperform Larger Ones on Domain-Specific TasksОлег Ивченко, Iryna Ivchenko2026 · Zenodo (CERN European Organization for Nuclear Research) In Library

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model SizesCheng-Yu Hsieh, Chunliang Li, Chih‐Kuan Yeh et al.2023 · arXiv (Cornell University) · 990 citations In Library

Small Models Struggle to Learn from Strong ReasonersYuetai Li, Xiang Yue, Zhangchen Xu et al.2025 · Annual Meeting of the Association for Computational Linguistics · 109 citations In Library

Model Distillation on Multi-domained DatasetsА. В. Орлов, Kamil Bayazitov, Andrey Grabovoy2023 In Library

Benchmarking Distilled Language Models: Performance and Efficiency in Resource-Constrained SettingsS. Wani, Eric Page, Ajay Dholakia et al.2026 · TPC Technology Conference · 0 citations In Library

No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot RetrievalGuilherme Moraes Rosa, Luiz Bonifacio, Vitor Jeronymo et al.2022 · arXiv (Cornell University) · 29 citations In Library

Source of record

OpenAlex

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal