Who Benefits from Using Intelligent Tutoring Systems, and How Much Use is Optimal? Examining the Effects of School Socioeconomic Status and Intelligent Tutoring System Use on Student Mathematics Learning Outcomes in Australian High Schools

Intelligent Tutoring Systems (ITS) show promise for improving student outcomes, but their effectiveness may vary by school socioeconomic status (SES). This study asks who benefits from ITS platforms and how much use is optimal. Using learning analytics from an Australian mathematics ITS with 7,147 Year 7 students across 500 classes and 127 schools, we examined how internal ITS achievement scores relate to external standardised assessments (NAPLAN), and how these relationships shift with school SES and ITS usage. We first tested a Conditional Mediation Structural Equation Model of direct, indirect, and interaction effects among school SES, student ITS usage, and class-level outcomes, then estimated a Mixture Model with piecewise splines across known-class SES quartiles to capture non-linear usage effects. Across both analyses, student ITS achievement scores were positively associated with NAPLAN outcomes, but the strength of this relationship depended on school SES and student ITS usage intensity. Specifically, lower SES schools reached a point of diminishing returns in their usage of the ITS platform, far sooner than their high SES counterparts. These findings highlight two key contributions: first, that the benefits of ITS may not be uniform, but rather depend on who is using the platform; and second, that there are practical limits to how much ITS use may be beneficial.