The Impact of Looking Further Ahead: A Comparison of Two Data-driven\n Unsolicited Hint Types on Performance in an Intelligent Data-driven Logic\n Tutor

Research has shown assistance can provide many benefits to novices lacking\nthe mental models needed for problem solving in a new domain. However, varying\napproaches to assistance, such as subgoals and next-step hints, have been\nimplemented with mixed results. Next-Step hints are common in data-driven\ntutors due to their straightforward generation from historical student data, as\nwell as research showing positive impacts on student learning. However, there\nis a lack of research exploring the possibility of extending data-driven\nmethods to provide higher-level assistance. Therefore, we modified our\ndata-driven Next-Step hint generator to provide Waypoints, hints that are a few\nsteps ahead, representing problem-solving subgoals. We hypothesized that\nWaypoints would benefit students with high prior knowledge, and that Next-Step\nhints would most benefit students with lower prior knowledge. In this study, we\ninvestigated the influence of data-driven hint type, Waypoints versus Next-Step\nhints, on student learning in a logic proof tutoring system, Deep Thought, in a\ndiscrete mathematics course. We found that Next-Step hints were more beneficial\nfor the majority of students in terms of time, efficiency, and accuracy on the\nposttest. However, higher totals of successfully used Waypoints were correlated\nwith improvements in efficiency and time in the posttest. These results suggest\nthat Waypoint hints could be beneficial, but more scaffolding may be needed to\nhelp students follow them.\n