Stanford Researchers Develop AI Model To Predict Wheel-Spinning Among Students

Global edtech investments are projected to reach $350 billion by 2025 from around $18.66 billion in 2021. The adoption of language apps, video conferencing tools, virtual tutoring, and online learning software is already through the roof.

Wheel spinning is a term coined by researchers Joseph Beck and Yue Gong referring to students that were repeatedly trying and failing to successfully complete a specific skill after many attempts in an intelligent tutoring system.

Recently, Stanford University, in collaboration with non-profit War Child Holland, has proposed a method that trains a model to predict wheel spinning and utilises a popular tool in interpretable machine learning, Shapley values, to provide individualised credit attribution over the features of the model, including actionable features like possible gaps in prerequisites. 

Explainable ML to rescue

According to researchers, the goal is to provide a method for using past student log data to predict when a student will need intervention to prevent wheel-spinning. "In our work, we use Shapley values and the python implementation SHAP package to inform interventions," as per the paper.

Researchers first trained a machine learning model using an existing dataset of student log data to predict wheel-spinning. In this case, researchers used the popular gradient boosting method XGBoost. Keeping the current state of the student in consideration, the algorithm uses the trained model to look for the possibility of wheel-spinning by looking at the student-activity pair. The number of failed attempts by the students above a threshold limit prompts the algorithm to provide a potential intervention.

Interventions are proposed using a method of feature attribution from explainable machine learning. "We use Shapley values to assign a contribution value to each feature used in the wheel spinning prediction model. A subset of these features are designed to be actionable and correspond to an intervention," as per the paper.

Shapley values were first introduced in the field of game theory in 1953, with the purpose of solving the following scenario: A group of people with varying levels of expertise are working together towards a common goal. How should the reward be fairly distributed among the members of the group?

The performance of the method presented in the paper is assessed in simulation. They create true cases of wheel-spinning, which they define as needing ten or more attempts on one educational activity to match prior works and Can't Wait to Learn (CWTL) data. CWTL is a programme by War Child Holland that provides self-paced educational software on tablets primarily to children in or coming from conflict-affected regions (Uganda as an example).

One limitation of this method is causal inferences cannot be made. However, "we found our method had high accuracy and was able to choose an accurate intervention for more than 80% of the time in the simulations before the students begin an activity," the paper concluded.

AI to rescue

Today, AI is widely used to identify the retention capacity of students and the level of participation during online learning sessions. However, there are few reasons behind its wider adoption in edtech, such as:

• Amount of data available: The internet of things and digitisation have led to an explosion of data. By crunching volumes of data, AI models are able to predict outcomes in a better way and make right recommendations.
• Advances in cloud computing: The availability of low-cost computing has made AI more affordable and practical.
• Better mathematical models and tools: Bots replying to queries and robustness of AI models at handling a variety of inputs has further simplified its deployment.

Teachers solely cannot solve the wheel-spinning problem. It's critical to have mechanisms in place to proactively identify children at risk, and AI might have just cracked the code.