Classroom

Q&A: Georgia Tech Researcher Discusses How AI can Improve Student Success

New findings hold significant potential for student online learning success and retention.

Amelia Pang is a journalist and an editor at EdTech: Focus on Higher Education. Her work has appeared in the New Republic, Mother Jones, and The New York Times Sunday Review, among other publications.

The Georgia Institute of Technology’s AI teaching assistant Jill Watson turned 5 years old in January. Since the birth of Jill, Georgia Tech has gone on to produce groundbreaking new research that reveals how conversations between humans and bots can be used to improve user experiences. A recent research paper titled “Towards Mutual Theory of Mind in Human-AI Interaction: How Language Reflects What Students Perceive About a Virtual Teaching Assistant” explores how to build chatbots that can conduct natural and prolonged conversations.

In a Q&A with EdTech: Focus on Higher Education, Qiaosi Wang, lead author of the paper and a Ph.D. student in human-centered computing at Georgia Tech’s Design and Intelligence Lab, discusses the implications of the paper’s findings for higher education.

EDTECH: What are the implications of your research for higher education, particularly for online learning engagement and student success?

WANG: Our research focuses on the design of natural communication between online learners and virtual teaching assistants (VTAs) being deployed in online learning contexts. As we try to scale up higher education to reach more students, there has been increasing use of different conversational agents in online learning, and VTA is a popular one.

VTAs can help answer student questions on course logistics such as assignment due dates, as well as common questions about class materials from previous semesters. All of these answers are largely delivered through text-based communications with online learners.

For VTAs like Jill Watson, the goal is to improve student engagement by providing timely and accurate responses to ensure student questions don’t go unanswered. However, similar to our less-than-satisfactory interactions with existing conversational agents like Amazon Alexa or Google Assistant in our daily lives, student communications with VTAs like Jill Watson are often viewed as unnatural and incoherent.

MORE ON EDTECH: Ashok Goel on using Jill Watson in higher education.

One reason for this is the gap between high user expectations of VTAs and the VTAs’ actual capabilities. In our case, with Jill Watson, students sometimes could not grasp what Jill could do, and therefore asked questions like, “Can you predict the score distribution at the end of the semester?” Jill, of course, could not answer this type of question.

When this kind of interaction happens more and more often, students became confused with Jill’s capability and frustrated with Jill over time. Our research focuses on how to help students and VTAs build a mutual understanding of VTA capabilities. This can ensure smoother and more natural communications.

When students can smoothly communicate with Jill, the interactions will be more enjoyable and engaging. More important, students will be able to get more out of the VTA’s ability to answer questions about online classes.

EDTECH: How can human-centered AI help support student retention?

WANG: One of the crucial factors in supporting student retention is to bring a personal touch that is commonly seen in physical classrooms to online classrooms. In physical classrooms, especially small classes, instructors can give personal attention to individual students through customized feedback and suggested learning strategies.

In the online learning environment, this kind of personal touch is almost completely lost, since the instructor-student ratio is often highly imbalanced. One potential way that human-centered AI could help with that is to not only create a VTA for the entire online class but also personal VTAs for every student in that class.

There are many ways this type of personal VTA can help. For instance, the VTA can identify student knowledge gaps by monitoring student learning activities and then suggest customized study strategies. It can help students navigate available learning tools. This can all help increase student retention in online learning significantly.

Our vision is that by knowing how the students perceive VTAs, future VTAs can potentially adjust their behavior.”

Qiaosi Wang lead author of “Towards Mutual Theory of Mind in Human-AI Interaction,” Georgia Institute of Technology

Another factor that affects the student retention rate is that many students often feel socially isolated during online classes. At Georgia Tech’s Online Master of Science in Computer Science program, there are often hundreds or thousands of students per class. It can be very difficult to build social connections and personal relationships with other students. So, that’s one issue we are actively exploring using conversational agents.

In that project, we designed and developed a conversational agent called SAMI, which stands for Social Agent-Mediated Interactions. It can perform personalized social matching between online students based on their shared hobbies, interests, location, etc. We have deployed SAMI in several online classes so far to help students build social connections with each other. While it’s still a work in progress, students are generally very fond of this initiative.

DIVE DEEPER: Read about higher education's increasingly nuanced, AI-powered chatbots.

EDTECH: Many institutions are already using AI chatbots to connect with students who have questions for a variety of departments, from enrollment to academics and housing. How can your research help improve this technology?

WANG: There are mainly two kinds of limitations when using AI chatbots in this context: One is social, and the other is technical. The social limitation is whenever we think about AI chatbots, we tend to have very high expectations of their capabilities. We might think that chatbots can understand and respond to everything we say, and that they can communicate with us like any other human being.

The truth is, the current chatbot technology is not there yet, and most chatbots are not able to conduct coherent and natural conversations with users. This is the technical limitation that I was alluding to earlier.

This mismatch between user expectations of the chatbot and the chatbot’s actual capabilities often leads to user frustration. Sometimes, this causes the student to abandon using chatbots entirely. This phenomenon is also common when using AI chatbots in higher education.

Our research tries to tackle this problem by examining how to help online students and AI chatbots, especially VTAs, build a mutual and shared understanding of the VTAs’ capability and avoid communication frustrations. My team and I examined this space by looking into the feasibility of automatically inferring student perceptions of the VTA Jill Watson through the language they used to talk with Jill. We were able to infer perceptions (such as how humanlike students think Jill is) from some linguistic traits (such as how many unique words students use in the questions) by extracting this information from the students’ language.

Our vision is that by knowing how the students perceive VTAs, future VTAs can potentially adjust their behavior, or let students know their perception is incorrect to avoid frustration during communication.

Peshkova/Getty Images