Using AI Chatbots in Education Recent Advances Challenges and Use Case
Using AI Chatbots in Education Recent Advances Challenges and Use Case
Proceedings Paper:
Aleedy, M, Atwell, E orcid.org/0000-0001-9395-3764 and Meshoul, S (2022) Using AI
Chatbots in Education: Recent Advances Challenges and Use Case. In: Artificial
Intelligence and Sustainable Computing: Proceedings of ICSISCET 2021. 3rd International
Conference on Sustainable and Innovative Solutions for Current Challenges in
Engineering & Technology ICSISCET 2021, 13-14 Nov 2021, Online. Algorithms for
Intelligent Systems . Springer Singapore , pp. 661-675. ISBN 978-981-19-1652-6
https://doi.org/10.1007/978-981-19-1653-3_50
This version of the article has been accepted for publication, after peer review (when
applicable) and is subject to Springer Nature’s AM terms of use
(https://www.springernature.com/gp/open-research/policies/accepted-manuscript-terms),
but is not the Version of Record and does not reflect post-acceptance improvements, or
any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-
981-19-1653-3_50.
Reuse
Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless
indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by
national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of
the full text version. This is indicated by the licence information on the White Rose Research Online record
for the item.
Takedown
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by
emailing [email protected] including the URL of the record and the reason for the withdrawal request.
[email protected]
https://eprints.whiterose.ac.uk/
Using AI Chatbots in Education: Recent Advances
Challenges and Use Case
1 Introduction
Technologies have evolved over the years; what we saw ten years ago as a technological
revolution is now becoming ordinary. We are currently in the era of AI, where science
is behind every development that simulates human thinking. Some common scientific
fields where AI is predominant include computer vision, pattern recognition, natural
language processing (NLP), understanding (NLU) and generation (NLG), robotics, and
planning to name just a few. Moreover, AI technologies are at the pace of creating
revolutions and paradigm shifts in almost all areas such as education, healthcare, busi-
ness, engineering, automotive, etc. A chatbot is an AI technology powered by natural
language processing techniques to learn and understand human language. It has become
one of the most important tools governments and private sectors use to provide contin-
uous communication to users 24 hours a day. Recently, many universities and educa-
tional institutions have started to use chatbot technologies to help students in the ad-
mission process and academic advising along with the learning process. Faculty mem-
bers and teachers can use chatbots to follow up with students, respond to inquiries,
review students’ backgrounds about subjects, and assess homework, among many other
2
tasks. The role of chatbots in education is clear, but their design and use by either teach-
ers or students face many challenges and difficulties.
The contribution of this work is twofold. First and as a review, the progress in de-
veloping AI chatbots for education is described, emphasizing up-to-date algorithms
used proposed for their design and the challenges related to their implementation. Sec-
ond, through a use case, we show how an AI chatbot can be leveraged to collect feed-
back from students regarding a course, namely machine learning in our case, and ana-
lyze the results.
This paper will be structured as follows: section 2 reviews chatbots, their categories
based on the design approaches, the uses of chatbots in education, and discusses chal-
lenges in languages, implementations, and education. In section 3, a use case in using
chatbots in education is conducted to support this research. A discussion about the use
case is provided in section 4. Finally, section 5 will conclude the overall paper.
2 Literature Review
Generally, chatbots simulate the interaction between humans. They are widely used in
education, banking, e-commerce, and business as tools to help customer support [1]. A
chatbot can be defined as a conversational tool that allows users to operate computers
in a simple natural language that people can understand [2]. Another attempt to define
the chatbot describes it as a programmed tool that interacts naturally with the user on a
specific topic or subject. This interaction can be by voice or text [3][4].
Before getting deep into chatbot technology, it is important to understand its struc-
ture. The chatbot architecture depends on the domain specified for it, but the basic
workflow remains the same. As shown in Fig. 1, a chatbot takes input (text, voice, or
both) from the user. Then, the input is passed to the Natural Language Processing (NLP)
component in the form of text to learn and understand the user’s input. Response con-
structor uses different algorithms to process the pre-defined Knowledge base, presents
a set of suitable responses, and then passes it to the response selector. Response selector
uses machine learning and AI algorithms to choose the most appropriate response [5].
Chatbots can be classified into two broad categories based on their use. According to
[2], chatbots can either be service chatbots or social chatbots. This classification depends
on the primary function it performs and the way that its programs operate. Moreover,
service oriented chatbots provide users’ services in a friendly and fast way, such as
online customer service, banking service, weather service, etc. On the other hand, social
chatbots care more about the emotional side of customers and their satisfaction.
Other researchers classify chatbots based on the knowledge domain, service-
provided, goals, and input/responses generated [6]. The knowledge domain defines the
chatbot access domains if it is open or close. The open domain chatbots make
conversations on general topics and give meaningful responses. In the closed domain,
the chatbots answer questions from a specific domain and fail to respond to other
domains’ questions. On the other hand, service-based chatbots include chatbots that
provide personalized, in-kind, and overlapping services between agents. Moreover, goal-
based chatbots include informative, conversational, and specific task chatbots. Finally,
the input method and responses generated chatbots accept input, then process it to
generate output in natural language, process input based on rules, or use both techniques
[6].
As can be seen on Fig. 2 below, chatbots can be classified in different ways depending
on some parameters such as: functionality, knowledge domain, service provided, goals,
and input method/ generated responses. Therefore, the development of a chatbot can be
viewed as a combination of choices related to the aforementioned parameters.
Input
Knowledge Service
Functionality Goals Method/Respo
Domain Provided
nses Generated
Accept
Personalized Informative Input/Generate
Services Output
Open Domain
Chatbots
Accept
Input/Use Rules
In-Kind Conversational
to Generate
Output
Chatbots can be classified based on the developing approach into three main groups:
rule-based, deep learning, and ensemble approaches [5]. Table 1 presents a summary
of the used approaches in the design of chatbots.
4
Machine
Learning • Do not rely on pre-defined
approach Generative-Based generates responses.
proper responses during the
• Useful for involving a person
conversation; it does not depend
in informal open-domain
on pre-defined questions and CAiRE [8],
conversations.
answers. Instead, it uses neural [11], [13]
• Not ideal for closed-domain
network models or deep learn-
communications.
ing techniques to develop a dia-
logue with the user. • Can generate mistakes.
• Difficult to be trained.
• Pattern Matchers: Used in traditional chatbots, it answers the user only if there is
a pattern (lexical) matching between the user query and set of question-answer stored
in its knowledge base. This kind of techniques is considered inflexible; it lacks the
5
intuitive ability of humans to see meaning, and the answers are given using a set of
pre-defined responses [11].
• Natural Language Processing (NLP): Natural language processing (NLP) is a field
of artificial intelligence that studies how computer systems can interpret and control
natural language. Information about understanding and using human language is col-
lected to create appropriate technologies for computer systems to manage human
language and perform many tasks. Most NLP techniques are based on machine learn-
ing. They consist of Natural language understanding (NLU) which develops the task
to understand text, and Natural language generation (NLG) which presents the re-
sponsibility for text generation that is commonly performed by neural networks [17].
• Machin Learning (ML): Algorithms that learn from conversations. They consider
the whole dialog context, not just the current turn, and do not require a pre-defined
response for each possible user input. However, it needs an extensive training set,
which can be very difficult to provide [17].
• Deep Learning (DL): Chatbots built using artificial neural networks learn every-
thing from data and conversations with humans. It is trained to develop its own opin-
ion on the text and be more effective by using more and more data. Retrieval-based
and generative-based are the two major types of bots created using deep learning [5].
• Reinforcement Learning (RL): is a machine learning framework for learning and
continuously improving policies by a computer algorithm through interacting with
its environment. Reinforcement learning algorithms that incorporate deep learning
(implemented as a neural network) are known as deep reinforcement learning (DRL)
[18][19].
In recent years, the chatbot’s technologies powered by combining more than one
technique in the field of AI and NLP to accurately mimic the real-life conversations
between humans and overcome the limitations of the traditional chatbot.
Many researchers studied and evaluated the benefits of using chatbots in educational
institutions. They present how the educational agent participates directly in the online
discussions, assists the teachers in following the chat discussion, uses natural language
processing (NLP) to understand and find the main points in the student discussion, and
applies sentiment analysis of the chat responses [20].
A study on Facebook messenger educational chatbots focuses on topics and language
used in conversation. The development platform showed that (89%) of the chatbots
under investigation use English as their primary communication language. Moreover,
about (46%) of chatbots cannot be used to discuss any technical issues or simulate hu-
man conversation [3]. Most of the chatbots’ topics were learning languages, economic
topics, simulate conversation with historical figures, and learning computer science
concepts [1].
A chatbot offers an interactive way of learning, similar to the one-to-one interaction
between student and teacher. The chatbot provides educational features, including solv-
ing individual problems (e.g., current student, parents, graduated). The chatbot has
6
become more important from another perspective, as it saves costs by replacing human
assistants and increasing user satisfaction because of the speed response and availability
24 hours. From the students’ perspective, and by using the chatbot, students can expand
their knowledge, exchange ideas, reduce learning loads, and feel more connected, es-
pecially the first-year students[21][22]. Chatbots become suitable for teaching individ-
uals at different levels and abilities because it increases their confidence. From the
teachers’ perspective, they can use the chatbot to send announcements to students, help
correct homework (e.g., identify spelling and grammar mistakes), and track student
progress [22].
Moreover, chatbots are used in education to handle frequently asked questions, ad-
ministrative tasks, student mentoring, motivation, student learning assessments, simu-
lations, training, and providing feedback. For example, in a real case in Cardenal Her-
rera University, a chatbot acts as a personal assistant to monitor students, predict their
behavior, answer administrative questions, and give advice [6].
The most important characteristic of a chatbot is its availability for learners on their
mobile devices. It provides an educational method suitable for modern students in terms
of speed and convenience, where communication is possible through mobile at any time
and place. It also provides immediate feedback and follow-up to students based on their
understanding and repetition of information according to their needs. Shy students in-
teract greatly through chatbots, especially learning new languages; it helps them speak
fluently and comfortably without judgment because they are sure they are not talking
to real people [23].
The educational content is usually huge, and thus students are less interested in start-
ing to learn due to a large number of scientific materials. For example, the database
systems course contains many exercises, explanations, examples, and tasks that are of-
ten considered difficult for students. However, it is possible to motivate students to start
solving them by gradually presenting these tasks, which helps the student search
quickly through a chatbot teacher, which can be easily accessed in several ways, in-
cluding websites or social media [24].
In some higher education institutions in India, an experimental research has been
conducted to explore the factors that affect chatbot technology adoption. A sample of
47 students and a series of 10 questions have been used to conduct the study. One of
the questions in the study measures the response time by the communication channels,
around 61% of the students choose chatbot as the quickest communication tool, online
chat take second place with 19%, phone calls 8.7%, 6.5% for face-to-face communica-
tion, social media and e-mail get 2.2% for each [6].
Another experiment has been conducted on a group of undergraduate students in the
university in Hong Kong for the course principles of networks while using a chatbot
named Infobot for 15 minutes. Infobot answers questions related to the educational ma-
terial, students’ grades, academic schedule, and the teacher’s office hours. The experi-
ment uses a questionnaire to measures students’ satisfaction with the information pro-
vided by this chatbot; this information has been previously entered into the database by
the teacher. The vast majority of students, nearly 60%, agreed on this chatbot’s effi-
ciency and how it helped them get the required information faster and relieve pressure
on the teacher by facilitating communication and accelerate learning [25].
7
There are numerous public and private universities in Saudi Arabia to teach the
growing number of students. Therefore, an increase in the number of students means
an increase in the number of inquiries 24 hours a day. This requires a sufficient number
of employees to achieve the required level of service quality. After investigating the
websites of Saudi universities, the results indicated that chatbots are not fully utilized
[29].
8
2.4 Challenges
Many challenges face the effective use of chatbots; this section covers some of them in
terms of languages, implementations, and education.
Language Challenges:
. Chatbots are developed for many languages such as English, Arabic, French, etc.
However, each language has its own sentence structure, punctuation rules, and the use
of spaces, which is a barrier for current chatbots to deal with [11][30]. For English
chatbots, the authors [11] mentioned several points that affect the efficiency of chatbot
conversations; the most important ones include the inability to recognize grammatical
errors and similar meanings questions. Moreover, information retrieval from a database
is not realistic; two questions may look different in terms of words but have the same
meaning, so similarity measures should be used to eliminate differences [11][31].
On the other hand, the Arabic language chatbots have many linguistic complexities,
it consists of several variants that are quite different from each other: Modern Standard
Arabic, the official written/read language, and several dialects. Moreover, Arabic writ-
ers make very common mistakes in spelling some problematic letters, such as Alf
Hamza and Ta Marbouta. Morphological richness Arabic words are influenced by a
large number of features such as gender and numbers. Also, verbs, adjectives, and pro-
nouns are all gender-specific, requiring the chatbot to have two different responses sys-
tems - one for male users and one for female users. As in other languages, Arabic has
its own set of unique dialogue expressions, for example, while the English greeting
expression “good morning” gets the answer “good morning”, the Arabic equivalent
greeting ““ ”صباح الخيرMorning of Goodness” gets the answer ““ ”صباح النورMorning of
Light” [32][33][34]. A collaboration between linguists and computer scientists is re-
quired to overcome these challenges.
Implementation Challenges:
. Users sometimes tend to initiate domain queries during a conversation with a chatbot
and then suddenly move to another domain; this may lead to significantly weak accu-
racy [10][11]. Moreover, the performance of a chatbot is dependent on the size and
accuracy of the databases, so the higher the database size, the better the performance.
The semantically similar questions unidentified by the system would be real problems
unless the developer included the semantically equivalent words in the terminology
section [10][11][31]. Moreover, lack of chatbot personality can also push users away
from the dialog, this risk can be reduced by giving the chatbot a name and an avatar.
Gender and time recognition are real issues; the chatbot has no sense of time and often
replies with the same “Have a good day” or “Good morning”, regardless of the time of
day [10]. In the research [35], some issues regarding the Question Answering system
(retrieval-base) are found. The research mentioned that the system answer questions
related to only a particular domain. The user must follow a specific format when asking
questions. They suggest using a sequence-to-sequence model (generative-base) to over-
come this issue.
9
Educational Challenges:
. In educational institutions, many challenges are facing the implementation of chatbots.
Students feel that they are not interested in speaking with chatbots because the conver-
sations have become monotonous with the long period of use. Therefore, more attention
needs to be paid to improving chatbots to be more like chat agents [31]. Furthermore,
the chatbot does not understand students’ feelings and their satisfaction or anger. Also,
it does not give advice or ask questions [11].
3 Use Case
Hubert is an artificial intelligence chatbot that replaces the traditional way of filling
surveys by chatting with users. Currently, Hubert is aimed towards three different do-
mains: customer experience, human resources, and education. Depending on the choice,
the survey is set up using specific questions about that domain. Hubert consists of two
main parts: the chatbot itself and the dashboard. In the dashboard, the user chooses the
domain and then customizes the pre-defined questions. Finally, it gives the user options
to start talking with the chatbot (copy the link or e-mail the participants). When the
participant clicks on the chatbot link, a window with a typical chat layout opens, and
Hubert starts the conversation [35].
To get a closer look at the importance of chatbots in education, A practical experi-
ment was performed on Hubert chatbot to get student feedback about a machine learn-
ing course. Since it was the middle of the semester, while applying the experiment, we
chose the informal course evaluation – during the semester version of Hubert.
This paper focuses on using AI in education, and Hubert provides a great tool to
serve this goal. The education domain of the Hubert chatbot allows teachers to get direct
feedback from the student. The students’ feedback allows the teachers to identify gaps
in their teaching efforts and do better. The Huber chatbot makes inquiries like how the
course can be improved, what to change, what is functioning admirably, and what is
not. The chatbot investigates the conversations, does some analysis, and then sends it
to the instructors [36].
Feedback Hubert website contains several models that provide feedback based on
the need of the teacher: stop/start/continue evaluation for mid-term feedback, students’
self-reflection – beginning/middle/end of the semester, informal course evaluation –
during the semester/at the end of the semester and customized template for the Edubots
project.
All participants are postgraduate students (26 female students) from Princess Nourah
bint Abdulrahman University in Riyadh, Saudi Arabia. During the lecture, the instruc-
tor shared a link with the student. The link opens the Hubert chatbot webpage directly
and asks the students if they are ready to start (see Fig. 3).
10
The Hubert chatbot starts asking students about their opening on the course and what
the instructor should do and stop doing. Moreover, Huber encourages students to give
some solutions and not participate only with yes and no. If a student does not provide
a real answer, Hubert tries to encourage him by asking a follow-up question (see Fig.
4).
Fig. 3. The start of the conversation Fig. 4. Follow-up question to encourage student
At the end, Hubert asks the students to evaluate the experience of talking to it and
then terminates the conversation (see Fig. 5).
11
On the other hand, the instructor can view all the transcripts with the students, and
an analysis of them in the dashboard. The dashboard provides the most discussed topic
and sentiment analysis for each student’s answer, if positive, negative, or neutral. The
instructor can change the sentiment analysis if he sees it incorrect.
The feedback chatbot asks seven main questions as follows:
• What is working well with _______ and should continue in the same way?
• What is working well with _______ and should continue in the same way?
• What could _______ stop doing, that would improve _______?
• What class activities or assignments help you learn the most so far?
• What is your overall experience of _______ so far?
• Are there any other points you would like to comment on?
• How much did you enjoy using me (Hubert), from 1 to 5? [37]
Our experiment got 100% responses from the class students; they were excited to talk
to the chatbots. Using the dashboard makes it easy to know the strengths and weak-
nesses of the course and clarify the most important points in students’ conversations
12
with the chatbot to develop the course. If the students interact with negative responses
such as “nothing” and “no”, they are encouraged by the chatbot to participate more by
asking a follow-up question, “Surely there must be something that could be im-
proved?”.
In our experiment, around 42% of the students responded negatively at first, but the
percentage reduced to about 7% after motivating them with another question. At the
end of the experiment, students were asked to rate their talking experience with the
chatbot. Hubert received a 3.6 out of 5, with 19 students rating 5, 4 students rating 4,
and 3 students rating 3 (see Fig. 6).
5 Conclusion
This study explores the existing chatbots’ domain of use, language characteristics, and
design approaches. The study also focuses on the most important practices of using
educational AI chatbots as a parallel educational tool, virtual teaching assistant, trainer
to practice English as a second language, and virtual agent to get student feedback. In
addition to their ability to serve some issues related to registration and administrative
problems for students. Educational chatbots are still in their early stages, and many
researchers have used recent AI techniques such as in deep learning and reinforcement
13
learning to improve the performance of chatbots. However, there are still major tech-
nical, linguistic, and psychological challenges. Recent researches suggested combining
more than one technique in the field of AI and NLP to overcome the challenges of the
existing chatbots.
References