Applied Computer Systems

ISSN 2255-8691 (online)

ISSN 2255-8683 (print)
December 2023, vol. 28, no. 2, pp. 232–244
https://doi.org/10.2478/acss-2023-0023
https://content.sciendo.com

AI Chatbot for Tourist Recommendations: A Case

Study in Vietnam
Hai Thanh Nguyen *, Thien Thanh Tran, Phat Tan Nham, Nhi Uyen Bui Nguyen, Anh Duy Le
College of Information and Communication Technology, Can Tho University, Can Tho, Vietnam

Abstract – Living standards are rising due to a more developed increasing unemployment and underemployment; reducing FDI
society, and recreation, particularly tourism, is becoming more and hurting the tourism sector. The results show that
critical. Expanding the tourist industry is one of the most multinational sight-seer arrivals to destinations worldwide were
significant concerns in economic growth. Tourism revenue has
helped increase residents’ income, leading to socio-economic 1.4 billion in 2018, representing a growth of 6.0 % compared to
development. In recent years, emerging Vietnamese tourism spots 2017 [2], indicating steady growth in tourism. Traveling is
like Hon Son, Sapa, Hue, Phu Quoc in Vietnam, and others have considered a hobby that is enjoyed by everyone, often seen as
consistently drawn travellers to visit and experience through social an enthusiastic leisure activity after extended periods of dealing
networking platforms. Tourism potential is tremendous, but with assignments and daily pressures. Travel is cherished not
foreign visitors’ information about tourist destinations still needs only for its unexpected benefits to mental health but also as a
to be improved. This work proposes an approach to integrating
machine learning algorithms into an information system to consult
means of stress relief that individuals can share with their
tourism traveling. Machine learning algorithms can classify family and friends. It serves as a way to strengthen relationships
question topics, predict user intent, and predict conversation and create unforgettable memories with loved ones.
scenarios to give appropriate responses. Our method is evaluated In addition, tourism brings not only mental health benefits to
on the dataset, including 7319 samples on 11 topics collected from people but also many great economic benefits and contributes
the TWCS dataset, using three algorithms: Bag of Words, BERT, to the development of an entire country [3]. From an economic
and RoBERTa. BERT achieved the highest performance among
point of view, numerous countries with favourable climatic and
the surveyed algorithms with 90 % in accuracy and 90.1 % in F1-
Score. From the trained model, the team built a mobile application geographical locations have chosen tourism as one of the
on Android to deploy the chatbot application with the Flutter spearheads to develop their national potential. The authorities
framework based on Dart, an object-oriented programming make policies to support and develop tourism and the
language developed by Google using the concept of containers. The infrastructure to support tourism, which can be mentioned in
system’s functionality serves two primary user groups: famous tourist countries such as France, Vietnam, Finland,
administrators and application users. Administrators can utilize
Sweden, and Denmark, among others. The fact that people take
the application’s primary functions to manage content set up, and
train a chatbot. Users can access information about locations, read advantage of vacations for tourism has contributed to creating
location articles, check hotel prices, and use chatbots to find more jobs and income for individuals in countries with solid
answers to their location-related questions. Administrators can tourism development. Specifically, in compliance with the
also train the chatbot model to expand its knowledge. World Travel and Tourism Council (WTTC), the Travel and
Tourism assiduousness contributed to the creation of 22 million
Keywords – BERT, chatbot, recommendation system, tourist. further employment in 2022, an expansion of 7.9 % compared
to 2021 and only 11.4 % more inferior than in 2019 and
I. INTRODUCTION
contributed 7.6 % to international GDP 2 . Besides, tourism
Recently, along with the remarkable advancement of development also plays a vital role in promoting the whole
civilization, individuals have also been caught in the whirlwind world about the history and culture of a country, contributing to
of this progress, perpetually bearing the invisible burdens of the creating opportunities for exchange between cultures around
economy, family, and social connections. This is particularly the world and helping people understand each other and connect
prevalent in developing countries, where people must race better. From the above benefits, it is easy to see tourism
against time, often sacrificing their health due to the stress of importance and long-term advantages to countries’ economic
obtaining money to improve life’s quality. As a consequence, development and cultural exchange.
individuals are increasingly seeking ways to alleviate stress and In recent years, with the outstanding development of modern
fatigue accumulated from prolonged career challenges, with science and technology, especially in computer science with the
travel being a likely choice. Based on statistics, COVID-19 has strong development of Artificial Intelligence (AI), a
wreaked havoc across the globe: by slowing down economic considerable number of countries have used AI to contribute to
growth; decreasing global trade; hurting health sector; the simplification and enhancement of efficiency in many
*
Corresponding author’s e-mail: [email protected]
2
https://wttc.org/research/economic-impact

©2023 Hai Thanh Nguyen, Thien Thanh Tran, Phat Tan Nham, Nhi Uyen Bui Nguyen, Anh Duy Le.
This is an open access article licensed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0).
232
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

fields, such as agribusiness, healthcare, and education [4], and Ctrip.com, have made it more leisurely for clients to reserve
it is applied to support the expansion of the spearhead economic travel packages and airline tickets using chatbots integrated into
sectors. As a trend, AI is also being developed and their websites [11]. As a result, realising the high applicability
experimented with in all areas of the travel and tourism and great effectiveness that chatbots bring to the travel and
industry, including personalization and guidance systems, tourism industry, especially in the context of the rapidly
confidential travel attendants, robotics, prognosis, and developing tourism industry today, we have collected the
projection systems, speech translation applications, speech essential information and conducted a series of experiments
tribute, and natural terminology processing techniques [5]. AI with different approaches, finally giving birth to a travel
is also expected to motivate more endurable tourism [6] by support chatbot for foreign tourists based on three web service
exploiting clients for a better social perspective. The above platforms, mobile apps, and machine learning (ML).
information shows the feasibility and potential of AI in With the desire to bring outstanding contributions and
particular, as well as computer science in general, in developing provide multi-dimensional perspectives to support further
the field of tourism and travel, promoting tourism development investigations, we have analysed numerous aspects to bring
in countries, creating more favourable conditions, and readers the most intuitive view. Hypothetically, we explore
enhancing the experience for international tourists. aspects such as the theory of machine learning and deep
AI is highly feasible in travel and tourism for several learning, the transformer model in natural language processing,
advantages. Firstly, tourists must make a series of the attention mechanism [12], mask attention [13], and research
determinations about their forthcoming journey. For on the bidirectional encoder representations from transformers
illustration, selecting a terminus is the first consideration; (BERT) model [14], robustly optimized BERT approach
means of transportation are also essential, and the location of (RoBERTa) [15]. Technically, we explore aspects such as how
lodging and activities during the trip also need to be scheduled. to install and deploy RASA framework [16], install and
Besides, there are numerous other things to arrange, especially configure BERT models, train and deploy trained models, and
for family voyages. These determinations will particularly program management software to manage and track
influence tourists’ pleasure with their trip. Nevertheless, the conversations. After the research process, we completed a
factors mentioned above have almost endless ways to choose tourism project with a tourist information search and lookup
from, and it takes a wise plan to ensure the best experience for application with considerable valuable features:
the trip. Secondly, travel instruments and associations meet the • The research created an application that includes an
same challenge when finding the most suitable match between integrated chatbot, helping users quickly find information
clients and travel packages that suit their requirements [5]. about tourist destinations.
Organisations have an almost endless source of potential • The chatbot recommends recent places based on the user’s
clients. Therefore, fulfilling customer requirements with travel interests and location. To do this, we have integrated the
services is a highly intricate process and seems reasonably chatbot into the application, allowing users to interact
conformed to take advantage of AI ability to handle vast directly and conveniently. The application’s database is
amounts of information. Thirdly, when arriving at their managed through a web interface, making it easy to
terminus, visitors are exposed to an entirely new culture, change or add new tables and data fields as needed.
characterised by completely different conventions, languages, • We have successfully integrated BERT into RASA to
social standards, and cuisines than where they usually live, and optimize user intent prediction.
considerable other communication components that may be
As a result, the user’s experience of searching and looking
strange to them. Last but not least, AI can help travellers freely
up tourist information is improved, responding more accurately
integrate into unfamiliar cultures, such as by suggesting travel
and quickly. Regarding the architecture, the article is
itineraries and helping overcome language impediments [5].
systematized as follows. We provide an exhaustive analysis of
Additionally, AI can assist organisations in optimising the
the application of AI in tourism support through reference and
customer experience by recording most customers’ travel habits
evaluation of related studies in Section II. This helps elucidate
and displeasures, improving service quality.
the importance and impact of applying AI in solving the
From the difficulties that the tourism industry encounters
existing problems of the tourism industry today. In Section III,
above and the feasibility of integrating AI to unravel the issue,
we describe the applied research approach and illustrate an
numerous solutions have been researched and developed, the
outline of the system structure. This section focuses on
most prominent of which is the approach of researching and
introducing the overview architecture of the system in
launching chatbots. The basic concept of chatbots was dated to
particular and transparently about the main components of the
around 1921 when robotics was presented [7]. So far, chatbots
procedure. Through its relentless and logical expression, this
have been involved in numerous ambitions. Over 85 % of
section helps the reader better comprehend the sophistication
consumer interactions were predicted to be superseded by
and feasibility of the involved research method. We describe
chatbots in the late 2020 [8]. For the travel and tourism industry,
the data used in Section IV, including how we split the data for
chatbots have shown to be sufficient in supporting services such
the problem and how we apply the data set to the given problem.
as booking/reservation, giving suggestions, and other valuable
Section V provides a straightforward overview of the methods
services [9], [10]. To illustrate, foremost online travel agencies
used to conduct system performance evaluations. In Section VI,
(OTAs) in China, such as Figgy.com, Qunar.com, and
we provide a detailed description of how to perform the test in

233
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

the given test scenarios to evaluate the system performance, recommendation and planning aid designed to optimize travel
thereby ensuring the scientificity and reliability of the results. experiences. Regarding usefulness, the app provides advice on
We comprehensively summarise and present the outcomes tourist attractions based on the traveller’s current interest and
obtained during the experiments in Section VII. Importantly, context. For hotel selection, travellers often look at online
this section is expected to provide a thorough and complete reviews of hotels by other users who have experienced the
reflection on the value and significance of the research. service there, making decisions about choosing a suitable hotel.
However, the increasing number of online hotel reviews leads
II. RELATED WORK to details overload. Understanding this, the authors in [23] used
There have been numerous studies on integrating a classification-based technique to consider the usefulness of
information technology into solving the remaining problems of online hotel reviews amassed from TripAdvisor.com. The
the tourism industry; the research has clarified numerous issues. outcomes indicate that reviewer factors are useful forecasters of
Moreover, it contributed to bringing the multi-dimensional review usefulness, while review quality and emotion are
view along with logical approaches to solving existing imperfect predictors of review usefulness of assessment.
problems and supporting the sustainable development of the Furthermore, the authors in [24] used a qualitative scale
tourism and travel industry. Authors in [17] involved three approach based on media-end-chain analysis. With a new
supervised ML models for the emotion category of reviews for perspective, the authors explored the ultimate values visitors
the preferred travel destinations. These models are Naïve Bayes expect when utilizing gamified elements on a destination-
classifier, SVM classifier, and dynamic language model related mobile app. Regarding data collection, 18 in-depth
classifier. The research utilized a dataset based on journey blog individual consultations were executed from March to August
reviews of seven widespread travel destinations in the United 2017 to serve the research process and use ATLAS.ti as data
States and Europe. Regarding precision, all three algorithms investigation software. As for the results, research demonstrates
can precisely achieve more than 80 % of classification. Besides, that gamified features help visitors gain some more elevated
the authors in [18] proposed a modern recommendation motivational purposes and boost their concentration in the
approach based on multi-criteria Collaborative Filtering (CF) to vacation destination. Meanwhile, research in [25] has
sweeten the predictive precision of recommender methods in developed an AI-integrated innovative map to aid the tourism
the tourism field. The experiments were conducted based on the industry. Brilliant map works based on visitors’ wishes; after
TripAdvisor dataset. In terms of contributions, the examination being processed by an AI algorithm, the system will construct
has dramatically assisted in utilizing the clustering set method several recommendations for suitable tourist attractions. As for
in the presented model. Meanwhile, research [19] presented a the outcomes, the system suggestions get an average of 73 %
travel recommendation system that extracts user appreciation to precision compared to the user’s cravings. Also, through the
supply personalized advice. The recommendation system is application of AI, the authors in [26] proceeded to develop an
estimated on a dataset from the TripAdvisor platform and built intelligent tourism method based on AI. The study conducted
on Python. Research has constructed a context-aware travel an in-depth analysis of the application of AI in China’s shrewd
suggestion technique. Considering several contextual factors, tourism, analysis of actual benefits, and outstanding issues.
the system can instruct attractions appropriate for the user’s After the analysis, the authors proposed an AI-based innovative
current state. With a different perspective, the authors in [20] travel system, which consists of four subsystems to enhance the
introduced a system called POST-VIA 360, which was operations of the existing method.
developed to collect data from the first visit using standard Similarly, besides the research on recommendation systems
approaches. After the analysis, the system could generate and intelligent maps for tourism, the chatbot is also considered
suitable colonial and contextual mobile suggestions based on an optimal solution to improve the efficiency of the tourism
place and a bio-inspired recommendation system. The study industry. Specifically, the authors in [27] used a qualitative
was conducted with samples from natural experiments based on research approach through semi-structured interrogation with
14 individuals participating (eight men and six women). As for 15 travellers who had used travel chatbots to find a connection
the consequences, the accuracy of the suggestions made by the between the readiness to use chatbots and the philosophy of
system is extraordinary compared to other analyses performed loneliness. Research has contributed to solidifying the
in the same field. functional applications of travel chatbots. With a different
In addition, the writers in [21] have proposed a unique approach, Giancarlo Sperlí in [28] proposed a cooperative data
framework called DCC-PersIRE to determine user selections model for tourism, including voice assistance and courtesy bus.
and instruct personalized journeys consequently. The proposal Using the micro-service architecture, the system supplies
also includes multiple Points of Interest (POIs) to maximize services over the Seq2Seq model. Regarding the effectiveness
user pleasure while remaining within the term budget. evaluation, the writer invited 50 users to interact with the
Comprehensive examinations on eight real-world datasets chatbot, proving that the presented method was superior to the
demonstrate the usefulness of DCC-PersIRE. In terms of modern method, combining biological language processing,
efficiency, the authors’ algorithm makes straightforward user silhouette details, and activity chronology.
predictions about user preferences and shows high feasibility. Also, Ashay Argal et al. in [29] proposed research on
Also, in support of trip personalization, research in [22] chatbots in tourism. Specifically, they analysed and deployed
presented an app named PSiS Mobile, a mobile an intelligent chatbot strategy in the travel field on the Echo

234
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

platform, which would collect user selections, model the user’s administrators. The application’s main functionality is
available scoop base, and use suggestions – restricted providing a reflexive and easy-to-use interface, permitting
Boltzmann Machine (RBM) with CF. The chatbot strategy administrators to conveniently and efficiently update and
solved fundamental problems such as identifying user intent modify the application’s content. In addition, the application
and searching for hotel room demand. Besides, the research in supplies a set of quality RESTful Application Programming
[30] was directed toward creating a web-based chatbot Interface (API) [34] services specifically designed to interact
application by implementing the Random Forest (RF) and smoothly with the application on mobile devices. These APIs
Rapid Automatic Keyword Extraction (RAKE) algorithm. The guarantee security and trustworthiness and provide flexible and
chatbot application would evolve into a virtual tour guide efficient data retrieval. Integrating APIs into mobile apps
conveyed to visitors in Bandung, Indonesia. Evaluation optimizes the user experience, allowing them to conveniently
operating the confusion matrix indicated that the average and quickly access information and interact with the app,
precision significance of the chatbot application using RF and regardless of device or geographic location.
RAKE was 98 %, the accuracy about was 96 %, and the recall
B. Chatbot Host System
about was 96 %. At the same time, social networks are
developing, and the integration of chatbots on social Regarding the server structure of the chatbot, after receiving
networking platforms is one of the priorities of businesses to messages from the user through the application, the user’s
support customers directly on these platforms. Support systems specific requirements are fed into the ML model system.
were born as an inescapable trend, typically in [31] of Chatfuel. Through this process, the system conducts a deep analysis,
This application lets developers quickly set up and install a makes accurate predictions, and proposes responses
chatbot based on a pre-set scenario. corresponding to each situation.
Overall, the application of AI to solve existing problems in If the chatbot needs to achieve more complex reactions or
the travel and tourism industry is of great significance [32]. AI behaviours in reply to a user request, the chatbot will
will determine how tours and journeys will be more disseminate with the server, requesting detailed actions to be
uncontroversial and can be improved. Regarding travel and executed. The servers will then receive the request, perform the
tourism, the foremost thing that comes to intellect is how assigned tasks, and return the outcomes to the chatbot so that it
technology has made everything for us. To summarise, the can respond to the user comprehensively and effectively. By
studies mentioned earlier have shown the positivity and doing this, the chatbot can provide the user with a personalized
necessity of applying information technology to support the interactive experience.
tourism industry, significantly when human tourism needs are C. Database Server
growing as high as it is now.
The area where the system’s central database storage
centralization has been realized plays an essential role in storing
III. PROPOSED APPROACH
and maintaining important data from the various components of
Figure 1 presents the general architecture of the system. The the system. A database server is where the database from the
components include five parts: content management system, content management system and the chatbot’s APIs are safely
chatbot host system, database server using PostgreSQL [33], and efficiently stored and preserved.
action server, and mobile applications. This database performs the role of interaction between
various parts of the system, allowing data to be retrieved,
updated, and interacted synchronously. This database is
managed and maintained by complying with security and data
management standards, ensuring the goodness and availability
of vital information to sustain the system’s function constantly
and stably.
The database server uses the PostgreSQL database
management system. This allows the process of handling and
storing data effectively, ensuring the performance and
reliability of the system during the procedure. PostgreSQL
offers robust data processing capabilities and adaptable
scalability, meeting the requirements of the research process.
This choice has made it possible for administrators to examine
conversations, label, and optimize the chatbot in the most
pleasing form.
Fig. 1. Proposed general system. D. Action Server
In the context of the complex requirements encountered by
A. Content Management System
the chatbot, such as sophisticated Structured Query Language
The web application being introduced is a comprehensive (SQL) queries or the execution of moderately complicated API
solution specifically optimized to meet the needs of calls, the server hosting the chatbot will conduct an intuitive and

235
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

intelligent examination. When receiving a request from the • Action server: The server performs pre-set actions to
chatbot, this server will continue to perform an intricate series complete a particular request from the user, such as calling
of processing operations. an API or SQL query to get data. This server should run
Remarkably, these movements include gathering and independently of the Agent to reduce load and latency for
managing the required SQL queries or API calls. The results of chatbots.
these operations are then received by the server containing the • Tracker store: SQL Server stores each user’s conversation
chatbot and then forwarded to the chatbot so that it can give a to track the conversation to deliver the correct answer.
suitable answer, thereby ensuring precision and improving • Lock store: This is a short-term data storage server to
efficiency during user interaction. determine the order in which messages from the same
person are processed in the correct order. This component
E. Mobile Application
can be deployed on a standalone Redis service or use RAM
The research has successfully developed a unique mobile sticks with the Agent.
application that integrates an intelligent chatbot, providing an • Filesystem: Component that stores training data or trained
efficient travel information search experience. In order to models. The Agent will perform the model upload from
optimize the user experience, the application is designed with here on the initial launch. The filesystem can be hosted
an intuitive interface, the ability to interact, and deliver unique independently or together with the Agent server.
suggestions of the chatbot combined with a diverse database of
tourist destinations to promote the user’s travel experience in a IV. DATA DESCRIPTION
positive direction. The application is a critical stage in taking
This analysis utilises the Customer Support on Twitter
advantage of technology to satisfy the requirements of finding
(TWCS) dataset [35]. The dataset contains more than three
tourist details quickly and with quality.
million tweets and replies on the social networking platform
The architecture of the chatbot system consists of seven main
Twitter of global businesses in customer care. The dataset is
components as described in Fig. 2.
distributed under the CC BY-NC-SA 4.0 license 3. The dataset
was collected and aggregated from 20 major brands in different
sectors, as illustrated in Fig. 3. Nevertheless, this research
focuses on tourism and only extracts data from American Air,
British Airways, and Southwest Air. It includes about 40,000
lines of dialogue and is hand-processed and labelled. Details
about the dataset after being labelled with 7319 lines and
11 classes are described in Table I.

Fig. 2. RASA chatbot system [16].

Regarding the specific tasks of the components, it can be

understood thoroughly as follows:
• Agent: This is the central processing core of the chatbot, Fig. 3. Top 20 brands in the dataset [35].
responsible for coordinating input data, making calls to
other services to receive answers, and responding to This research performs cross-validation with several folds
answers about the user. of 5. Cross-validation means the training dataset will be divided
• NLU Pipeline: This is a place for developers to install into five parts (20 % each). Then, with each run, the model will
machine learning services to perform train–predicted key- learn from the remaining 80 %, use 20 % to test, and cycle
words. In this study, we use the BERT model to optimize through all the data sets. The final result will be the average of
the prediction. the trials. Cross-validation is a statistical method used to
• Dialogue Policies: Acting as a supervisor, it will evaluate the performance of machine learning models. It is
continuously monitor the conversation and give often used to compare and select the best model for a problem.
appropriate replies in exceptional cases. This technique is easy to understand and implement and gives

3
”Customer Support on Twitter”
https://www.kaggle.com/datasets/thoughtvector/customer- support-on-twitter

236
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

more reliable estimates than other methods. The k-fold cross- From the above analysis, this irregularity poses an essential
validation technique is also applied in many studies [36]–[39]. challenge in developing and optimizing the classification
model. Considerations about how to process and model the data
TABLE I to ensure balance and consider data augmentation strategies are
DATA FOR TRAINING AND TESTING essential factors in achieving the best performance of the model
Amount in a given situation, such as unequal data situations.
No. Label Description Training Testing
Total
(80 %) (20 %) V. EVALUATION
1 greet Greeting 433 108 541 A. Checking User’s Intent Classification Accuracy
2 goodbye Goodbye 776 194 970
In order to evaluate the accuracy of the chatbot in recognising
3 affirm Affirm 770 193 963 user intent, the study applied the Cross-validation test method,
4 deny Deny 462 115 577 evaluating an independent data set with an average of 5-fold
5 mood great Good mood 234 58 292 accuracy. The results achieved are reflected in the following
6 mood unhappy Bad mood 750 187 937 scales:
7 bot challenge Ask about bot 273 68 341 • Accuracy scale: Also known as model accuracy. This is a
simple but easy scale to evaluate a model. The formula for
8 get suggest Get suggest 593 148 741
calculating accuracy is shown below:
Give type
9 give interests 460 115 575
of location
TP + TN
10 give city Give locations 432 108 540 Accuracy = .
11 giving range Give range 674 168 842
TP + TN + FP + FN

The parameters are presented as follows:

In Fig. 4, we unquestionably and intuitively see that the – TP (true positive): Number of samples predicted to
distribution of data between classes is not uniform, an important belong to that class and their actual label values to
aspect that can affect the processing and classification of data. belong to that class.
This is demonstrated by corresponding the percentages of each – TN (true negative): The number of predicted samples
intent. For instance, we have two intents: “goodbye” and that do not belong to that class and their true label values
“mood_unhappy” with equivalent percentage, representing a also do not belong to that class.
relative harmony. Nonetheless, when we pay attention to the – FP (false positive): Number of samples that are
“mood_great” intent, we see the lowest percentage, which predicted to belong to that class, and their actual label
means that the presence of this layer in the data set is less than values also do not belong to that class.
that of other intents. – FN (false negative): The number of samples predicted
that do not belong to that class, and their true label
values also belong to that class.
• Confusion matrix [40]: The calculation using accuracy as
above only tells us what percentage of the data is correctly
classified without specifying how each type is classified
and which class is classified most correctly. Data
belonging to one class is often misclassified into another
class. To be able to evaluate these values, we use a matrix
called the confusion matrix. The confusion matrix
represents how many data points belong to a class and are
expected to fall into a class.
• Precision-Recall scale: For classification problems where
the data sets of classes are very different, there is a
commonly used efficiency measure called Precision-
Recall. To define a class as positive, Precision is defined
as the ratio of the number of true positives among those
classified as positive (TP + FP). Recall is the ratio of true
positives out of those actual positives (TP + FN). We have
the formula for calculating precision described below:

TP
Fig. 4. Amount of data for each intent.
Precision = .
TP + FP

237
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

For Recall, we have the formula: VII. RESULTS

The section presents in detail the main results achieved
TP during the research process. These results represent the effects
Recall = .
TP + FN in the examination process, as well as this is the scale for the
effectiveness of the introduced chatbot. The research data
• F1-Score: F1-Score is a scale with values in the range presented and our approach can serve as a premise for the
(0, 1]. The higher the F1-Score, the better the classification development of further deconstructions.
model. To calculate the F1-Score, we have the formula:
A. Checking User’s Intent Classification Accuracy
P× R Through testing on three different models, including Bag-of-
F1= 2 × . word, BERT, and RoBERTa, we conducted a series of
P+R quantitative tests to evaluate performance. The scales used
include Accuracy, F1-Score, and Precision. The result is the
The parameters are presented as follows: average of cross-validation with folds of 5. We experimented
– P: The result of the Precision scale. with various epochs, including 20, 50, 100, and 200, to
– R: The result of the Recall scale. demarcate the consequence of epoch counts on the outcomes.
The unambiguous consequences of the testing and
VI. EXPERIMENTS
comparison procedure are shown in Table II.
As for the experiment, the conversation script is a solid and
essential framework that forms the core foundation for the TABLE II
chatbot operation. Assessing the effectiveness of chatbots for RESULT OF THREE MODELS
user experience, testing and evaluation have gone through many Accuracy F1-Score Precision
times. We need a more comprehensive approach in which Model Epochs
Train Test Train Test Train Test
careful consideration of each pre-established scenario and
20 0.839 0.636 0.831 0.595 0.884 0.644
experimentation with random interactions play a vital position.
50 0.977 0.715 0.977 0.676 0.981 0.676
By taking this approach, the analysis provides a Bag-of-word
straightforward and unpretentious view of the chatbot ability to 100 1.000 0.795 1.000 0.764 1.000 0.756
function in predictable situations, interact, and flexibly react to 200 1.000 0.848 1.000 0.832 1.000 0.831
changes in the context. This process reviews the results against 20 0.588 0.509 0.562 0.477 0.601 0.503
predetermined scenarios and randomized trials, providing an 50 0.869 0.729 0.857 0.695 0.892 0.713
objective and realistic approach to evaluating chatbot BERT
100 0.998 0.834 0.998 0.813 0.998 0.831
performance. Experimental scenarios are presented below.
200 1.000 0.900 1.000 0.893 1.000 0.901
• Scenario 1: Greetings, casual chat: In this test, we test the
20 0.601 0.544 0.566 0.494 0.598 0.486
chatbot with common conversations such as hello,
introduction to the chatbot, thank you, and goodbye. This 50 0.864 0.636 0.853 0.587 0.880 0.594
RoBERTa
test scenario aims to check if the chatbot can correctly 100 0.990 0.788 0.990 0.770 0.991 0.788
recognise the intent in the user’s dialogue and then give 200 1.000 0.808 1.000 0.794 1.000 0.807
the appropriate response.
• Scenario 2: Greetings; suggest the type of place in a The results in Table II show that on the test set, the BERT
certain city. In the second scenario, the difficulty will model produces more accurate results in Accuracy, F1-Score,
increase. The chatbot needs to recognise the keywords in and Precision at epochs 20, 50, 100, 200 than the other two
the user’s dialogue to get the information to perform a models, Bag-of-Word and RoBERTa
more complex action. The chatbot needs to classify When conducting batch highlighting of data for analysis, we
intents, identify keywords and entities inside the dialogue, found that the BERT model performed outstandingly on all
and perform extracting from the dialogue. three scales, showing the best accuracy when trained over 200
• Scenario 3: Finding detailed information about a place. epochs. These results provide a pearl of in-depth and instinctive
The chatbot supports searching for information on the wisdom into the potential of BERT and the significance of
application to serve users looking for information about a tuning the number of epochs to achieve the expected optimal
place. When users need to find information about a performance. Besides, Figs. 5 and 6 show that the BERT model
particular place, users will send a message requesting the demonstrates superior accuracy when increasing the number of
chatbot and provide the place’s name, after chatbot receive epochs from 50.
input from users, chatbot will response and provide
information about particular place.
• Scenario 4: Random scenario test. To test this scenario, the
user will randomly enter non-business lines of the chatbot
to test the chatbot behaviour when receiving out-of-scope
requests.

238
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

According to Fig. 9, the Bag-of-Word model effectively

recognises different sentences. Nonetheless, for negative
sentences (red circle), the model will often give confusing
results with negative sentence determination.
For the BERT model, we obtain a classification matrix as
shown in Fig. 10. Since BERT is a model that takes care of the
two-dimensional context of words, it gives better results when
recognising negative classes.
For the RoBERTa model, we get the results as shown in
Fig. 11. For RoBERTa, this model does not use next sentence
prediction (NSP) from BERT but uses dynamic masking,
whereby mask tokens will be changed during training.
Fig. 5. Accuracy on each model. Therefore, it will reduce the accuracy of the model when
training with a small data set.

Fig. 6. F1-Score on each model.

When compared at 200 epochs (Fig. 7), BERT delivers

extremely impressive results on all three metrics on both the
training and test datasets. Figure 8 shows time of three models.
Again, the BERT model gives the best time processing results.

Fig. 9. Confusion matrix on Bag-of-Word model.

Fig. 7. Comparison of models with 200 epochs.

Fig. 10. Confusion matrix on BERT model.

Fig. 8. Time processing of three models.

239
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

TABLE III
A TEST CASE IN SCENARIO 1 USING COMMAND LINE INTERFACE (CLI)
# Bot You
1 action_listen
hello
2
intent: greet 1.00
utter_greet 1.00
3 Hey! How are you?
action_listen 1.00
great
4
intent: mood_great 1.00
utter_happy 1.00
5 Great, carry on!
action_listen 1.00
i'm talking with human or bot
6
intent: bot_challenge 1.00
utter_iamabot 1.00
7 I am a bot, powered by Rasa.
action_listen 1.00
Fig. 11. Confusion matrix on RoBERTa model. nice, thank you
8
intent: thank 0.83
B. Scripted Accuracy Check utter_thank 1.00
9 You're welcome
After the test execution with predefined and random action_listen 1.00
scenarios, we obtained the aggregate outcomes provided below. bye
10
For Scenario 1, the experiment was conducted with the utter_goodbye 1.00
essential communication and greeting scenario illustrated in intent: goodbye 1.00
11
Fig. 12. It can be seen in Fig. 12 that for basic communication Bye
situations such as greetings, questioning about health status,
expressing gratitude and ending the conversation, the chatbot For Scenario 2, the experiment was conducted with
demonstrates the ability to understand and respond suitably for greetings, suggesting the type of place in a specific city as
common types of dialogue. Understanding, reacting, and giving illustrated in Fig. 13.
the correct answers are acceptable, thereby constructing a
smooth and fluid experience for the user.
As can be seen in Table III, the chatbot correctly recognises
the user’s messages with an approximate value of 1.00. Only
the message “nice, thank you” intentionally dilutes the data, so
the accuracy is only 0.83, but it also brings high accuracy.

(a) Extract the entity from the response about the desired place type

Fig. 12. A test case in Scenario 1 on app.

240
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

TABLE IV
SOME TEST CASES IN SCENARIO 2 USING CLI
# Bot You
1 action_listen
hello
2
intent: greet 1.00
utter_greet 1.00
3 Hey! How are you?
action_listen 1.00
great
4 intent: mood_great
1.00
utter_happy 1.00
5 Great, carry on!
action_listen 1.00
sugest me a place
6 intent: get_suggest
1.00
suggestion_form 1.00
active_loop{"name": "suggestion_form"}
slot{"requested_slot": "interests"}
Please tell me what you interests, mountain,
beach or
cultural place?
slot{"interests": "beach"}
(b) Predict and extract words after the answer about the location/city slot{"requested_slot": "city"}
What the city/province you will go to?
Fig. 13. Some test cases in Scenario 2 on app. slot{"city": "Vung Tau"}
slot{ "requested_slot": "range"}
Specify search radius, in km. slot{"range":
In the case of Fig. 13a, the chatbot will ask the user what kind "4"}
of place the user wants to search for. After receiving the answer, 7 slot{ "requested_slot": null}
the chatbot will extract an entity from the sentence to store it in active_loop{"name":null}
the clipboard. When the user enters “a beach” the chatbot will action_suggest_destinations 1.00
Beach
extract “beach” in the sentence and save it in memory.
-Vung Tau Front Beach
Continuing, the chatbot will ask about the location/city the user -Paradise Beach
wants to search for, as shown in Fig. 13b. When the user names -Vung Tau Back
a place or a city, the chatbot will continue to make predictions Beach
and extract exactly what words the user wants. In another -Pullman Beach Club
example, if the user enters “I want to find a place in Can Tho” -VT beach
-Bãi Sau
the chatbot will have to recognise the desired keyword as “Can -Bãi Thùy Vân
Tho”. -Bien Dong
The results show that the chatbot can recognise user dialogue -Khu du lịch Gió Biển
entities. It makes an API call to look up a list of desired
locations. Specifically, Table IV shows the steps at the server. For Scenario 3, the experiment was conducted by finding
When the application launches the form crawler, the user will detailed information about a place as illustrated in Fig. 14. In
ask to enter the requested slots in turn until the necessary the case of Fig. 14, the chatbot successfully handles when the
information is collected. The server then executes action- user requests to get information about any location. Whenever
suggest-destinations to call the location search API. a user requests to search for a specified place, the chatbot will
perform a location search through the API and respond with
information about that place. However, there is a limit on the
amount of API information. The chatbot cannot get other
contact information, such as phone numbers or email addresses.
Besides, Table V shows that, not only in forms, chatbots can
still extract information from regular messages. For example,
after receiving the get-detail request, the chatbot will extract the
place the user wants to search for, “Ninh Kieu Restaurant” and
perform the action-get-detail action.

241
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

Fig. 14. Search information about a location in Scenario 3 on app.

Fig. 15. A random test in Scenario 4.
TABLE V
SEARCH INFORMATION ABOUT A LOCATION IN SCENARIO 3 ON CLI VIII. CONCLUSION
# Bot You After conducting a series of scenarios, including pre-
1 action_listen planning and random interactions, we can draw some
2
hello significant conclusions about the ability of chatbots. Preferably,
intent: greet 1.00 the chatbot has demonstrated an outstanding and precise
utter_greet 1.00 response-ability, understanding, and accurate answer to
3 Hey! How are you?
inquiries within the programmed content. This represents a
action_listen 1.00
lavish effort in building and training the model.
great, thank
4
intent: thank 1.00 In addition, the chatbot ability to handle random interactions
utter_thank 1.00 is also remarkable. The model has been designed to respond
5 You're welcome. correctly in the circumstances unrelated to the immediate
action_listen 1.00 function by applying automated responses that inform its
I want to get details about [Ninh limitations. This allows ensuring translucency and efficiency in
6 Kieu Restaurant] (place) intent: interchanges.
get_detail 0.99
slot{"place": "Ninh Kieu
Restaurant"} CONFLICT OF INTEREST STATEMENT
action_get_detail 0.99 All authors declare that they have no conflict of interest.
7 Ninh Kiều Restaurant Address:
2, Hai Bà Trưng, Cần Thơ,
Thành Phố Cần Thơ Category: REFERENCES
Vietnamese Restaurant [1] A. Khan, N. Khan, and M. Shafiq, “The Economic Impact of COVID-19
from a Global Perspective,” Contemporary Economics, vol. 15, no. 1, pp.
For Scenario 4, the experiment was conducted with a random 64–75, 2021. https://doi.org/10.5709/ce.1897-9254.436
[2] E. Saluveer et al., “Methodological framework for producing national
scenario trial as illustrated in Fig. 15. An essential part of the tourism statistics from mobile positioning data,” Annals of Tourism
testing process is the face-to-face encounter to test the chatbot Research, vol. 81, Mar. 2020, Art. no. 102895.
responsiveness to conversations unrelated to the primary https://doi.org/10.1016/j.annals.2020.102895
[3] N. Comerio and F. Strozzi, “Tourism and its economic impact: A
assignment. In such situations, such as a question that is not literature review using bibliometric tools,” Tourism Economics, vol. 25,
functionally relevant or a subject that is beyond the range of the no. 1, pp. 109–131, Aug. 2018. https://doi.org/10.1177/1354816618793762
chatbot, the system has been configured to achieve a [4] H. Herath and M. Mittal, “Adoption of artificial intelligence in smart
cities: A comprehensive review,” International Journal of Information
mechanical reaction. This response highlights that the chatbot Management Data Insights, vol. 2, no. 1, Apr. 2022, Art. no. 100076.
cannot handle this particular query or topic and makes it https://doi.org/10.1016/j.jjimei.2022.100076
unmistakable to the user that the chatbot will decline to supply [5] J. Bulchand-Gidumal, “Impact of artificial intelligence in travel, tourism,
and hospitality,” in Handbook of e-Tourism, Z. Xiang, M. Fuchs,
an answer. This ensures translucency and exactness in U. Gretzel, and W. Höpken, Eds. Springer, Cham, 2022, pp. 1943–1962.
interactions and delivers a comprehensive and quality user https://doi.org/10.1007/978-3-030-48652-5_110
experience.

242
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

[6] I. Tussyadiah and G. Miller, “Perceived impacts of artificial intelligence [24] A. Aebli, “Tourists’ motives for gamified technology use,” Annals of
and responses to positive behaviour change intervention,” in Information Tourism Research, vol. 78, Sep. 2019, Art. no. 102753.
and Communication Technologies in Tourism 2019, J. Pesonen and https://doi.org/10.1016/j.annals.2019.102753
J. Neidhardt, Eds. Springer, Cham, Dec. 2018, pp. 359–370. [25] I. D. Wahyono, K. Asfani, M. M. Mohamad, A. Aripriharta, A. P.
https://doi.org/10.1007/978-3-030-05940-8_28 Wibawa, and W. Wibisono, “New smart map for tourism using artificial
[7] A. Tlili, B. Shehata, M. A. Adarkwah, A. Bozkurt, D. T. Hickey, intelligence,” in 2020 10th Electrical Power, Electronics,
R. Huang, and B. Agyemang, “What if the devil is my guardian angel: Communications, Controls and Informatics Seminar (EECCIS), Malang,
ChatGPT as a case study of using chatbots in education,” Smart Learning Indonesia, Aug. 2020, pp. 213–216.
Environments, vol. 10, Feb. 2023, Art. no. 15. https://doi.org/10.1109/EECCIS49483.2020.9263435
https://doi.org/10.1186/s40561-023-00237-x [26] M. Xu, “Research on smart tourism system based on artificial
[8] J. Wirtz, P. G. Patterson, W. H. Kunz, T. Gruber, V. N. Lu, S. Paluch, and intelligence,” in 2023 IEEE 3rd International Conference on Information
A. Martins, “Brave new world: service robots in the frontline,” Journal Technology, Big Data and Artificial Intelligence (ICIBA), Chongqing,
of Service Management, vol. 29, no. 5, pp. 907–931, Sept. 2018. China, May 2023, pp. 201–205.
https://doi.org/10.1108/JOSM-04-2018-0119 https://doi.org/10.1109/ICIBA56860.2023.10165293
[9] F. Wotawa, I.-D. Nica, and O. Tazl, “Chatbot-based tourist [27] P. Wang and J. Shao, “Escaping loneliness through tourist-chatbot
recommendations using model-based reasoning,” in 20th International interactions,” in Information and Communication Technologies in
Workshop on Configuration, ConfWS, 2018. [Online]. Available: Tourism 2022, J. L. Stienmetz, B. Ferrer-Rosell, and D. Massimo, Eds.
https://ceur-ws.org/Vol-2220/05_CONFWS18_paper_31.pdf Springer, Cham, 2022, pp. 473–485. https://doi.org/10.1007/978-3-030-
[10] D. C. Ukpabi, B. Aslam, and H. Karjaluoto, “Chatbot adoption in tourism 94751-4_44
services: A conceptual exploration,” in Robots, Artificial Intelligence, and [28] G. Sperlí, “A cultural heritage framework using a deep learning based
Service Automation in Travel, Tourism and Hospitality, Emerald chatbot for supporting tourist journey,” Expert Systems with Applications,
Publishing Ltd., Oct. 2019, pp. 105–121. https://doi.org/10.1108/978-1- vol. 183, Nov. 2021, Art. no. 115277.
78756-687-320191006 https://doi.org/10.1016/j.eswa.2021.115277
[11] L. Li, K. Y. Lee, E. Emokpae, and S.-B. Yang, “What makes you [29] A. Argal, S. Gupta, A. Modi, P. Pandey, S. Shim, and C. Choo,
continuously use chatbot services? evidence from Chinese online travel “Intelligent travel chatbot for predictive recommendation in echo
agencies,” Electronic Markets, vol. 31, pp. 575–599, Jan. 2021. platform,” in 2018 IEEE 8th Annual Computing and Communication
https://doi.org/10.1007/s12525-020-00454-z Workshop and Conference (CCWC), Las Vegas, NV, USA, Jan. 2018,
[12] Z. Niu, G. Zhong, and H. Yu, “A review on the attention mechanism of pp. 176–183. https://doi.org/10.1109/CCWC.2018.8301732
deep learning,” Neurocomputing, vol. 452, pp. 48–62, Sep. 2021. [30] D. S. Maylawati et al., “Chatbot for virtual travel assistant with random
https://doi.org/10.1016/j.neucom.2021.03.091 forest and rapid automatic keyword extraction,” in 2021 IEEE 7th
[13] Z. Fan et al., “Mask attention networks: Rethinking and strengthen International Conference on Computing, Engineering and Design
transformer,” in Proceedings of the 2021 Conference of the North (ICCED), Sukabumi, Indonesia, Aug. 2021, pp. 1–6.
American Chapter of the Association for Computational Linguistics: https://doi.org/10.1109/ICCED53389.2021.9664876
Human Language Technologies, Jun. 2021, pp. 1692–1701. [31] H. K. Ahmed and J. A. Hussein, “Design and implementation of a chatbot
https://doi.org/10.18653/v1/2021.naacl-main.135 for Kurdish language speakers using Chatfuel platform,” Kurdistan
[14] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of Journal of Applied Research, vol. 5, no. 2, pp. 117–135, Dec. 2020.
deep bidirectional transformers for language understanding,” in https://doi.org/10.24017/science.2020.2.10
Proceedings of the 2019 Conference of the North American Chapter of [32] R. Goel, T. Singh, S. K. Baral, S. L. Sahdev, and S. Gupta, “The era of
the Association for Computational Linguistics: Human Language artificial intelligence reforming tourism industry in society 5.0,” in 2022
Technologies, vol. 1, Minneapolis, Minnesota, Jun. 2018, pp. 4171–4186. 10th International Conference on Reliability, Infocom Technologies and
https://aclanthology.org/N19-1423.pdf Optimization (Trends and Future Directions) (ICRITO), Noida, India,
[15] Y. Liu et al., “Roberta: A robustly optimized Bert pretraining approach,” Oct. 2022, pp. 1–4. https://doi.org/10.1109/ICRITO56286.2022.9964947
arXiv:1907.11692, Jul. 2019. https://doi.org/10.48550/arXiv.1907.11692 [33] B. Momjian, PostgreSQL: Introduction and Concepts, vol. 192. Addison-
[16] T. Bocklisch, J. Faulkner, N. Pawlowski, and A. Nichol, “Rasa: Open Wesley New York, 2001.
source language understanding and dialogue management,” in NIPS [34] X. Chen, Z. Ji, Y. Fan, and Y. Zhan, “Restful API architecture based on
Conversational AI Workshop, CA, USA, 2017. laravel framework,” Journal of Physics: Conference Series, vol. 910, Oct.
https://arxiv.org/pdf/1712.05181.pdf 2017, Art. no. 012016. https://doi.org/10.1088/1742-6596/910/1/012016
[17] Q. Ye, Z. Zhang, and R. Law, “Sentiment classification of online reviews [35] S. Axelbrooke, “Customer support on Twitter,” 2017. [Online].
to travel destinations by supervised machine learning approaches,” Expert Available: https://www.kaggle.com/datasets/thoughtvector/customer-
Systems with Applications, vol. 36, no. 3, pp. 6527–6535, Apr. 2009. support-on-twitter?trk=public_profile_project-title
https://doi.org/10.1016/j.eswa.2008.07.035 [36] H. L. Vu, K. T. W. Ng, A. Richter, and C. An, “Analysis of input set
[18] M. Nilashi, K. Bagherifard, M. Rahmani, and V. Rafe, “A recommender characteristics and variances on k-fold cross validation for a recurrent
system for tourism industry using cluster ensemble and prediction neural network model on waste disposal rate estimation,” Journal of
machine learning techniques,” Computers & Industrial Engineering, Environmental Management, vol. 311, Jun. 2022, Art. no. 114869.
vol. 109, pp. 357–368, 2017. https://doi.org/10.1016/j.cie.2017.05.016 https://doi.org/10.1016/j.jenvman.2022.114869
[19] Z. Abbasi-Moud, H. Vahdat-Nejad, and J. Sadri, “Tourism [37] Z. Wang, Y. Lei, H. Cui, H. Miao, D. Zhang, Z. Wu, and G. Liu,
recommendation system based on semantic clustering and sentiment “Enhanced RBF neural network metamodelling approach assisted by
analysis,” Expert Systems with Applications, vol. 167, Apr. 2021, Art. no. sliced splitting-based k-fold cross-validation and its application for the
114324. https://doi.org/10.1016/j.eswa.2020.114324 stiffened cylindrical shells,” Aerospace Science and Technology, vol. 124,
[20] R. Colomo-Palacios, F. J. García-Peñalvo, V. Stantchev, and S. Misra, May 2022, Art.no. 107534. https://doi.org/10.1016/j.ast.2022.107534
“Towards a social and context-aware mobile recommendation system for [38] X. Zhang and C.-A. Liu, “Model averaging prediction by k-fold cross-
tourism,” Pervasive and Mobile Computing, vol. 38, no. 2, pp. 505–515, validation,” Journal of Econometrics, Feb. 2022.
July 2017. https://doi.org/10.1016/j.pmcj.2016.03.001 https://doi.org/10.2139/ssrn.4032249
[21] L. Chen, L. Zhang, S. Cao, Z. Wu, and J. Cao, “Personalized itinerary [39] J. Roy and S. Saha, “Ensemble hybrid machine learning methods for gully
recommendation: Deep and collaborative learning with textual erosion susceptibility mapping: K-fold cross validation approach,”
information,” Expert Systems with Applications, vol. 144, Apr. 2020, Art. Artificial Intelligence in Geosciences, vol. 3, pp. 28–45, Dec. 2022.
no. 113070. https://doi.org/10.1016/j.eswa.2019.113070 https://doi.org/10.1016/j.aiig.2022.07.001
[22] R. Anacleto, L. Figueiredo, A. Almeida, and P. Novais, “Mobile [40] T. R. Shultz et al., “Confusion matrix,” in Encyclopedia of Machine
application to provide personalized sightseeing tours,” Journal of Learning, C. Sammut and G. I. Webb, Eds. Springer, Boston, MA, 2011.
Network and Computer Applications, vol. 41, pp. 56–64, May 2014. https://doi.org/10.1007/978-0-387-30164-8_157
https://doi.org/10.1016/j.jnca.2013.10.005
[23] P.-J. Lee, Y.-H. Hu, and K.-T. Lu, “Assessing the helpfulness of online
hotel reviews: A classification-based approach,” Telematics and
Informatics, vol. 35, no. 2, pp. 436–445, May 2018.
https://doi.org/10.1016/j.tele.2018.01.001

243
 Applied Computer Systems
_________________________________________________________________________________________________2023/28

Hai Thanh Nguyen is a Lecturer of the College of Phat Nham Tan, he was a student in information
Information and Communication Technology, Can systems at the College of Information and
Tho University, Vietnam. He received his Master Communication Technology of Can Tho
degree in Computer Science and Engineering from University, Can Tho, Viet Nam. He is experienced
the National Chiao Tung University, Taiwan, in in PHP, Laravel and Android, is a backend web
2014, and his PhD degree in Computer Science developer passionate about AI's role in enhancing
from Sorbonne University, France, in 2018. His everyday life through innovative software
current research includes bioinformatics, healthcare solutions.
systems, recommendation systems, and E-mail: [email protected]
applications using computer vision, machine
learning, and data analysis. Nhi Uyen Bui Nguyen, a student at College of
E-mail: [email protected] Information and Communication Technology, Can
ORCID iD: https://orcid.org/0000-0002-1386-1390 Tho University, Vietnam, is actively engaged in
research. Her primary areas of interest encompass
Thien Thanh Tran is a student at the College of machine learning, big data, and data analysis.
Information and Communication Technology of Committed to broadening her expertise in these
Can Tho University, Can Tho, Vietnam. His fields, she is eager to contribute to technological
research interests revolve around machine learning, innovation. Having a strong academic background
deep learning, and computer vision. He is dedicated and a passion for research, she is working towards
to expanding knowledge in these fields and aims to a machine learning-based network intrusion detection system, along with other
contribute to technological advancements. With a predictive systems.
solid academic foundation and a passion for Email: [email protected]
research, he strives to develop intelligent systems
that can improve various industries. Anh Duy Le is an Engineer in Information
E-mail: [email protected] Technology of the Can Tho University Publishing
House, Can Tho University, Vietnam. His interest
research includes recommendation systems,
machine learning, and deep learning.
E-mail: [email protected]

244