See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281904323

Evaluating the Usability of Educational Websites Based on Students'

Preferences of Design Characteristics

Article · January 2014

CITATIONS READS

58 3,692

1 author:

Layla Hasan
Universiti Teknologi Malaysia
35 PUBLICATIONS   661 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Usability of e-learning systems: the case of Moodle View project

Usability of Educational Websites View project

All content following this page was uploaded by Layla Hasan on 02 March 2016.

The user has requested enhancement of the downloaded file.

International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014 179

Evaluating the Usability of Educational Websites

Based on Students' Preferences of Design
Characteristics
Layla Hasan
Department of Computer Information Systems, Zarqa University, Jordan

Abstract: This research investigated the relative importance of specific design criteria developed for the purpose of this
research, in the evaluation of the usability of educational websites from the point view of students; it then evaluated the
usability of nine educational websites based on students’ preferences. The results showed that content and navigation were the

T
first and second preferred design categories to be considered while evaluating the usability of educational websites, while the
organisation/architecture was the least important category. Also, the results showed that there was a statistically significant
difference between males and females regarding only one category: the content. Females considered this to be the most
important category while males considered it as the second most important. By contrast, the results showed that there were no

e
statistically significant differences between the students of the two selected faculties (the Faculty of Information Technology
and Science, and the Faculty of Economics and Administrative Sciences) concerning the relative importance of the developed
criteria based on their majors/specialisations. In general, the results showed that the majority of the students were satisfied
with the usability of the Jordanian university websites. Specifically, the results showed the students were satisfied with the

J
content and navigation (ease of use) of the tested websites, but dissatisfied with the design of the websites.

Keywords: Usability, design criteria, educational websites, user testing, Jordan

Received February 3, 2013; Accepted March 17, 2013

A
whether it conforms to a set of usability principles
1. Introduction (namely 'heuristics') [9].

I
• Tool-based methods: involve software tools in the
Usability is one of the most important characteristics of process of identifying usability problems. The
any user interface; it measures how easy the interface software tools automatically assess whether a
is to use [11]. Usability has been defined as: "a website conforms to a set of specific usability
measure of the quality of a user's experience when guidelines. Most of these tools assess the quality of
interacting with a product or system - whether a web the HTML code of a website with regard to a
site, a software application, mobile technology, or any number of guidelines.
user operated device" [2]. A variety of usability
evaluation methods have been developed to evaluate Heuristic evaluation and user testing methods are two
human interaction with an interface; these aim to of the most frequently used methods for assessing
identify areas for improvements in the interactions in website usability as observed by Kantner and
order to increase usability [4]. The usability evaluation Rosenbaum [5].
methods could be categorised into three general Research has offered some advantages that can be
categories based on how the usability problems are gained if the usability of websites is considered or
identified: improved. Agarwal and Venkatesh [1] and Nielsen
[10] indicated that addressing the usability of sites
• User-based methods (user-testing methods): include could reduce the number of errors, enhance accuracy,
a set of methods that involves users in the process of and encourage positive attitudes toward the target
identifying usability problems. The aim of these interface. Furthermore, researchers indicated that
methods is to record users' performance (using addressing the usability of educational websites could
different types of observations) or satisfaction help students to enjoy the learning experience, increase
(using questionnaires and interviews) with the students’ confidence, and encourage students to use the
interface being tested. website [7].
• Evaluator-based methods: include methods that Despite the importance of making educational
involve evaluators in the process of identifying websites usable, few studies were found in the
usability problems. Heuristic evaluation is an literature that evaluated the usability of such sites,
example of a common usability method related to including Arabic websites [8, 15]. The studies that
this category. It involves having a number of were found stressed the importance of usability in the
evaluators assess the user interface and judge
180 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

design of educational websites and provided an outline content, ease of navigation, download speed,
of the design features that are important and that need customisation and personalisation, security, and
to be included in the design of educational websites [3, availability and accessibility. The investigation was
6-8, 12, 15]. However, these studies did not carried out by two web users only who evaluated a
investigate, and therefore consider, the relative total of 200 websites using the six design factors.
importance of the design features in the usability of These sites were selected from five different domains:
educational websites from the viewpoint of students. It portals and search engines, retail, entertainment, news
is worth mentioning, however, that research has been and information, and financial services (40 sites in
conducted which has investigated the relative each industry). Interestingly, the results showed that
importance of design features for the usability of the four design factors that influenced website usability
different types of website, such as: an e-commerce site were: information content, ease of navigation,
[13]; portals and search engines, retail, entertainment, download speed, and availability and accessibility.
news and information, and financial services [14]; However, the results showed that security and
online bookstores, automobile manufacturers, airlines customisation did not influence a website’s usability.
and car rental agencies [1]; financial, e-commerce, Agarwal and Venkatesh [1] also investigated the

T
entertainment, education, government, and medical relative importance of evaluation criteria in
[17] from the viewpoint of users. However, no research determining the usability of web sites for two types of
has been conducted specifically to investigate user (consumers and investors) across four industry
educational websites. The research described here aims sectors: online bookstores, automobile manufacturers,
to address the gap noted in the literature by evaluating airlines and car rental agencies. The criteria related to

e
the usability of nine Jordanian university websites. It the Microsoft Usability Guidelines (MUG), which
has used design criteria that were specifically includes five categories: content, ease of use,
developed for the purpose of this research after taking promotion, made-for-the-medium, and emotion. The

J
into consideration the relative importance of the results showed that content was the most important
developed criteria for the usability of educational category in all eight groups (four industries, two types
websites from the point view of students. of user). The second category of ease of use was
modestly moderately important across all eight groups.
2. Related Works Similarly, Zhang et al. [17] investigated user

A
perception regarding the relative importance of website
2.1. Relative Importance of Design Issues design features in six different website domains:

I
This subsection reviews studies that have investigated financial, e-commerce, entertainment, education,
certain design criteria and shed light on the relative government, and medical. The five most important
importance of design issues for different types of features were identified for each of the domains. The
website from the point of view of users. For example, results also showed that there were certain features
the study conducted by Pearson et al. [13] investigated equally important among different domains. For
the relative importance of five design criteria in the example, the results indicated that an ease of
evaluation of the usability of an e-commerce site from navigation feature was a must-have for all six domains,
the viewpoint of 178 web users. The objective of their while search tool was commonly ranked by the
research was to shed light on the criteria that influence following four domains as important: education,
successful web design, and to determine if gender has government, medical, and e-commerce. The results
an impact on the relative importance of these usability showed that education and medical domains required
criteria. The criteria related to navigation, download comprehensiveness of information which was not
speed, personalisation and customisation, ease of use, ranked within the list of the five most important
and accessibility. The results showed that these five features in the other four domains.
criteria were significant predictors of website usability Alternatively, Zhang and Dran [16] presented a two-
from the point of view of website users. Ease of use factor model that can be used to distinguish website
and navigation were the most important criteria in design factors into two types: namely, hygiene and
determining website usability, while personalisation motivator factors. Hygiene factors are those whose
and customisation were the least important. It was also presence makes a website functional, useful and
found that males and females viewed these web serviceable, and whose absence causes user
usability criteria differently. The two usability criteria, dissatisfaction (i.e. live/broken links). Motivator
navigation and ease of use, were found to have factors, however, are those whose presence will
significant differences based on gender. Females enhance users' satisfaction with the website and
placed greater emphasis on both of these web usability motivate them to return, while their absence will leave
criteria than did males. users feeling neutral, but not necessarily dissatisfied, as
Similarly, Tarafdar and Zhang [14] investigated the long as the fundamentals or hygiene factors are in
influence of six web design issues on the usability of place (i.e. the use of multimedia). Participants of the
websites using different criteria related to information study were asked to distinguish web site factors into
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 181

hygiene and motivator factors in the context of CNN’s problems that had not been traced in the website's
website. The clearly identified hygiene categories development phase and revealed users' lack of
include: technical aspects, navigation, and privacy and satisfaction with the website.
security, while the clearly identified motivator Furthermore, Toit and Bothma [15] investigated the
categories include: enjoyment, cognitive outcome, and usability of the website of an academic marketing
credibility. However, the results indicated that eighty- department in the University of South Africa using the
six percent of the participants believed that website heuristic evaluation method conducted by two expert
types do affect the way they judge hygiene or evaluators. The usability guidelines which were used in
motivator factors. For example, the participants the evaluation were adapted from an earlier research
specifically commented that they expected educational study [8] and consisted of five categories: content;
websites to have accurate, factual, nonbiased, and organisation and readability; navigation and links; user
richer materials. interface design; performance and effectiveness; and
educational information.
2.2. Evaluating the Usability of Educational However, Lencastre and Chaves [7] employed a
Websites questionnaire method aimed at students in the

T
evaluation of the usability of an educational website
This section summarises earlier research that evaluated used by Masters Degree students at Minho University,
the usability of educational websites using different Portugal. The evaluation was conducted by asking five
types of usability method. In fact, nearly all the studies students from the Masters course to reply to a
that evaluated the usability of educational websites questionnaire. This consisted of 49 questions divided

e
employed either user-based (i.e. user testing) or into seven categories: visual clarity, navigation,
evaluator-based (i.e. heuristic evaluation) usability content, control, feedback, errors, and consistency. The
evaluation methods. For example, Gonzalez et al. [3] questionnaire was designed to gather data about
evaluated the usability of academic websites in the

J
students' reactions to and perceptions of the
Spanish-Speaking Context of Use (SSCU) through the educational website.
heuristic evaluation and cognitive walkthrough The study conducted by Mustafa and Al-Zoua'bi [8]
methods. A specialised software tool was developed was similar to the study conducted by Lencastre and
based on heuristic evaluation techniques to support the Chaves [7] in the sense that they too employed a

A
usability evaluation of SSCU; this was used to evaluate questionnaire specifically to evaluate the usability of
the usability of 69 academic websites. The defined educational websites (the websites of nine Jordanian

I
heuristics consisted of twenty-five questions related to universities) which was also aimed at students (252).
four categories: design, content, navigation and search. However, Mustafa and Al-Zoua'bi [8] employed two
The evaluation team which carried out the usability online automatic tools (html toolbox and web page
evaluation comprised two usability experts and two analyze) to measure the internal attributes of the
advanced students with solid knowledge of heuristic websites which could not be perceived by users, The
evaluation. The results showed the feasibility of questionnaire was developed and designed based on 23
applying both the specialised software tool and the usability criteria divided into five categories: content,
particular cognitive walkthroughs while evaluating organisation and readability; navigation and links; user
academic websites. interface design; performance and effectiveness; and
Kostaras and Xenos [6] also employed the heuristic educational information. The results showed that the
evaluation method to evaluate the usability of the overall usability level of the studied websites was
website of the Hellenic Open University. The usability acceptable. However, there were some weaknesses in
assessment was conducted by five evaluators; two of some aspects of the design, interface and performance.
these were usability specialists while the other three The usability category content, organisation and
were experienced in heuristics evaluation. The readability exhibited the highest evaluation value,
heuristics used were the set of ten usability heuristics followed by the category of navigation and links; both
suggested by Nielsen et al. [9]. The results revealed were rated “good” according to the scale that was used.
that the heuristic evaluation method was an effective The other three categories (i.e. educational
and useful method which identified various usability information, user interface design, and performance
problems most of which were not previously detected. and effectiveness) were rated “moderate”. The results
Similarly, Papadopoulos and Xenos [12] evaluated obtained from the studies mentioned above proved the
the usability of the new version of the Hellenic Open usefulness of employing heuristic evaluation and user
University (HOU) website using heuristic evaluation testing methods in the evaluation of educational
by employing Nielsen's ten usability rules [9] and websites.
performance measurement (user testing). The The literature outlined above shows that research
evaluation was performed by experts and regular users which investigated the relative importance of design
(students of the HOU). The combination of the two issues or criteria in the evaluation of the usability of
evaluation methods identified several usability different types of website from the viewpoint of users,
182 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

did not specifically concern educational websites. The (weights) of the different categories and subcategories
literature considered above also shows that research of the developed usability criteria by asking students to
which evaluated the usability of educational websites distribute 100 points across the five major categories of
investigated the extent of the tested websites’ the criteria, and then to distribute the points assigned to
conformance to specific design issues. However, these each category across the corresponding subcategories
studies did not investigate the relative importance of (Appendix 1 presents two parts of the relative
the design criteria used in their evaluation from the importance survey; part 1 which shows the instructions
viewpoint of students before the evaluation. for assigning weights to the five main categories of the
developed criteria, and part 2 which shows the
3. Aims and Objectives instructions for assigning weights to the five
subcategories of the one of the main categories- the
The aim of this research is to investigate the relative navigation category). The second survey (the
importance of specific design criteria in the evaluation university ratings survey) was aimed at obtaining the
of the usability of educational websites from the ratings for every university website in terms of the
viewpoint of students and, based on students' various categories and subcategories of the developed
preferences, to evaluate the usability of nine Jordanian

T
usability criteria by asking students to provide ratings
university websites. for each website included in this study.
The specific objectives for the research are:
• To develop evaluation criteria for assessing the 4.2. Selection of the Websites

e
usability of educational websites. In order to select nine Jordanian educational websites,
• To ask students to judge the relative importance one of the major international university ranking
(weights) of the different categories of the websites was used; this was the 4 International
developed usability criteria. Colleges and Universities (4ICU.org). Universities and

J
• To determine if gender has an impact on the relative colleges worldwide are ranked by 4ICU by the
importance of the developed usability criteria. popularity of their websites. The list of Jordanian
• To determine if students’ majors/specialisations universities sorted by their web popularity, as provided
have an impact on the relative importance of the by 4ICU for the year of 2011, was used to select the
developed usability criteria.

A
sample for this research. Nine of these websites which
• To ask students to provide ratings for nine Jordanian had the highest ranking were then picked out, as shown
university websites on the developed usability in Table 1. This number was chosen to keep the

I
criteria and their categories. research at a manageable size for the students and
• To use the weights and ratings together to assess the researcher.
overall usability of each Jordanian university
Table 1. The Jordanian universities included in the research.
website.
University
University University Website
Symbol
4. Methodology 1
The University
www.ju.edu.jo U1
of Jordan
4.1. Research Instruments 2 Petra University www.uop.edu.jo U2
Jordan
Usability criteria for assessing the usability of 3
University of
www.just.edu.jo U3
Science and
educational websites were developed based on an Technology
extensive review of the literature [1, 3, 6-8, 10, 12-15, Al Balqa
17]. Section 5 presents the categories and Applied
4 University of newar.bau.edu.jo U4
subcategories of the criteria. Science and
In order to collect information regarding the Technology
characteristics (demographic information) of students 5
Mutah
www.mutah.edu.jo/ U5
University
who participated in the research, a pre-test survey was Al-albayt
developed. In order to achieve objectives 2, 5 and 6 6 www.aabu.edu.jo/ U6
University
(i.e., obtaining the relative importance (weights) of the 7
The Hashemite
www.hu.edu.jo U7
University
different categories of the developed usability criteria; Yarmouk
obtaining the rates for the selected Jordanian university 8 www.yu.edu.jo/ U8
University
websites; and using the weights and ratings to calculate 9
Philadelphia www.philadelphia.e
U9
University du.jo/
the overall usability of each Jordanian website),
Agarwal and Venkatesh’s [1] method for the
assessment of usability that includes weights and 4.3. Participants/Sample
ratings was adopted; based on this, two surveys were The participants in this study were undergraduate
developed. The first survey (the relative importance students enrolled on twelve classes related to two
survey) aimed to collect the relative importance faculties (the Faculty of Information Technology and
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 183

Science, and the faculty of Economics and number of surveys that needed to be filled in; and
Administrative Sciences) at one of the universities in students’ right to withdraw from the session at any
Jordan. Six classes were selected from each faculty. time. The students were then asked to fill in the pre-
The total number of students was 247; the number of test questionnaire in order to obtain information
males was 155, while the number of females was 92 regarding their background and experience. Then, the
(Table 2). 237 provided usable responses. Unusable students were asked to provide their perceptions of the
responses, which were ten, were primarily related to relative importance (weights) of the developed
incomplete information. In cases where some students usability criteria (5 categories) using the relative
were enrolled onto more than one of the classes importance survey (Appendix 1). Following this,
included in the sample, they were asked to leave the students were asked to distribute the points across the
session and to participate only once. Demographic various subcategories. Before filling out the surveys
information concerning the students is shown in Table related to the evaluation of three university websites
2. The students in each class carried out the procedure (i.e. three university rating surveys), the students were
(Subsection 4.5) on three of the websites included in asked to explore the website included in the first of the
the sample. Each website was evaluated by four classes three surveys for a maximum of 10 minutes. Thus, one

T
(two classes from each faculty). survey, the university ratings survey, was used to
evaluate each of the three websites. After the
Table 2. Demographic information of the research participants.
exploration, the students were asked to fill in the
Faculty survey for a particular website from the three test
Economics
websites. This related to rating the website in terms of

e
Science and
and
Information Total
Technology
Administrative its compliance to the different categories of the
Sciences developed usability criteria. The ratings were based on
Male 82 67 149
Sex a seven-point rating scale (Likert scale). A similar
Female 35 53 88

J
< one year 0 4 4 procedure was followed by the students while testing
Computer From one to
4 13 17
the second and third websites. The order of the three
Experience three years websites (i.e. the three university ratings surveys)
> three years 113 103 216
< one year 4 11 15 which were evaluated in each class/session was
Internet From one to changed so that each website was tested fairly by all

A
20 40 60
Experience three years the students. The average time spent in conducting the
> three years 92 70 162
Daily 101 84 185 session was one hour. All the evaluations were carried

I
Frequently Weekly 13 24 37 out in two weeks (during May 2011) to take into
Use of Monthly 3 6 9 consideration the possibility that the sites included in
Internet By semester 0 4 4
Yearly 0 2 2
the sample might change over time. The students were
instructed to leave blank any criterion they were unsure
4.4. Pilot of.

A pilot study was conducted before the 4.6. Analysis

experiment/main test to test the method of assigning
weights and ratings. Before conducting the pilot study, The data collected were analysed in several ways.
the surveys were translated into Arabic. The surveys Descriptive analysis was used to analyse the data
were pilot tested using ten Jordanian undergraduate collected from the pre-rest questionnaire to describe
students using the Arabic language version. The pilot the characteristics of the students. In order to find the
study identified some ambiguity in the surveys. Results relative importance (weight) for the developed criteria
from the pilot test were taken into consideration and (the five categories and their corresponding
minor changes were made to the surveys. subcategories) from the viewpoint of students, the
average weight (relative importance) was calculated.
4.5. Procedure Descriptive analysis (the mean and standard deviation)
of the weights (i.e. the relative importance) of the
All data collection sessions followed the same developed criteria based on gender and faculty
procedure. Data were gathered using five surveys in a (major/specialisation) was carried out. To determine if
university in Jordan where all students had access to there was a statistically significant difference in the
the Internet. It is worth mentioning that the university relative importance of the web usability criteria based
from which the sample of students was selected was on gender and faculty, the one-way analysis of
not included in the sample of the universities selected variance (ANOVA) was used for each category and the
and evaluated in this research. corresponding subcategories of the developed usability
The session began with the researcher welcoming criteria.
the students and explaining the objectives of the study; In order to find out the ratings of the nine Jordanian
the number of websites that would be evaluated; the university websites with regard to their conformance
184 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

with the subcategories of the developed usability • Architecture/organisation: This criterion relates to
criteria, Likert scores were calculated for each the structure of a site's information which should be
statement in the survey (the university ratings survey) divided into logical, clear groups; each group should
for each website. A Likert score of 1-3 was regarded as include related information.
a negative response, 5-7 a positive response, and 4 a Architecture/organisation consists of three
neutral one. Negative statements identified a number of subcategories. These are: Logical structure of site:
usability problems within the sites. The usability scores The structure of the site is simple and
of the nine Jordanian university websites with regard to straightforward; related information is grouped
each subcategory related to the five categories of the together; not deep architecture: Architecture is not
developed usability criteria were calculated by too deep so that the number of clicks to reach goals
multiplying the weight (the average of the is not too large, e.g. it does not require clicking
subcategory) by the ratings of the site (the Likert more than 3 links; and simple navigation menu: The
score). Then, the usability scores of the related navigation menu is simple and straightforward.
subcategories were added up for each website to • Ease of use and communication: This relates to
produce the usability scores for each site with regard to the cognitive effort required to use a website [1],

T
the five major categories of the developed usability and to the existence of basic information which
criteria. The usability score of the five categories facilitates communications with the university in
related to each website were then added up to produce different ways. Research has found that ease of use
an overall usability score for each website. is an important factor/issue in determining web
Qualitative data obtained from students’ responses

e
usability [1, 13, 14, 17]. Ease of use and
to the open-ended questions in the surveys (the communication comprises four subcategories. These
university ratings surveys) were taken into account in are: Quick downloading of web pages: The
determining the usability of the tested websites (i.e. download time of the pages is appropriate; easy

J
features on the site students liked and other features interaction with a website: Interaction with the
they disliked). Students’ answers were translated into website is easy for different groups of users, e.g.
English from Arabic. Several usability problems were navigating through the site's pages is easy; returning
identified from the answers of students, as well as to the home page from any page is easy; finding
some good features. information is easy; contact us information: Useful

A
information to enable easy communication with the
5. Criteria for Evaluating the Usability of university is displayed, e.g. contact us (e.g. name,

I
Educational Websites physical address, telephone number, fax number,
email details); and foreign language support: The
Specific criteria for evaluating the usability of site’s content is displayed in different languages.
educational websites were developed based on a • Design: This relates to the visual attractiveness of a
literature review. The developed criteria consisted of site's design; the appropriate design of a site's pages,
five main categories. This section presents the and the appropriate use of images, fonts and colours
categories and their corresponding subcategories: in the design of a site. Design comprises six
• Navigation: This assesses whether a site includes subcategories: Aesthetic design: The site is
the main tools (i.e. navigation menu, internal search attractive and appealing so that it impresses the
facility) and links which facilitate the navigation of potential customer; appropriate use of images: The
users through a site, enabling them to reach the quality of images is adequate, there are no broken
required information quickly. Research showed that images, images make a contribution to the
navigation was one of the design factors that understanding and navigation of the site, image size
influenced website usability [13, 14, 17]. Navigation is relevant so that it has minimal effect on loading
comprised five subcategories. These were: time; appropriate choice of fonts: Font types are
navigation support: Navigational links are obvious appropriate and easy to read; appropriate choice of
in each page so that users can explore and find their colours: Choice of colours for both fonts and
way around the site and navigate easily; effective background is appropriate, the combination of
internal search: Internal search is effective: e.g. it is background and font colours is appropriate;
fast, accurate and provides useful, concise and clear appropriate page design: Pages are uncluttered,
results which are easy to interpret; working links: page margins are sufficient, the page title is
Links are discernible, working properly and not appropriate; and consistency: Page layout or style is
misleading so that the user knows what to expect consistent throughout the website: e.g. justification
from the destination page; no broken links: The site of text, font types, font sizes, colours, and position
has no broken links; and no orphan pages: The site of the navigation menu in each page.
has no dead-end pages. • Content: This assesses whether a site includes the
information users require. Research stresses the
importance of this factor and shows that it is one of
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 185

the most important factors that influence web It is worth mentioning that the students considered
usability [1, 14]. Content consists of seven information about departments to be more important
subcategories. These are: Up-to-date information: than information about colleges and the university, as
The information is up-to-date, current and often they gave it a higher weight (3.01) compared to the
updated; relevant information: The information is other two subcategories (2.51 and 2.79, respectively)
sufficient and relevant to user needs, e.g. content is (Table 3).
concise and non-repetitive, terminology/terms are
Table 3. The relative importance (weights) for the categories and
clear and unambiguous; no under-construction subcategories of the developed usability criteria and the total
pages: There are no ‘under construction’ pages; weight for each category.
accurate information: The information is accurate; Total
information about the university: Basic facts about Weights
Categories Subcategories Weight
the university are displayed, e.g. university for each
Category
overview, higher management, academic calendar,
Navigation Support 5.11
registration, description, photographs, etc.; Effective Internal
5.01 20.75
information about the colleges: Adequate Navigation Search Tool

T
information about the colleges is displayed, e.g. Working Links 4.49
No Broken Links 2.96
overview, department, specialisations, etc.; and No Orphan Pages 3.17
information about the departments: Adequate Logical Structure of a
7.16
information about the departments is displayed, e.g. Site
18.66
Organisation/ Not Deep
overview, academic staff, outlines, course

e
5.73
Architecture Architecture
descriptions, study plans, specialisations, etc. Simple Navigation
5.77
Menu
Quick Downloading
6. Results of Web pages
6.20

J
Easy Interaction with
6.1. Relative Importance of the Developed Ease of Use and a Website
5.38 19.88

Usability Criteria Communications Contact Us

4.43
Information
The results showed that the most important design Foreign Language
3.86
Support
category for the usability of educational websites from

A
Aesthetic Design 4.27
the viewpoint of users was the content as it has the Appropriate Use of
3.16
highest weight (as shown in Table 3). The results also Images

I
Appropriate Use of
showed that the navigation was the second most Fonts
2.57 19.16
Design
important category for the usability of educational Appropriate Choice of
2.74
websites. The results showed that ease of use, and Colours
Appropriate Page
communications and design were the third and fourth Design
3.35
important categories respectively in the usability of Consistency 3.06
educational websites from the viewpoint of students. Up-to-date
4.74
Information
Finally, the results showed that the Relevant Information 3.23
organisation/architecture was the least important No Under
2.07
category for the usability of educational websites from Construction Pages
Accurate Information 3.20 21.56
the viewpoint of students. Content
Information about the
2.79
Interestingly, the results showed that the weights of University
the subcategories varied; the highest weight assigned Information about
2.51
Colleges
to a single subcategory was 7.16, while the lowest Information about
3.01
weight was 2.07. The subcategories with the highest Departments
weights, which represented the design features that Total Weights 100
students preferred the most for a usable educational
website, included: logical structure of a site (7.16), The ANOVA test revealed no statistically
quick downloading of web pages (6.20), simple significant differences between males and females
navigation menu (5.77), not deep architecture (5.73) regarding the relative importance of four categories of
and easy interaction with a website (5.38). However, the criteria: navigation, organisation/architecture, ease
the design features which were the least important of use and communication, and design (Appendix 2).
from the viewpoint of users were: no under However, the ANOVA test showed that there was a
construction pages (2.07), information about colleges statistically significant difference between males and
(2.51), appropriate use of fonts (2.57), appropriate females regarding the relative importance of the
choice of colours (2.74), and information about the content category (Appendix 2). The females
university (2.79) (Table 3). considered this category as the most important and
gave it therefore the highest weight (23.58), while the
males considered this category as the second most
186 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

important category and therefore gave it the weight of Examples of the aspects students liked in this website
20.37. The descending order of the usability categories were: the attractive design of the site; the content of
based on their relative importance according to males this website; the fact that it was easy to navigate and
was: navigation, content, ease of use and interact with the site; and the electronic registration
communication, design, and organisation/architecture. provided by the site.
However, the descending order of the usability
categories based on their relative importance according 6.2.3. Jordan University of Science and Technology
to females was: content, navigation, ease of use and This website was ranked as the fifth website in order of
communication, organisation/architecture, and design. preference among the other eight websites and its
The ANOVA test showed that there were no overall usability score was 886.84. The students,
statistically significant differences between the through the answers of the open-ended questions,
students of the two faculties (the Faculty of Science defined some preferable aspects on the website (i.e. the
and Information Technology, and the Faculty of content of the site; the ease of use, navigation and
Economics and Administrative Sciences) concerning interaction with the site; and the electronic registration
the relative importance of the five categories of the feature), and some issues they did not like (i.e. the
criteria.

T
unattractive design of the site; inappropriate and
The descending order of the categories based on inconsistent colours and fonts used on the site). Most
their relative importance according to the students of students revealed that they were unhappy with the
the Faculty of Science and Information Technology website since it does not support Arabic.
was: navigation, content, ease of use and

e
communication, design, and organisation/architecture. 6.2.4. Al Balqa Applied University of Science and
However, the descending order of the categories based Technology
on their relative importance according to the students

J
of the Faculty of Economics and Administrative The overall usability score of this website was 877.32
Sciences was: content, navigation, ease of use and and it was ranked as the seventh preferred website
communication, organisation/architecture, and design. according to students' viewpoint. The qualitative data
6.2 The Usability of the Jordanian University obtained from the students showed their satisfaction
with some features of the site such as: the content of
Websites

A
the university website; it was easy to use, and the
This subsection presents the results obtained from the
electronic registration. However, the students
analysis of the quantitative and qualitative data

I
expressed their dissatisfaction with some issues on the
obtained from the surveys regarding the usability of the
website such as: the overall appearance of the site; and
nine Jordanian university websites. Details of the
the small number of images used on the site.
results obtained are available from the author; the
results from each website are summarised below. 6.2.5. Mutah University
6.2.1. The University of Jordan This website was ranked as the fourth preferred
website from the viewpoint of students; its overall
This website was ranked as the third website in terms
usability score was 897.05. The qualitative data
of preference among the other eight websites
obtained from the students stressed the fact that the
(Appendix 3 and Table 4). The overall usability score
students were satisfied with the content of the site; the
of this website was 910.69. The students, through the
quick downloading of the site's pages, ease of
qualitative data obtained from the open-ended
navigation and ease of use; and the availability of an
questions, provided more information regarding issues
electronic registration system. Regarding the issues
they liked (i.e. the content of the website; the ease of
with which students were dissatisfied, these included:
navigation and interaction with the site; and the interest
poor and unaesthetic design of the site; inappropriate
in graduate students where pages were allocated for
choice of colours and fonts; and inconsistent colours.
them to keep in touch with each other) and disliked
(i.e. the design of the site; the large amount of 6.2.6. Al-albayt University
information presented on the home page with a lack of
order) regarding the design of the site. This website was the least preferred website among the
other eight websites; it had the lowest overall usability
6.2.2. Petra University score compared to the other eight websites (Appendix
3 and Table 4). The qualitative data obtained from the
This website was ranked as the first preferred website
open-ended questions proved that the students were
from the viewpoint of the students and had the highest
dissatisfied with this website. The students provided
overall usability score: 962.64 (Appendix 3 and Table
information regarding the issues that made them
4). The students, via the answers of the open-ended
dissatisfied with the site such as: the inappropriate,
questions, provided useful information regarding issues
unaesthetic and unattractive design; complex structure;
they liked and others they disliked on this website.
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 187

irrelevant content; inconsistency (colours and page 6.3. Common and Unique Design Issues
design); and slow downloading of web pages.
This subsection summarises common and unique
6.2.7. The Hashemite University strengths and weaknesses related to design issues in the
investigated websites.
The overall usability score of this website was 873.99;
it was ranked as the eighth preferred website according 6.3.1. Common Strengths
to the students' preferences. The qualitative data
obtained from the open-ended questions shed light on The results showed that the investigated websites were
the features of the site which the students were strong in the following design areas:
satisfied with, such as: the content of the site • Content: Nearly all the investigated websites (eight
(especially the academic staff contact information); out of the nine) had satisfactory content from the
and quick downloading of the web pages. However, viewpoint of the students. Specifically, the students
the students indicated there were issues which made liked the content of the sites as it was: clear,
them dissatisfied with the site; these mainly related to: accurate, detailed and relevant, and related to the
the design of the site; and the fact that information information about the university, colleges and

T
about higher education was not provided in the Arabic departments (presented by all universities). The
language. students also liked the information about the
academic staff as presented by The Hashemite
6.2.8. Yarmouk University University, Yarmouk University and Philadelphia

e
This website was ranked as the second preferred University.
website from the viewpoint of students, and its overall • Ease of use/navigation/interaction with the site: The
usability score was 937.58 (Appendix 3 and Table 4). students indicated that almost all the sites (eight out
of the nine) had an interface that was easy to use/

J
The qualitative data obtained from the open-end
questions showed that the students were satisfied with: navigate and interact with.
the design of the site; content of the site; easy to • Quick downloading: The students indicated that
navigate and interact with the site; electronic learning; they liked the quick downloading of the web pages
and the well-designed library website. However, the related to six out of the eight of the investigated

A
students mentioned some issues with the website websites.
which they did not like, such as: its deep architecture; • Arabic language support: The students liked the

I
and inconsistency between Arabic and English support of most of the investigated websites for the
interfaces. Arabic language (eight out of the nine).

6.2.9. Philadelphia University 6.3.2. Common Weaknesses

This website was ranked as the six preferred website The results showed that the investigated websites
from the students' viewpoint with its overall usability shared the following common weaknesses:
score being 880.28. The students, via their answers to
• Design: The students indicated that they did not like
the open-end questions, listed the areas they liked in
the design of nearly all the investigated websites
this website, such as: the content of the website; the
(seven out of the nine). The students did not like the
availability of contact us information (email addresses)
colours, fonts and images used throughout the sites.
of the academic staff; ease of interacting with the site;
• Consistency: The students disliked, on most of the
electronic learning; and electronic registration.
investigated websites, the fact that the websites
However, the students did not like some features on the
were inconsistent in: colours, fonts, the Arabic and
site, such as: its design; inappropriate use of colours;
English language interfaces, and the design of the
fonts; images; and inconsistency.
pages.
Table 4. The descending order of the nine Jordanian university
websites with regard to their overall usability scores. 6.3.3. Unique Strengths
No. Jordanian University Overall Usability Score
The results showed that the students made some
1 Petra University (U2) 962.64
2 Yarmouk University (U8) 937.58 comments regarding design factors which they liked
3 The University of Jordan (U1) 910.69 that were presented on some of the websites. These
4 Mutah University (U5) 897.05 included :
Jordan University of Science and 1. The availability of a graduate students’ section, as
5 886.84
Technology (U3) featured on the websites of the University of Jordan
6 Philadelphia University (U9) 880.28
and Yarmouk University.
Al Balqa Applied University of
7
Science and Technology (U4)
877.32 2. The connection to Facebook, and/or YouTube,
8 The Hashemite University (U7) 873.99 and/or Twitter, as provided on the websites of the
9 Al-albayt University (U6) 749.51
188 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

University of Jordan, Mutah University and There was agreement between the results obtained
Philadelphia University. by this research and earlier research regarding the
3. The support of electronic registration, as provided importance of the ease of use/ease of navigation design
by the websites of Petra University, Jordan category while designing/ evaluating the usability of
University of Science and Technology, Mutah websites. The results of this research revealed that ease
University, Yarmouk University and Philadelphia of use was the third most important design category
University. which influences the usability of educational websites
from the viewpoint of students. Other research which
6.3.4. Unique Weaknesses investigated this category across different types of
The results showed that the students raised some issues website (e-commerce, portals and search engines,
which they disliked regarding design features that were entertainment, news and information, financial
present on some websites. These included: services, financial, entertainment, government, and
medical) [1, 13, 14, 17] also stressed the importance of
• The fact that one website, Jordan University of this category. For example, Zhang et al. [17] found that
Science and Technology, did not support the Arabic ease of use was a must-have feature for all six of the
language. Also, the lack of support for the Arabic

T
domains they investigated. This stressed the
language in the higher education information by The importance of the ease of use category when designing
Hashemite University. a usable website and/or evaluating the usability of
different types of website.
7. Discussion The results of this research showed that the least

e
important category that influenced the usability of
7.1. Relative Importance of Design Features educational websites from the viewpoint of students
As indicated in Section 2, earlier research which was the organisation/architecture of the site. The

J
investigated users’ preferences on the relative students also rated the design category as the fourth
importance of website design features, did not most important category that influences the usability of
specifically concern educational/academic websites. educational websites. These results, together with the
However, this research has addressed the gap noted in previous ones, shed light on the design categories and
the literature and focused primarily on investigating subcategories that must be taken into consideration

A
the relative importance of website design features on when designing and/or evaluating the usability of
the usability of educational websites from the educational websites, as well as the design categories

I
viewpoint of students. and subcategories which should have less focus when
The results of this research revealed that the content designing and/or evaluating the usability of such
category was the most important category that websites (Table 3).
influenced the usability of educational websites from The results of this research were comparable with
the point of view of students. This is in agreement with other research [13] regarding the rating of the design
the results obtained from earlier research [1, 14, 16, categories of the suggested criteria differently by males
17]. This result stressed the importance of the content and females. However, there was some inconsistency
design category, not only not only in the domain of e- between the results of this research and the results
commerce websites, as shown by [1, 17] and other obtained by Pearson et al. [13] regarding the types of
domains (financial, entertainment, government, and category which were significantly different based on
medical) [17], but also in the educational website gender. The results of this research showed that
domain. content was the only category which showed
The results of this research showed that navigation significant differences based on gender, since females
was the second most important category in terms of the placed a greater emphasis on this than did males, while
usability of educational websites from the point of the results of Pearson et al. [13] showed that the
view of students. This was in agreement with the navigation and ease of use categories had significant
results obtained by Person et al. [13] and Zhang et al. differences based on gender, where females placed
[17]. Zhang et al. [17] found that search tools were greater emphasis on them than did males. The
ranked important in four domains (education, differences between the results might relate to the fact
government, medical, and e-commerce); the search that the research conducted by Person et al. [13]
tools constituted one of the subcategories of the concerned e-commerce websites, while this research
navigation category suggested and used in this considered educational websites. This suggests that
research. This stressed the importance of considering universities and/or academic institutions which are
navigational issues when designing educational especially for females should give the content category
websites, as well as e-commerce, education, first priority when designing usable educational
government, medical websites as shown by earlier websites, or when evaluating the usability of their
research [13, 17]. websites. However, universities and/or academic
institutions for males should give the navigation
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 189

category the first priority. Furthermore, universities from the viewpoint of students. Furthermore, Section
and/or academic institutions could take into 6.3 shows, based on the qualitative data obtained from
consideration the order of the design categories, from the open-ended questions, that the common strengths
the first to the least important from the viewpoint of of the tested websites include the content, while the
students, which was different based on gender, as common weaknesses of the tested websites include the
discussed in Section 6.1. design category
This research, unlike earlier research, also However, the study of Mustafa and Al-Zou’bi [8]
investigated whether the relative importance of the did not report the specific strengths and weaknesses of
design categories of the suggested criteria differed each Jordanian university website; it reported the
from the viewpoint of students based on the differences overall results of all the investigated websites with
in their major/specialisation. The results, as discussed regard to five categories without explaining the
in Section 6.1, showed that all the design categories usability level of each website. Furthermore, Mustafa
did not have statistical significant differences based on and Al-Zou’bi [8] employed only a quantitative
faculty (major/ specialisation). However, the order of method. However, unlike the earlier research [8], this
the design categories, from the first to the least research highlighted the specific weaknesses and

T
important from the point of view of students, was strengths of each Jordanian university website with
different based on faculty, as discussed in Section 6.1. regard to their level of conformance with five design
This provides evidence for universities and/or categories related to the suggested criteria and their
academic institutions to consider the preferences of corresponding subcategories; this provides useful
design categories from the viewpoint of students based results for each university.

e
on their specialisation/major. For example, websites of Furthermore, unlike earlier research [8], this
academic institutions specialising in scientific faculties research not only depends on quantitative data, but also
could consider content and ease of use as the most employed a qualitative method through the use of

J
important design categories when evaluating the open-ended questions. This provided additional
usability of their websites or designing usable important evidence regarding features that students
websites. However, specialist websites for liked and disliked in the investigated sites. Section 6.3
administrative faculties could consider navigation and summarised common and unique design features that
content as the most important categories when students liked about the design of the tested websites,

A
designing and/or evaluating the usability of their and common and unique weaknesses that students
websites. disliked concerning the design of the tested websites.

I
This provides useful information for universities and/or
7.2. Usability Evaluation of Jordanian academic institutions while designing their websites.
Educational Websites
The results of this research agreed with the results 8. Conclusions
obtained by the research conducted by Mostafa and Al- This research provides empirical evidence for
Zou’bi [8] regarding the fact that the majority of the academic institutions and universities regarding the
students were satisfied with the usability of the relative importance of specific design features on
Jordanian university websites that were investigated which to focus when designing and/or evaluating the
(seven of the websites included in the sample of usability of their educational websites. The results
Mustafa and Al-Zou’bi’s study [8] were included in the showed that content and navigation were the most and
sample of this research). This indicated that the second most important design categories respectively
majority of the students were satisfied with most for the usability of educational websites from the
features relating to the usability of the websites that viewpoint of students. The results also showed that the
were evaluated. third, fourth and least important categories for
Also, there was agreement between the results of the educational websites were: ease of use and
earlier research [8] and this research regarding the communications; design; and organisation/architecture,
design factors (categories) the students believed that respectively.
the tested websites conformed to. Earlier research This research also investigated whether gender and
showed that the two design factors (categories) to major/specialisation had an impact on the relative
which the tested websites conformed were: content, importance of the developed usability criteria. The
organisation and readability, and navigation and links. results showed that there was a statistically significant
The two categories in which the tested websites difference between males and females regarding only
conformed less, however, related to the design, one category: the content. Females considered it as the
performance and effectiveness categories. The results most important category while males considered it as
of this research also showed that Jordanian university the second most important category. By contrast, the
websites conformed to the content and navigation results showed that there were no statistically
categories, but did not conform to the design category significant differences between the students of the two
190 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

selected faculties concerning the relative importance of References

the developed criteria based on majors/specialisations.
[1] Agarwal R. and Venkatesh V., "Assessing a
This research employed students to evaluate the
Firm’s Web Presence: A Heuristic Evaluation
usability of nine Jordanian university websites using
Procedure for the Measurement of Usability",
specific criteria and then, based on the relative
Information Systems Research, vol. 13, no. 2, pp.
importance of the developed criteria, the overall
168–186, 2002.
usability score for each Jordanian university website
[2] Anonymous, Step-by-Step Usability Guide,
was calculated. Students were also asked to provide
2006, <http://www.usability.gov> [accessed
qualitative data regarding issues they liked and disliked
20.09.2011].
on the tested websites. The results provided detailed
[3] Gonzalez M., Granollers T., and Pascual A.,
information for each tested website regarding their
"Testing Website Usability in Spanish-Speaking
conformance with the developed usability criteria. In
Academia through Heuristic Evaluation and
general, the results showed that the majority of the
Cognitive Walkthrough", Journal of Universal
students were satisfied with the usability of the
Computer Sciences, vol. 14, no. 9, pp. 1513-
Jordanian university websites. Specifically, the results
1528, 2008.
showed the students were satisfied with the content and

T
[4] Gray W. and Salzman C., "Damaged
navigation (ease of use) aspects of the tested websites,
Merchandise? A Review of Experiments that
and were dissatisfied with the design of the websites.
Compare Usability Evaluation Methods",
The results also showed that Petra University had the
Human-Computer Interaction, vol. 13, pp. 203-
highest overall usability score while Al-bayt University

e
261, 1998.
had the lowest. The results provided by this research
[5] Kantner L. and Rosenbaum S., "Usability Studies
offered evidence for the owners of the tested Jordanian
of WWW sites: Heuristic Evaluation vs.
university websites regarding the strong and weak
Laboratory Testing", in the Proceedings of ACM

J
features of their websites.
15th International Conference on Systems
The developed usability criteria, which are specific
Documentation, Salt Lake City, UT, USA, pp.
for the evaluation of educational websites, provide
153-160, 1997.
guidance for the designers and evaluators of such
[6] Kostaras N. and Xenos M., "Assessing
websites regarding website features that should be
Educational Web-site Usability using Heuristic

A
taken into consideration while designing and/or
Evaluation Rules", in the Proceedings of 11th
evaluating educational websites.
Panhellenic Conference in Informatics, Corfu,

I
This research, however, has four limitations, which
Greece, pp. 197-201, 2006.
could influence the results obtained. The first is related
[7] Lencastre J. and Chaves J., "A Usability
to the participants/sample used in this research. It was
Evaluation of Educational Websites", in the
limited to students, while other stakeholders of
Proceedings of EADTU Conference, France,
educational websites (i.e. faculty staff, parents) were
2008.
not taken into consideration.
[8] Mustafa S. and Al-Zoua'bi L., "Usability of the
The second limitation is related to the fact that this
Academic Websites of Jordan's Universities", in
research employed only one user testing method,
the Proceedings of the International Arab
which was questionnaire, in the process of evaluating
Conference on Information Technology, Tunisia,
the usability of the Jordanian university websites.
pp. 2-9, 2008.
Other usability methods, such as the heuristic
[9] Nielsen J., "Heuristic Evaluation". In J. Nielsen
evaluation, which involves evaluators/experts in the
and R. L. Mack (Eds.); Usability Inspection
process of identifying usability problems, were not
Methods, John Wiley & Sons, New York, pp. 25-
employed.
64, 1994.
The third limitation is concerned with the selection
[10] Nielsen J., Designing Web Usability: The
of only nine Jordanian university websites to conduct
Practice of Simplicity, New Riders Publishing,
this research. The evaluation of all the Jordanian
2000.
university websites was not undertaken
[11] Nielsen J., Usability 101: Introduction to
The fourth limitation is related to the developed criteria
usability. Useit.com, 2003.
that were used for evaluating the usability of the
<http://www.useit.com/alertbox/20030825.html>
websites. Some issues were not considered while
, [accessed 14.02.2006].
developing the criteria, which might influence the
[12] Papadopoulos T. and Xenox M., "Quality
results, such as: number of clicks
Evaluation of Educational Websites Using
Heuristic and Laboratory Methods", in the
Acknowledgment Proceedings 2nd Panhellenic Scientific Student
This research was funded by the Deanship of Research Conference on Informatics, Related
and Graduate Studies in Zarqa University /Jordan.
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 191

Technologies, and Applications, Samos, Greece, Layla Hasan holds a BSc degree in
pp. 43-54, 2008. computer science from the
[13] Pearson J., Pearson A., and Green D., University of Jordan, Jordan in
"Determining the Importance of Key Criteria in 1996; an MBA degree in business
Web Usability", Management Research News, administration from the University
vol. 30, no. 11, pp. 816-828, 2007. of Jordan, Jordan in 2004; and a
[14] Tarafdar M. and Zhang J., "Analyzing the PhD degree in computer science
Influence of Website Design Parameters on from the Loughborough University,
Website Usability", Information Resources UK in 2009. Her research interests include web-based
Management Journal, vol. 18, no. 4, pp. 62 - 80, human computer interaction, website usability
2005. methods, web analytics, and website quality. She is
[15] Toit M. and Bothma C., "Evaluating the currently an assistant professor in the Department of
Usability of an Academic Marketing Computer Information Systems at Zarqa University,
Department’s Website from a Marketing Jordan. She was the Editorial Secretary of the
Student’s Perspective", International Retail and International Arab Journal of Information Technology

T
Marketing Review, vol. 5, no. 1, pp. 15-24, 2010. (IAJIT) and the Director of the General Secretariat of
[16] Zhang P. and von Dran G., "Satisfiers and Colleges of Computing and Information Society (2011-
Dissatisfiers: A Two-Factor Model for Website 2012). She also worked as Director of the General
Design and Evaluation", Journal of the American Secretariat of the International Arab Conference on
Society for Information Science, vol. 51, no. 14, Information Technology (ACIT) and the Editorial

e
pp. 1253-1268, 2000. Secretary of the International Arab Journal of
[17] Zhang P., von Dran G., Blake P., and Information Technology (IAJIT) before obtaining her
Pipithsuksunt V., "A Comparison of the Most PhD (2005-2006). Furthermore, she worked as Head of

J
Important Website Features in Different the Information Center at the Association of Arab
Domains: An Empirical Study of User Universities before joining Zarqa University. She had
Perceptions", in the Proceedings of Americas more than 17 publications in international conferences
Conference on Information Systems and journals, and she is a reviewer for many
(AMCIS'2000), Long Beach, CA. August 10-13, international conferences and journals specialized in

A
pp. 1367-1372, 2000. computer science and information technology.

I
192 International Arab Journal of e-Technology, Vol. 3, No. 3, January 2014

Appendix 1: The relative importance survey

Part 1: Instructions for assigning weights to the five main categories of the developed criteria.
Please distribute 100 points, which represent weights, across the five major categories presented in the Table below based on the importance
of these categories in the evaluation of the usability of any educational website from your perspective. For example, if navigation is the most
important category then give it a higher weight.

No. Criteria Explanation Weight

This assesses whether a site includes the main tools (i.e. navigation menu, internal search facility) and links
1 Navigation
which facilitate the navigation of users through a site, enabling them to reach the required information quickly.
Architecture/ This criterion relates to the structure of a site's information which should be divided into logical, clear groups;
2
organization each group should include related information.
Ease of use and This relates to the existence of basic information which facilitates communications with the university in
3
communication different ways.
This relates to the visual attractiveness of a site's design; the appropriate design of a site's pages, and the
4 Design
appropriate use of images, fonts and colours in the design of a site.
5 Content This assesses whether a site includes the information users require.

T
Total weights

Part 2: Instructions for assigning weights to the five subcategories of the navigation category.

e
Please distribute the ---- points you have assigned to the navigation category (Part 1) across its five corresponding subcategories based on the
importance of these subcategories in the evaluation of the usability of any educational website from your perspective. For example if the
navigation support is the most important subcategory then give it a higher weight.

J
No. Criteria Explanation Weight
Navigational links are obvious in each page so that users can explore
1 Navigation support
and find their way around the site and navigate easily
Internal search is effective: e.g. it is fast, accurate and provides
2 Effective internal search
useful, concise and clear results which are easy to interpret
Links are discernible, working properly and not misleading so that

A
3 Working links
the user knows what to expect from the destination page
4 No broken links The site has no broken links

I
5 No orphan pages The site has no dead-end pages

Total weights
Evaluating the Usability of Educational Websites Based on Students' Preferences of Design Characteristics 193

Appendix 2: Descriptive and ANOVA results which show the impact of gender on the
relative importance of the categories of the developed usability criteria.
Descriptive Results ANOVA Results
Sum of Mean
Category N Mean St. Deviation df F Sig.
Squares Square
Between
Male 149 21.05 8.919 37.785 1 37.785 0.440 0.508
Groups
Within
Navigation Female 88 20.23 9.823 20169.025 235 85.826
Groups
Total 237 20.75 9.253 Total 20206.810 236
Between
Male 149 18.91 7.663 24.643 1 24.643 0.428 0.514
Groups
Organisation/ Within
Female 88 18.24 7.468 13542.673 235 57.628
Architecture Groups
Total 237 18.66 7.582 Total 13567.316 236

T
Between
Male 149 19.92 7.851 0.702 1 0.702 0.009 0.923
Groups
Ease of Use and Within
Female 88 19.81 9.867 17592.749 235 74.863
Communications Groups
Total 237 19.88 8.634 Total 17593.451 236

e
Between
Male 149 19.75 11.616 142.332 1 142.332 1.101 0.295
Groups
Within
Design Female 88 18.15 10.945 30390.892 235 129.323
Groups

J
Total 237 19.16 11.374 Total 30533.224 236
Between
Male 149 20.37 9.831 570.222 1 570.222 5.286 0.022
Groups
Within
Content Female 88 23.58 11.269 25352.141 235 107.881
Groups

A
Total 237 21.56 10.480 Total 25922.363 236

I
Appendix 3: The usability scores of the nine Jordanian university websites with regard to
their performance on the five categories of the developed usability criteria.
Criteria U1 U2 U3 U4 U5 U6 U7 U8 U9

Navigation 110.32 103.81 103.17 100.92 108.99 87.62 109.72 108.56 100.65
Organisation/
87.84 97.59 91.00 91.72 89.28 80.09 82.39 94.16 91.00
Architecture
Ease of Use and
110.41 110.25 100.14 103.47 103.80 82.23 102.86 105.46 103.38
Communications
Design 88.70 107.46 92.8 86.98 88.80 76.52 82.01 100.15 86.75

Content 116.13 124.40 112.46 111.14 115.31 96.59 120.04 120.92 116.73

Overall Usability 910.69 962.64 886.84 877.32 897.05 749.51 873.99 937.58 880.28

View publication stats