MUHAMAD SAIFUL BAHRI YUSOFF

ONLINE ASSESSMENT
Department of Medical Education, School of Medical Sciences,
Universiti Sains Malaysia, email: [email protected]. CHALLENGES & SOLUTIONS
https://www.researchgate.net/profile/Muhamad_Saiful_Bahri_Yusoff 4th ICHPE-2020 | SYMPOSIUM 01 | Challenge of assessing the students online
 Today’s Focus
Trends
Revisit the trends
Current Practices
REVISIT REVIEW Review the current
leading to new norms
in higher education practices of assessment

Challenges Solutions
REFLECT ENVISION Envision solutions to
Reflect the challenges
address challenges of
of online assessment
online assessment
 Trends Leading to New Norms in Higher
Education
Internet, web
technologies, e-
learning, Evolution of
Technology ICT, 4IR,
Enhanced Pandemic New
1980’ 1990’
Learning 2000’ 2010 onwards Norms in
Technical skills Mobile devices, video streaming, Higher
needed to work with
world-wide technology, wide band
data network, powerful processors,
Education
computer hardware and gamification, AI, augmented reality
software
Informational Reproduction literacy/
skills/literacy content creation skills

(Westera, 2010, Guardia et al, 2017; Gui, Fasoli & Carradore, 2017; Burr & Floridi, 2020)
 New Norms in Higher Education
01 SOCIAL
DISTANCING

02 PERSONAL
SPACE

03 SMALL
GROUP

04 DIGITAL
PLATFORM
 Technology Enhanced Learning
Digital
Technology
Teaching &

01 02
enhanced
Learning
classrooms
The use of technology to
optimise teaching and Teachers teach by
learning experience using technology

Flexible Learning with

04
Pedagogies
Application of ICT to
enable new choices for
teaching and learning
03 technology
Learners learn through
application of ICT

(Higher Education Academy, 2014; Adrian & Linda, 2014; Gordon, 2014; Carretero et al., 2017)
 Program Learning
Outcomes Starting with a clear picture of
what is important for students to
• Cognitive Domain (C) be able to do (COMPETENCY).
• Affective Domain (A) Course Learning
• Psychomotor Domain (P) Outcomes

Online Face-to-face
Flipped Learning STUDENTS assessment
Classroom (are the focus)

Teaching &
Assessment
Learning Activities

Project Others
Labs Other learning Online assessment
experiences
 3-5 years
COMPETENCY post training

Competent Medical
Final year Novice Competent Practitioner
Medical Practitioner (Medical Officer)
(House Officer)

Clinically Skillful
(Advanced beginner)
3rd & 4th year

Applied Knowledge
1st & 2nd year
(Novice)

Student
(Incompetent)
 3-5 years Competent Medical
F2F Assessment post training Practitioner (Medical Officer)

SBA Patient management

OSCE Clinical procedures Final year training
STEPS Ethical reasoning
Novice Competent Medical
Clinical viva Diagnostic skills
Practitioner (House Officer)
Short/Long cases Professionalism

SBA History taking skills

OSCE Physical examination
STEPS Clinical Reasoning 3rd & 4th year training
Logbook Diagnostic skills Clinically Skilful
Short/Long cases Professionalism (Advanced beginner)

SBA Application of knowledge

MEQ Problem-solving skills 1st & 2nd year training
OSCE Communication skills Applied Knowledge (Novice)
Basic clinical skills
Student
(Incompetent) Competency
 3-5 years Competent Medical
Online Assessment post training Practitioner (Medical Officer)

SBA Patient management

OSCE Clinical procedures Final year training
STEPS Ethical reasoning
Novice Competent Medical
Clinical viva Diagnostic skills
Practitioner (House Officer)
Short/Long cases Professionalism

SBA History taking skills

OSCE Physical examination
STEPS Clinical Reasoning 3rd & 4th year training
Logbook Diagnostic skills Clinically Skilful
Short/Long cases Professionalism (Advanced beginner)

SBA Application of knowledge

MEQ Problem-solving skills 1st & 2nd year training
OSCE Communication skills Applied Knowledge (Novice)
Basic clinical skills
Student
(Incompetent) Competency
 Challenges of Online Assessment
Learning Outcomes ICT facility
Not all learning outcomes It is a necessary condition
can be assessed via online for the deployment of online
assessment assessment, e.g., the
internet connection quality
Rules & Regulations
Exams come in a digitized Integrity & Security
form yet the same set of rules
The ability of ensure the
and regulations still applies
integrity and security of
online assessment
Limited Alternatives
Lack of knowledge about Preparedness
alternatives to the traditional
Preparedness, skills &
assessment as to how digital
technology required for online
technology can be used in
assessment
assessing students
(Embi, 2011; Raaheim et al, 2019; Bala et al., 2020; Fuller et
al., 2020; Haider & Salman, 2020; Baharuddin et al, in press)
 (Technolog
y euphoria)

1
Everyone are excited
with new technologies
2
1 Things slowing down due
2
(New Norms) to the unmet expectations
3 something that is
usual, typical, or
standard.
After short period,
3
eventually it achieved a
(Technostress) stable level of acceptance

Fenn J., M. Raskino. Understanding Gartner’s Hype Cycles. 2009. http://www.gartner.com/

The adaptation to the new norms requires

 Program Learning
Outcomes Starting with a clear picture of
what is important for students to
• Cognitive Domain (C) be able to do (COMPETENCY).
• Affective Domain (A) Course Learning
• Psychomotor Domain (P) Outcomes

Online Face-to-face
Flipped Learning STUDENTS assessment
Classroom (are the focus)

Teaching &
Assessment
Learning Activities

Project Evidence of Others

Labs Other learning Online assessment
experiences Learning
 Evidence of Learning (Assessment Task)

Direct Indirect
Assessment that Assessment that
demonstrates learning describes learning
(observed) (unobserved)

Some examples, Some examples,

• Online oral examination • Online Quiz (MCQ, Short Answer)
(interview, viva) • Project assignment (project report,
• Online presentation (final year pre-recorded video, e-poster, etc)
project, progress presentation, etc) • Written assignment (essay, case
• 360 Assessment (supervisor, report, reflective journal, etc)
lecturer, peer, etc) • e-Portfolio
The Assessment Goals (Epstein, N Engl J Med, 2007)

Training Further Training

To provide a basis for
Protect Publics advanced training
To protect the public by Publics
identifying incompetent
graduates

Capability
Optimize capabilities
To optimize the capabilities of all
learners by providing motivation
and direction for future learning
The 11 Public Medical Schools for UG Medical Education in Malaysia:
High-Stakes Assessment Recommendations
(i) Recommendation 1 is based on the situation where the Movement Control Order (MCO) is lifted. Students will return to
the campus and the examination will be conducted with strict measures to prevent the spread of COVID-19 (WHO). The date for
Professional examinations are conducted in the current academic year, depending on each university.

(ii) Recommendation 2 is based on the situation where the MCO is extended and students are not allowed to return to the
campus until a later date. The date for Professional examinations are required to be done within the current academic year. In
this scenario, the conduct of the examination will be fully online, adhering to the MMC Online Exam Guidelines. The Task Force
decided not to accept pre-recorded videos as students might not be comfortable to speak and act in front of the camera, as well
as differences among students regarding technology skills. To certify the clinical competency of the graduates, a refresher or
supplementary course will be conducted before their internship placement. If assessment data from the candidates’ previous
postings could be retrieved, it could also be considered as indicators of their clinical competency.

(iii) Recommendation 3 is based on the situation where the MCO is extended and students are not allowed to return to the
campus. However, in this scenario, the final professional examination will be deferred to a later date, probably early next year.
Universities may consider conducting theory examinations online at an earlier date. Those who pass will be awarded a
conditional pass until they sit for their clinical examination.

(Baharuddin et al, Recommendations for High-stakes Examinations for Public

Malaysian Medical Faculties during the COVID-19 Pandemic, Educ Med J, in press)
 High-Stakes Assessment Recommendations

MCO is lifted MCO is extended

Students can return to campus Student cannot return to campus

• Professional Examination will be • Professional Examination will be

• Professional Examination conducted fully online according deferred to the next year
(f2f) will be conducted to MMC guidelines • Theory examination will be
• Clinical competency will be conducted online
• Adhered to strict COVID-19
certified during refresher course • Those who pass will be awarded
SOP (smaller group) • Assessment data from previous a conditional pass until they sit
Recommendation 1 clinical rotations can be indicators for their clinical examination
of clinical competency
Recommendation 3
Recommendation 2

(Baharuddin et al, Recommendations for High-stakes Examinations for Public

Malaysian Medical Faculties during the COVID-19 Pandemic, Educ Med J, in press)
 The Malaysian Case: Competency vs. Assessment Tasks
Competency Online Assessment Modalities
Cognitive Psychomotor Affective
Application of knowledge    
MEQ, SBA
Problem Solving    
Clinical reasoning    
Ethical reasoning    
Diagnostic ability Online clinical viva    
Patient management    
History taking skills   Online History Taking  
Communication skills   (real/simulated patients)  
Physical examination   Pre-recorded video  
(students demonstrate
physical examination) +
online mini clinical viva
Professionalism     Online Clinical Viva
Pre-recorded video
(students demonstrate
physical examination)

(Baharuddin et al, Recommendations for High-stakes Examinations for Public

Malaysian Medical Faculties during the COVID-19 Pandemic, Educ Med J, in press)
 Assessment Tasks

Appropriatenes Relevance
s of assessment Fairness

Motivation
Students’ perceived Students’ Learning effort
experience pertaining learning
to the assessment Learning approach
Student’s coping

Quantity
Feedback
Quality

(Adam SK, 2020, Thesis)

 Solutions
Learner-focused L E A D Defensible Methods
What learners able to Assessment methods are
do is the central focus defensible for the attainment
(competency) of learning outcomes

Evidence of learning Authentic Task

Verify the attainment of
Students are assessed
knowledge, skills &
by valid & meaningful
abilities based on the
assessment tasks
learning evidence
 Final Thoughts
Technology-driven Adaptation Time
The need to leverage on Adaptation to the new
technology for enhancing normal in assessment
assessment is inevitable requires time

Real Challenge Local Solutions

Online assessment Finding solutions that
imposes a real challenge suit with the local needs
to higher education is important and critical
Thank You
MUHAMAD SAIFUL BAHRI YUSOFF, MD,MSC,PHD
Department of Medical Education, School of Medical Sciences,
Universiti Sains Malaysia, email: [email protected].

https://www.researchgate.net/profile/Muhamad_Saiful_Bahri_Yusoff

This ppt slide can be downloaded from

https://tinyurl.com/yx9es993