Knowledge Is Not Enough To Solve The Problems - The Role of Diagnostic Knowledge in Clinical Reasoning Activities
Knowledge Is Not Enough To Solve The Problems - The Role of Diagnostic Knowledge in Clinical Reasoning Activities
Abstract
Background: Clinical reasoning is a key competence in medicine. There is a lack of knowledge, how non-experts
like medical students solve clinical problems. It is known that they have difficulties applying conceptual knowledge
to clinical cases, that they lack metacognitive awareness and that higher level cognitive actions correlate with
diagnostic accuracy. However, the role of conceptual, strategic, conditional, and metacognitive knowledge for
clinical reasoning is unknown.
Methods: Medical students (n = 21) were exposed to three different clinical cases and instructed to use the think-
aloud method. The recorded sessions were transcribed and coded with regards to the four different categories
of diagnostic knowledge (see above). The transcripts were coded using the frequencies and time-coding of the
categories of knowledge. The relationship between the coded data and accuracy of diagnosis was investigated
with inferential statistical methods.
Results: The use of metacognitive knowledge is correlated with application of conceptual, but not with conditional
and strategic knowledge. Furthermore, conceptual and strategic knowledge application is associated with longer
time on task. However, in contrast to cognitive action levels the use of different categories of diagnostic knowledge
was not associated with better diagnostic accuracy.
Conclusions: The longer case work and the more intense application of conceptual knowledge in individuals with
high metacognitive activity may hint towards reduced premature closure as one of the major cognitive causes of
errors in medicine. Additionally, for correct case solution the cognitive actions seem to be more important than the
diagnostic knowledge categories.
Keywords: Medical problem-solving, Metacognition, Knowledge categories, Clinical reasoning, Diagnostic reasoning
© The Author(s). 2016 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Kiesewetter et al. BMC Medical Education (2016) 16:303 Page 2 of 8
Table 1 Diagnostic knowledge dimensions according to Schmidmaier [7], van Gog [9], Krathwohl [10]
Knowledge dimension Definition Examples of knowledge
Conceptual Knowledge The basic elements one must know to be acquainted Knowledge of terminology, specific details, and elements.
- what- with a discipline or solve problems in it.
Strategic Knowledge How to execute something; methods of inquiry, Knowledge of subject-specific skills and algorithms,
- how - and criteria for using skills, algorithms, subject-specific techniques and methods, and criteria
techniques and methods. for determining when to use appropriate procedures.
Knowledge about problem solving
Conditional Knowledge The interrelationships among the basic elements Knowledge of classifications and categories, principles
- why - within a larger structure that enable them to and generalizations, theories, models, and structures.
function together. Knowledge about the rationale behind.
Metacognitive Knowledge How to think about thinking; knowledge about Knowledge about cognition in general as well as
- selfcognition- cognitive tasks, and self-knowledge awareness of one’s own knowledge. Knowledge about
one’s own cognition
The Revision of Bloom’s Taxonomy added a fourth cat- them. More specifically we wanted to answer the follow-
egory: Metacognitive knowledge, which “involves know- ing research questions:
ledge about cognition in general as well as awareness of
one’s own knowledge about one’s own cognition” [9, 10]. 1. How are diagnostic knowledge categories
While handling a case, medical students or doctors are interrelated?
able to externalize their thoughts about the strategies of The interplay of knowledge categories gives insight
problem-solving or their application of knowledge [11]. how students store clinical knowledge and whether
Metacognition in this sense includes the judgements of some categories seem more important to them than
how easily one believes one learns and whether one has others. Further, it has not been investigated how
the feeling of knowing something. knowledge categories relate to previous knowledge.
Surprisingly, little is known about the assessment and 2. How is the use of the diagnostic knowledge categories
applicability of metacognition within the medical context related to time on task?
and its relation to the knowledge categories in the situ- It is important to understand how much time
ated learning contexts of medical students. the application of the different knowledge
Whereas several methods are used to assess “classic” categories takes.
knowledge categories (e.g. multiple choice tests, key 3. How is the use of diagnostic knowledge categories
feature problems, interviews, questions, stimulated re- related to diagnostic accuracy?
call) it has proven difficult to measure and observe Especially, it seems interesting to identify the role
metacognition in a realistic setting [7]. Since metacogni- each plays to solve a clinical case.
tion cannot be observed directly in students [12], self- 4. How are the knowledge categories divided over the
report methods like questionnaires, rating scales and course of a case solution?
stimulated recall are used. However, these self-reporting It is interesting to see if some of the knowledge
measures already reflect that, to be able to talk what one categories are used more frequently in the beginning
thinks, the student’s metacognitive activities and one’s and others are used more towards the end of the
verbal capacity are of importance [13]. When students case solutions.
are thinking aloud, registering the metacognitive ac-
tivities without the student’s awareness is possible and To answer this research questions we conducted a study
the otherwise implicit cognitive processes can be ob- where medical students first received a short knowledge
served [14]. training for clinical nephrology and a subsequent know-
In clinical problem solving research, traditionally only ledge test to standardize previous knowledge. After that
little parts of knowledge are investigated in relation to the students worked on paper-based, clinical case scenar-
the correct diagnosis. Thus far, there is no model of clin- ios while thinking a-loud. The think-a-loud protocols were
ical reasoning that, if applied, can explain how and why transcribed and coded according to the aforementioned
successful students come to the correct diagnosis, while knowledge categories. In the following paragraphs each
unsuccessful students do not. However, it seems worth- step of the methodology is explained in detail.
while to create evidence for such a holistic model of
clinical problem solving of medical students that should Methods
include all knowledge categories. We therefore set out Participants
to observe all aforementioned knowledge categories sim- Twenty-one medical students (female = 11) of two
ultaneously in order to identify the relationship between German medical faculties in their third, fourth and fifth
Kiesewetter et al. BMC Medical Education (2016) 16:303 Page 3 of 8
year (M = 23.9 years; range 20–34) volunteered to take Figure 1 shows the course of the study with knowledge
part in the study. These curricular years were chosen be- training, a subsequent knowledge test, and work on the
cause the participants would have finished their internal paper-based, clinical case scenarios.
medicine curriculum and should have enough prior All students were recorded and recordings were tran-
knowledge to solve clinical problems but would not have scribed and coded according to the defined knowledge
experienced the final sixth clinical year of full-time categories. Codings were analysed for accuracy of the
electives that usually elevates students’ problem-solving diagnosis.
substantially. This study was approved by the Ethical
Committee of the Medical Faculty of LMU Munich. Pre-study questionnaire
Written, informed consent was obtained from all partici- All participants completed a questionnaire containing
pants and all participants received a small monetary items about their socio-demographic data, gender and
compensation for participation. age to control possible confounders. Further the partici-
pants were asked their overall grade of the preliminary
medical examination. The reliability of this national
Coding scheme
multiple-choice exam is very high (Cronbachs α = .957)
A coding scheme was established on the foundation of
[16]. The performance of participants in this exam was
the knowledge type definitions [7, 10, 15]. The defin-
used as an indicator for general prior knowledge in
ition used in the coding scheme is illustrated in
medicine. The results of the questionnaire and all other
Table 2. The coding scheme had an overall interrater
obtained data were pseudonymized.
reliability of k = .79; SD = .9 for the categories. One in-
vestigator (R.E.) coded all transcripts; a random 10%
Knowledge training and test
sample of the text was double coded.
Although all participants had successfully passed their
internal medicine curriculum a standardized learning
Course of study tool was provided to refresh the textbook knowledge.
Students arrived and first filled out a pre-study ques- Thirty flashcards were used containing 98 items with
tionnaire (see below), then students received a three factual information on clinical nephrology and more
hours of practicing a standardized learning unit in precisely to acute renal failure and chronic renal insuffi-
the field of clinical nephrology and upon completion, ciency. This content matches with the pathomechanisms
the students’ retention of content specific medical of the used cases. The content of the flashcards was pre-
knowledge was tested using a multiple choice test. viously published in another study (appendix S1 (online)
Then participants were instructed on the think-aloud of Schmidmaier et al. [17]). Within a 3 h electronic
method in a short practice exercise. Finally, students learning module it was ensured by testing that all partic-
then solved three cases in clinical nephrology with ipants could retrieve the contents of each flash card at
the think-aloud method (see below). least once. This was to help ensure that all students were
Table 3 Descriptive data of diagnostic knowledge dimensions 1. How are the diagnostic knowledge categories
used by medical students during the cases interrelated?
Knowledge dimension Frequency Percent To answer this research question the frequency
Conceptual knowledge (CcK) 432 44% per case of the use of knowledge categories was
CcK only 325 33% correlated. Results show that conceptual and
strategic knowledge are not significantly related
With other knowledge dimensions 107 11%
(rCcK;SK = .23;n.s.; rCcK;CdK = .00;n.s.). Conceptual
Strategic knowledge (SK) 349 36%
knowledge and metacognitive knowledge
SK only 279 28% (rCcK;MK = .35) are significantly related, as are
With other knowledge dimensions 70 8% conditional and strategic knowledge (rCdK;SK = .27).
Conditional knowledge (CdK) 202 21% The results are presented in Table 4.
CdK only 121 12% Interestingly prior knowledge (grades of PME and
assessment of the learning phase in the field of
With other knowledge dimensions 81 9%
clinical nephrology) was significantly correlated to
Metacognitive knowledge (MK) 568 58%
metacognitive knowledge (rMK;PME = .41, rMK;
MK only 0 0%
LEARNING PHASE = .28).
With other knowledge dimensions 568 58% 2. How is the use of diagnostic knowledge categories
Percent refer to the overall use of knowledge dimensions (CcK, SK, and CdK related to time on task?
equal 100%) To answer this research question the time-on-task
(TT) was correlated with the use of knowledge cat-
frequency, conditional knowledge (CdK) was used egories. In three cases the students had to be inter-
with a 36% frequency, strategic knowledge (SK) with rupted after 10 min. These students were included in
a 21% frequency. Metacognition was identified most the analysis with the maximum time. The overall
frequently (58%) but always in combination with time-on-task was not correlated with diagnostic
other knowledge categories. Metacognition was used accuracy (rTT; DIAGNOSTIC ACCURACY = -.13;n.s.).
in every case with a mean of M = 9.02 per case (SD = However, conceptual and strategic knowledge is sig-
6.21). Figure 2 shows the time-line graphs of two par- nificantly correlated to TT (see Table 5).
ticipants, exemplifying little and extensive use of 3. How is the use of diagnostic knowledge categories
metacognition. related to diagnostic accuracy?
Fig. 2 Time-line graph (Gantt-charts) of two participants of a session with a clinical case. The Gantt-chart shows the distribution of the use of dif-
ferent diagnostic knowledge categories over time. As metacognitive knowledge was only in use in combination with other knowledge categories
its use is presented additively on top. The upper part of the figure shows a participant with only little use of the knowledge categories and the
lower part of the figure a participant with much use of the knowledge categories
Kiesewetter et al. BMC Medical Education (2016) 16:303 Page 6 of 8
Table 4 Pearson’s Correlations of the use of diagnostic conditional knowledge throughout the cases. None of
knowledge dimensions the knowledge categories on its own has a crucial role
Knowledge dimension Strategic Conditional Metacognitive for good performance. Further, prior knowledge was not
knowledge knowledge knowledge directly related to the correct diagnosis. These results
Conceptual knowledge .23 .00 .35* supports the claim that it is not simply knowledge which
Strategic knowledge .27* .09 solves clinical cases and more in this sense does not dir-
Conditional knowledge .15 ectly mean better. Instead, it is the goal-directed applica-
Significant results (p < .05) are marked with an asterisk tion of knowledge in a certain order that helps to solve
cases. Over the course of the cases it seems that the ap-
When correlating the use of the four knowledge plication of conceptual and strategical knowledge de-
categories to the correct solution none of them clines, while the importance of metacognitive knowledge
showed a significant result. As well, Chi squared increases. We found that oftentimes the last two cat-
tests of socio-demographic data of the participants egories before a diagnostic decision was made by the
(age, year of studies) and correct versus incorrect participants consisted of a pattern of conditional and
diagnosis yielded no significant result. strategic knowledge at the closure of cases, named
4. How are the knowledge categories divided over the sequence-at-closure, which correlated with the correct
course of a case solution? solution of the case. This result relates to our previous
We found that frequencies of the used categories are findings regarding the so called higher loop of cognitive
not equally distributed over the case. Interestingly, in actions, which was associated with better diagnostic per-
the first two sixth of the case the students used formance [20]. The higher loop consisted of the cogni-
more conceptual and strategic knowledge. From the tive actions Evaluation, Representation and Integration.
third sixth the students used more metacognition It seems that students who are ready to state a correct
than any other category. Of course, metacognition diagnosis evaluate and summarize their represented
could only be coded together with other categories, knowledge about the case with this final pattern of con-
so there is a dependency of this category. However, ditional and strategic knowledge before integrating into
the frequencies of conceptual and strategic the correct solution. If students have a clear representa-
knowledge decline in the fifth and sixth sixths. tion of the case in relation to their predefined clinical
All frequencies over the course of the cases are knowledge they know, why the patient’s symptoms and
depicted in Fig. 3. clinical findings occur and how to deal with them, then
We found the occurrence of a pattern of conditional they have a very good chance to correctly diagnose the
and strategic knowledge right before the closure of patient. This finding has direct implications for instruc-
cases, named sequence-at-closure (s@c). This tional medical education research, which we will discuss
sequence-at-closure appeared in 24 of the 63 case further below.
solutions (=38%) and is significantly correlated with Metacognition could be coded in all participants.
the correct solution of the case (rϕ. S@C; CORRECT SO- However, it always appeared in conjunction with
LUTION = .37). other knowledge categories. This result seems plaus-
ible as the application of metacognition cannot be
Discussion separated from the content of a case. The coding of
In this study the different knowledge categories includ- metacognition was worthwhile; it significantly corre-
ing metacognition in case work of medical students were lated with conceptual knowledge and with two dis-
empirically coded and described. The diagnostic know- tinct measures of prior knowledge. People who know
ledge categories were applied for the first time to med- more and scored better in their previous studies seem
ical students problem-solving in a realistic environment. to have additional capacity to control and monitor
The result was application of conceptual, strategic and their solution in a better way. Knowledge regularly is
measured in assessment and learning research [6].
Table 5 Pearson’s correlations knowledge dimensions and Thus far only a few studies take metacognitive know-
time-on-task ledge into account. The few available studies take into
Knowledge dimension Correlation with time-on-task regard interventional aspects, namely reflective prac-
tice [21–23]. There are many ways to assess metacog-
Conceptual knowledge .27*
nition. With our method, we tried to go one step
Strategic knowledge .35*
beyond the current approaches to understand what is
Conditional knowledge .16 happening in the mind of medical students. It shows
Metacognitive knowledge .24 that high and low performers are not distinguished
Significant results (p < .05) are marked with an asterisk simply by their use of knowledge categories.
Kiesewetter et al. BMC Medical Education (2016) 16:303 Page 7 of 8
Fig. 3 Diagnostic knowledge dimensions used by medical students over the course of the cases