UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 24
Development and Validation of the Household Food
Safety Questionnaire
Daniel Maestro*1,2,3, Arzija Pašalić2,3, Sabina Šegalo4, Amar Žilić3,5
*1
Institute of Public Health for Federation of Bosnia and Herzegovina, Bosnia & Herzeegovina
2
Faculty of Health Studies, University of Sarajevo, Bosnia and Herzegovina
3
Association for Sanitary Engineering for Bosnia and Hercegovina, Bosnia and Herzegovina
4
Faculty of Medicine, University of Sarajevo, Bosnia and Herzegovina
5
Federal Inspection Administration, Bosnia and Herzegovina
danielmaestrobih@gmail.com
Abstract— Health problems associated with the
consumption of foods that do not meet the hygiene and
epidemiological standards are not of recent date and
have been occurring continuously throughout the
history of human existence. The incidence of food
poisoning and foodborne transmissible diseases is three
times more common in-home kitchens and households.
Restaurant poisoning generally involves a larger
number of people, whereas a home-based one involves
individuals or a small number of people, so its
likelihood of identification by the competent authority
or public health organizations and services are
significantly smaller. The development of the
household food safety questionnaire (HFSQ) for the
general population went through five phases
(preparation of the questionnaire, distribution of the
questionnaire to panelists and then the respondents,
statistical analysis and the formation of the final
version of the validated questionnaire). A total of 58
particles that formed the basis of the questionnaire
were divided into four segments: demographic (10),
knowledge (17), opinion (14) and food safety practice
(17). Overall, the validity of the questionnaire in
examining practice, knowledge and attitude was
determined with Cronbach's alpha = 0.842. The total
number of particles adequate for the questionnaire is 29
questions. This questionnaire is a good instrument for
assessing the knowledge, attitudes and practices of
food handlers in their households.
Keywords— food safety, households, questionnaire,
development and validation.
I. INTRODUCTION
1.1 Food Safety in Households
Health problems associated with the consumption of
foods that do not meet the hygiene and epidemiological
standards are not of recent date and have been
occurring continuously throughout the history of
human existence. More than 95% of cases of
foodborne transmissible diseases (FTD) are
characterized by sporadic occurrence [1]. According to
World Health Organization (WHO) reports, the actual
prevalence is 300-350 times the number of reported
cases [2]. An estimated 1.8 million people die annually
from diarrheal diseases, with the majority attributed to
contaminated water or food [3]. Although governments
around the world are making great efforts to improve
the health safety of accessible food [1], the emergence
of FTD remains a significant public health problem in
all countries, regardless of the degree of economic
development.
Borneff and associates believe that the incidence of
food poisoning and FTD is three times more common
in-home kitchens and households. [4] Restaurant
poisoning generally involves a larger number of
people, whereas a home-based one involves individuals
or a small number of people, so its likelihood of
identification by the competent authority or public
health organizations and services are significantly
smaller [5]. Although experts generally agree that
households are the primary places where FTD occurs,
consumers do not consider their homes a risky place
[6]. Unlike restaurants, home kitchens are
multifunctional areas where a variety of activities take
place. Pets, items not exclusively used for food
preparation, dirty laundry, houseplants and the likes are
commonly found in kitchens. Sinks are used for
washing hands, various objects, dishes, clothes,
children and pets, as well as wetting and soaking
cloths. Dirty dishes can be easily stacked together with
clean ones on kitchen countertops [7] Currently, most
purchased foods are considered safe to consume,
however, there is a need for consumers to properly
handle, prepare and store household foods [8].
Inadequate handwashing, food handling and
preparation, short cooking times and long storage
under inadequate conditions are considered to be a
cause of significant FTD in households, which can lead
to proliferation of microorganisms [9]. Proper food
preparation can prevent most FTD s [3], so research
into the knowledge, attitudes and practices of food
handlers is essential to formulate and design programs
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 25
to promote and educate on food safety in households
[10]. It is only when there is knowledge of food safety
attitudes and practices that it is possible to plan
effective strategies to encourage and reinforce
desirable food handling behaviors and discourage those
that are inappropriate [11]. Recognizing the need to
study the sources of infection and the possible causes
of FTD in households [12], our study is primarily
directed in this direction. In Bosnia and Herzegovina,
according to the available literature, no research based
on testing and research into household food safety
knowledge, practices and attitudes has been conducted,
so we designed and developed a questionnaire for the
purpose of the survey and evaluated its validity using
adequate statistical methods.
1.2 Survey Questionnaire
Every scientific research is based on some form of
measurement. Like all other measuring instruments,
questionnaires and the answers they collect do not have
value in themselves, but solely as a way of measuring
the subjective phenomenon they seek to capture [13].
The name of the questionnaire refers to a set of
procedures by which data obtained from respondents is
collected and analyzed, which provide details about the
behavior, attitude, opinion, knowledge, intention,
interests of the respondents. They are used for official
statistics, various surveys and public / market surveys.
The basic aim of the questionnaire must be clearly
defined, therefore, when designing the questionnaire, it
is necessary to study the interdependence of the
variables that occur in the research goals and to be
guided by asking questions that provide the basis for
providing important data on multiple variables [14]. A
good questionnaire is characterized by a high
correspondence between the response received and the
measurement object and the consistency of the data in
different comparable situations. These characteristics
belong to two basic metric features: validity and
reliability. Instruments without adequate metric
characteristics lead to insufficiently accurate estimates
of the state of affairs which significantly diminishes
their useful value [13]. When we talk about the relation
between reality and the picture of reality that we get by
using a questionnaire, we come to the concept of
validity of the questionnaire. Validity is, in the
broadest sense, related to the question: "Do we really
measure what we think we are measuring?" This term
refers to the appropriateness, meaningfulness and
usefulness of conclusions and interpretations derived
from test results [15]. Content validity is the extent to
which the content of the questionnaire is thematically
tailored to the area it should cover. Criterion validity
can be defined as the relationship between the applied
questionnaire and the result of another known variable,
while constructive validity is the degree to which a
questionnaire result indicates a particular trait [16].
Equally important is the question of the reliability of
the questionnaire, and it relates to the consistency of
the respondent's answers to the same or similar
question. The reliability determinations are different -
some emphasize measurement accuracy, second
consistency, and third the absence of measurement
error [17].
The terms validity and reliability often overlap. A non-
reliable answer may not be valid either, but a reliable
answer may not always be valid. For the credibility of
the results, it is essential that the questionnaire as a
whole is both reliable and valid. This makes it easier to
further equalize the measurement conditions for each
individual subject, and the objectivity of the
measurement procedure. The reliability and validity
data of the test provide insight into the structure of
each measuring instrument and provide assurance that
the measuring instrument really measures the
appearance / trait we want to measure and that the
results obtained reflect the real characteristics of the
subjects. Validity, with reliability, is the metric
characteristic that receives the most attention and is
considered to be the most important characteristics of
instruments used in the public health and social science
[13].
II. MATERIALS AND METHODS
The development of the household food safety
questionnaire (HFSQ) for the general population went
through five phases (preparation of the questionnaire,
distribution of the questionnaire to panelists and then
the respondents, statistical analysis and the formation
of the final version of the validated questionnaire). The
preparation of a pool of potential questions entailed
detailed insight into the available literature and studies.
Part of the question was derived in part from similar
studies [18, 19, 20. 21, 22, 23, 24. 25], and additional
particles were formed in accordance with
recommendations from the household food safety
literature [26, 27, 28]
A total of 58 particles that formed the basis of the
questionnaire were divided into four segments:
demographic (10), knowledge (17), opinion (14) and
food safety practice (17). Demographic issues included
the collection of data on gender, age of respondents
with a particular focus on persons under 18 and over
65, educational attainment, marital status and number
of household members, total monthly household
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 26
income, and frequency of meal preparation in
households. The answers to the questions in the
remaining segments were mostly formed on the Likert
scale with a series of statements devoted to different
aspects of knowledge, attitude and practice, with the
respondents being tasked with expressing the degree of
agreement or disagreement for each individual
statement on a four-step scale in the 4- 1 (I totally
agree, I agree, I don't know, I disagree). Due to the
complexity of the questions, individual items were
offered answer options, where the respondents,
depending on their knowledge, attitude and practice,
had to choose one that they thought was correct.
In the second phase, ten experts in the field were
selected who were willing to participate as panelists.
They were asked to clearly state their ratings of the
importance of including each particle in the
questionnaire. In a specially formed panel for panelists
/ experts in the field, each particle is offered answers:
necessary, not necessary, but may be useful and not
necessary. Based on their results, Content Validity
Ratio (CVR) was calculated using the Lawshe method
[29].
The third phase involved the distribution of the
questionnaire of the respondents by random selection
method. Of the 106 respondents who participated in the
pilot survey, six were excluded because of incomplete
answers. The results of 100 subjects were included in
the study and analyzed quantitatively and qualitatively.
For the purpose of validating the questionnaire,
Cronbach's alpha was calculated, which measured the
internal consistency of each factor and its total value.
Factor analysis was also performed, tested using
Bartlett's test of sphericity, and determined by Kaiser-
Meyer-Olkin (KMO) as an indicator of sample
adequacy. A higher Cronbach's alpha coefficient
indicates greater reliability, that is, attributes of the
same factor measure the same phenomenon.
Cronbach's alpha does not have strictly established
values that confirm that a group of questions is
consistent and usable for examining a particular topic,
but a value of 0.7 is taken as the most commonly used
distinction.
The analysis is considered justified if the Bartlett test is
significant p <0.05, because then there is confirmation
of the existence of correlations at least among some
variables. The KMO value is a measure for quantifying
the correlation among variables, and values> 0.8
indicate very strong correlation, 0.7-0.8 strong, 0.6-0.7
medium strong, 0.5-0.6 weak, while smaller values
indicate the inadequacy of variables for factor analysis
[30]
III. RESULTS AND DISCUSSIONS
This is the first study to measure the validity and
reliability of HFSQ, which contained a diverse and
broad range of particles to evaluate knowledge,
attitudes, and practices.
To evaluate the validity of the content of the
questionnaire, a CVR (Content Validity Ratio) was
calculated, which tells us the validity of the individual
particles and the CVI (Content Validity Index), which
assesses the validity of the entire questionnaire. For all
three segments of USHD - food safety knowledge,
food safety attitude and food safety practice, the CVR
of each particle and the total CVI of each segment were
calculated. In addition to analyzing the validity of each
segment, the CVI of the entire HFSQ was also
calculated. The panelists' scores were used to calculate
the CVR for each questionnaire particle (N = 48) using
the Lawshe method (33):
where n is the number of panelists who identify a
particle as "required" and N is the total number of
panelists (N / 2 is half the total number of panelists).
A CVR value <0.62 was the exclusive factor, that is, if
2 out of 10 experts found the question unnecessary, the
same was removed from further analyzes. The number
of experts recommended when determining content
validity in areas where it is difficult to find experts is
three, and as a rule a panel of 5-10 experts / panelists is
preferred. The use of over 10 experts / panelists is in
principle unnecessary [31]
As a simple measure, CVI represents the mean of all
CVR values of those particles that meet the CVR
threshold of 0.62 and were retained in the
questionnaire. All CVI values> 0.70 are considered
acceptable [32]. In many situations, it is more efficient
to present results on the total CVI than to display the
CVR values of each particle [33]. Two questions were
excluded from the analysis of the validity of
knowledge questions, Z12 - When buying groceries
when is the best time to buy frozen food? with CVR =
0.2 and Z13 - What is the optimum temperature for
frozen food storage? with CVR = 0.6. Overall CVI of
the segment - food safety knowledge, after excluding
these two questions is 0.96 indicates significant
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 27
validity of the content of the question, and the value
exceeds the preferred CVI value = 0.8 [34].
The analysis of attitude questions excluded two
questions: S8 - If, when purchasing groceries, on the
declaration we check the date indicated under "best use
until" we reduce the possibility of food poisoning
(CVR = 0.6) and S9 - food poisoning is not a "normal"
life event (CVR = 0). Overall CVI of the Segment -
Food Safety Attitude, after excluding these two
questions is 0.95, which indicates the significant
validity of the content of the questions being tested.
The value exceeds the preferred CVI value of 0.8 [34,
35].
Analyzing the practice segment, you found that
question P6 - Do you sometimes clean your chicken
with paper towels instead of washing it in the kitchen
sink? CVR was 0.6 and was excluded from the
questionnaire, so the total CVI of this segment is 1,
indicating significant validity of the content of the
question.
A CVI of 0.95 for the whole questionnaire indicates a
high degree of validity [29].
In a similar study conducted in Iran [34] after an expert
evaluation, the total CVI of the questionnaire was 0.75,
indicating that HFSQ has a high degree of
representativeness and clarity, thus confirming its
content validity. In order to test the reliability of USHD
for assessing household food safety knowledge,
attitudes, and practices, the value of the Cronbach's
alpha coefficient was determined. A higher coefficient
value indicates greater reliability, that is, the attributes
of the same factor really measure the same
phenomenon [36].
An analysis of the internal consistency of the questions
was conducted for each segment separately and for the
whole questionnaire. After the first analysis of the
knowledge questions remaining after the analysis by
the experts, a total of 15 questions were given for
completion in the pilot study. Analyzing the answers to
them, it was found that certain questions reduce the
homogeneity, and therefore the validity of the
questionnaire. Questions excluded from the
questionnaire to increase Cronbach's alpha are: Z11,
Z14, Z16, and Z17. After excluding the question,
Cronbach's alpha was found to be 0.745. The ANOVA
test found a significant association between response
and subjects at p <0.001. Knowledge questions were
also subjected to a sample adequacy analysis, which
determined a Kaiser-Meyer-Olkin correlation of KMO
= 0.782, factor analysis was justified by a significance
of p <0.001.
Table 1. Analysis of quality and validity of knowledge
questions
Question CVR
Cronbach's
alpha
Status
Z1 Harmful microbes
that are not visible to the
naked eye can lead to
food poisoning
1,00 0,588 Remains
Z2 If we eat meat that is
raw or semi-cooked, we
are at a high risk of food
poisoning
1,00 0,562 Remains
Z3 Non-hygienic
practices are a source of
microorganisms that can
cause food pollution
1,00 0,598 Remains
Z4 If we eat cooked food
stored at room
temperature for more
than 4 hours, we are at
high risk of food
poisoning
1,00 0,583 Remains
Z5 Direct contact of
unwashed hands and
ready-to-eat food leads
to bacterial
contamination of food
0,80 0,574 Remains
Z6 Defrosting frozen
meat on the lower shelf
of the refrigerator is the
correct way of defrosting
1.00 0,622 Remains
Z7 Eating half-cooked
eggs (soft yolk) can
cause food poisoning
that often ends in
hospitalization
1.00 0,560 Remains
Z8 Insects such as
cockroaches and flies
can transmit bacteria that
lead to food poisoning
1.00 0,595 Remains
Z9 Cooked food
residues should be
reheated, thoroughly
heated / boiled before
consumption
1,00 0,571 Remains
Z10 In addition to 0,80 0,570 Remains
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 28
Question CVR
Cronbach's
alpha
Status
bacteria, fungi and
viruses can also cause
foodborne diseases
Z11 Do you think that
bacteria in food can be
destroyed by freezing at
-18 ° C?
0,80 0,630 Rejected
Z12 When buying
groceries, when is the
best time to buy frozen
food?
0,20 - Rejected
Z13 What is the optimal
storage temperature for
frozen foods?
0,60 - Rejected
Z14 What to do with
freshly cooked food
consumed 3 hours later?
1,00 0,675 Rejected
Z15 Do you think it is
correct to defrost meat
again?
1,00 0,630 Remains
Z16 Which of the
following is the least
certain way of defrosting
raw meat?
1,00 0,669 Rejected
Z17 How long can
cooked food be stored in
the refrigerator?
1,00 0,629 Rejected
Cronbachs alpha after
excluding rejected questions
(n=11)
0,745
The pilot study included 12 attitude particles.
Following the responses provided by the respondents,
Cronbach's alpha = 0.552 was determined, which did
not indicate the reliability of the attitude assessment
questions. After standardization, the Cronbach alpha
value was 0.580, but still indicated poor validity of
attitude questions. Subsequent analyzes based on the
Cronbach's alpha value of each question excluded three
questions - S11, S12, S13. By excluding these
questions, the degree of certainty of the attitude
questions that they truly represent the real world has
increased to the degree of Cronbach's alpha = 0.714.
The number of questions was reduced to 9, but their
reliability increased. The correlation matrix established
the connection between most questions. The very
existence of the correlation of the answers between the
questions, and their heterogeneity, indicates that there
are more trends in the answers. A total correlation was
found within the question with a mean of 0.217 and a
range of -0.02-0.460, which classifies the same into
weak and moderately strong correlations. Question
validity would be increased by excluding question S14,
but not significantly. An ANOVA with the attitude
question test found a significant association between
questions at p <0.001. Attitude questions were also
subjected to a sample adequacy analysis, which
established a Kaiser-Meyer-Olkin correlation of KMO
= 0.732, Factor analysis was justified by a significance
of p <0.001.
Table 2. Analysis of the quality and validity of attitude
questions
Question CVR
Cronbach's
alpha
Status
S1 People who prepare
food in their home can
infect themselves and
their families with
foodborne bacteria
1,00 0,505 Remains
S2 It is important to
check the temperature of
the refrigerator to
prevent spoilage and
food poisoning
1,00 0,500 Remains
S3 Raw food should be
kept separate from
cooked food
1,00 0,471 Remains
S4 Hair, jewelry, and
long nails can
contaminate food with
bacteria
1,00 0,491 Remains
S5 Raw vegetables and
meat should not be cut
using the same knife
unless previously
washed
1,00 0,525 Remains
S6 Raw fruits and
vegetables are safer to
eat if washed with cold
running water
0,80 0,510 Remains
S7 If you wash your
hands with liquid soap
and warm water for 40-
45 seconds before
handling food, it will be
1.00 0,496 Remains
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 29
Question CVR
Cronbach's
alpha
Status
enough to get rid of the
bacteria on your hands
S8 If, when purchasing
groceries, on the
declaration we check the
date indicated under
"best use until" we
reduce the possibility of
food poisoning
0,60 - Rejected
S9 Food poisoning is not
a “normal” life event
0,00 - Rejected
S10 In your opinion,
what is the incidence of
food poisoning due to
improper household
preparation?
1,00 0,487 Remains
S11 In your opinion,
what is the incidence of
food poisoning in a
restaurant compared to
home-cooked meals?
1,00 0,593 Rejected
S12 What do you think
is right way to deal with
leftover cooked food?
0,80 0,603 Rejected
S13 What do you think
is the best measure to
prevent food poisoning?
1,00 0,592 Rejected
S14 From the following
two statements, which is
closer to your view: a)
Certain types of people
have a higher risk of
food poisoning, b) All
people have the same
risk of food poisoning
0,80 0,557 Remains
Cronbachs alpha after
excluding the rejected
questions (n=9)
0,714
Based on the results of the analysis of practice
questions by the experts in the pilot study, the practice
was examined with 16 questions. After the results
obtained, Cronbach's alpha = 0.510 was determined.
The value indicated that the given group of questions
had great heterogeneity and that as such it was not
adequate to represent the reliable practice of the
respondents. Subsequent analysis of the reliability of
the questions to allow the examination of the
respondents' practice revealed that it was necessary to
exclude the questions: P9, P11, P12, P13, P14, P15,
P16. After excluding them, a Cronbach's alpha of 0.683
was determined. Although not> 0.7, the Cronbach's
alpha values do not have a strictly set benchmark, and
the Cronbach's alpha value = 0.683 is consistent with
the results of [37, 38, 39] suggesting that the given
value can be used as a reference.
After standardization of the data, Cronbach's alpha =
0.724 was determined. The correlation matrix
determined the existence of a positive relationship
between the answers to the questions. The correlation
itself ranges from extremely weak: 0.046 between for
P1 and P10 (classified as extremely weak), to 0.579
(moderately strong correlation) between P2 and P4.
The maximum value of Cronbach's alpha was found to
be set to practice questions, and standardization yielded
Cronbach's alpha values = 0.724. Further exclusion of
the questionnaire could not increase the value of the
questionnaire reliability factor in examining
respondents ’practices. Sample adequacy analysis
showed a strong correlation of response and sample,
and Bartlett's test of sphericity was p <0.001. The
Kaiser-Meyer-Olkin analysis determined the degree of
correlation KMO = 0.734.
Table 3. Analysis of the quality and validity of practice
questions
Question CVR
Cronbach's
alpha
Status
P1 Do you avoid
cooking and preparing
food if you have open
wounds on your hands?
1,00 0,517 Remains
P2 Do you clean
surfaces that come in
contact with food
before and after food
preparation?
1,00 0,461 Remains
P3 Do you use different
planks for cutting raw
meat and vegetables?
1,00 0,389 Remains
P4 Do you wash your
hands with soap and
warm water before
starting food
preparation?
1,00 0,498 Remains
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 30
P5 Do you defrost meat
on the bottom shelf of
the refrigerator?
1,00 0,472 Remains
P6 Do you sometimes
clean your chicken with
paper towels instead of
washing it in the
kitchen sink?
0,60 - Rejected
P7 Do you separate raw
and cooked foods in
your fridge?
1.00 0,460 Remains
P8 Do you use a food
thermometer to check
that the food is cooked
enough?
1.00 0,517 Remains
P9 Do you leave hot
cooked food at room
temperature for more
than 4 hours?
1.00 0,556 Rejected
P10 How often do you
disinfect your kitchen
sink?
1.00 0,472 Remains
P11 After cracking and
opening raw eggs, what
do you usually do?
1.00 0,496 Rejected
P12 After handling raw
meat, what do you
usually do?
1.00 0,523 Rejected
P13 If you pre-cook
large quantities of food
with meat additives,
when do you put food
in the refrigerator?
1.00 0,522 Rejected
P14 How do you wash
vegetables and fruits?
1.00 0,498 Rejected
P15 If you have
chopped or cut meat on
the board and now
want to cut the fruit,
which of the following
do you do?
1.00 0,421 Rejected
P16 How do you wash
your hands?
1.00 0,535 Rejected
P17 In your opinion, in
terms of food safety,
for you and your
1.00 0,503 Remains
household members.
Cronbachs alpha after
excluding the rejected
questions (n=9)
0,683
Overall, the validity of the questionnaire in examining
practice, knowledge and attitude was determined with
Cronbach's alpha = 0.842. The total number of particles
adequate for the questionnaire is 29 questions. The
ANOVA test found a significance level of p <0.001.
The result of the ANOVA test indicates that the
questionnaire can be used to test the attitude,
knowledge and practice of food safety in the subjects.
The correlation matrix values in all segments were less
than 0.85, indicating that there were no
multicollinearity problems among the factors [40]
In the aforementioned study, the Cronbach alpha
coefficient for each factor ranged from 0.549-0.798,
indicating satisfactory internal consistency criteria.
Based on research by Bautista et al [41]. A Cronbach
alpha of 0.5 or greater is considered an acceptable
criterion for internal consistency in the questionnaire
reliability analysis.
In order to test the validity of factor analysis, Bartlett's
test of sphericity was applied and the Kaiser-Meyer-
Olkin (KMO) indicator of sample adequacy was
determined. The analysis is considered justified if the
Bartlett test is significant (p <0.05), because then there
is confirmation of the existence of correlations at least
among some variables. The Kaiser-Meyer-Olkin
(KMO) sample adequacy indicator has values from 0 to
1 and is a measure for quantifying intercorrelations
among variables. KMO values above 0.8 indicate a
very strong correlation; values 0.7-0.8 strong
correlation; values 0.6-0.7 moderate correlation; values
0.5-0.6 have a weak correlation, while values below
0.5 indicate the inadequacy of conducting factor
analysis. In this study, a KMO of 0.6 was defined as
the minimum acceptable. The strength of the
relationship between HFSQ particles is very good and
fulfills the criteria for factor analysis, as demonstrated
by the high KMO (0.734) of the whole questionnaire
and of each segment separately [36].
IV. CONCLUSIONS
For the purpose of the research, a questionnaire was
specifically created that was adapted to the research in
the general population. The instrument was developed
based on a thorough literature review with expert
consultation. The Household Food Safety
Questionnaire (HFSQ), which measures food safety
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 31
knowledge, attitudes, and practices, met the
psychometric criteria for the reliability and validity
test. This questionnaire is a good instrument for
assessing the knowledge, attitudes and practices of
food handlers in their households. An insight into the
available literature indicates that the study conducted
was the first study of its type in Bosnia and
Herzegovina, and may provide a starting point for
future research and a scientific platform for
considering measures and activities necessary to
improve household food safety.
REFERENCES
[1] Food Standards Agency. Foodborne Disease:
Developing a Strategy to Deliver the Agencies
Targets. Paper no. FSA 00/05/02. Agenda item 4,
2000.
[2] World Health Organization: Fact sheet No 237:
Food Safety and Food Born Illnesses, Geneva
2006.
[3] World Health Organization, Five keys to healty
food, Geneva - Switzerland, 2008.
[4] Borneff, J., R. Hassinger, J. Wittig, and R.
Edenharder. Effective Hygienic Measurements in
Households Today. Zentralbl. Bacteriol. 2008.
[5] Worsfold, D., and C. J. Grif. th. Food Safety
Behaviour in the Home. Br. Food J, 1997.
[6] Maurer, J., Byrd-Bredbenner, C., Wheatley, V.,
Cottone, E., Clancy, M. Young Adults Report
Better Handwashing Behaviors than they Actually
Practice. Food Prot. Trend, 2008.
[7] Byrd-Bredbenner, C., Maurer, J., Wheatley, V.,
Cottone, E., Clancy, M. Food Safety Hazards Lurk
in the Kitchens of Young Adults. J. Food Prot.
2007.
[8] Tomaszewska, M., Trafialek, J., Suebpongsang, P.,
Food Hygiene Knowledge and Practice of
Consumers in Poland and in Thailand - A survey,
Food Control Vol. 85, 2018.
[9] Fischer, A., Frewer, L., Nauta, M. Toward
Improving Food Safety in the Domestic
Environment: A Multi-item Rasch Scale for the
Measurement of the Safety Efficacy of Domestic
Food-Handling Practices. Risk Anal. 2006.
[10] Westaway, M. S., Vijoen, E. Health and Hygiene
Knowledge, Attitudes and Behaviour. Health
Place, 2000.
[11] Foster, G. M., Kaferstein F.K.. Food Safety and
the Behavioural Sciences. Soc. Sci. Med 1995.
[12] Iranjia-Hurdoyal, S., Latouche, C. Factors
Affecting Microbial Load and Profile of Potential
Pathogens and Food Spoilage Bacteria from
Household Kitchen Tables, Hindawi Publishing
Corporation Canadian Journal of Infectious
Diseases and Medical Microbiology, 2016.
[13] Milas, G. Research Methods in Psychology and
Other Social Sciences. Jastrebarsko: Naklada Slap,
2005.
[14] Ksenija Dumičić, Berislav Žmuk, Internet User
Characteristics in Croatia and Representativeness
of Internet Surveys, Proceedings of the Faculty of
Economics and Business in Zagreb, year 7, br.2,
2009.
[15] Association, A.P. Standards for Educational and
Psychological Testing, A.P. Association, Editor.
Washington DC, 1992.
[16] Nunnally, J.C. and I.H. Bernstein, Psychometric
Theory. 3rd ed. McGraw-Hill Series in
Psychology. New York, 1994.
[17] Cronbach, L.J. The Dependability of Behavioral
Measurements: Theory of Generalizability for
Scores and Profiles. 1972.
[18] Carol Byrd-Bredbenner, Virginia Wheatley,
Donald Schaffner, Christine Bruhn, Lydia Blalock,
and Jaclyn Maurer,. Development and
Implementation of a Food Safety Knowledge
Instrument, Food Science Education Research, Vol
6, 2017.
[19] Paul T. Brown. Food Safety Knowledge of
Undergraduate Nutrition Majors vs. Hospitality
Management majors, M.S., Kent State University,
2017.
[20] Elizabeth C. Redmond, Christopher J. Griffith,
Consumer Food Handling in the Home: A Review
of Food Safety Studies, Journal of Food
Protection, Vol. 66, No. 1, 2003.
[21] Elisa Langiano, Maria Ferrara, Liana Lanni,
Viviana Viscardi, Angela Marie Abbatecola,
Elisabetta De Vito, Food Safety at Home:
Knowledge and Practices of Consumers, Journal
Public Health, 2012.
[22] Ayaz, W.O., Priyadarshini, A., Jaiswal, A.K. Food
Safety Knowledge and Practices Among Saudi
Mothers. Foods, 2018.
[23] Nora A. Moreb, Anushree Priyadarshini, Amit K.
Jaiswal. Knowledge of Food Safety and Food
Handling Practices Amongst Food Handlers in the
Republic of Ireland, Food Control, 2017.
[24] Naureen Naeem, Shahid Raza, Hira Mubeen,
Shoaib Ahmad Siddiqui, Raana Khokhar, Food
Safety Knowledge, Attitude, and food Handling
Practices of Household Women in Lahore, Journal
of Food Safety, 2018.
[25] Nevin Sanlier, The Knowledge and Practice of
Food Safety by Young and Adult Consumers,
Food Control, June 2009.
UIJRT | United International Journal for Research & Technology | Volume 01, Issue 03, 2019
All rights are reserved by UIJRT.COM. 32
[26] Croatian Food Safety Agency, European
Commission. Food Safety in My Home European
Union Policy: Food Safety, Belgium, 2014.
[27] World Health Organization, Department of Food
Safety, Zoonoses and Foodborne Diseases, Five
Keys to Safe Food - A Guide, Geneva, 2006.
[28] Food and Agriculture Organization of the United
Nations/World Health Organization Guidelines on
the Application of General Principles of Food
Hygiene to the Control of Viruses in. Food, 2008.
[29] C. H. Lawshe, A Quantitative Approach to
Content Validity, Phrsonnhl Psychoi.Ogy, 1975.
[30] Cronbach, L.J., Coefficient Alpha and the Internal
Structure of Tests. Psychometrika, 1951.37
[31] Lynn, M. Determination and Quantification of
Content Validity. Nursing Research, 1986.
[32] Tilden, V., Nelson, C., May, B. Use of Qualitative
Methods to Enhance Content Validity. Nursing
Research, 1990.
[33] Gregory E. Gilbert, Susan Prion. Making Sense of
Methods and Measurement: Lawshe’s Content
Validity Index, Clinical Simulation in Nursing,
2016.
[34] Fatemeh Esfarjani, Hedayat Hosseini, Fatemeh
Mohammadi Nasrabadi, Alireza Abadi, Roshanak
Roustaee, Haleh Alikhanian, Marjan Khalafi,
Mohammad Farhad Kiaee, Ramin Khaksar,
Development of a Home Food Safety
Questionnaire Based on the PRECEDE Model:
Targeting Iranian Women, Journal of Food
Protection, Vol. 79, No. 12, 2016.
[35] Davis L.L. Instrument Review: Getting the Most
From a Panel of Experts. Applied 1'iursing
Research , 1992.
[36] Milica T. Zeković, Validation of a questionnaire
for the evaluation of dietary folate intake and
analysis of knowledge, attitudes and practices of
the pharmaceutical team regarding counseling
women of reproductive age about adequate
nutrient intake, University of Belgrade, Faculty of
Pharmacy, Doctoral dissertation, Belgrade, 2018.
[37] Whitley, B. E. Principle of Research in
Behavioural Science. McGraw-Hill 2002.
[38] Robinson, C. C., Mandelco, B., Olsen, S. F., &
Hart, C. H. The Parenting Styles and Dimensions
Questionaire (PSQD), 2001.
[39] Lance, C.E., Butts, M.M., Michels, L.C, The
Sources of Four Commonly Reported Cutoff
Criteria: What Did They Really Say?
Organizational Research Methods, 2006.
[40] Nik Rosmawati NH, Wan Manan WM, Noor Izani
NJ, Nik Nurain NH, Validity and Reliability of
Food Safety Knowledge and Practices
Questionnaire among Food Handlers, Health and
the Environment Journal, 2015.
[41] Bautista, R.E.D., Rundle-Gonzalez, V., Awad,
R.G. and Erwin, P.A. Determining the Coping
Strategies of Individuals with Epilepsy. Epilepsy
& Behavior, 27(2): 2013. 286-291.

Development and Validation of the Household Food Safety Questionnaire

  • 1.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 24 Development and Validation of the Household Food Safety Questionnaire Daniel Maestro*1,2,3, Arzija Pašalić2,3, Sabina Šegalo4, Amar Žilić3,5 *1 Institute of Public Health for Federation of Bosnia and Herzegovina, Bosnia & Herzeegovina 2 Faculty of Health Studies, University of Sarajevo, Bosnia and Herzegovina 3 Association for Sanitary Engineering for Bosnia and Hercegovina, Bosnia and Herzegovina 4 Faculty of Medicine, University of Sarajevo, Bosnia and Herzegovina 5 Federal Inspection Administration, Bosnia and Herzegovina [email protected] Abstract— Health problems associated with the consumption of foods that do not meet the hygiene and epidemiological standards are not of recent date and have been occurring continuously throughout the history of human existence. The incidence of food poisoning and foodborne transmissible diseases is three times more common in-home kitchens and households. Restaurant poisoning generally involves a larger number of people, whereas a home-based one involves individuals or a small number of people, so its likelihood of identification by the competent authority or public health organizations and services are significantly smaller. The development of the household food safety questionnaire (HFSQ) for the general population went through five phases (preparation of the questionnaire, distribution of the questionnaire to panelists and then the respondents, statistical analysis and the formation of the final version of the validated questionnaire). A total of 58 particles that formed the basis of the questionnaire were divided into four segments: demographic (10), knowledge (17), opinion (14) and food safety practice (17). Overall, the validity of the questionnaire in examining practice, knowledge and attitude was determined with Cronbach's alpha = 0.842. The total number of particles adequate for the questionnaire is 29 questions. This questionnaire is a good instrument for assessing the knowledge, attitudes and practices of food handlers in their households. Keywords— food safety, households, questionnaire, development and validation. I. INTRODUCTION 1.1 Food Safety in Households Health problems associated with the consumption of foods that do not meet the hygiene and epidemiological standards are not of recent date and have been occurring continuously throughout the history of human existence. More than 95% of cases of foodborne transmissible diseases (FTD) are characterized by sporadic occurrence [1]. According to World Health Organization (WHO) reports, the actual prevalence is 300-350 times the number of reported cases [2]. An estimated 1.8 million people die annually from diarrheal diseases, with the majority attributed to contaminated water or food [3]. Although governments around the world are making great efforts to improve the health safety of accessible food [1], the emergence of FTD remains a significant public health problem in all countries, regardless of the degree of economic development. Borneff and associates believe that the incidence of food poisoning and FTD is three times more common in-home kitchens and households. [4] Restaurant poisoning generally involves a larger number of people, whereas a home-based one involves individuals or a small number of people, so its likelihood of identification by the competent authority or public health organizations and services are significantly smaller [5]. Although experts generally agree that households are the primary places where FTD occurs, consumers do not consider their homes a risky place [6]. Unlike restaurants, home kitchens are multifunctional areas where a variety of activities take place. Pets, items not exclusively used for food preparation, dirty laundry, houseplants and the likes are commonly found in kitchens. Sinks are used for washing hands, various objects, dishes, clothes, children and pets, as well as wetting and soaking cloths. Dirty dishes can be easily stacked together with clean ones on kitchen countertops [7] Currently, most purchased foods are considered safe to consume, however, there is a need for consumers to properly handle, prepare and store household foods [8]. Inadequate handwashing, food handling and preparation, short cooking times and long storage under inadequate conditions are considered to be a cause of significant FTD in households, which can lead to proliferation of microorganisms [9]. Proper food preparation can prevent most FTD s [3], so research into the knowledge, attitudes and practices of food handlers is essential to formulate and design programs
  • 2.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 25 to promote and educate on food safety in households [10]. It is only when there is knowledge of food safety attitudes and practices that it is possible to plan effective strategies to encourage and reinforce desirable food handling behaviors and discourage those that are inappropriate [11]. Recognizing the need to study the sources of infection and the possible causes of FTD in households [12], our study is primarily directed in this direction. In Bosnia and Herzegovina, according to the available literature, no research based on testing and research into household food safety knowledge, practices and attitudes has been conducted, so we designed and developed a questionnaire for the purpose of the survey and evaluated its validity using adequate statistical methods. 1.2 Survey Questionnaire Every scientific research is based on some form of measurement. Like all other measuring instruments, questionnaires and the answers they collect do not have value in themselves, but solely as a way of measuring the subjective phenomenon they seek to capture [13]. The name of the questionnaire refers to a set of procedures by which data obtained from respondents is collected and analyzed, which provide details about the behavior, attitude, opinion, knowledge, intention, interests of the respondents. They are used for official statistics, various surveys and public / market surveys. The basic aim of the questionnaire must be clearly defined, therefore, when designing the questionnaire, it is necessary to study the interdependence of the variables that occur in the research goals and to be guided by asking questions that provide the basis for providing important data on multiple variables [14]. A good questionnaire is characterized by a high correspondence between the response received and the measurement object and the consistency of the data in different comparable situations. These characteristics belong to two basic metric features: validity and reliability. Instruments without adequate metric characteristics lead to insufficiently accurate estimates of the state of affairs which significantly diminishes their useful value [13]. When we talk about the relation between reality and the picture of reality that we get by using a questionnaire, we come to the concept of validity of the questionnaire. Validity is, in the broadest sense, related to the question: "Do we really measure what we think we are measuring?" This term refers to the appropriateness, meaningfulness and usefulness of conclusions and interpretations derived from test results [15]. Content validity is the extent to which the content of the questionnaire is thematically tailored to the area it should cover. Criterion validity can be defined as the relationship between the applied questionnaire and the result of another known variable, while constructive validity is the degree to which a questionnaire result indicates a particular trait [16]. Equally important is the question of the reliability of the questionnaire, and it relates to the consistency of the respondent's answers to the same or similar question. The reliability determinations are different - some emphasize measurement accuracy, second consistency, and third the absence of measurement error [17]. The terms validity and reliability often overlap. A non- reliable answer may not be valid either, but a reliable answer may not always be valid. For the credibility of the results, it is essential that the questionnaire as a whole is both reliable and valid. This makes it easier to further equalize the measurement conditions for each individual subject, and the objectivity of the measurement procedure. The reliability and validity data of the test provide insight into the structure of each measuring instrument and provide assurance that the measuring instrument really measures the appearance / trait we want to measure and that the results obtained reflect the real characteristics of the subjects. Validity, with reliability, is the metric characteristic that receives the most attention and is considered to be the most important characteristics of instruments used in the public health and social science [13]. II. MATERIALS AND METHODS The development of the household food safety questionnaire (HFSQ) for the general population went through five phases (preparation of the questionnaire, distribution of the questionnaire to panelists and then the respondents, statistical analysis and the formation of the final version of the validated questionnaire). The preparation of a pool of potential questions entailed detailed insight into the available literature and studies. Part of the question was derived in part from similar studies [18, 19, 20. 21, 22, 23, 24. 25], and additional particles were formed in accordance with recommendations from the household food safety literature [26, 27, 28] A total of 58 particles that formed the basis of the questionnaire were divided into four segments: demographic (10), knowledge (17), opinion (14) and food safety practice (17). Demographic issues included the collection of data on gender, age of respondents with a particular focus on persons under 18 and over 65, educational attainment, marital status and number of household members, total monthly household
  • 3.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 26 income, and frequency of meal preparation in households. The answers to the questions in the remaining segments were mostly formed on the Likert scale with a series of statements devoted to different aspects of knowledge, attitude and practice, with the respondents being tasked with expressing the degree of agreement or disagreement for each individual statement on a four-step scale in the 4- 1 (I totally agree, I agree, I don't know, I disagree). Due to the complexity of the questions, individual items were offered answer options, where the respondents, depending on their knowledge, attitude and practice, had to choose one that they thought was correct. In the second phase, ten experts in the field were selected who were willing to participate as panelists. They were asked to clearly state their ratings of the importance of including each particle in the questionnaire. In a specially formed panel for panelists / experts in the field, each particle is offered answers: necessary, not necessary, but may be useful and not necessary. Based on their results, Content Validity Ratio (CVR) was calculated using the Lawshe method [29]. The third phase involved the distribution of the questionnaire of the respondents by random selection method. Of the 106 respondents who participated in the pilot survey, six were excluded because of incomplete answers. The results of 100 subjects were included in the study and analyzed quantitatively and qualitatively. For the purpose of validating the questionnaire, Cronbach's alpha was calculated, which measured the internal consistency of each factor and its total value. Factor analysis was also performed, tested using Bartlett's test of sphericity, and determined by Kaiser- Meyer-Olkin (KMO) as an indicator of sample adequacy. A higher Cronbach's alpha coefficient indicates greater reliability, that is, attributes of the same factor measure the same phenomenon. Cronbach's alpha does not have strictly established values that confirm that a group of questions is consistent and usable for examining a particular topic, but a value of 0.7 is taken as the most commonly used distinction. The analysis is considered justified if the Bartlett test is significant p <0.05, because then there is confirmation of the existence of correlations at least among some variables. The KMO value is a measure for quantifying the correlation among variables, and values> 0.8 indicate very strong correlation, 0.7-0.8 strong, 0.6-0.7 medium strong, 0.5-0.6 weak, while smaller values indicate the inadequacy of variables for factor analysis [30] III. RESULTS AND DISCUSSIONS This is the first study to measure the validity and reliability of HFSQ, which contained a diverse and broad range of particles to evaluate knowledge, attitudes, and practices. To evaluate the validity of the content of the questionnaire, a CVR (Content Validity Ratio) was calculated, which tells us the validity of the individual particles and the CVI (Content Validity Index), which assesses the validity of the entire questionnaire. For all three segments of USHD - food safety knowledge, food safety attitude and food safety practice, the CVR of each particle and the total CVI of each segment were calculated. In addition to analyzing the validity of each segment, the CVI of the entire HFSQ was also calculated. The panelists' scores were used to calculate the CVR for each questionnaire particle (N = 48) using the Lawshe method (33): where n is the number of panelists who identify a particle as "required" and N is the total number of panelists (N / 2 is half the total number of panelists). A CVR value <0.62 was the exclusive factor, that is, if 2 out of 10 experts found the question unnecessary, the same was removed from further analyzes. The number of experts recommended when determining content validity in areas where it is difficult to find experts is three, and as a rule a panel of 5-10 experts / panelists is preferred. The use of over 10 experts / panelists is in principle unnecessary [31] As a simple measure, CVI represents the mean of all CVR values of those particles that meet the CVR threshold of 0.62 and were retained in the questionnaire. All CVI values> 0.70 are considered acceptable [32]. In many situations, it is more efficient to present results on the total CVI than to display the CVR values of each particle [33]. Two questions were excluded from the analysis of the validity of knowledge questions, Z12 - When buying groceries when is the best time to buy frozen food? with CVR = 0.2 and Z13 - What is the optimum temperature for frozen food storage? with CVR = 0.6. Overall CVI of the segment - food safety knowledge, after excluding these two questions is 0.96 indicates significant
  • 4.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 27 validity of the content of the question, and the value exceeds the preferred CVI value = 0.8 [34]. The analysis of attitude questions excluded two questions: S8 - If, when purchasing groceries, on the declaration we check the date indicated under "best use until" we reduce the possibility of food poisoning (CVR = 0.6) and S9 - food poisoning is not a "normal" life event (CVR = 0). Overall CVI of the Segment - Food Safety Attitude, after excluding these two questions is 0.95, which indicates the significant validity of the content of the questions being tested. The value exceeds the preferred CVI value of 0.8 [34, 35]. Analyzing the practice segment, you found that question P6 - Do you sometimes clean your chicken with paper towels instead of washing it in the kitchen sink? CVR was 0.6 and was excluded from the questionnaire, so the total CVI of this segment is 1, indicating significant validity of the content of the question. A CVI of 0.95 for the whole questionnaire indicates a high degree of validity [29]. In a similar study conducted in Iran [34] after an expert evaluation, the total CVI of the questionnaire was 0.75, indicating that HFSQ has a high degree of representativeness and clarity, thus confirming its content validity. In order to test the reliability of USHD for assessing household food safety knowledge, attitudes, and practices, the value of the Cronbach's alpha coefficient was determined. A higher coefficient value indicates greater reliability, that is, the attributes of the same factor really measure the same phenomenon [36]. An analysis of the internal consistency of the questions was conducted for each segment separately and for the whole questionnaire. After the first analysis of the knowledge questions remaining after the analysis by the experts, a total of 15 questions were given for completion in the pilot study. Analyzing the answers to them, it was found that certain questions reduce the homogeneity, and therefore the validity of the questionnaire. Questions excluded from the questionnaire to increase Cronbach's alpha are: Z11, Z14, Z16, and Z17. After excluding the question, Cronbach's alpha was found to be 0.745. The ANOVA test found a significant association between response and subjects at p <0.001. Knowledge questions were also subjected to a sample adequacy analysis, which determined a Kaiser-Meyer-Olkin correlation of KMO = 0.782, factor analysis was justified by a significance of p <0.001. Table 1. Analysis of quality and validity of knowledge questions Question CVR Cronbach's alpha Status Z1 Harmful microbes that are not visible to the naked eye can lead to food poisoning 1,00 0,588 Remains Z2 If we eat meat that is raw or semi-cooked, we are at a high risk of food poisoning 1,00 0,562 Remains Z3 Non-hygienic practices are a source of microorganisms that can cause food pollution 1,00 0,598 Remains Z4 If we eat cooked food stored at room temperature for more than 4 hours, we are at high risk of food poisoning 1,00 0,583 Remains Z5 Direct contact of unwashed hands and ready-to-eat food leads to bacterial contamination of food 0,80 0,574 Remains Z6 Defrosting frozen meat on the lower shelf of the refrigerator is the correct way of defrosting 1.00 0,622 Remains Z7 Eating half-cooked eggs (soft yolk) can cause food poisoning that often ends in hospitalization 1.00 0,560 Remains Z8 Insects such as cockroaches and flies can transmit bacteria that lead to food poisoning 1.00 0,595 Remains Z9 Cooked food residues should be reheated, thoroughly heated / boiled before consumption 1,00 0,571 Remains Z10 In addition to 0,80 0,570 Remains
  • 5.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 28 Question CVR Cronbach's alpha Status bacteria, fungi and viruses can also cause foodborne diseases Z11 Do you think that bacteria in food can be destroyed by freezing at -18 ° C? 0,80 0,630 Rejected Z12 When buying groceries, when is the best time to buy frozen food? 0,20 - Rejected Z13 What is the optimal storage temperature for frozen foods? 0,60 - Rejected Z14 What to do with freshly cooked food consumed 3 hours later? 1,00 0,675 Rejected Z15 Do you think it is correct to defrost meat again? 1,00 0,630 Remains Z16 Which of the following is the least certain way of defrosting raw meat? 1,00 0,669 Rejected Z17 How long can cooked food be stored in the refrigerator? 1,00 0,629 Rejected Cronbachs alpha after excluding rejected questions (n=11) 0,745 The pilot study included 12 attitude particles. Following the responses provided by the respondents, Cronbach's alpha = 0.552 was determined, which did not indicate the reliability of the attitude assessment questions. After standardization, the Cronbach alpha value was 0.580, but still indicated poor validity of attitude questions. Subsequent analyzes based on the Cronbach's alpha value of each question excluded three questions - S11, S12, S13. By excluding these questions, the degree of certainty of the attitude questions that they truly represent the real world has increased to the degree of Cronbach's alpha = 0.714. The number of questions was reduced to 9, but their reliability increased. The correlation matrix established the connection between most questions. The very existence of the correlation of the answers between the questions, and their heterogeneity, indicates that there are more trends in the answers. A total correlation was found within the question with a mean of 0.217 and a range of -0.02-0.460, which classifies the same into weak and moderately strong correlations. Question validity would be increased by excluding question S14, but not significantly. An ANOVA with the attitude question test found a significant association between questions at p <0.001. Attitude questions were also subjected to a sample adequacy analysis, which established a Kaiser-Meyer-Olkin correlation of KMO = 0.732, Factor analysis was justified by a significance of p <0.001. Table 2. Analysis of the quality and validity of attitude questions Question CVR Cronbach's alpha Status S1 People who prepare food in their home can infect themselves and their families with foodborne bacteria 1,00 0,505 Remains S2 It is important to check the temperature of the refrigerator to prevent spoilage and food poisoning 1,00 0,500 Remains S3 Raw food should be kept separate from cooked food 1,00 0,471 Remains S4 Hair, jewelry, and long nails can contaminate food with bacteria 1,00 0,491 Remains S5 Raw vegetables and meat should not be cut using the same knife unless previously washed 1,00 0,525 Remains S6 Raw fruits and vegetables are safer to eat if washed with cold running water 0,80 0,510 Remains S7 If you wash your hands with liquid soap and warm water for 40- 45 seconds before handling food, it will be 1.00 0,496 Remains
  • 6.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 29 Question CVR Cronbach's alpha Status enough to get rid of the bacteria on your hands S8 If, when purchasing groceries, on the declaration we check the date indicated under "best use until" we reduce the possibility of food poisoning 0,60 - Rejected S9 Food poisoning is not a “normal” life event 0,00 - Rejected S10 In your opinion, what is the incidence of food poisoning due to improper household preparation? 1,00 0,487 Remains S11 In your opinion, what is the incidence of food poisoning in a restaurant compared to home-cooked meals? 1,00 0,593 Rejected S12 What do you think is right way to deal with leftover cooked food? 0,80 0,603 Rejected S13 What do you think is the best measure to prevent food poisoning? 1,00 0,592 Rejected S14 From the following two statements, which is closer to your view: a) Certain types of people have a higher risk of food poisoning, b) All people have the same risk of food poisoning 0,80 0,557 Remains Cronbachs alpha after excluding the rejected questions (n=9) 0,714 Based on the results of the analysis of practice questions by the experts in the pilot study, the practice was examined with 16 questions. After the results obtained, Cronbach's alpha = 0.510 was determined. The value indicated that the given group of questions had great heterogeneity and that as such it was not adequate to represent the reliable practice of the respondents. Subsequent analysis of the reliability of the questions to allow the examination of the respondents' practice revealed that it was necessary to exclude the questions: P9, P11, P12, P13, P14, P15, P16. After excluding them, a Cronbach's alpha of 0.683 was determined. Although not> 0.7, the Cronbach's alpha values do not have a strictly set benchmark, and the Cronbach's alpha value = 0.683 is consistent with the results of [37, 38, 39] suggesting that the given value can be used as a reference. After standardization of the data, Cronbach's alpha = 0.724 was determined. The correlation matrix determined the existence of a positive relationship between the answers to the questions. The correlation itself ranges from extremely weak: 0.046 between for P1 and P10 (classified as extremely weak), to 0.579 (moderately strong correlation) between P2 and P4. The maximum value of Cronbach's alpha was found to be set to practice questions, and standardization yielded Cronbach's alpha values = 0.724. Further exclusion of the questionnaire could not increase the value of the questionnaire reliability factor in examining respondents ’practices. Sample adequacy analysis showed a strong correlation of response and sample, and Bartlett's test of sphericity was p <0.001. The Kaiser-Meyer-Olkin analysis determined the degree of correlation KMO = 0.734. Table 3. Analysis of the quality and validity of practice questions Question CVR Cronbach's alpha Status P1 Do you avoid cooking and preparing food if you have open wounds on your hands? 1,00 0,517 Remains P2 Do you clean surfaces that come in contact with food before and after food preparation? 1,00 0,461 Remains P3 Do you use different planks for cutting raw meat and vegetables? 1,00 0,389 Remains P4 Do you wash your hands with soap and warm water before starting food preparation? 1,00 0,498 Remains
  • 7.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 30 P5 Do you defrost meat on the bottom shelf of the refrigerator? 1,00 0,472 Remains P6 Do you sometimes clean your chicken with paper towels instead of washing it in the kitchen sink? 0,60 - Rejected P7 Do you separate raw and cooked foods in your fridge? 1.00 0,460 Remains P8 Do you use a food thermometer to check that the food is cooked enough? 1.00 0,517 Remains P9 Do you leave hot cooked food at room temperature for more than 4 hours? 1.00 0,556 Rejected P10 How often do you disinfect your kitchen sink? 1.00 0,472 Remains P11 After cracking and opening raw eggs, what do you usually do? 1.00 0,496 Rejected P12 After handling raw meat, what do you usually do? 1.00 0,523 Rejected P13 If you pre-cook large quantities of food with meat additives, when do you put food in the refrigerator? 1.00 0,522 Rejected P14 How do you wash vegetables and fruits? 1.00 0,498 Rejected P15 If you have chopped or cut meat on the board and now want to cut the fruit, which of the following do you do? 1.00 0,421 Rejected P16 How do you wash your hands? 1.00 0,535 Rejected P17 In your opinion, in terms of food safety, for you and your 1.00 0,503 Remains household members. Cronbachs alpha after excluding the rejected questions (n=9) 0,683 Overall, the validity of the questionnaire in examining practice, knowledge and attitude was determined with Cronbach's alpha = 0.842. The total number of particles adequate for the questionnaire is 29 questions. The ANOVA test found a significance level of p <0.001. The result of the ANOVA test indicates that the questionnaire can be used to test the attitude, knowledge and practice of food safety in the subjects. The correlation matrix values in all segments were less than 0.85, indicating that there were no multicollinearity problems among the factors [40] In the aforementioned study, the Cronbach alpha coefficient for each factor ranged from 0.549-0.798, indicating satisfactory internal consistency criteria. Based on research by Bautista et al [41]. A Cronbach alpha of 0.5 or greater is considered an acceptable criterion for internal consistency in the questionnaire reliability analysis. In order to test the validity of factor analysis, Bartlett's test of sphericity was applied and the Kaiser-Meyer- Olkin (KMO) indicator of sample adequacy was determined. The analysis is considered justified if the Bartlett test is significant (p <0.05), because then there is confirmation of the existence of correlations at least among some variables. The Kaiser-Meyer-Olkin (KMO) sample adequacy indicator has values from 0 to 1 and is a measure for quantifying intercorrelations among variables. KMO values above 0.8 indicate a very strong correlation; values 0.7-0.8 strong correlation; values 0.6-0.7 moderate correlation; values 0.5-0.6 have a weak correlation, while values below 0.5 indicate the inadequacy of conducting factor analysis. In this study, a KMO of 0.6 was defined as the minimum acceptable. The strength of the relationship between HFSQ particles is very good and fulfills the criteria for factor analysis, as demonstrated by the high KMO (0.734) of the whole questionnaire and of each segment separately [36]. IV. CONCLUSIONS For the purpose of the research, a questionnaire was specifically created that was adapted to the research in the general population. The instrument was developed based on a thorough literature review with expert consultation. The Household Food Safety Questionnaire (HFSQ), which measures food safety
  • 8.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 31 knowledge, attitudes, and practices, met the psychometric criteria for the reliability and validity test. This questionnaire is a good instrument for assessing the knowledge, attitudes and practices of food handlers in their households. An insight into the available literature indicates that the study conducted was the first study of its type in Bosnia and Herzegovina, and may provide a starting point for future research and a scientific platform for considering measures and activities necessary to improve household food safety. REFERENCES [1] Food Standards Agency. Foodborne Disease: Developing a Strategy to Deliver the Agencies Targets. Paper no. FSA 00/05/02. Agenda item 4, 2000. [2] World Health Organization: Fact sheet No 237: Food Safety and Food Born Illnesses, Geneva 2006. [3] World Health Organization, Five keys to healty food, Geneva - Switzerland, 2008. [4] Borneff, J., R. Hassinger, J. Wittig, and R. Edenharder. Effective Hygienic Measurements in Households Today. Zentralbl. Bacteriol. 2008. [5] Worsfold, D., and C. J. Grif. th. Food Safety Behaviour in the Home. Br. Food J, 1997. [6] Maurer, J., Byrd-Bredbenner, C., Wheatley, V., Cottone, E., Clancy, M. Young Adults Report Better Handwashing Behaviors than they Actually Practice. Food Prot. Trend, 2008. [7] Byrd-Bredbenner, C., Maurer, J., Wheatley, V., Cottone, E., Clancy, M. Food Safety Hazards Lurk in the Kitchens of Young Adults. J. Food Prot. 2007. [8] Tomaszewska, M., Trafialek, J., Suebpongsang, P., Food Hygiene Knowledge and Practice of Consumers in Poland and in Thailand - A survey, Food Control Vol. 85, 2018. [9] Fischer, A., Frewer, L., Nauta, M. Toward Improving Food Safety in the Domestic Environment: A Multi-item Rasch Scale for the Measurement of the Safety Efficacy of Domestic Food-Handling Practices. Risk Anal. 2006. [10] Westaway, M. S., Vijoen, E. Health and Hygiene Knowledge, Attitudes and Behaviour. Health Place, 2000. [11] Foster, G. M., Kaferstein F.K.. Food Safety and the Behavioural Sciences. Soc. Sci. Med 1995. [12] Iranjia-Hurdoyal, S., Latouche, C. Factors Affecting Microbial Load and Profile of Potential Pathogens and Food Spoilage Bacteria from Household Kitchen Tables, Hindawi Publishing Corporation Canadian Journal of Infectious Diseases and Medical Microbiology, 2016. [13] Milas, G. Research Methods in Psychology and Other Social Sciences. Jastrebarsko: Naklada Slap, 2005. [14] Ksenija Dumičić, Berislav Žmuk, Internet User Characteristics in Croatia and Representativeness of Internet Surveys, Proceedings of the Faculty of Economics and Business in Zagreb, year 7, br.2, 2009. [15] Association, A.P. Standards for Educational and Psychological Testing, A.P. Association, Editor. Washington DC, 1992. [16] Nunnally, J.C. and I.H. Bernstein, Psychometric Theory. 3rd ed. McGraw-Hill Series in Psychology. New York, 1994. [17] Cronbach, L.J. The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. 1972. [18] Carol Byrd-Bredbenner, Virginia Wheatley, Donald Schaffner, Christine Bruhn, Lydia Blalock, and Jaclyn Maurer,. Development and Implementation of a Food Safety Knowledge Instrument, Food Science Education Research, Vol 6, 2017. [19] Paul T. Brown. Food Safety Knowledge of Undergraduate Nutrition Majors vs. Hospitality Management majors, M.S., Kent State University, 2017. [20] Elizabeth C. Redmond, Christopher J. Griffith, Consumer Food Handling in the Home: A Review of Food Safety Studies, Journal of Food Protection, Vol. 66, No. 1, 2003. [21] Elisa Langiano, Maria Ferrara, Liana Lanni, Viviana Viscardi, Angela Marie Abbatecola, Elisabetta De Vito, Food Safety at Home: Knowledge and Practices of Consumers, Journal Public Health, 2012. [22] Ayaz, W.O., Priyadarshini, A., Jaiswal, A.K. Food Safety Knowledge and Practices Among Saudi Mothers. Foods, 2018. [23] Nora A. Moreb, Anushree Priyadarshini, Amit K. Jaiswal. Knowledge of Food Safety and Food Handling Practices Amongst Food Handlers in the Republic of Ireland, Food Control, 2017. [24] Naureen Naeem, Shahid Raza, Hira Mubeen, Shoaib Ahmad Siddiqui, Raana Khokhar, Food Safety Knowledge, Attitude, and food Handling Practices of Household Women in Lahore, Journal of Food Safety, 2018. [25] Nevin Sanlier, The Knowledge and Practice of Food Safety by Young and Adult Consumers, Food Control, June 2009.
  • 9.
    UIJRT | UnitedInternational Journal for Research & Technology | Volume 01, Issue 03, 2019 All rights are reserved by UIJRT.COM. 32 [26] Croatian Food Safety Agency, European Commission. Food Safety in My Home European Union Policy: Food Safety, Belgium, 2014. [27] World Health Organization, Department of Food Safety, Zoonoses and Foodborne Diseases, Five Keys to Safe Food - A Guide, Geneva, 2006. [28] Food and Agriculture Organization of the United Nations/World Health Organization Guidelines on the Application of General Principles of Food Hygiene to the Control of Viruses in. Food, 2008. [29] C. H. Lawshe, A Quantitative Approach to Content Validity, Phrsonnhl Psychoi.Ogy, 1975. [30] Cronbach, L.J., Coefficient Alpha and the Internal Structure of Tests. Psychometrika, 1951.37 [31] Lynn, M. Determination and Quantification of Content Validity. Nursing Research, 1986. [32] Tilden, V., Nelson, C., May, B. Use of Qualitative Methods to Enhance Content Validity. Nursing Research, 1990. [33] Gregory E. Gilbert, Susan Prion. Making Sense of Methods and Measurement: Lawshe’s Content Validity Index, Clinical Simulation in Nursing, 2016. [34] Fatemeh Esfarjani, Hedayat Hosseini, Fatemeh Mohammadi Nasrabadi, Alireza Abadi, Roshanak Roustaee, Haleh Alikhanian, Marjan Khalafi, Mohammad Farhad Kiaee, Ramin Khaksar, Development of a Home Food Safety Questionnaire Based on the PRECEDE Model: Targeting Iranian Women, Journal of Food Protection, Vol. 79, No. 12, 2016. [35] Davis L.L. Instrument Review: Getting the Most From a Panel of Experts. Applied 1'iursing Research , 1992. [36] Milica T. Zeković, Validation of a questionnaire for the evaluation of dietary folate intake and analysis of knowledge, attitudes and practices of the pharmaceutical team regarding counseling women of reproductive age about adequate nutrient intake, University of Belgrade, Faculty of Pharmacy, Doctoral dissertation, Belgrade, 2018. [37] Whitley, B. E. Principle of Research in Behavioural Science. McGraw-Hill 2002. [38] Robinson, C. C., Mandelco, B., Olsen, S. F., & Hart, C. H. The Parenting Styles and Dimensions Questionaire (PSQD), 2001. [39] Lance, C.E., Butts, M.M., Michels, L.C, The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? Organizational Research Methods, 2006. [40] Nik Rosmawati NH, Wan Manan WM, Noor Izani NJ, Nik Nurain NH, Validity and Reliability of Food Safety Knowledge and Practices Questionnaire among Food Handlers, Health and the Environment Journal, 2015. [41] Bautista, R.E.D., Rundle-Gonzalez, V., Awad, R.G. and Erwin, P.A. Determining the Coping Strategies of Individuals with Epilepsy. Epilepsy & Behavior, 27(2): 2013. 286-291.