English

Name: Paula Andrea Melo Erazo

Mixed Method Research: Instruments, Validity, Reliability and Reporting Findings

NOTES:

1. INTRODUCTION
- Quantitative designs use tests and closed-ended questionnaires in order to analyze and
interpret the data.
- Qualitative methods use interviews, diaries, journals, classroom observations and open-
ended questionnaires to analyze and interpret the data.
- The researchers can obtain information through different procedures to heighten the
dependability and integrity of the dada and their interpretation.
- The researchers need to consider the validity and reliability of their data, they will be
described in detail.
- The final section exemplifies how the inquirers can write and report their study.
2. RESEARCH INSTRUMENTS
- The mixed method approaches qualitative and quantitative data, they are simultaneously
collected, analyzed and interpreted.
- The main research instruments used are questionnaire, interview and classroom
observation.
- It is believed that using different types of procedures for collecting data can augment the
validity and reliability of the data and their interpretation.
- The items of the questionnaires are mainly developed based on the research objectives
and research questions.
2.1. Questionnaires
- Questionnaires are one of the primary sources of obtaining data.
- The critical point is when the researcher should ensure that it is valid, reliable and
unambiguous.
- Questionnaires can appear in three types:
 Closed-ended (or structured) questionnaires
They provide the inquirer with quantitative data and open-ended questionnaires
with qualitative or text information.
They are more efficient because of their ease of analysis.
Usually those questionnaires are divided into seven basic question types: quantity
or information, category, list or multiple choice, scale, ranking, complex grid or
table, and open-ended.
 Open-ended (or unstructured) questionnaires
Questions will reflect better what the respondent wants to say.
Qualitative data indicates that its open nature made it more difficult to compare
debate and interview reports.
 Mix of closed-ended and open-ended questionnaires.
 Closed-ended and open-ended questions complement each other.
- Advantages of questionnaires
- They are efficient collecting data on a large-scale basis.
- They can be sent simultaneously to a great number of people.
- Anonymity makes responders to share information more easily.
- Disadvantages of questionnaires
- Sometimes the answers are inaccurate and questionable.
- Ambiguity and unclearness of some questions might lead to inaccurate and unrelated
responses.
- Wording of the questions might affect the answers.
- Shipping ways
- When the respondents are not within the reach of the evaluator, they might be sent by
post or internet.
- By telephone instead of physical way the respondent is rather obliged to answer the
questions.
- Face-to-face, the respondents are relatively compelled to answer the questions.
- Administering questionnaires
- Self-administered: mailed out to respondents.
 Respondents usually hardly return the questionnaire.
 If any misunderstandings arise, or if there be any vague questions, the researcher is
not available to clarify them.
 The researcher has no idea how the questions were answered.
- Administered questionnaire: administered to the groups of individuals all at one time and
place.
 The return rate is high, the researcher is present to explain any unclear questions and
he/she knows the conditions under which the questionnaires were filled out.
2.2. Interviews
- Used to collect qualitative data.
- Information is recollected from knowledgeable informants.
- It is a key to understand what and how people perceive and interpret the world around
them, answers will become accessible to interpretation.
- Goal oriented conversation.
- The important issue in any interview encounter is the type of the questions that are asked.
- The different types of questions allow the researchers to deal more explicitly with the
presuppositions.
- Technical jargons can be used in the scientific circles and the interviewer should use
ordinary everyday language during the interview process.
- Yes-or-no questions hardly provide any useful and relevant information.
- The interviewer should explain to the respondents the purpose of the interview.
- Ways of identifying suitable informants.
 The researcher can find a knowledgeable person based on his/her experience and on-
site observations.
 Asking other participants to introduce a reliable informant.
 The introduced person can be asked to introduce other key informants in the field.
- Interview can be conducted in two forms:
 Person-to-person.
  Group or collective formats.
- Strengths
 Good for measuring attitudes and most other content of interest.
 Can provide in-depth information.
 Allow good interpretative validity.
 Very quick turnaround for telephone interviews.
 Moderately high measurement validity for well-constructed and well-tested interview
protocols.
 Relatively high response rates often attainable.
 Useful for exploration and confirmation.
- Weaknesses
 In-person interviews are expensive and time-consuming.
 Perceived anonymity by respondents is possibly low.
 Data analysis sometimes is time-consuming for open-ended items.
- Types of interviews.
 Informal conversation interview
Conducted without any predetermined questions and without any order.
Exploratory in nature and indeed difficult for the new researchers.
 Interview guide approach
The topics and questions are specified and they can be reworded in any sequence
based on the situation.
Collected information can be compared and contrasted.
Data collection is rather systematic and conversational.
 Structured open-ended interview
The questions are predetermined with almost fixed order.
 Closed, fixed response interviews
The respondent only answers the interviewer’s questions in a fixed format and
order.
This form of interview is too rigid and predetermined questions may not allow
perspectives and understandings of the world.
Mechanistic.
- Interviews based on the amount of control over the conversation.
 Structured
 Semi structured
It is flexible and allows the interviewee to provide more information.
It is a moderate form in which a great amount of data can be elicited from the
interviewee.
 Unstructured.
- Important points during the interview process
 The interviewees should be provided with scope to express their opinions.
 The interviewer should be nonjudgmental and neutral during the interview.
 The interviewer should be respectful, natural and nonthreatening.
 The interviewer should create rapport.
 The interviewer should not interrupt.
- Recording
 Record the interview.
 Take notes: write the important points during the interview process.
 Write down every detail of the interview immediately: not recommended because it is
difficult and incorrect way of collecting information.
2.3. Classroom Observations
- Observation is a preplanned research tool which is carried out purposefully to serve
research questions and objectives.
- It is a kind of data triangulation in order to substantiate the findings.
- Represent a natural field obtain contextual factors.
- Steps
 Determine the setting.
 Identify what to be documented.
 Obtain a general presentation of the field.
 Conduct focused and selected observations which are pertinent to the research
objectives and questions.
- Methods
 Nonparticipant
The observer only watches and records the classroom activities without any
involvement.
 Participant.
The observers enter the classroom and integrate with the students directly.
The observer becomes too much involved in the classroom processes which
consequently cannot concentrate on selected behaviors and activities.
The crucial issue is what to be observed.
the researcher must be selective and choose randomly in order to maintain the
representativeness criterion.
- Types of observations
 Narrow focus
Usually concentrates on a single element and broad focus on an overall picture of
what is happening in the classroom.
Based on observer’s goal, the research objectives, research questions, the
inquirer’s field of study, practical issues, researcher’s personal impressions, the
topic under study and so on.
 Broad focus observations.
- Categories of classroom activities
 student activities
 sources of input to students
 student behavior
 the distribution of classroom time
- The amount of time spent on observation is essential: depends on the research objectives
and questions. The more time spent observing, the better and reliable information can be
obtained.
- The observers must try hard to remain nonjudgmental and control their bases.
- The inquirers should prepare a form to put the information: design a data, one of the
advantages of this checklist is that it is fairly detailed and the observer can easily tick the
appropriate boxes according to his/her observations. This method of collecting
information is used to complement field notes.
3. VALIDITY
- Validity is trustworthiness, utility and dependability for the evaluator and the different
stakeholders.
- The researcher and research participants must attempt to build validity into the different
phases of the research from data collection through to data analysis and interpretation.
- Validity is concerned with whether our research is believable and true and whether it is
evaluating what it is supposed or purports to evaluate.