Chapter 6

Descriptive Designs:
Survey and Observation
 Focus
• classification of survey methods
• criteria for evaluating survey methods
• types of observation methods
• observation versus survey methods
• use of the Internet
 Survey Method
• uses a structured questionnaire
– formal instrument
– questions are asked in a pre-arranged order
• advantages: simple to administer, yields
reliable data, coding and analysis are simple
• disadvantages: respondents may be unwilling
or unable to answer, wording of questions is
not easy
Modes of Administering Surveys
• telephone interviews - traditional, CATI
• personal interviews - in-home, mall
intercept, CAPI
• mail interviews - traditional, mail panels
• electronic - e-mail or Internet
Traditional Telephone Interviews
• phone a sample of respondents and ask a
series of questions
• use a paper-pencil questionnaire
• nationwide telephone interviewing from a
central location has been made feasible with
WATS service
 Computer-assisted telephone
interviews
• uses a computerized questionnaire
• administered to respondents over the telephone
• record answers directly into the computer
• only one question at a time appears on the
computer screen
• questionnaire can be personalized based on
responses
• feasible to prepare interim and update reports
 Personal in-home interviews
• respondents are interviewed in their homes
• not in use today because of high cost,
although still popular in other countries
• used by syndicated firms
 Mall-intercept personal
interviews
• mall shoppers are intercepted and brought
to test facilities in the mall
• is more efficient
• several hundred mall research facilities
• useful when the respondent has to see,
handle or consume the product before they
can give meaningful information
 Computer-assisted personal
interviews
• respondent sits in front of a computer
terminal and answers a questionnaire on the
CRT screen
• increases the involvement and interest level
of the respondent
• interviewer is usually present to guide
 Mail interview
• questionnaires are mailed to pre-selected
respondents
• send the whole package - cover letter,
return envelope, incentive
• no verbal interaction between the researcher
and the respondent
• uses mailing lists (appropriate and accurate)
 Mail Panels
• comprises of a large, nationally representative
sample of households that has agreed to
participate in periodic mail questionnaires,
product tests and telephone surveys
• marketing research companies maintain these
panels
• are very useful for implementing longitudinal
designs
 E-Mail Interviews
• generate a list of e-mail addresses
• survey is written within the body of the
message
• uses pure text (ASCII) to represent the
questionnaire
• data entry is required when survey is received
• somewhat limited: skip patterns,
randomization, length
 Internet Interviews
• uses HTML
• survey is posted on a Web site
• respondents recruitment: on-line, mail or
telephone, visitors
• several advantages: can prevent more than one
response, skip patterns can be programmed,
validate responses, complex survey stimuli can
be built
 Comparative Evaluation of
Survey Methods
• Table 6.2 - p. 188
 Flexibility of Data Collection
• extent to which the respondent can interact with
the interviewer and the survey questionnaire
• highest flexibility: personal interview methods
– face-to-face
– administer complex questionnaires
– explain/clarify difficult questions
– utilize unstructured questionnaires
 Continued..
• Moderate flexibility: traditional telephone interview
method
– more difficult to use unstructured techniques, ask complex
questions or obtain in-depth answers to open-ended questions
• somewhat higher flexibility: CATI, CAPI, Internet surveys
– use of an interactive mode
– personalize and use skip patterns easily
• Low flexibility: mail surveys, mail panels, e-mail surveys
– no interaction possibility
 Diversity of Questions
• depends on the degree of interaction with the
interviewer and the respondent’s ability to actually
see the questions
• most diversity: in-home, mall intercept, CAPI
• moderate to high: Internet surveys
• less diversity: mail surveys, mail panels, e-mail
surveys
• least diversity: traditional telephone and CATI
– the respondent cannot see the questions
 Use of Physical Stimuli
• when you want respondents to view a
stimuli (e.g., product, ad) and react to it
• personal interviews conducted at central
locations (mall, CAPI) are the best
• moderate: mail surveys, mail panels
• limited: telephone methods, e-mail surveys
 Sample Control
• ability to direct the survey to the right person and
get that person’s cooperation
• Best: in-home interviews
– difficult to find respondents during the day
– safety concerns
• Moderate: mall intercept interviews
– interviewer has control over which person to intercept
– but limited to mall shoppers (frequent)
 Continued ..
• Moderate to high: telephone methods
– offer access to geographically-dispersed respondents
and hard-to-reach areas
– problems in using a telephone directory as a
sampling frame (use of RDD)
• Low: mail survey
– access may not be a problem, but getting cooperation
would be
– mail panel may be better
 Continued ..
• very low: Internet survey research
– general population is a poor fit
– ability to meet quotas restricted
– may not control people taking a survey multiple
times
 Control of the Data Collection
Environment
• by the researcher
• greatest control: personal interviews
conducted at central locations
• moderate to high - in-home personal
interviews
• moderate - telephone methods
• little control - all other methods
 Control of Field Force
• eliminates field force problems: mail, e-
mail and Internet surveys
• moderate control: telephone methods, mall
intercept and CAPI
– supervision is possible
• low control: in-home personal interviews
(e.g., Census Bureau)
 Quantity of Data
• Largest: in-home personal interviews (up to 75
minutes)
– social relationship
– home environment
– less effort for the respondent
• Large: mail panels
• Moderate: mall intercept and CAPI (< 30 minutes)
• Moderate: mail surveys
• Most limited: telephone methods (< 15 minutes)
 Response Rate

• % of total attempted interviews that are

completed
• Highest: personal (> 80%)
• Moderate: telephone (60-80%)
• Poor: Mail surveys (< 15%)
• how to improve response rates ??
 Perceived Anonymity

• High: mail surveys, mail panels, Internet

surveys
• Low: personal interviews
• Moderate: telephone methods, e-mail
survey
 Social Desirability
• tendency of the respondent to give answers that
are socially acceptable, whether or not they are
true
• least susceptible: mail surveys, panels, Internet
surveys
• moderate: telephone methods
• less moderate: e-mail
• most susceptible: personal methods
 Interviewer Bias

• selection of respondents, manner of asking

questions, and recording answers
• extent of interviewer’s role is important
• high: In-home and mall intercept interviews
• moderate: telephone methods
• low: computer-assisted interviews
• free of bias: mail methods, Internet surveys
 Speed
• fastest: Internet survey, e-mail survey
• next: telephone methods
• next: mall-intercept and computer-assisted
interviews
• slower: in-home personal interviews (dead time)
• slowest: mail surveys
• somewhat faster: mail panels
 Cost

• Lowest: Internet survey

• Highest: in-home personal interviews
 Selection of Survey Method

• no method is superior on all criteria

• will depend on:
– information requirements
– budgetary constraints
– respondent characteristics
• can use a combination of methods
 Observation Methods
• recording the behavioral patterns of people,
objects, and events in a systematic manner to
obtain information about the phenomenon of
interest
• no direct contact with the stimuli being
observed
• information recorded as the events occur or
from past records
 Structured Observation
• specify in detail what is to be observed
• how the measurements are to be recorded
– reduces observer bias
– enhances data reliability
– useful in conclusive research when the problem is well
defined
 Disguised Observation
• respondents are unaware that they are being
observed
• enables them to behave naturally
 Natural Observation
• observing behavior as it takes place in the
environment
• contrived - observe in an artificial environment
• accuracy versus cost of waiting
 Personal Observation
• observers record the phenomenon as it occurs
• no attempt to control or manipulate
• e.g. traffic flows in a store-- layout decisions
• e.g. mystery shopper, sales call monitoring
 Mechanical Observation
• mechanical devices record the phenomenon of interest
• typically used for continuous recording of ongoing behavior
• e.g. Audimeter, people meters, traffic counters, on-site
cameras, UPC system
• may also be used in situations that require respondent
cooperation
• e.g., eye-tracking monitors, pupilometers,
psychogalvanometers, response latency devices
• assumption: physiological reactions are associated with
cognitive and affective responses
 Audit
• examine physical records or perform inventory
analysis
• retail and wholesale audits - discussed earlier
• pantry audit
 Content Analysis
• objective, systematic, and quantitative description
of the manifest content of a communication
• unit of analysis: words, characters, themes, space
and time measures, topics
• classify each unit into categories
• communication: ads, newspaper articles, TV/radio
programs, 10k reports
 Survey Vs Observation Methods
• actual behavior vs reports of behavior: reporting
bias, social desirability bias (+)
• behaviors that the respondent is unaware or unable
to communicate (+)
• reasons for the behavior (-)
• observer’s selective perception (-)
• time and cost (-)
• ethicality (-)
 Internet Surveys
• low cost, non-intrusive, quick response time, ability
to target specific populations
• non-representative, verification, how to motivate
participants
• types of Web observations
– # of times Web page is visited
– time spent on the page
– links accessed most often
– originating links