Research Design and Methodology

Measurement and scaling

Gebeyehu B. (Dr. of Eng.) Associate professor

[email protected]

BDU: Bahir Dar Institute of Technology: Computing Faculty

 Concept and Levels of Measurement
• Researchers strive for objectivity: not influenced by own perceptions,
impressions, and biases.

• Therefore, must identify systematic way of measuring a phenomenon

• Which means that, if it exists, then it can be measured

• If it is researchable, then data must be measurable

• Many ways we can measure data

• Measuring data has nothing to do with physical measurement

2
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Concept and Levels of Measurement
 Measurement can also be part of the research design

 In Computer research, measurement and scaling of variables is an

indispensable requirement
 However, defining what is to be measured, and how it is to be accur
ately and reliably measured are challenging
 Therefore, choosing facts or key concepts which are inherently abs
tract in their nature (e.g. programming performance, user satisfactions, adapt
ability, brand loyalty of consumers) are more difficult to measure than
concepts which can be assigned numerical values (e.g. sales volume for
objects X, Y and Z)

 It is the process of assigning numbers or scores to attributes of abstr

acts, concepts and/or objects.

3
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Concept and Levels of Measurement
 Or
 It is the process of describing some property of a phenomenon
of interest by assigning numbers in a reliable and valid way

 Hence, precise measurement requires:

 Careful conceptual definition – i.e. careful definition of the con
cept (e.g. performance) to be measured

 Operational definition of the concept

 Assignment rules by which numbers or scores are assigned to

different levels of the concept that an individual (or object) pos
sesses.

 It is the process of creating variables often requires creativity

4
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Concept and Levels of Measurement
 Conceptual definitions:
• Measurement is quantifying of any phenomenon, substantial or insubstantial,
concrete or abstract, and involves the comparison of the data being measured
to a pre-established standard.

• Quantifying mean “how much”, how many”, “to what degree” you think of the
world and its manifestations through the data observed in terms of magnitude an
d significance.

 Concept - A generalized idea about a class of objects, attributes, occurrences, or

processes.
 Concepts must be precisely defined for effective measurement.
 Examples: effectiveness, type, performance, fault, user satisfactions, …

 Construct - A concept that is measured with multiple variables.

 Examples: Brand loyalty, satisfaction, attitude, market orientation, socio-economic
status

 Variable - Anything that varies or changes from one instance to another; can exh
ibit differences in value, usually in magnitude or strength, or in direction.
5
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
• How to measure abstract concepts? -also depends on level of
analysis.

• Operational:
• It gives meaning to a concept by specifying what the researcher
must do (i.e. activities or operations that should be performed)
in order to measure the concept under investigation.

• Operationalization: is the process of identifying scales that corr

espond to variance in a concept.
o convert abstract theoretical notions into concrete terms, thereby all
owing measurement.
o process of applying measuring instrument in order to assign values
to some characteristic or property of the phenomenon being studied
. (Turn concepts into variables and then into data)

6
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
• Sometimes, it depending on the context of the research study - it may
be difficult to make operational definitions

• Therefore, there must be an operational guidelines that establish by

the students or researcher for assigning numbers or scores to differen
t levels of the concept (or attribute) that different individuals (or obje
cts) possess

• It is the process of assigning the the numbers 1, 2, 3 or scaling (perce

ntile) to different levels of attributes or abstracts towards the measure
ment of that specific task.

• A scale is any series of items (numbers) arranged along a continuous

spectrum of values for the purpose of quantification (i.e. for the purp
ose of placing objects based on how much of an attribute they possess).
7
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
 There are three ways in which the word “scale” we use in research
measurement:

 The level at which a variable is measured (Level of scale measure

ment)
o the arithmetical properties implied by the numbers assigned to levels of an att
ribute possessed by an object (i.e. the unit of analysis)

 An index, or composite measure of a construct

o Multiple statements used to measure a construct (also called a multi-item mea
sure of the construct)

 The response categories provided for a close-ended question in a q

uestionnaire, e.g.
o Subjects expressed their agreement / disagreement on or other way of present
ation

8
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
 Different level of scaling:
 Nominal Scale: a scale in which the numbers or letters assigned t
o an object serve only as labels for identification or classification,
e.g. Gender (Male=1, Female=2)
 Ordinal Scale: a scale that arranges objects or alternatives accord
ing to their magnitude in an ordered relationship, e.g. Academic statu
s (Sophomore=1, Freshman=2, Junior=3, etc
 Interval Scale: a scale that both arranges objects according to the
ir magnitude, distinguishes this ordered arrangement in units of e
qual intervals, but does not have a natural zero representing absen
ce of the given attribute, e.g. the temperature scale (40oC is not twice as h
ot as 20oC)
 Ratio Scale: a scale that has absolute rather than relative quantiti
es and an absolute (natural) zero where there is an absence of a gi
ven attribute, e.g. income, age.

9
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
Type of Data Numerical Descriptive
Examples
Scale Characteristics Operation Statistics
Nominal Classification but no Counting Frequency in Gender (1=Male,
order, distance, or each category 2=Female)
origin Percent in each
category, Mode
Ordinal Classification and Rank ordering Median Academic status
order but no Range (1=Freshman,
distance or unique Percentile 2=Sophomore,
origin ranking 3=Junior, 4=Senior)
Interval Classification, order, Arithmetic Mean Temperature, in degr.
and distance but no operations that Standard Satisfaction on
unique origin preserve order and deviation semantic differential
magnitude Variance scale
Ratio Classification, order, Arithmetic Geometric mean Age in years
distance and unique operations on Coefficient of Income in Saudi riyals
origin actual quantities variation

Note: All statistics appropriate for lower-order scales (nominal being lowest) are appropriate for higher
-order scales (ratio being the highest)
10
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
 Index and composite measures are also other type of scaling, which could apply a
s combinations (or collection) of several variables to measure a single construct (
or concept); they are multi-item measures of constructs.
 However, for index measures, the variables need not be strongly correlated with
each other, whilst for composite measures, the variables are typically strongly cor
related as they are all assumed to be measuring the construct in the same way
 Example 1: Index Measure
o Construct: Social class
o Measures: Linear combination (index) of occupation, education, income. Social class
= β1Education + β2Occupation + β2Occupation
 Example 2: Composite Measure
o Construct: Attitude Toward Brand A
o Measures: Extent of agreement/disagreement with multiple statements:
o a.“I like Brand A very much”
o b.“Brand A is the best in the market”
o c.“I always buy Brand A”
 Statements a), b), c), constitute a “scale” to measure attitudes toward brand A

11
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
 If a concept is simple, it can be measured easily usually with one qu
estion or observation: index measurement

o Example: To what extent do consumers of Product X like the product’s packa

ging material? (very much, somewhat, not at all)

 If, however, the concept to be measured is complex and abstract, tw

o or more questions or observations may be required in order to get
accurate data: composite or both (combination) scaling
o Example: The level of a salesperson’s motivation depends on (1) job satisfact
ion (2) workplace environment (3) family life

o Indexes (and/or composite measures) are meant to deal with the issue of mult
idimensionalty (e.g. an index of social class may be the variables residence,
occupation and education)

12
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
 Computing scale value for composite measures
• Summated Scale
• A scale created by simply summing (adding together) the response to each ite
m making up the composite measure.

• In MIS and other social science research, Likert-type interval scales (ratings)
are used to assess subjects attitudes, opinions, etc. toward an object by summi
ng, and sometimes averaging, over all items.

• Reverse Coding
• Means that the value assigned for a response is treated oppositely from the ot
her items.

13
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Rules of measurement
• Measurement scales
• If you can say that:
• One object is different from another, you have a nominal scale

• One object is bigger or better or more of anything than another,

you have an ordinal scale

• One object is so many units (degrees, inches) more than anoth

er, you have an interval scale

• One object is so many times as big or bright or tall or heavy as

another, you have a ratio scale

14
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement

 Reliability: is the degree to which measure is free from rando

m error and therefore gives consistent results
o It is an indicator of the measure’s of internal consistency

Test - retest
Stability (re
peatability)
Splitting half
Reliability
Internal con
sistency Equivalent
forms

15
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
 Stability  the extent to which results obtained with the meas
ure can be reproduced.

 Test-Retest Method
o Administering the same scale or measure to the same respondents at
two separate points in time to test for stability.

 Test-Retest Reliability Problems

o The pre-measure, or first measure, may sensitize the respondents and su
bsequently influence the results of the second measure.

o Time effects that produce changes in attitude or other maturation of the

subjects.

16
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
• Internal Consistency: the degree of homogeneity among the
items in a scale or measure
 Split-half Method
o Assessing internal consistency by checking the results of one-half
of a set of scaled items against the results from the other half.
o Coefficient alpha (α): The most commonly applied estimate of a
multiple item scale’s reliability.
o Represents the average of all possible split-half reliabilities for a
construct.
 Equivalent forms
o Assessing internal consistency by using two scales designed to be
as equivalent as possible.

17
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
• Validity: is the ability of a scale or measuring instrument to meas
ure what it is intended to measure (e.g. is absenteeism from work
a valid measure of job satisfaction or are there other influences like
a flu epidemic which is keeping employees from work)
 The accuracy of a measure or the extent to which a score truthfully
represents a concept.
 The ability of a measure (scale) to measure what it is intended me
asure.
 Establishing validity involves answers to the ff:
o Is there a consensus that the scale measures what it is supposed to measure?
o Does the measure correlate with other measures of the same concept?
o Does the behavior expected from the measure predict actual observed behav
ior?

18
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
• Assessing validity:
 Content validity: The subjective agreement among professionals that a scale lo
gically appears to measure what it is intended to measure.
 Criterion Validity: the degree of correlation of a measure with other standard
measures of the same construct.
o Concurrent Validity: the new measure/scale is taken at same time as criterio
n measure.
o Predictive Validity: new measure is able to predict a future event / measure
(the criterion measure).
 Construct Validity: degree to which a measure/scale confirms a network of relat
ed hypotheses generated from theory based on the concepts.
o Convergent Validity.
o Discriminate Validity.

19
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
• A relationship between reliability and validity
 A measure that is not reliable cannot be valid, i.e. for a measure to
be valid, it must be reliable  Thus, reliability is a necessary condi
tion for validity

 A measure that is reliable is not necessarily valid; indeed a measure

can be but not valid  Thus, reliability is not a sufficient condition
for validity

 Therefore, reliability is a necessary but not sufficient condition for

Validity.

20
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
 Sensitivity – Sensitivity is the ability of a measurement instrument
to accurately measure variability in stimuli or responses (e.g. on a s
cale, the choices very strongly agree, strongly agree, agree, don’t a
gree offer more choices than a scale with just two choices - agree a
nd don’t agree – and is thus more sensitive)
o The ability of a measure/scale to accurately measure variability in stim
uli or responses;
o Composite measures allow for a greater range of possible scores, they
are more sensitive than single-item scales.

• Sensitivity is generally increased by adding more response poi

nts or adding scale items.

21
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Criteria for good measurement
• It summarize as:
o Understanding of the questions

o Discriminatory power of scale descriptors

o Balanced versus unbalanced scales

o Forced or nonforced choice scales

o Desired measure of central tendency and dispersion

22
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Basics of measurement and scaling process
• Measurement process:
• Generate Items/Questions
• Wording
• Response format
• Layout and design questionnaire
• Pretest and refine
• Basic measurement issues
• Concept or Construct
• A generalized idea about a class of objects, attributes,
occurrences, or processes
• Concrete –traffic patterns, purchase quantity
• Abstract – loyalty, personality, satisfaction, leadership

23
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Basics of measurement and scaling process
• Variables operational definitions:

• A statement which defines a variable by specifying the operations

used to measure and manipulate it.
• Example:
o Variable in research Problem: Computer usage by end users
o Labeling the variable: Attitudes towards computer usage
o Operational Definition: An individual’s score on the attitudes to
wards computer usage instrument

24
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Basics of measurement and scaling process
• Measurement: Assigning numbers to objects or events ac
cording to rules

• Measurement is used in every research project

o Determine the level of measurement for each variable

o Decide if the level of measurement could be changed to a higher level, a

nd if this would improve the research

o Review validity and reliability in order to critique the literature and for l
ater use at the instrumentation stage

25
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Basics of measurement and scaling process
• Measurement implies:
• Selecting observable empirical events
• Developing a mapping rule: a scheme for assigning numbers or symbols to r
epresent aspects of the events being measured
 Applying the mapping rule(s) to connect the observation to the symbol

 Example:
 Assume you are studying people who attend a computer show where all of the ye
ar’s new products are on display. You are interested in learning the male-to-fema
le ratio among attendees. You observe those who enter the show area. If a person
is female, you record an F; if male, an M. Any other symbols such as 0 and 1 or
% and # also may be used if you know what group the symbol identifies.

26
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Basics of measurement and scaling process
• What is measured?
• Variables being studied may be classified as objects or pr
operties.
• Objects include:
• Things of ordinary experience (e.g., people, tables, books, computer syst
ems, etc.)
• Some things not concrete (e.g., attitudes, genes, neutrons, etc.)
• Properties are:
• Characteristics of objects
• Physical properties (e.g., weight, height, etc.)
• Psychological properties (e.g., attitudes, perceptions, etc.)
• Social properties (e.g., leadership ability, status, etc.)

27
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Basics of measurement and scaling process
• Characteristics of measurement data:

• Mapping rules for data have four characteristics:

• Classification
• Numbers are used to group or sort responses. No order exists.
• Order
• Numbers are ordered. One number is greater than, less than, or equal to
another number.
• Distance (interval between numbers)
• Differences between numbers are ordered. The difference between any p
air of numbers is greater than, less than, or equal to any other pair of nu
mbers.
• Origin of number series
• The number series has a unique origin indicated by the number zero.

28
BDU: Bahir Dar Institute of Technology: Computing Faculty
 Types of Rating Scales

• Comparative Rating Scales

• format that requires a judgment comparing one object, person,
or concept against another on the scale

• Noncomparative Rating Scales

• format that requires a judgment without reference to another
object, person, or concept

29
BDU: Bahir Dar Institute of Technology: Computing Faculty
 30
BDU: Bahir Dar Institute of Technology: Computing Faculty