PSY 239 401 CHAPTER 3 SLIDES
- 1. Chapter 3: Personality Psychology as Science: Research Methods The Personality Puzzle Sixth Edition by David C. Funder Slides created by: Tera D. Letzring Idaho State University © 2013 W. W. Norton & Company, Inc. 1
- 2. Objectives • Discuss research methods that are particularly important to personality psychology • Discuss the difference between scientific education and technical training • Discuss aspects of data quality • Discuss common research designs • Discuss some statistical issues • Discuss research ethics © 2013 W. W. Norton & Company, Inc. 2
- 3. Psychology’s Emphasis on Methods • Psychologists sometimes seem to know more about research methods than about the mind and behavior. • Goal: improving hypotheses • Question everything, be skeptical, think analytically. © 2013 W. W. Norton & Company, Inc. 3
- 4. Scientific Education and Technical Training • Technical training • Scientific education – Question what is already “known” – Learning to explore the unknown – Research: the exploration of the unknown © 2013 W. W. Norton & Company, Inc. 4
- 5. Quality of Data: Reliability • Definition • Measurement error – Also called error variance – The cumulative effect of extraneous influences • States versus traits © 2013 W. W. Norton & Company, Inc. 5
- 6. Quality of Data: Factors that Undermine Reliability • • • • Low precision of measurement The state of the participant The state of the experimenter The environment © 2013 W. W. Norton & Company, Inc. 6
- 7. Quality of Data: Enhancing Reliability • Be careful • Use a standardized procedure or protocol • Measure something that is important and engages participants © 2013 W. W. Norton & Company, Inc. 7
- 8. Quality of Data: Enhancing Reliability • Aggregation – Allow random influences to cancel each other out – Especially important for predicting behavior – Spearman-Brown formula © 2013 W. W. Norton & Company, Inc. 8
- 9. Quality of Data: Validity • Definition • A “slippery” concept – Reliability is necessary but not sufficient for validity. – Invokes the idea of “ultimate truth” © 2013 W. W. Norton & Company, Inc. 9
- 10. Quality of Data: Validity • Difficulty of measuring a construct – Assessing personality is similar to testing a theory. • Construct validation – Gather as many measurements as possible. – Look for the ones that hang together. © 2013 W. W. Norton & Company, Inc. 10
- 11. Quality of Data: Generalizability • The distinction between reliability and validity is regarded as “fuzzy” by some. • Definition © 2013 W. W. Norton & Company, Inc. 11
- 12. Quality of Data: Generalizability • Generalizability over participants – Gender bias: Women are more likely to volunteer and show up. – Shows versus no-shows – Cohort effects: the tendency of a group of people living at a particular time to be different in some way from those who lived earlier or later – Ethnic and cultural diversity © 2013 W. W. Norton & Company, Inc. 12
- 13. Quality of Data: Generalizability • The burden of proof – Avoid simplistic generalizations to members of other cultures and people in different times (including differences). – Those who question the generalizability of a study should propose when, how, and why it is not generalizable. © 2013 W. W. Norton & Company, Inc. 13
- 14. Research Design: Case Method • Definition • Can yield explanations of particular events, general lessons, and scientific principles • Case studies of ourselves © 2013 W. W. Norton & Company, Inc. 14
- 15. Research Design: Case Method • Advantages – Describes the whole phenomenon – Source for ideas – Sometimes necessary for understanding an individual • Disadvantages – No control – Findings must be confirmed by other cases, which is not usually possible. 15 © 2013 W. W. Norton & Company, Inc.
- 16. Research Design: Experimental Method • Definition: a research technique that establishes the causal relationship between an independent variable (x) and a dependent variable (y) by randomly assigning participants to experimental groups characterized by differing levels of x, and measuring the average behavior y that results in each group © 2013 W. W. Norton & Company, Inc. 16
- 17. Research Design: Experimental Method • Test differences between groups with statistical tests to determine if the difference is larger than would be expected by chance © 2013 W. W. Norton & Company, Inc. 17
- 18. Research Design: Experimental Method Leaders (high power) Leaders’ assistants (low power) © 2013 W. W. Norton & Company, Inc. Rank list of items needed to survive in a lifeboat on the open sea Measure interpersonal sensitivity 18
- 19. Research Design: Experimental Method F (1, 72) = 4.91, p = .03 © 2013 W. W. Norton & Company, Inc. 19
- 20. Research Design: Correlational Method • Definition • Scatter plot • Correlation coefficient © 2013 W. W. Norton & Company, Inc. 20
- 21. Research Design: Correlational Study Measure power Measure interpersonal sensitivity Determine the relationship r = .25 © 2013 W. W. Norton & Company, Inc. 21
- 22. Research Design: Comparing the Experimental and Correlational Methods • Both attempt to assess the relationship between two variables. • The statistics (with two groups) are interchangeable. • The experimental method manipulates the presumed causal variable, and the correlational method measures it. © 2013 W. W. Norton & Company, Inc. 22
- 23. Research Design: Comparing the Experimental and Correlational Methods • Only experiments can assess causality. – Correlational studies: unknown direction of cause; third-variable problem © 2013 W. W. Norton & Company, Inc. 23
- 24. Research Design: Comparing the Experimental and Correlational Methods • Complications with experiments – Uncertainty about what was really manipulated • Third-variable problem – Can create unlikely or impossible levels of a variable – Often require deception – Not always possible • Experiments are not always better. © 2013 W. W. Norton & Company, Inc. 24
- 25. Research Design: Representative Design • Frequent concern: representativeness of participants • Less frequent, but important, concerns – Representativeness across stimuli – Representativeness across responses © 2013 W. W. Norton & Company, Inc. 25
- 26. Research Design: Representative Design • Solution: use a representative design – Seldom done because it is expensive and timeconsuming © 2013 W. W. Norton & Company, Inc. 26
- 27. Thinking About Representativeness • How is the psychology of today’s college students different from that of their parents? Would the conclusions of research done with college students apply to their parents? What areas are most likely to be different? © 2013 W. W. Norton & Company, Inc. 27
- 28. Thinking About Representativeness • Is research done with the predominantly white college students in Western cultures also relevant to members of ethnic minorities or to people who live in other cultures? In what areas would you expect to find the most differences? © 2013 W. W. Norton & Company, Inc. 28
- 29. Significance Testing • Statistical significance: a result that would only occur by chance less than 5% of the time • p-level: probability level of obtaining a result from a statistical test if there really is no difference between groups or no relationship between variables • Null-hypothesis significance testing (NHST) © 2013 W. W. Norton & Company, Inc. 29
- 30. Significance Testing: Problems with NHST • The logic is difficult to describe (and understand). • “Significant” does not necessarily mean strong or important. • The criterion for significance is an arbitrary rule of thumb. • Chances of significance vary with sample size. © 2013 W. W. Norton & Company, Inc. 30
- 31. Significance Testing: Problems with NHST • Nonsignificant results are often interpreted as “no result.” • Only provides information about the probability of one type of error – Type I error vs. Type II error • Cannot really tell you if a result is important © 2013 W. W. Norton & Company, Inc. 31
- 32. Correlations and Effect Sizes • Effect size definition • More meaningful than a significance (p) level • Correlation coefficient – Can be used for correlational and experimental studies • Between -1 and +1 – 0 = no relationship • Positive and negative correlations © 2013 W. W. Norton & Company, Inc. 32
- 33. Correlations and Effect Sizes • Use for prediction • Interpreting correlations – Look at the actual size – r2 = percent of variance explained; “a terrible way to evaluate effect size” (p. 92) – Binomial Effect Size Display (BESD) © 2013 W. W. Norton & Company, Inc. 33
- 34. BESD r = .00 Drug No drug Lived 50 50 100 © 2013 W. W. Norton & Company, Inc. Died 50 50 100 100 100 200 34
- 35. BESD r = .30 Drug No drug Lived 50 + (r*100)/2 = 65 50 - (r*100)/2 = 35 100 © 2013 W. W. Norton & Company, Inc. Died 50 - (r*100)/2 = 35 50 + (r*100)/2 = 65 100 100 100 200 35
- 36. BESD r = .25 High I.S. Low I.S. 62.5 37.5 100 37.5 62.5 100 High power Low power 100 100 200 If you are worried about being interpersonally sensitive, do you want to have high power? © 2013 W. W. Norton & Company, Inc. 36
- 37. Thinking About Statistical Issues • Let’s say we find that you score 4 points higher on a “conscientiousness” test than does another person. Alternatively, imagine that women score 4 points higher on the same test, on average, than men do. In either case, is this difference important? What else would we have to know to be able to answer this question? © 2013 W. W. Norton & Company, Inc. 37
- 38. Research Ethics • The uses of psychological research – Make sure it is not harmful, or at least that the potential harm does not outweigh the potential good. • Truthfulness – Avoid plagiarism and fabrication of data. © 2013 W. W. Norton & Company, Inc. 38
- 39. Research Ethics: Deception • • • • Definition Purpose: usually to make the research realistic APA guidelines Review by the Institutional Review Board (IRB) or Human Subjects Committee (HSC) © 2013 W. W. Norton & Company, Inc. 39
- 40. Research Ethics: Deception • Arguments in favor of deception – Informed consent – It usually does no harm. – Certain topics cannot be investigated without deception. © 2013 W. W. Norton & Company, Inc. 40
- 41. Research Ethics: Deception • Arguments against deception – Informed consent for deception is not possible. – When does the deception stop? – Harms credibility of psychology – Alternative: Investigate topics in the real world. • What do you think about deception? Is it justified? © 2013 W. W. Norton & Company, Inc. 41
- 42. Clicker Question #1 In order to say that one variable caused another, a researcher must a)calculate the correlation between the variables. b) conduct an experiment. c) construct a BESD. d) use deception. © 2013 W. W. Norton & Company, Inc. 42
- 43. Clicker Question #2 In order for data to have a high degree of validity a)they must also have a high degree of reliability. b) they must come from an experiment. c) they must have low generalizability. d)one must know the ultimate truth about the construct being assessed. © 2013 W. W. Norton & Company, Inc. 43
- 44. Clicker Question #3 The case method should be used when a)the researcher is especially concerned that the results have high generalizability. b)the researcher wants to establish the cause of a particular behavior. c)the researcher wants to have lots of control. d)there is an individual that the researcher wants to understand as fully as possible. © 2013 W. W. Norton & Company, Inc. 44
Editor's Notes
- #3: Discuss aspects of data quality: reliability, validity, and generalizability Activity 3-1: What Kind of Thing is Personality?
- #4: Goal: improving hypotheses (not finding truth)
- #5: Technical training: conveying what is already known about a subject so that the knowledge can be applied Scientific education: teaching what is known and how to find out what is not yet known
- #6: Definition: the tendency of a measurement instrument to provide the same comparative information on repeated occasions; getting the same result more than once Measurement error: the cumulative effect of extraneous influences on a test score States vs. traits: differ in expected level of stability, in terms of what are considered extraneous influences or sources of measurement error; with traits – can you get the same result more than once? State vs. trait anxiety: Is the current situation (e.g., party, employment interview) an extraneous influence? How much stability do you expect? How much individual variation do you expect? (most people will be anxious at an interview, few at a party)
- #7: Precision: Make measurements as exact as possible; data should be correctly recorded, scored, and entered into a database. State of participant: may not depend on the study (illness, affect, distraction, fatigue, etc.) State of experimenter: may not treat all participants the same; participants may respond differently to aspects of the experimenter (gender, age, attractiveness) The environment: temperature of the room, the weather, noise in the building Activity 3-3: Interruption Olympics: Bill O’Reilly Versus Jon Stewart
- #8: Be careful: double-check measurements, check data entry, check accuracy of scoring Use a standardized procedure or protocol: adequate training of research assistants and monitoring to ensure procedures are followed Measure something important: Do you care about the health care debate? If you do, then your opinion will be more easy to measure reliably than that of someone who doesn’t care, whose opinion is more likely to be influenced by extraneous factors.
- #9: Aggregation: especially important when measurements contain a lot of error Especially important for predicting behavior: because single behaviors are influenced by many factors other than personality. What influences whether you make your bed in the morning? (lots of things, but over time, conscientious people will make their beds more often) Spearman-Brown formula: a mathematical formula that predicts the degree to which the reliability of a test can be improved by adding more items Activity 3-2: Why So Many Observations of the Same Thing?
- #10: Definition: degree to which a measurement actually reflects what one thinks or hopes it does Ultimate truth: It’s difficult, or impossible, to know what constructs (intelligence, honesty) really are, so it’s difficult to know if they are measured validly.
- #11: Construct definition: an idea about a psychological attribute that goes beyond what might be assessed through any particular method of measurement Construct validation: the strategy of establishing the validity of a measure by comparing it to a wide range of other measures Cronbach & Meehl article from the reader: Construct Validation in Psychological Tests
- #12: Fuzzy distinction: How much can a test be changed before it becomes a different test and assessments of reliability become assessments of validity? Definition: the degree to which a measurement or result of an experiment applies to other tests, situations, or people
- #13: Gender bias: Much early research (pre-1970s) only used male participants. Are women importantly different from men, which is why they are more likely to volunteer and show up? Are the men who volunteer and show up importantly different from other men? Shows vs. no-shows: There’s a problem if people in these groups are systematically different. What if you are trying to study the effects of punctuality but can only use participants who show up within 5 minutes of the start of the experiment? Cohort effect definition: the tendency of a group of people living at a particular time to be different in some way from those who lived earlier or later How does growing up with the current technology (Internet, cell phones, iPods) influence people differently from people who grew up without it? How does growing up during a time of war differ from growing up during a time of peace? Ethnic and cultural diversity: Most research is with white middle-class college students.
- #15: Ongoing example: relationship between power and interpersonal sensitivity (Schmid Mast, Jonas, & Hall, 2009) Definition: closely studying a particular event or person of interest in order to find out as much as possible Case studies of ourselves: might help us understand others better Case method example: observe that someone in a high-power position has a high level of sensitivity
- #16: No control: impossible to determine which facts and variables are crucial or incidental to understanding the person
- #17: Activity 3-4: Experimental and Correlational Studies
- #19: Measure of interpersonal sensitivity: inferences of thoughts and feelings of a videotaped interaction (Ickes paradigm)
- #21: Definition: a research technique that establishes the relationship between two variables by measuring both variables in a sample of participants Scatter plot: chart on which each point represents an individual’s scores on two variables Correlation coefficient: reflects the strength and the direction of the relationship
- #23: Statistics are interchangeable: t can be converted to r.
- #25: Uncertainty about what was really manipulated: Manipulation may affect a variable other than the one intended (competence instead of power; but this was checked and did not differ between high- and low-power groups). Third-variable problem: Manipulation may have affected an unintended variable, and this could affect both variables of interest (people who are more confident may feel more powerful and also have higher interpersonal sensitivity). Can create unlikely or impossible levels of a variable: results in exaggerated group differences (if one person has all of the power and the other has none); correlational studies measure natural levels Not always possible: especially true for personality, so experiments are rarely used by personality psychologists
- #26: Representativeness across stimuli: Schmid Mast, et al. manipulated power in more than one way (also with a word completion priming task—leadership vs. service, writing about a time when the person had high or low power) Representativeness across responses: Schmid Mast, et al. measured IS in more than one way; also with the Diagnostic Analysis of Nonverbal Sensitivity (DANVA) —decoding of emotional facial expressions—and Profile of Nonverbal Sensitivity (PONS) , decoding of nonverbal cues
- #27: Representative design: sampling across the domains to which you wish to generalize
- #30: I left the definitions on the slides because these are confusing. NHST: the traditional method of statistical data analysis
- #31: Chances of significance vary with sample size: Nature hasn’t changed, just the conclusion about nature has changed.
- #32: Chances of significance vary with sample size: Nature hasn’t changed, just the conclusion about nature has changed. Type I error: deciding that one variable has an effect on or a relationship to another variable, when really it does not; crying wolf Type II error: deciding that one variable does not have an effect on or a relationship to another variable, when really it does
- #33: Effect size: an index of the strength of the relationship between the variables Correlation coefficient: the most commonly used measure of effect size Positive correlation = as one variable goes up, so does the other Negative correlation = as one variable goes up, the other goes down
- #34: .25 squared = .0625, so power explained 6.25% of the variance in interpersonal sensitivity Binomial Effect Size Display (BESD): one method for displaying and understanding more clearly the magnitude of an effect reported as a correlation
- #35: Rosenthal & Rubin article from the reader: A Simple, General-Purpose Display of Magnitude of Experimental Effects
- #37: I.S. = interpersonal sensitivity
- #39: Avoid plagiarism and fabrication of data, which undermine the foundation of science.
- #40: Definition: telling research participants something that is not true APA guidelines: “Psychologists strive to benefit those with whom they work and take care to do no harm. . . . Psychologists respect the dignity and worth of all people, and the rights of individuals to privacy, confidentiality, and self-determination.” (http://www.apa.org/ethics/code/index.aspx)
- #42: Alternative is to investigate topics in the real world that cannot be manipulated in the lab (but this will only be correlational)
- #43: Correct answer: b
- #44: Correct answer: a
- #45: Correct answer: d