Below the fold is the transcript of my interview with Dr. Keith Stanovich. I didn't realize how much work it was to transcribe a conversation! I'm thinking of farming out future episodes to a transcription service, but that runs $40-$50/episode, and I'm not interested in spending a whole lot of money into this project quite yet.  I haven't even forked over the money to pay for hosting (hence the GoDaddy banner ad up top).  Isn't it great to live at a time when this is possible — for nearly free? I've peppered the transcript with links expanding on some topics or terms we referenced in passing, including some blog posts I've written (or will soon write).  Some are links to Wikipedia, some are just fun.  I may come back and add links as I produce more content and come across more cool stuff; feel free to offer suggestions in the comment section for stuff I can add.  I hope you enjoy, leave a comment and show some love!

David Bradley: Hello, and welcome to episode one of Psyconoclasm, a skeptical exploration into the field of psychology, recorded April 17, 2009. My name is David Bradley, thank you for joining me. I couldn't think of a better person to kick off this podcast series than Dr. Keith Stanovich. Dr. Stanovich is a professor of applied cognitive science at the University of Toronto, He's a leading educational psychologist, and an expert in the fields of rational cognition, reading, and intelligence . He's the author of several books, including The Robot's Rebellion: Finding Meaning in the Age of Darwin. His most recent book is What Intelligence Tests Miss: The Psychology of Rational Thought. We'll be focusing on what he discusses in what might be his best known book: How to Think Straight about Psychology, which is in its eighth edition. Dr Stanovich, it's an honor and a pleasure to have you here.

Dr. Keith Stanovich: Well thanks for having me, David. I'm glad to be here.

Bradley: So you begin How to Think Straight About Psychology by talking about what you term The Freud Problem. What's wrong with Freud? What's the problem?

Stanovich: Freud presents many problems for the student's understanding of psychological science. First of all, there's the unrepresentativeness. Freud's the best known name that people free associate with psychology, including the introductory student, who is the focus of my book, How to Think Straight About Psychology. And yet, Freud and Psychoanalysis represents a very small proportion of the interest and work done in psychology. First of all, there's the statistical unrepresentativeness of Freud. But in fact in my book, I stress some other problems. Knowing only Freud sets the student down the wrong track to understanding how psychological research is done. Freud's work is best known for his famous case studies. But ,of course, one of the most important principles that a methodology instructor in psychology or a statistics instructor in psychology wants to get across to their students is that you can't make causal inferences from case studies. One needs controlled observation on a large number of people. Those controls being the things like the true experiment, where we invoke random assignment of subjects to groups, where we isolate variables for study, where we manipulate an independent variable and look for its effect on a dependent variable. All of the sort of normal science – and I say normal science because these types of methods are in no way unique to psychology, they're the normal method of science in all the other social sciences and the physical sciences. So knowledge of Freud and the idea that one can make large theoretical statements from simply a case study of an individual fundamentally puts students on the wrong track of understanding what psychology is about. Psychology proceeds normally like the other sciences do by trying to isolate variables for study. First, you have to find out what the important variables are. That's where case studies do come in to play, because they may give people insight about what might be important to study. That's a different thing entirely, the insight of what might it be important to study is a very different thing from saying “from this case study, I derive this causal conclusion.” The world of research methodology, those are two very different things. Freud's work fuses those together in a way that is atypical and misleading for our students.

Of course, one of the basic introductory methodological things that a psychology student learns is that to test theories you have to design an experiment that differentiates theories. It will enable you to tell not only that the data supports A, but that the data rules out some other explanation B. That's the reason for the structured observations that we call the true experiment in science. All of those insights are obscured for the student if they have the misleading impression that Freud is somehow typical of psychology.

Before I leave this topic, I want to get in one other misleading aspect, and that is the nature of the theorizing. Freudian psychoanalysis has been deeply criticized by a number of different authors for being unfalsifiable. Falsifiability – and now we're moving to the domain of theory – falsifiability is the fundamental criterion of theories in all sciences, but, again, something we have to stress in psychology. For a theory to be useful, it must specify a prediction about the world, and that prediction must have some specificity to it, and a prediction having specificity means that it must rule something out. This is the notion of falsifiability that, again, we've stressed in introductory methodology in the study of psychology. Freud's theories are dangerously close to being not falsifiable in the sense that good theory in the social sciences is. So the reason I started this book that way is to take on some of those misconceptions very directly.

Bradley: There's a lot in Freud that you have to criticize. I wonder if the reason that Freud gets so much play in psychology texts books and introductory psychology courses is because it shows up so often in the culture. For example, when you started to talk about Freud, you said that Freud is what students free associate with when they think about psychology. Free association is, in my mind, linked to Freud. Did he come up with that concept, or was that something he . . . ?

Stanovich: I wasn't using that in a Freudian sense. I mean, association in psychology also has roots separate from Freud. But I agree with your point at the outset, the idea that the general culture is saturated with Freudian references. But again, the instructor can use that to make the point in the sense that Freud is of more interest to the literary theorist than he is to the psychological theorist.

Bradley: So someone taking a course in English literature, for example, might find more use than someone in a psychology lecture?

Stanovich: That's exactly right. If someone is interested in Freud, that may be a better route for them.

Bradley: Interesting. So you said that the main way the field has changed since Freud, generally speaking, is in the methods that psychology uses. Instead of using case studies to prove things, they might use them to ask questions instead of answer them, is that a fair characterization?

Stanovich: Yes, exactly, that's a fair characterization. It's not just in the methods. But again, Freud gives a misleading idea of the range of topics that we study in psychology. Yes, psychology studies psychopathology and problematic behavioral traits of all kinds, but again the bulk of psychological research is not focused in that manner. The bulk of psychological research is focused on normal cognition, and normal personality traits that are manifest in all individuals. It's a more universal conception of mind.

Bradley: So you also speak, in addition to Freud and the problems that he raises, there are some intuitions that we have about psychology, what you term folk wisdom and folk psychology. What are the problems with those? Are they the same sorts of problems as far as nonfalsifiability and those sorts of issues?

Stanovich: Yes, very parallel issues there. Again, we all have folk theories or folk wisdom about various things in the world. I think psychology is a little different in the sense that our folk theories engage us more in the domain of human behavior than they do perhaps in some other endeavors. For example, physics instructors study folk physics. People have folk notions of how objects move in the world (PDF), and about fire and various other things, and you know many of them are mistaken. Ideas of inertia and gravity and so on and so forth, there has been a reasonable amount of study in the educational literature on this, and how physics instructors should approach a student who brings to the study of physics a folk physics that is incorrect. It is true that the student might come to physics with folk notions of physics, but they are never usually held with the emotional fervor that folk theories of human behaviors are held. So although all disciplines have this problem, it's particularly acute in psychology, because the folk notions of human behavior are very close to us. The idea of scientific objectivity, then, looms so much larger in psychology than it would loom in geology, where the student has little problem in geology with approaching the study of sediments with objectivity. It's much more of problem for psychology getting students to approach issues of personality, of behavior, of thinking, of interpersonal relationships, with a fresh eye, with an objective eye, and — in fact, in the book I label this the preexisting bias problem — without imposing their preexisting biases on everything. And the other aspect of folk wisdom and folk psychology is the one that you correctly pointed to, it's a similar one to Freud, in that a lot of the folk theories we have turn out to be unfalsifiable.

Of course, every intro textbook plays to this. I do this in my own book, where many of our theories are simply folk cliches that we trot out on the appropriate occasion to explain an instance of behavior. Most of us are pretty good at it. We have a cliché that can cover everything. The problem is that that seeming strength of folk psychology is actually its weakness, because if it can explain everything after the fact, then it must not be very falsifiable. I have a list in my own book of these folk cliches that are totally contradictory, but we use no matter what the evidence is. “Opposites attract,” except for “birds of a feather flock together.” Regardless of what happens, if like people cluster together, then we say “birds of a feather flock together.” If two people have very different traits, then we say that “opposites attract,” but the theory gives us no way of saying which is going to happen under which circumstances. “Absence makes the heart grow fonder,” but “out of sight out of mind.” “Better safe than sorry,” but “nothing ventured, nothing gained.” And then, of course, in folk psychology are all the what are sometimes called “psychofacts,” or another author , I'm forgetting who, has called them “factoids,” these things that are passed around folk culture. This notion that “people only use 10% of their brains.” That's one of those factoids that doesn't map anything that we know in psychological neuroscience, but every person in the street, quote, “knows” it. Those are some of the problems that folk psychology presents for instructor in psychology.

Bradley: Are there any examples of those factoids that you've had to give up as you've learned more and more about psychology?

Stanovich: That's an interesting one. I think I have held some of the factoid beliefs. I did hold some of them in my earlier life prior to an intense study of psychology. Let's take an example, this is one I think that I did hold many many years ago: Ideas about the causal efficacy of self-esteem, for instance. The idea, very popular, and it's still very popular, that if we raise people's self-esteem, then that will result, causally, in increases in some types of target behavior. The very common one is in schools, the idea that people's school achievement will go up if you first raise their self-esteem. And I emphasize the “first” to emphasize that I'm talking about people who say this is a causal connection, not just a correlational one. We've had a lot of recent work in the last 10 years, some by Roy Baumeister of Florida State, has a big, big review article on this (PDF). If anything, the causal connection goes in the other direction.

Bradley: People do well, then they have self-esteem?

Stanovich: It's an outcome of superior achievement, not the cause of superior achievement. I can plead guilty to believing the standard causal, folk model about that one some years ago, until I became familiar with Baumeister and others' evidence on that. So I certainly do think that I've been affected by some of these folk theories. Another kind of factoid belief that I believed until being exposed to the scientific evidence was in one of my research areas. You mentioned in your introduction many years I studied the psychology of reading and reading acquisition, and also reading difficulties. It's still a folk view that people with reading difficulties, so called dyslexia, have their problem because they reverse letters visually. I believed that for a long time too, until I did studies in the area, and found out that that was not the primary cause of reading difficulties. The primary cause of the dyslexia lie in the speech domain and the domain of speech segmentation and philological awareness, and not in the visual domain. Troubled speech regions outnumber visual difficulties 9 to 1 at least. And yet, the folk belief that visual reversals are the cause of dyslexia is still out there. And again, I understand why it is, because I held it myself until actually doing research in this area.

Bradley: It's really interesting, just how important it is even in a field of expertise, to be willing to be disabused of these notions we have.

Stanovich: Right. Again, the thinking attitude we try to instill in students when we teach them about falsifiability. We teach is as a technical concept having to do with theories and so on, but actually, the way I teach of it is also as a personal thinking tool, that one should allow beliefs that haven't stood the test of evidence to fall by the wayside, that it's no sin to give up a belief. That may make us less reticent to actually try to say something with more specificity knowing that we may have to admit error in the future.

Bradley: Right. That's been something that I think a lot of people getting into the scientific way of thinking struggle with, is that they realize how many wrong ideas they've had, and so they become a little more timid about sharing the ideas that they do have that they think are right.

Stanovich: Yes, that can be a problem, and there is a scale that we use in our rational thinking research in my lab called the Belief Flexibility Scale, and its trying to tap how easy or hard it is for people to give up beliefs that have been falsified. It's a very important intellectual trait in a psychologist.

Bradley: So speaking of wrong ideas and people being persuaded by evidence, in the book you talk about someone named Clever Hans – I love this story. Could you explain, who is "Clever Hans" and what does he teach us?

Stanovich: Clever Hans is a horse, actually, in the early 20^th Century, and it's a beautiful illustration why mere observation – and by using this phrase “mere observation” I mean simply looking unaided by the experimental manipulations and controls and interventions, the experimental interventions that a researcher actually does, those interventions can sometimes look funny and artificial to an observer, all the controls and random assignment and this and that – clever Hans teaches us the reason for the intervention, and that is unaided observation can be misleading. Hans was a horse who garnered tremendous publicity in early 20^th Century Germany when his owner presented him as a horse who had mathematical abilities. The trainer would shout out mathematical problems, or indeed even just show the horse a mathematical problem (simple addition, subtraction, that type of thing, division). Then the horse, with one front hoof, if you can imagine, you've seen this kind of thing in the movies – would slam the hoof on the ground and tap out the answer to the mathematical problem. And indeed, in most of the cases, the horse was correct. The trainer would show 5-3, and the horse would give two taps.

This was in the newspapers, and learned individuals traveled to come see this horse. It was a sensation. And indeed, a panel was set up of very eminent lawyers and judges and people of that type who came to formally attest that the horse indeed had mathematical ability. And of course, the problem is, and some of the psychology students will be shaking their heads right now, they'll know the answer, how this goes, whether they've heard of Clever Hans or not. You needed actually a psychologist. There is a hero, an experimental psychologist by the name of Oskar Pfungst. He came and pointed out all of the methodological issues that a modern psychologist would now see in this situation. First of all, there was a little conceptual trick going on here, in that we've made a very quick jump from observation to theory, and that is what we've seen is the horse tap out the correct answers to mathematical problems, but immediately the trainer jumped to the theory or tried to sell the theory that the horse had mathematical abilities, but that is a theory, and there may be other theories that could account for the horse's behavior, i.e. an alternative theory that then the psychologist would want to match against the theory that the horse had actual mathematical abilities. That's exactly what Pfungst did.

The alternative theory that a behavioral scientist would wonder about was whether the horse wasn't getting some type of cues from the trainer. These can be of various types; that is, cues consciously given by the trainer, i.e. perhaps this whole set up about the mathematical horse is a fraud; but a real behavioral scientist would think of a more subtle possibility, that there's no conscious fraud going on, that the trainer actually thinks that the horse has this ability, but is somehow giving off some type of subtle behavioral cue that is telling the horse when to stop the hoof counting. And that indeed turned out to be the case; that is, the horse wasn't a mathematician, but the horse was a very good behavioral scientist, in that he had very acute observational powers, and what was happening, of course, was that the trainer was giving a subtle little tilt of the head when the horse got to the right answer, well the horse had learned how to pick up this little tilt and hence had learned exactly when to stop.

What the psychologist, Oscar Pfungst, did, was set up experimental procedures that would differentiate these two theories; that is, the theory that the horse was picking up behavioral clues from the trainer, from the theory that most people had was the horse actually had mathematical abilities. If the horse actually has mathematical abilities, it shouldn't matter who was presenting the problem. One test would be to give matched problems, one given by the trainer but the other given by someone who didn't know the horse to see if the horse could solve the problem. There's a whole very interesting book on this. Pfungst did all sorts of setups with this similar logic to it, and it turned out that the horse couldn't perform when he wasn't getting the subtle cue from the trainer. Note that Clever Hans often shows up in methodological textbooks because, I want to reiterate the point that I made in the introduction to it, that mere observation was not enough. The lawyers and the judges who came, all they were able to say was that the horse did X, the horse actually did tap out the correct answer to these mathematical problems. Not thinking like scientists, they did not realize that that's not enough proof for the theory that the horse actually had mathematical abilities. That theoretical leap requires controlled observation of the type that the experimental psychologist Oskar Pfungst carried out. Controlled interventions. That's why science, scientific testing, and the experimental method is more than just raw observation, or observation of the natural world as it naturally occurs, but involves interventions and manipulations. So that's the main theme of the Clever Hans example as he appears in many methods textbooks in psychology

Bradley: It's interesting, It reminds me of an episode of The Simpsons, where Maggie, the baby, is taking an IQ test. Lisa Simpson, one of the other children, is there. Maggie does really well on the test, but then they slow down the frames and you can see Lisa Simpson giving the answers to the test to Maggie, subtly, without anyone noticing.

Stanovich: I'd say that's analogous; yes, it's very analogous.

Bradley: We're running short on time. I know we only got a chance to talk about a few of the topics of your book. I was wondering if there were any other essential points we should examine before winding down.

Stanovich: The point I ended on at the end of the Clever Hans example, where I said that observation in the natural world is not enough. I have a chapter in the book that explains some of the misconceptions that people have about laboratory research and why its done and why its necessary. That chapter is designed to address the lay person's criticism of psychology. You often hear, “how can you find out anything in a laboratory?” that's a very ironic criticism. If you think very hard about the Clever Hans example, and you can see just how ironic that criticism is. The artificiality of the laboratory is not a problem for psychology, it's not an Achilles' Heel, it's not an issue for criticism, it's an issue of strength. It's nothing to apologize for, it exemplifies the advance of psychology as a science. We have laboratories, and what's done in laboratories doesn't always look like the natural world, because we need to intervene, because we need to manipulate, because we need to isolate variables that in the natural world are all entangled with each other. That makes a nice addendum to our discussion of Clever Hans, so I thought that I would throw that in to end things.

Bradley: Is it possible that there is a drawback to some of these controls that we put on some research? For example, what pops to my mind is research on depression. In a lot of clinical trials, they remove a large number for being too suicidal, or for having too severe a depression, so for ethical reasons those subjects are removed from the groups. Is that an example of somewhere where we have to be cautious of the conclusions we draw from studies like that?

Stanovich: Yes, there are certainly differential attrition issues in real life study of something of that type. But absolutely right, and you told us the step that we take from that, that is when various methodological strictures can't be met, we appropriately increase the tentativeness of the conclusions. But I'd like to throw in another thing here, that is another principle from my book, and that is the principle of converging evidence. In any complex problem, we often have very different types of investigation that we use to converge on the conclusion. We can study things in the field, we can study things epidemiologically, we can set up a field experiment – that's why we're all taking the little aspirins now, because in the past someone set up a study where a random group got the little aspirin, and another randomized group received the placebo, and of course there are problems there, there are attrition issues in medical research as well. Ah!, but then we add to that laboratory trials, laboratory models with nonhuman animals. That involves an extrapolation. Each of these methodologies for getting at the problem have fundamental problems of the type that you're alluding to, Dave. But when very different investigations all point in the same direction, and they are so different that they have very different weaknesses, instead of each one having exactly the same weaknesses. It's not that they don't have weaknesses, they do, it's that each of the weaknesses is a little different, and despite that, the bulk of evidence points in one direction, we then start to feel confident in science of drawing a conclusion despite the flaws in any single study. Each of our methods have flaws, but when things all point in the same direction, we say that the evidence has converged. We look for converging evidence, not necessarily a single knock-down study that proves everything.

Bradley: Thank you very much. Dr. Stanovich is the author of How to Think Straight about Psychology. You can find links to his home page and links to his look on Amazon.com on the website for this podcast, Psyconoclasm.com. Dr. Stanovich, it's been an enlightening discussion. Thank you so much for coming out on a limb and being my guest today.

Stanovich: It's been great talking with you, Dave. I've enjoyed participating.

Bradley: Thank you again to Dr. Stanovich for being my first guest on Psyconoclasm. His book, How to Think Straight About Psychology, is very readable and accessible to the lay reader, I recommend it. And thank you, for taking the time to listen. I hope you'll drop me a line and let me know how I did: host@psyconoclasm.com. I'm still getting my footing as far as audio and editing, interview tempo, interview length, and personal tone, so any feedback, positive or negative, is appreciated. So send me an e-mail or leave a comment on the website, where you'll find a transcript of this interview later in the week. Psyconoclasm is distributed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 license. You can reach me by e-mailing host@psyconoclasm.com, I promise to get back to you. Opening and closing music by Spiraling, from the Podsafe Music Network, available at music.podshow.com. I'll end this week with aquote from ancient Greek philosopher Epicurus: “One must not pretend to philosophize, but philosophize in reality. For we do not need the semblance of health but true health.” Talk to you soon.