This was going to be my notes on all the rationality references, but I also added a few on quotes that seem important to the story / generally interesting / whatever.
Decide what evidence justifies what conclusion before the experiment. No changing afterwards.
Observation trumps theory.
Cognitive biases exist (bystander effect).
Emotions will take your rationality when you most need it.
Robert Cialdini's Influence: The Psychology of Persuasion may be worth reading.
""I ask the fundamental question of rationality: Why do you believe what you believe? What do you think you know and how do you think you know it?"
"Learning to admit you're wrong." is key to true science.
Always questioning yourself, always taking another look at things you've always taken for granted
"Conscientiousness is just about as important as raw intelligence in determining life outcome"
Watch out for fantasies of greatness?
I got nothing
Try not to be the dark lord.
"You couldn't change history. But you could get it right to start with. Do something differently the first time around."
"what should be, and what is, are two different things"
Anthropic bias - is this the right reference?
"Up until this point Harry had lived by the admonition of E. T. Jaynes that if you were ignorant about a phenomenon, that was a fact about your own state of mind, not a fact about the phenomenon itself; that your uncertainty was a fact about you, not a fact about whatever you were uncertain about; that ignorance existed in the mind, not in reality; that a blank map did not correspond to a blank territory. There were mysterious questions, but a mysterious answer was a contradiction in terms. A phenomenon could be mysterious to some particular person, but there could be no phenomena mysterious of themselves. To worship a sacred mystery was just to worship your own ignorance."
DO NOT MESS WITH TIME
Win the long game, not the short game.
""Reality is always lawful," said Harry, "even if we don't know the law.""
privileging the hypothesis - don't promote one hypothesis without reason.
"You know," Harry said icily, "in one of my quite fascinating Muggle books, they describe a study in which people managed to make themselves look very smart by asking questions about random facts that only they knew. Apparently the onlookers only noticed that the askers knew and the answerers didn't, and failed to adjust for the unfairness of the underlying game.
Maintaining the current status hierarchy and enforcing its rules seems ever so much more wise and moral and important when you are on the top and doing the enforcing than when you are on the bottom, and I can cite studies to this effect if required. I could go on for several hours about this point, but I will leave it at that."
Learn how to lose.
"Professor Quirrell gaze seemed to come straight out at Harry from the repeater screen. "What you demonstrated today, Mr. Potter, is that - unlike those animals who keep their claws sheathed and accept the results - you do not know how to lose a dominance contest. When a Hogwarts professor challenged you, you did not back down. When it looked like you might lose, you unsheathed your claws, heedless of the danger. You escalated, and then you escalated again. It started with a slap at you from Professor Snape, who was obviously dominant over you. Instead of losing, you slapped back and lost ten points from Ravenclaw. Soon you were talking about leaving Hogwarts. The fact that you escalated even further in some unknown direction, and somehow won at the end, does not change the fact that you are an idiot.""
The import of an act lies not in what that act resembles on the surface, Mr. Potter, but in the states of mind which make that act more or less probable."
Harry blinked. He'd just had the dichotomy between the representativeness heuristic and the Bayesian definition of evidence explained to him by a wizard.
If you had a medical test that was only wrong one time in a thousand, sometimes it would still be wrong anyway.
preference utilitarianism - mentioned but not necessarily advocated.
"Yes," Harry said. "I shall achieve my objectives through the power... of Science"!
science is how you go about understanding and controlling the universe
"It's a new idea to me. And one of the hidden secrets of science, passed down from a few rare teachers to their grad students, is how to avoid flushing new ideas down the toilet the instant you hear one you don't like."
"She's getting you to make an advance commitment now, and then the pressure of consistency will make you agree with whatever she says afterward!"
"I offer you power," said the shadowy figure, "and I will tell you of that power and its price. The power comes from knowing the shape of reality and so gaining control over it. What you understand, you can command, and that is power enough to walk upon the Moon. The price of that power is that you must learn to ask questions of Nature, and far more difficult, accept Nature's answers. You will do experiments, perform tests and see what happens. And you must accept the meaning of those results when they tell you that you are mistaken. You will have to learn how to lose, not to me, but to Nature. When you find yourself arguing with reality, you will have to let reality win. You will find this painful, Draco Malfoy, and I do not know if you are strong in that way. Knowing the price, is it still your wish to learn the human power?"
"The power of science comes from finding out the way Nature really is that can't be changed by arguing!"
He was violating the experimental procedure he'd written down beforehand, which was a sin, and he was violating it because he didn't like the results he was getting, which was a mortal sin, you could go to Science Hell for that
Nowadays it was called "blinding" and it was one of the things modern scientists took for granted. If you were doing a psychology experiment to see whether people got angrier when they were hit over the head with red truncheons than with green truncheons, you didn't get to look at the subjects yourself and decide how "angry" they were. You would snap photos of them after they'd been hit with the truncheon, and send the photos off to a panel of raters, who would rate on a scale of 1 to 10 how angry each person looked, obviously without knowing what color of truncheon they'd been hit with. Indeed there was no good reason to tell the raters what the experiment was about, at all. You certainly wouldn't tell the experimental subjects that you thought they ought to be angrier when hit by red truncheons. You'd just offer them 20 pounds, lure them into a test room, hit them with a truncheon, color randomly assigned of course, and snap the photo. In fact the truncheon-hitting and photo-snapping would be done by an assistant who hadn't been told about the hypothesis, so he couldn't look expectant, hit harder, or snap the photo at just the right time.
""Fine. But if my books were worth a carp, that's a kind of fish not anything bad, they would have given me the following important piece of advice: When there's a confusing problem and you're just starting out and you have a falsifiable hypothesis, go test it. Find some simple, easy way of doing a basic check and do it right away. Don't worry about designing an elaborate course of experiments that would make a grant proposal look impressive to a funding agency. Just check as fast as possible whether your ideas are false before you start investing huge amounts of effort in them. How does that sound for a moral?""
Asch's Conformity Experiment, you might find it quite amusing. For now I'll just note that it's dangerous to worry about what other people think on instinct, because you actually care, not as a matter of cold-blooded calculation.
it was simply a rule of how scientists operated that you had to try to disprove your own theories, and if you made an honest effort and failed, that was victory.
life isn't fair
Litany of Gendlin: What's true is already so, owning up to it doesn't make it worse
"Now when you're dealing with a confusing problem and you have no idea what's going on, the smart thing to do is figure out some really simple tests, things you can look at right away. We need fast tests that distinguish between these hypotheses. Observations that would come out a different way for at least one of them compared to all the other ones."
But Harry Potter held up a hand. "Law of science, Draco. First I tell you the theory and the prediction. Then you show me the data. That way you know I'm not just making up a theory to fit; you know that the theory actually predicted the data in advance. I have to explain this to you anyway, so I have to explain it before you show me the data. That's the rule. So put on your cloak and let's sit down."
Lots about DNA, but this is not interesting.
"To become a scientist. You questioned one of your beliefs, not just a small belief but something that had great significance to you. You did experiments, gathered data, and the outcome proved the belief was wrong. You saw the results and understood what they meant." Harry Potter's voice was faltering. "Remember, Draco, you can't sacrifice a true belief that way, because the experiments will confirm it instead of falsifying it. Your sacrifice to become a scientist was your false belief that wizard blood was mixing and getting weaker."
Belief in belief
You might wish you believed in blood purism, but you'll always expect to see happen just exactly what would happen if there was only one thing that made you a wizard. That was your sacrifice to become a scientist."
that which can be destroyed by the truth should be.
You're awakened as a scientist now, and even if you never learn to use your power, you'll always," Harry gasped, "be looking, for ways, to test, your beliefs -"
One case of true curiosity had the same sort of redeeming power in rationality that one case of true love had in movies.
And a strange thought came to Draco then, something Harry had kept talking about yesterday.
And the thought was: Test it.
That was when Father had told Draco about the Rule of Three, which was that any plot which required more than three different things to happen would never work in real life.
Father had further explained that since only a fool would attempt a plot that was as complicated as possible, the real limit was two.
There was a story from the dawn days of Artificial Intelligence - back when they were just starting out and no one had yet realized the problem would be difficult - about a professor who had delegated one of his grad students to solve the problem of computer vision.
Harry was beginning to understand how that grad student must have felt.
What method had the Dark Lord used? Come to think, the fact that the Dark Lord had somehow managed to survive the death of his first body was almost infinitely more important than the fact that he'd tried to take over magical Britain -
I hadn't noticed the immortality theme so early before.
she had a time and a place and a beetle to be
"Miss Skeeter," said Quirrell, "I had hoped to find some lever that would prove persuasive. Yet I find that I cannot deny myself the pleasure of simply crushing you."
Harry then launched into an explanation of a test done by someone named Norman Maier, who was something called an organizational psychologist, and who'd asked two different sets of problem-solving groups to tackle a problem.
The problem, Harry said, had involved three employees doing three jobs. The junior employee wanted to just do the easiest job. The senior employee wanted to rotate between jobs, to avoid boredom. An efficiency expert had recommended giving the junior person the easiest job and the senior person the hardest job, which would be 20% more productive.
One set of problem-solving groups had been given the instruction "Do not propose solutions until the problem has been discussed as thoroughly as possible without suggesting any."
The other set of problem-solving groups had been given no instructions. And those people had done the natural thing, and reacted to the presence of a problem by proposing solutions. And people had gotten attached to those solutions, and started fighting about them, and arguing about the relative importance of freedom versus efficiency and so on.
The first set of problem-solving groups, the ones given instructions to discuss the problem first and then solve it, had been far more likely to hit upon the solution of letting the junior employee keep the easiest job and rotating the other two people between the other two jobs, for what the expert's data said would be a 19% improvement.
Starting out by looking for solutions was taking things entirely out of order. Like starting a meal with dessert, only bad.
(Harry also quoted someone named Robyn Dawes as saying that the harder a problem was, the more likely people were to try to solve it immediately.)
"But that is improbable," said Harry. "To quote Douglas Adams, the impossible often has a kind of integrity which the merely improbable lacks."
Quirrel has a suspicious interest in prophecy:
"He didn't have any choice," said Harry. "Not if he wanted to fulfill the conditions of the prophecy."
"Give me that," said Professor Quirrell, and the newspaper leaped out of Harry's hand so fast that he got a paper cut.
It was like the man had cast off a pretense of mortality.
Coming at things from the wrong angle:
"I have a feeling," Harry said finally, "that we're coming at this from the wrong angle. There's a tale I once heard about some students who came into a physics class, and the teacher showed them a large metal plate near a fire. She ordered them to feel the metal plate, and they felt that the metal nearer the fire was cooler, and the metal further away was warmer. And she said, write down your guess for why this happens. So some students wrote down 'because of how the metal conducts heat', and some students wrote down 'because of how the air moves', and no one said 'this just seems impossible', and the real answer was that before the students came into the room, the teacher turned the plate around."
"That your strength as a rationalist is your ability to be more confused by fiction than by reality," said Harry. "If you're equally good at explaining any outcome, you have zero knowledge. The students thought they could use words like 'because of heat conduction' to explain anything, even a metal plate being cooler on the side nearer the fire. So they didn't notice how confused they were, and that meant they couldn't be more confused by falsehood than by truth. If you tell me that the centaurs were under the Imperius Curse, I still have the feeling of something being not quite right. I notice that I'm still confused even after hearing your explanation."
Professor Quirrell waved a hand in the direction of the door, and a bolt slid shut. "Note the bolt on the inside. This room, Mr. Potter, is known as Mary's Room. It happens to be proof against all scrying, and I do mean all; Dumbledore himself could detect nothing of what happens here. Mary's Room is used by two kinds of people. The first sort are engaged in illicit dalliances. And the second sort lead interesting lives."
Nestled up against the wall, where Professor Quirrell had stumbled, glistened the crushed remains of a beautiful blue beetle.
Your dissociative talent is so strong that I am surprised to find no other signs of childhood abuse,
Why is Snape so unwashed anyway? Such a bizarre character trait.
"All right," Harry said slowly. It was hard to see how having a conversation and being unable to tell anyone could be more constraining than not having it, in which case you also couldn't tell anyone the contents. "I promise."
Never give anyone wise advice unless you know exactly what you're both talking about. Got it.
Harry made a disgusted face. He was still having trouble getting used to that sort of thing, it shouldn't matter what something was named if you knew what it was.
"Congratulations indeed," said Dumbledore. "Even I did not make any original discoveries in Transfiguration before the age of fourteen. Not since the day of Dorotea Senjak has any genius flowered so early."
There was a legendary episode in social psychology called the Robbers Cave experiment . It had been set up in the bewildered aftermath of World War II, with the intent of investigating the causes and remedies of conflicts between groups. The scientists had set up a summer camp for 22 boys from 22 different schools, selecting them to all be from stable middle-class families. The first phase of the experiment had been intended to investigate what it took to start a conflict between groups. The 22 boys had been divided into two groups of 11 - - and this had been quite sufficient.
Fast. Creative. Unpredictable. Non-homogenous. Don't just obey orders, think about whether what you're doing right now makes sense.
Harry wasn't quite as sure as he'd pretended that this was the optimum of military efficiency... but he'd been given a golden opportunity to change how some students thought about themselves, and that was how he intended to use it.
"You know," Harry said lightly, knowing that none of the others would understand the real message passing between them, "it just goes to show that you should always question everything you see your role models doing, and ask why it's being done, and whether it makes sense in context for you to do it too. Don't forget to apply that advice to real life, by the way. And thanks for the slow-moving clustered targets."
questioning authority was a merely practical technique for real life.
I got nothing.
"Yesss," hissed Harry, like the boy thought he was a Parselmouth.
Imposition of order equaled escalation of chaos
His right hand stretched out, fingers open and spread. "Division is weakness," said the Defense Professor. His hand closed into a tight fist. "Unity is strength."
Reversed stupidity is not intelligence; the world's stupidest person may say the sun is shining, but that doesn't make it dark out...
The Rise and Fall of the Third Reich
Mr. Potter, the stupidity of Quidditch is transparent to you because you did not grow up revering the game. If you had never heard of elections, Mr. Potter, and you simply saw what is there, what you saw would not please you
"You must learn to blur your vision until you can see the forest obscured by the trees. "
"Harry?" called a thin, blonde woman whose perfectly smooth and unblemished skin made her look a good deal younger than thirty-three; and Harry realized with a start that it was magic, he hadn't known the signs before but he could see them now. And whatever sort of potion lasted that long, it must have been terribly dangerous, because most witches didn't do that to themselves, they weren't that desperate...
You know, Draco, just as the fundamental question of rationality is 'What do I think I know and how do I think I know it?', there's also a cardinal sin, a way of thinking that's the opposite of that.
"Or perhaps," Harry said more softly, "it is the foe that makes the Gryffindor, as it is the friend that makes the Hufflepuff, and the ambition that makes the Slytherin. I do know that it is always, in every generation, the puzzle that makes the scientist."
"Death is bad," said Harry, discarding wisdom for the sake of clear communication. "Very bad. Extremely bad. Being scared of death is like being scared of a great big monster with poisonous fangs. It actually makes a great deal of sense, and does not, in fact, indicate that you have a psychological problem."
See, there's this little thing called cognitive dissonance, or in plainer English, sour grapes. If people were hit on the heads with truncheons once a month, and no one could do anything about it, pretty soon there'd be all sorts of philosophers, pretending to be wise as you put it, who found all sorts of amazing benefits to being hit on the head with a truncheon once a month. Like, it makes you tougher, or it makes you happier on the days when you're not getting hit with a truncheon. But if you went up to someone who wasn't getting hit, and you asked them if they wanted to start, in exchange for those amazing benefits, they'd say no.
The young boy stood very straight, his chin raised high and proud, and said: "There is no justice in the laws of Nature, Headmaster, no term for fairness in the equations of motion. The universe is neither evil, nor good, it simply does not care. The stars don't care, or the Sun, or the sky. But they don't have to! We care! There is light in the world, and it is us!"
Harry had chosen the question carefully. Not, do you believe in an afterlife? but simply Is there an afterlife? What people really believed didn't seem to them like beliefs at all. People didn't say, 'I strongly believe in the sky being blue!' They just said, 'the sky is blue'. Your true inner map of the world just felt to you like the way the world was...