The Trolley Problem With Science

 

By Neuroskeptic in Discover Magazine

 

Imagine a scientist who does an experiment, and doesn’t like the results. Perhaps the scientist had hoped to see a certain pattern of findings and is disappointed that it’s not there.

Suppose that this scientist therefore decided to manipulate the data. She goes into the spreadsheet and adds new, made-up data points, until she obtains a statistically significant result she likes, and publishes it.

That’s bad.

Now, start this scenario over. Suppose that rather than making up data, the scientist throws it out. She runs the experiment again and again (without changing it), throwing out the results every time they’re wrong, until eventually, by chance, she obtains a statistically significant result she likes, and publishes it.

Is that bad?

Yes, but isn’t it less obviously bad than data fabrication? I’m talking about an intuitive level. We feel that fabrication is clearly outrageous, fraudulent. Cherry-picking is bad, no-one denies it, but it doesn’t generate feelings to the same extent.

Cherry-picking goes on in science, and I don’t know a scientist who doubts that it’s more common than fraud. Yet we treat fraud much more harshly, regardless of the extent of the manipulation or the amount of money and prestige at stake.

One fabricated point of data is misconduct; a thousand unpublished points of data is merely a ‘questionable practice‘.

Why?

I think it’s a trolley problem.

Imagine an out of control trolley was going to smash into five people and kill them. You happen to be standing by a lever that would divert the trolley onto another track, where it would hit a person, and kill him. Do you pull it?

Many people say that they would not pull the lever, even though this means that four extra deaths occur. They wouldn’t pull it because they don’t like the idea of committing the act of killing someone. Even though the decision not to act causes more harm, it doesn’t feel as bad, because it’s (in some intuitive sense) an act of omission.

Even people who do pull the lever feel conflicted about it.

I think the mentality is the same in science. Data fabrication is like pulling the lever – it’s committing deception. Cherry-picking is a sin of omission – you wait for the right data to come to you, and report that, omitting to report the rest of the data that points to a different conclusion.

The end result is the same – misleading results. But they feel different.

 

The entire post with comments can be found at:  http://blogs.discovermagazine.com/neuroskeptic/?p=4070