Cherry picking

Whose science?

Eating eggs is like smoking a pack of cigarettes a day…

http://www.freerepublic.com/focus/f-news/2919284/posts

(C 20) So, somehow eating eggs, and presumably the controversial cholesterol in them, does as much harm to bodily fluids and organs as smoking a pack of cigarettes? Seriously? This isn’t science, it’s hyperbole.

Western science typically spins its story in a proscribed way. Another type of science, one based mostly on observation and intuition, is the subject of my next book; I discuss it some in the research of Ton Baars, at the end of this chapter. For now, we’ll look at our problematic Western science.

Typically, there’s a statement of a hypothesis or idea or question, a review of what’s been done, selection of data methods, and discussion of data (Results). Oh yes, and there’s recommendations for future research. And a requisite self-bashing on the limits of the work (why you can’t generalize form the research – like saturated fat causes a rise in cholesterol, of course maybe Twinkies would, too. Or maybe sleep deprivation, arguing with your spouse, or basking in the light of a full moon (we don’t know, unless someone has researched these questions). So, if the low-cholesterol (a sign of disease to begin with) mice, show drops in blood (serum) cholesterol when eating only lean meat, it’s interesting, but none of this really amounts to much in terms of a dietary recommendation (although many have tried).

One reason for this is that many elements of bias can be introduced at any point along the way – scientists are supposed to account for these in their work. More importantly, selection of the research question itself is hardly unbiased. One easily-recognizable and flagrant type of bias is what’s known as “cherry-picking.” Cherry-picking is the purposeful selection of certain data to prove one’s hypothesis, and ignore the data that do not.

Here’s a hypothetical example: Study on the effects of fast food on weight. I put together my research design and gather data (with the help of high school students), using a simple research method (systematic random sampling). In this study, every sixth person is asked about his or her weight at a Mac Donald’s for a study linking weight and number of visits to MacDonald’s each week.  A positive correlation is expected (more visits, more weight), but what if that’s not what you get? You start throwing out some data that don’t make sense or don’t fit – that’s cherry picking.