Anecdotal Evidence – Not Worth The Screen It’s Displayed On

Anecdotal evidence is everywhere in our daily lives. Most new products depend on it to make sales and most commercials use it to convince you to buy. In gardening, it is a very common way for people to get information about a problem. Go to any gardening social media site and ask about curing something. A dozen people will come back with different cures that they have tried – and THEY ALL WORK!

Baloney!

Why is anecdotal evidence not worth much? Why do we believe it so much? Good questions to ask if you are trying to understand why we have so many garden myths.

Anecdotal Evidence – What Is It?

I thought this was a simple question before I started to research it. There are a lot of different definitions so I will just come up with one of my own which tries to encompass and simplify many of the others.

Anecdotal evidence is an observation someone has made which has lead them to a conclusion or a belief.

In the gardening world it usually takes the form of: I did ABC, and noticed that DEF happened. Based on this observation I believe that GHI is correct. An example that I just read on a Face Book site is; “I put crushed egg shells under my hosta, and it stopped the slugs from eating my hosta leaves.”

No further details are given. We don’t know the changes in weather that took place, the type of hosta, the location, how the number of holes were counted, or how much damage there was before the egg shells were added. We don’t even know how the egg shells were crushed or placed on the soil. We really do not know anything, and in most cases, even the person making the claim can not answer any of these questions. This was simply an observation – I use that term loosely – at one point in time.

But ….. it was enough to convince this person that egg shells = no slug damage.

Science, government and industry does not really care about gardening and almost no funding is available for scientific studies about gardening. As a result much of our information comes from anecdotal evidence.

Anecdotal evidence is NOT scientific evidence, and in almost every case, the observation does NOT logically lead to the conclusion reached.

In the egg shell example, there are several other possible explanations that are just as logical given the evidence. A toad could have moved into the area and eaten the slugs. Or, the egg shells might have been added just when the garden went into a hot and dry session, in which case slugs go underground (ref 2) to wait it out.

Anecdotal Evidence vs Scientific Fact

Given a choice between anecdotal evidence and scientific fact, many people believe the anecdotal evidence.

I discussed DEET in DEET – Is It Safe? It has been used for 55 years with no health issues. Most government bodies consider it safe. And yet people refuse to believe the science. Anecdotal evidence recently showed that it might cause autism – it doesn’t. It might cause brain damage – it doesn’t. But people are ignoring the science in favor of some vague anecdotal evidence.

Why is anecdotal evidence believed so strongly?

Michael Shermer (ref 1), put it this way. “we have evolved brains that pay attention to anecdotes because false positives (believing there is a connection between A and B when there is not) are usually harmless, whereas false negatives (believing there is no connection between A and B when there is) may take you out of the gene pool.” ie you die.

This may be true for something like DEET, but why do people believe egg shells keep slugs away?

For the person who tries egg shells, on their own, without any information from others, it becomes a predetermined beleif. They would only try it if they had some confidence that egg shells will work. Why try the experiment if you don’t believe it will work? Once they have this belief – results would have to be very negative to destroy the belief.

When someone sees statements in social media saying egg shells work, they also start forming the belief. We trust people. Why would anyone say egg shells work, when they don’t work? That’s not logical. So if a stranger says they work – they must work. Now that the belief is set in the mind, it is difficult to reverse.

Brain research has shown that we create a new belief easily. Our brains are designed to work on patterns, and patterns rely on us having beliefs. We function better if we believe something, than having a void. We would rather have a false belief, than no belief at all. Once a belief is formed, we are very reluctant to change it.

The average person sees more statements on a daily basis that are based on anecdotal evidence than ones based on scientific evidence. So their beliefs are mostly based on anecdotal evidence.

Anecdotal Evidence is Illogical

Someone responded to a social media question about keeping slugs from eating the hostas, by reporting that crushed egg shells work. They have tried them and they no longer have a slug problem.

Sounds very convincing and believable.

I asked them for more information on how they did their test. What kind of hosta was used? What kind of control was used? The reply was no surprise to me: “My testing was not that scientific – it is just an anecdotal observation”.

Let me describe my own anecdotal evidence about a different garden problem. I have a viburnum that for three years running got demolished by viburnum beetle larvae. I had to do something. In spring the following year, I took out my lawn chair, sat down in front of the bush, and had a beer – a Canadian of course. I warned the shrub not to get eaten again.

It worked! In fact it worked so well that I had no problems for the next 4 years.

Do you believe my anecdotal evidence? Probably not. Why not? Because you can use common sense to conclude that there was no cause and effect here. Drinking beer can’t possibly keep the beetle from laying eggs.

What if my anecdotal evidence was a bit different. Instead of drinking the beer, I poured the beer onto the shrub, and that kept the beetle from laying eggs.

Now that is a bit more believable. After all, beer is used in all kinds of garden home remedies so it must have some powers. Maybe the beetle doesn’t like the taste of the beer? And then there is all of the pseudo scientific mumble jumble I could invent to convince you. Are you a believer now?

What if my anecdotal evidence went like this. I made a special concoction containing beer, Epsom salts, bone meal, and a dash of Ivory soap. Poured it on the shrub and it worked!

Now I have most of the people on social media convinced that it must work. The concoction is so complex few people can use logic to argue against it. The evidence is clear – it worked.

Why is this last example of anecdotal evidence so much more convincing?

It is important to understand that humans are not very logical thinkers. Most people read, and then either believe or don’t believe. When asked why they made the choice they did, most can’t explain it logically. Most people base their conclusion on their former beliefs – which don’t need an explanation.

Anecdotal Evidence – Not Worth The Screen It is Displayed On

There are many reasons why anecdotal evidence is not worth much, and I will look at a couple of them.

People are poor observers. Our preconceived ideas cloud our judgement. We can’t remember what we saw. This is made very clear when people are asked to describe a previous event. If the evidence does not include some kind of numerical values – like counting holes in leaves – the observation is not very reliable.

People are biased by their former beliefs – I discussed this above.

People don’t use controls. This is my biggest complaint about anecdotal evidence. It almost always lacks a control. It is not that hard to use two hostas, growing in the same garden, with the same amount of shade, watering and size. Put egg shells under one and not the other. Count holes before and after adding the egg shells.

If someone explained this experiment, and presented data for the number of holes they found, it would be much more believable. Some of the human bias has been removed. Quite a few variables such as temperature, precipitation and time of year have been removed by using a control.

It still would not be scientific evidence, since the number of plants used is not statistically significant, and there are too many other variables that need to be excluded, but it would be a huge step in the right direction.

Consider my anecdotal evidence for the Viburnum bush. If I had included a control, one bush treated, and another not treated, there is a very good chance I would have found my treatment was no more effective than doing nothing – which is in fact what really happened. The problem just went away on its own, until this year.

Without a proper control – most observations are of little value except in helping someone design a proper experiment with a good control.

Next time you read something about gardening, the environment, food, government, health or any other aspect of your life, check for and understand the control. If it is not described – don’t believe the conclusion.

By the way – egg shells do not work, no matter how many people say they do. See the post Egg Shells Control Slugs – Do They Really Work? for more details.

reference:

1) How Anecdotal Evidence Can Undermine Scientific Results; http://www.scientificamerican.com/article/how-anecdotal-evidence-can-undermine-scientific-results/

2) Slug and Snail FAQ’s;  http://www.allaboutslugs.com/faq/