The Scientific Method and Why Controls Are So Important

My regular readers will understand that this is a science based gardening blog and that I rely heavily on the scientific method to debunk gardening myths, but what does “the scientific method” really mean and why is it so important? In today’s post I am going to focus on one aspect of this, the control.

What is the Scientific Method?

The scientific method is a formal way to find the answer to a question. It is well established and accepted as the best way to understand the world. All scientific studies follow the same process.

At the heart of the scientific method are two key concepts; measurements and controls.

The measure part is critical. Instead of looking at a field of potatoes and using a gut feeling to decide how well they are growing, the experiment is designed to actually measure something. It might count the number of potatoes produced, the number of damaged leaves, the height of the plants, or the amount of potassium in each leaf. The thing that is measured is decided upon before the experiment is started and the measuring process must have zero biases or as few as possible.

The second key aspect of the scientific method is the control. Instead of using one field of potatoes for the test, the field is divided into several plots; some are treated, and others are not. All plots are then measured in exactly the same way.

In agriculture or gardening, the control is especially important, because nature has too many variables. One summer is sunnier than the next, or it might rain more this year than last year. There might be more grasshoppers this year. You can’t control nature.

Why are Controls Needed?

I was out in the garden a few days ago, thinking about my vegetable harvest this year. This year I used extra fertilizer in the hopes of increasing my yield. A soil test earlier in the year showed that phosphate and potassium were a bit low. I had not used fertilizer or manure for a few years and so this was not a surprise.

Did the extra fertilizer work?

My tomato harvest is better than ever. I have never had so many tomatoes, but to be honest the plants have lost more leaves than normal. I always get septoria leaf spot, but this year it seems really bad. It looks as if extra fertilizer does produce more tomatoes, but it also makes the leaves more susceptible to disease. That last point corresponds well with what I have read – too much nitrogen makes leaves softer and more disease prone.

On the other hand my green bean yield is way down. I thought beans were heavy feeders, but I guess not. Too much nitrogen must be making more leaves and less fruit.

What is the problem with these conclusions?

I assumed that all variables, except fertilizer, were the same between the two years. My test plot was this year, and my control was last year.

That is faulty logic and does not follow the scientific method.

Oh….. I left out one minor thing. We had an unusual, very early, warm spring, and then when our normal spring usually starts, it turned quite cold. June and July were extremely hot and not just for a day or two which is normal, but for a week at a time. We never have weather like that.

On closer inspection the bean plants flowered well, but not all flowers got pollinated. Maybe that is due to the hot weather?

Did the Fertilizer Work?

1) Without a control I can’t reach any conclusion.

2) I never measured anything. My observations are based on my gut feelings. If I had counted the number of beans I might have found that both years produced the same number?

I can’t reach any conclusion, because I did not have a proper control. You can not compare one year to the next!

Unreliable Anecdotal Evidence

What I have described above is called anecdotal evidence. Most gardeners reach conclusions based on such incomplete information. They then form beliefs and post their so-called facts on social media, where other gardeners accept them as truths.

This is how myths are born.

When these same gardeners are asked for their evidence, two thing invariable happen. They have no proof, and they get offended.

The reason for a lack of proof is simple. They had no controls and they did not measure anything. After many years of online discussions, only one or two people who did a control.

Most people don’t understand the scientific method and have no idea that their experience is unreliable. They truly believe. So any questioning of their belief is perceived as a person attack – it rarely is.

If people understood the scientific method better, they would be less likely to be offended.

Scientific Facts are Always Under Scrutiny

Another aspect of the scientific method is that all work is under review, all the time. All “certain facts” are up for debate.

Does humus exist? It’s existence was a “scientific fact” for over 100 years, although some scientist always disagreed. More recently, science has mostly come to the conclusion it does NOT exist in nature. It is an artifact that is only found in the lab. For more on this see my post: Humus Does Not Exist – Says New Study

Scientists don’t get offended when their work is questioned. OK, maybe they do a little bit, but they know that it is part of the process and that it is the only way we can move our knowledge base forward.

Should Gardeners Follow the Scientific Method?

It is unrealistic for gardeners to carry out full scientific studies because they rarely have enough plants or space. They also don’t have the time to do it properly.

However, in many cases they can improve the value of their testing by doing simple side by side controls.

My cucumbers normally get powdery mildew by late summer and I wanted to see if a bicarbonate spray might work. Rather than spraying all of my plants, which is what most gardeners would do, I selected one of my three plants and sprayed it. All plants were planted at the same time, grew in the same soil quite close together, and got the same weather and watering. It is not fully scientific because the there are not enough replicates and I was not going to count actual spots of mildew, but at least there is a control.

So did the spray work? I don’t know. I have no powdery mildew this year (not yet anyways), but I may have some downy mildew. What I can say is that the spray did not harm the leaves enough to cause visual signs.

This illustrates the value of a control. If I had sprayed all the plants, I would have concluded that the bicarbonate spray works, at least for powdery mildew.

I don’t expect gardeners to follow a rigorous scientific method, but they can do much better than they are. And they can learn about the scientific method so that they at least understand the limitations of anecdotal evidence, both theirs and that of others.

Pest Control and the Scientific Method

Pests are a great concern to gardeners who like using all kinds of home remedies. If you believed everything you read on social media you would conclude that just about anything in the kitchen reduces any pest problem.

I’ve found that beer is the best pest control agent.

I had a terrible infestation of viburnum sawfly on one of my shrubs for several years in a row. I also get mugo pine sawflies on another plant. I sprayed the plants with a 10% beer/water solution and it not only got rid of the current years infestation but it prevented their return the following year.

If you read this on social media, you might believe the story – may gardeners would.

What if I changed the story a bit. Instead of spraying the beer, I took out a lawn chair, sat beside the plants and drank the beer. After all, who in their right mind wastes beer on a shrub?

Would you still believe the story? Probably not.

Neither story is any more creditable than the other. Neither one has a control – there was only one viburnam and one mugo pine involved – no control. From a scientific point of view I can’t reach any conclusion.

Except for the part about the beer, this story is perfectly true. I did have these infestations for several years. I did absolutely nothing and they stopped. I have no idea why, but Mother Nature must be helping me out. Any home remedy that I might have tried would be declared as “working”, because the problem went away.

Most of the home remedies on social media have no scientific basis and any report you see is just anecdotal evidence with no controls, so they have little or no value.

Tomatoes Don’t Ripen Better on the Vine

A few weeks ago I read that tomatoes can be harvested early, brought inside and they ripen and taste as good as vine ripened tomatoes. What nonsense is that? I have been gardening for many years and I know they taste better when they ripen on the vine. And my dad says the same thing!

I could have been insulted; but wasn’t. Instead, I thought, “well, maybe I am wrong and I can learn something here. In the worst case I can find proof that this is a myth and write about it”.

By the way – I never believe the first person, or website that tells me something new.

So I researched tomato ripening. I looked for scientific evidence for or against the above idea. Turns out I was wrong. Something I believed all my life, turned out to be wrong.

I wrote about it in Ripening Tomato Myths – Both on the Vine and in the Home.

When tomatoes are picked at breaker stage they ripen perfectly inside. You don’t even need a banana or paper bag.

Is the Scientific Method Such a Big Deal?

Yes it is. Perhaps it does not really matter in the garden. Most of us garden for fun. Nobody will be harmed if you spray your shrub with beer.

But when it comes to health products and nutrition we have the same problem. There are all kinds on stupid products being promoted that have zero scientific basis. People will write 5 star reports because “they tried it and it works”, without ever running a control.

We are in the middle of a pandemic. One day soon, someone will announce a vaccine. What if that vaccine was tested on only one person and the manufacturer declared, “it works”, would you take it?

Most of you would not use the vaccine without a proper scientific study including controls and measurements and that is smart.

Apply the same logic to the nonsense on social media and the countless new products that don’t work. Unless the testing includes controls or measurements, don’t believe it.