This subject already seems to have come up, so let's discuss it.
I love scientific studies. I have read many of them on many subjects from cover to cover. There is much to be learned by them. I often disagree with the conclusions drawn by the researchers though.
"Post hoc ergo procto hoc" (After this therefore because of this) is the primary error in logic and is a trap fallen into by humans and animals alike. The big temptation of this error is that "Post hoc erog procto hoc" is a good basis for a theory. The error isn't using it for a theory it's using it as proof.
Let's examine the error of this, first. Every morning at my house, the roosters crow. Every morning after the roosters crow, the sun comes up. Does this mean that the roosters cause the sun to come up? Because we can't see any mechanism to connect them other than the sequence of events, most of us would assume that the roosters are not the cause.
Every culture I know of has folk tales and or jokes to make fun of this error. One in our culture is "pull my finger". Because you pull the finger and immediately afterwards something happens, your brain makes a connection and for a second you fall prey to this error. Then after a second or two you brain catches up with itÂ’s processing and the absurdity of that connection hits you and you laugh. The Africans often tell the Â“roosters causing the sun to come upÂ” story and the Lakota tell it as the horses whinnying. Foolish anthropologists often record these stories as if the people telling them believe this connection, but my experience with primitive cultures is that they tell these stories to teach the error of that way of thinking. Of course they watch to see if the anthropologists believe the foolish conclusion and after watching them diligently write it down without so much as a comment, or a chuckle the natives shake their heads at the foolishness.
I have done things while driving that were immediately followed by some noise. My first conclusion is that I caused the noise and IÂ’m wondering what it is IÂ’ve caused. After trying a couple of more times and the noise does not follow it, I find out it was one of my children making the noise. It was mere coincidence that they happened simultaneously.
Any Â“statistical proofÂ” really constitutes no proof. As I collect a larger and larger sample it becomes more and more likely that what IÂ’m seeing statistically is an actual connection and not a coincidence, but it never constitutes analytical proof. Unless I have a mechanism and can prove that mechanism is the cause, by some means other than simple statistics, then I only have an increasing likelihood.
I can prove this to anyone who understands basic probability. What are the odds that if I flip this quarter it will land on heads? 50/50. So I flip it and it comes up heads. What are the odds if I flip this quarter again that it will come up heads again? 50/50 same as before. So I flip it and it comes up heads. I personally have flipped a quarter 27 times in a row and got heads every time. Does this prove that the odds are not 50/50? No it proves my sample was too small to be statistically valid. How many times to I have to flip it before my results are an absolute fact? No matter how many times I do it, I only get closer and closer to the actual answer. It is not a matter of absolute proof but a matter of accumulating large enough sample. The larger the sample the closer I get to the answer, but itÂ’s like the old math problem of going half way and half the remaining way and half of that and so on. When will I get to the end? Never. I can only get closer and closer.
This was just trying to prove that flipping a quarter has odds of 50/50. The life cycle of any organism is infinitely more complex than flipping a quarter and affected by more things than we can know. If I do a certain thing and get a certain result how many times will it take to prove absolutely that what I did contributed to that result? If I have a very large sample and I have a very large success rate compared to a very large control group with a very small success rate, it is very likely that my theory is correct. The smaller the sample, the smaller the difference in success rate and the more other variables that could contribute to success or failure, or even worse, the more skewed those variables are in favour of either group, the less valid my results are.
This is all assuming a lack of prejudice on the part of the researcher. As one of my teachers (he was a carpenter with a lot of wisdom, not a professor) once said, Â“everyone thinks their idea is better because they thought of itÂ”. This seems intuitively obvious, but it is important. I have a natural prejudice to my ideas because they fit my way of thinking. If they didnÂ’t, I wouldnÂ’t have thought of them! This is why in the scientific community it is important to be able to reproduce the results. Reproducibility is a good test, especially if someone else is doing the second or third study than did the first. I may eliminate some of the prejudice and also it may change some of the other unmeasured and unaccounted for variables.
The second problem with research is the motivation for doing it. The motivation for doing research is almost (but not always) personal gain. A few actual altruistic people have a love of some fellow creature, or some fellow humans and is actually involved because they want to reduce suffering or solve someoneÂ’s problems. Unfortunately these people are not well funded and their research is usually not well received.
Most research is funded by and prejudiced by some entity that has an agenda to prove their solution and that solution has to be something they can market and sell, preferable with a patent of copyright or some other protection to provide them with a monopoly. There is no profit in, and therefore no money for, research into simple common solutions to problems.
IÂ’m sure some will disagree with me, but I think some entities, such as the USDA, have their own agenda that has been revealed by observing them over time. The big agenda of any government agency is to get more money, more power and try to appear that they are serving the purpose they were put there for. In the case of the USDA, itÂ’s obvious they have favored chemicals over natural solutions. They favor anything that appears to help the economy of agribusiness. This doesnÂ’t mean just the small farmer/beekeeper etc. but the whole of agribusiness. They seem to like to see money changing hands because it helps the economy.
Just because research was done on a subject and the researchers came to some conclusion, does not make that conclusion the truth.
Now, while we are talking about facts, letÂ’s talk about one of the reasons some people do not like science and prefer their own opinions. I covered one above, which is that we always like our own ideas because they fit our way of thinking, but the other is that people are fond of saying that something has not been proven scientifically as if that means it is NOT true because it has NOT been proven. Anything we have not proven is simply something that has not been proven. Because I have not proven it true does not make it false.
In 1847 Dr. Ignaz Philipp Semmelwis instituted the practice of hand washing before delivering babies. He came to this conclusion simply by the statistical evidence that mothers and babies who were attended by doctors who washed had less mortality than ones attended by doctors who did not wash. This was "Post hoc ergo procto hoc" . The doctors washed and less babies died. This is not scientific proof and therfore his colleages did not consider it scientific proof. Why? Because he could not provide a mechanism to explain it nor an experiment to prove that mechanism. Because he was a proponent of something he would not prove absolutely, he was driven out of the medical community as a quack. This is an example of something that had not been proven scientifically.
In the 1850Â’s when Louis Pasteur and Robert Koch created the science of microbiology and the Â“germ theoryÂ” of disease Dr SemmelwisÂ’s theory finally was proven scientifically. Now there was a mechanism and they were able to create experiments to prove that mechanism. My point is, that it was true before they proved it and it was true after they proved it. The truth did not change because they proved it. There was, previous to this proof, evidence that would lead to the practice of hand washing, but not proof.
We live our lives and make decisions all the time based on our view of the world. This view is not truth, but it is based on our experience and our learning. Sometimes something comes along to change that view and we accept it because the evidence is strong enough. To ignore evidence that fits the pattern of what we see around us because it has not been proven is foolish. To ignorantly hang on to things that are proven to not be true is equally foolish. But just because the majority believe something to be proven does not mean has been. Just because the majority of people believe something is true does not make it true.
In conclusion, I would say, read research with a grain of salt. Look at their methods. Think about the contributing issues that they have overlooked. Pay attention to anything that would skew the population they are studying or the population of the control group. Look into whether or not the study has been duplicated and were the results similar or contrary. What was the size of the population? What was the difference in success? If there is only a small difference it may not be statistically important. Even if there is a large difference, was it duplicated at that large a difference? Also what might be the prejudices of the people doing the research?