The Scientific Method
Randomized Double-Blinded Trials

http://www.humantruth.info/randomized_double_blind_trials.html

By Vexen Crabtree 2018

Scientific experiments that involve human subjects are influenced by psychological effects that often skew the results. People simply behave differently and report symptoms differently, if they think they are part of an experiment. If you divide people into one group who are exposed to one factor, and a different group that is not (the control group, the first group performs better no matter what the tested factor is. There are three steps to solving this problem:

Blinding: If the subjects do not know who is in the control group and who is in the experimental group, then, both groups are affected by the same psychology, and the difference between them must be due to the factor being tested.₁_,₂
Randomization: When it comes to dividing subjects into groups, experimenters' subconscious biases or assumptions may lead to them selecting 'the right kind' of person for the experimental group, and this can bias the results. The solution is to randomly divide patients into groups.₂
Double-blinding: Experimenters tend to accidentally give away clues as to which group is which, and so double-blind trials are even better, where the experimenter and observers themselves do not know which group the subjects are in.₃

The result, double-blinded randomized trials, are the standard mode of operation for all experiments that involve humans - be it drugs trials, tests on sociological effects, psychological investigations, taste-trials, or any other field of research.

1. What is Science and the Scientific Method?

#epistemology #knowledge #science #truth

The "Scientific Method" is a set of steps taken to ensure that conclusions are reached sensibly, experiments designed carefully, data is interpreted in accordance with the results of tests, and that procedures can be verified independently. The system is designed to reduce as much Human error and bias as possible₄. Ideas and theories must be subject to criticism, and counter-evidence must be taken into account in order to produce new and more accurate theories₅. Everything should be questioned. Most people cannot "do" science and do not have the skills to analyse data in an adequate manner₆. The Scientific Method is hard and demanding, with high standards of ethical conduct expected - Daniel C. Dennett wrote that "good intentions and inspiration are simply not enough" (2007)₇. The effects of science can impact on all human development, changing entire societies₈. Science has been responsible for a staggering increase in human knowledge, human technology and human capabilities over the last few centuries.₉

See: What is Science and the Scientific Method?.

2. Blind Trials

When it comes to testing theories that involve humans, many psychological factors come into play. These can change the results of experiments. Good experimental designs will try to minimize indirect effects of psychology and other confounding factors in order to test just the hypothesis.

One technique to remove psychological factors is blinding. This means that the patients do not know if they are receiving the actual treatment, or a placebo₁. Otherwise, patients who think they're receiving new treatments tend to behave differently, and skew the results.

“A review of blinding in all kinds of trials of medical drugs [in particular,] found that trials with inadequate blinding exaggerated the benefits of the treatments being studied by 17 per cent.”
"Bad Science" by Ben Goldacre (2008)₂

3. Randomisation

When it comes to dividing subjects into groups, experimenters' subconscious biases or assumptions may lead to them selecting 'the right kind' of person for the experimental group, and this can bias the results. The solution is to randomly divide patients into groups, using a system that does not allow individual humans to make a choice about who goes into what group.

“Randomisation is not a new idea. It was first proposed in the seventeenth century by John Baptista van Helmont, a Belgian radical who challenged the academics of his day to test their treatments like blood-letting and purging. [...] Does randomisation matter? As with blinding, people have studied the effect of randomisation in huge reviews of large numbers of trials, and found that the ones with dodgy methods of randomisation overestimate treatment effects by 41 per cent.”
"Bad Science" by Ben Goldacre (2008)₂

4. Double Blinding

In their interactions with patients, or with observers who are recording the results of tests, experimenters tend to accidentally give away clues as to which group is which. If those who record the results know which group is which, they are liable to pay more attention to the experimental group, or, in many ways, to treat the groups differently. This distorts the results.

Double blind trials are even better than blind ones, as the experimenters and observers themselves do not know which group the subjects are in. Therefore they are much more likely to treat both groups the same, and to process data about each group in a fairer manner.

Only at the end of the trial when the results are in, does an independent team reveal which group of patients was the control group and which were receiving active factors. This last-minute revelation also helps make it difficult to withhold results if the experiment didn't go the way that the researcher wanted - because towards the end, the results are already known to many people involved.

“A placebo is most likely to work if the doctor genuinely believes it to be a cure and communicates that conviction to the patient. Quack doctors who have a talent for invoking the placebo effect sometimes attract huge followings. [...] The randomized placebo-controlled, double-blind trial is perhaps the most important advance in medical research. Neither patient nor the doctor knows whether a treatment is real or a sugar pill. By eliminating even unconscious bias, it makes it possible to determine what works and what doesn't.”
"Superstition: Belief in the Age of Science" by Robert L. Park (2008)₃

Sir Francis Galton (1822-1911) was a British academic who is credited with creating the methods discussed above:

“Sir Francis Galton... Always on the lookout for ways to keep from being misled, Galton was at least a century ahead of his time. [...] He studied medicine at his father´s wish, but was dismayed by the lack of data on which to judge whether common medical procedures employed in Victorian England, such as purgatives and bleeding, actually helped. [...]
Recognizing the capacity of humans for self-deception, Galton played a key role in what is arguably the most important advance in the history of medical research: the randomized placebo-controlled, doubleblind trial.”
"Superstition: Belief in the Age of Science" by Robert L. Park (2008)₁₀

5. Case Study: Acupuncture

#acupuncture #placebo_effect #scientific_method

Ben Goldacre studied the evidence surrounding acupuncture and found that properly controlled studies show no advantage of acupuncture above the placebo effect, but poorly controlled studies where psychological factors are not properly accounted for end up showing that acupuncture is effective.₂. Poor experimental design can have real effects - and in this day and age where most news outlets don't have scientifically qualified staff that examine trials before they are printed, this means that large numbers of people can be easily duped into trusting something that ought to be debunked.

Details: Critical Thinking on Acupuncture: How Does it Work?: 2. Scientific Studies of Acupuncture.

1. What is Science and the Scientific Method?

2. Blind Trials

3. Randomisation

4. Double Blinding

5. Case Study: Acupuncture

6. Other Elements of the Scientific Method