The Human Truth Foundation

What is Science and the Scientific Method?

By Vexen Crabtree 2014

#epistemology #knowledge #science #truth

The "Scientific Method" is a set of steps taken to ensure that conclusions are reached sensibly, experiments designed carefully, data is interpreted in accordance with the results of tests, and that procedures can be verified independently. The system is designed to reduce as much Human error and bias as possible1. Ideas and theories must be subject to criticism, and counter-evidence must be taken into account in order to produce new and more accurate theories2. Everything should be questioned. Most people cannot "do" science and do not have the skills to analyse data in an adequate manner3. The Scientific Method is hard and demanding, with high standards of ethical conduct expected - Daniel C. Dennett wrote that "good intentions and inspiration are simply not enough" (2007)4. The effects of science can impact on all human development, changing entire societies5. Science has been responsible for a staggering increase in human knowledge, human technology and human capabilities over the last few centuries.6

One of the most ednearing descriptions of science is that of E. O. Wilson:

Book CoverScience, to puts its warrant as concisely as possible, is the organized, systematic enterprise that gathers knowledge about the world and condenses the knowledge into testable laws and principles. The diagnostic features of science that distinguish it from pseudoscience are first, repeatability: The same phenomenon is sought again, preferable by independent investigation, and the interpretation given to it is confirmed or discarded by means of novel analysis and experimentation. Second, economy: Scientists attempt to abstract the information into the form that is both simplest and aesthetically most pleasing - the combination called elegance - while yielding the largest amount of information with the least amount of effort. Third mensuration: If something can be properly measured, using universally accepted scales, generalizations about it are rendered unambiguous. Fourth, heuristics: The best science stimulates further discovery.

"Consilience: The Unity of Knowledge" by E. O. Wilson (1998)7

And some shorter comments:

Science is not merely a body of knowledge, but a method, [...] a progress report - one that changes constantly as new techniques and instruments allow us to probe the universe more deeply. [...] This method begins with many observations over a period of time. From the trends found through observations, scientists can model the particular phenomena we want to understand. Such models are always approximations of nature, subject to further testing.

"Astronomy" by Fraknoi, Morrison & Wolff (2017)2


1. The Scientific Method

1.1. To Turn a Hypothesis Into a Theory, It Must Be Tested8

#knowledge #science #scientific_method #theories

Science begins with evaluations of what might be true, given the existing evidence9 (a hypothesis)10,11. The implications of this idea are then compared to our existing knowledge, to see how well it fits12. Then, tests are devised to see if the new idea's predictions will match experimental results. If a hypothesis cannot be tested, then, it is not scientific13,14,15,16. If the hypothesis fails, then, it is wrong15. If it passes, then, it becomes (or supports) a theory17. All science, no matter how durable, remains a theory until proven wrong10. This way, science comprises of a continual series of adjustments and improvements to theories as they are adapted to fit new evidence. Theories that cannot be adjusted are replaced by theories that fit the evidence better.

For more, see: To Turn a Hypothesis Into a Theory, It Must Be Tested.

1.2. Systematic Literature Review and Meta-Analysis - Overcoming Selection Bias

#science #scientific_method #selection_bias #thinking_errors

To prevent errors it is best to conduct a methodical approach in evaluating evidence. In the scientific method this is called Systematic Review or Literature Review. The combining of statistical data from experiments into a single data-set is called a meta-analysis, a process which makes statistical analysis more accurate18,19. Specific search criteria are laid out before any evidence is sought and each source of information is methodically judged according to pre-set criteria before looking at the results. This eliminates, as much as possible, the human bias whereby we subconsciously find ways of dismissing evidence we don't agree with. Once we've done that, then we tabulate the results of each source, and it is revealed to us which sources confirm or disconfirm our idea. The Cochrane Collaboration does this kind of research on healthcare subjects and as a result of its gold-standard methodical approach they have "saved more lives than you can possibly imagine"20. Systematic Literature Review is thus one of the most important checks-and-balances of the scientific method.

For more, see: Systematic Literature Review and Meta-Analysis - Overcoming Selection Bias.

1.3. Peer Review - Overcoming Poor Design and Poor Argumentation

#experts #peer_review #science #scientific_method

Peer-review is an important part of the scientific method21. It is naive to believe that scientists act without passion, subconscious bias or social influences when they conduct studies - this is admitted by scientists themselves22. So scientific publications are sent to a number of recognized experts in appropriate fields for review. The publishing journal will wait for the results and feedback from those experts, and decide whether they want to publish the paper or not. Some get published straight away. Others might be sent back to their authors with the scientific concerns of the experts put forth, and the journal will wait for an edited version to be resubmitted, before (probably) sending it off for peer review again. Some studies are found to be "fatally flawed" and so never get published.

Likewise, some papers can be removed from the original publication even years after the journal was printed, such as where a study is later on found to be flawed, completely erroneous, or to be fraudulent. Sometimes, things such as undisclosed funding can cause an article to be withdrawn, such as when a scientist is secretly paid by an industry body to produce favourable "science" to support the industry in question. Such tactics have been employed by oil, tobacco, drinks, alternative therapies and lobby groups for other industries that are responding to criticism from governments and scientists by attempting to "buy" scientific credence for their activities. Being removed from previous publications is a serious indication that something was wrong with the study.

See:

Peer-review will look at the methods used in the research, the strength of the statistical analysis, whether proper care is taken with the wording of the conclusion, whether the blurb does indeed reflect what the data shows, etc. The idea is not to publish misleading or faulty papers. This quality-control boosts both the quality of the publication itself (hence, making the journal more trusted) and aids science in general: to publish in a scientific journal, you have to display the correct care and attention to detail, and to avoid the many pitfalls of bad science and poor methodology.

The scientist must convince colleagues, editors of journals, and ultimately a broad cross-section of other scientists that her hypothesis is provisionally correct.

"Astronomy" by Fraknoi, Morrison & Wolff (2017)23

The main strength of this approach is that the scientific methods involved, and the conclusions, are scrutinized by experts who do not have a vested interested in the quality of the study. It is well-known that those who conduct studies often believe their own stated conclusions and are biased towards seeing their own work in a positive light24, and peer-review is the process used to allow critical evaluation of work from others' point of view. As in all human endeavours, a second set of eyes will often reveal problems that the original author would never spot.

1.4. Reproducibility and Independent Verification of Results

#science

Reproducibility and independent verification are integral parts of the scientific method25,26,27. Whether it is research in physics, chemistry or psychology, the results of any experiment must be reproduced independently in another location. This checks that the results were not the result of unintentional but consistent human bias in the original experiments. There have been plenty of cases where a scientist declares results and describes his experiment in a scientific journal but other researchers fail to reproduce in their own experiments. If results cannot be duplicated then the data is not accepted as valid. This is why newspaper reports on single experiments should be heeded with care: any experimenter can claim results but if others around the world cannot verify the procedure then the chances are the experiment was flawed. Results should only be acclaimed once they have been verified and this is why sometimes public announcements are not made for some time, especially with highly technical or long-term experiments. Always think to check who done the original experiments, and who verified the methods.

Book CoverScience requires that a phenomenon be reliably produced in different laboratories for it to be accepted as genuine. Whoever claims to have discovered a phenomenon must describe in sufficient detail how it was produced so that other investigators, following similar steps, can reproduce it themselves. This requirement of replicability applies to all fields of science. [...]

Although the history of science contains numerous examples of an investigator's expectations clouding his or her vision and judgement, the most serious of these abuses are overcome by the discipline's insistence on replicability and the public presentation of results. Findings that rest on a shaky foundation tend not to survive in the intellectual marketplace. [...] The biggest difference between the world of science and everyday life in protecting against erroneous beliefs is that scientists utilize a set of formal procedures to guard against [...] sources of bias and error.

"How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life" by Thomas Gilovich (1991)27

1.5. Occam's Razor: Simplicity & Fewer Assumptions are Better

#assumptions #atheism #epistemology #philosophy #science #theism

The aim of science is, on the one hand, a comprehension, as complete as possible, of the connection between the sense experiences in their totality, and, on the other hand, the accomplishment of this aim by the use of a minimum of primary concepts and relations.

Einstein (1936) in Albert Einstein (1954)28,29

A hypothesis has assumptions which must then be backed up by evidence if the idea is to take ground. Clearly, the fewer such assumptions are, the better. In general this has led to a principal in science that the theory with fewest assumption and fewest complicated side-effects is probably a better theory than others. This is commonly called 'Occam's Razor':

Occam is best known for a maxim which is not to be found in his works, but has acquired the name of 'Occam's razor'. This maxim says: 'Entities are not to be multiplied without necessity.' Although he did not say this, he said something which has much the same effect, namely: 'It is vain to do with more what can be done with fewer'. That is to say, if everything in some science can be interpreted without assuming this or that hypothetical entity, there is no ground for assuming it. I have myself found this a most fruitful principal in logical analysis.

"History of Western Philosophy" by Bertrand Russell (1946)30

In philosophical arguments, it is frequently used to mean that if a particular belief or idea leads to the requirement for a massive amount of special explanation, other odd conclusions, and outstanding complexity, then such a belief is probably wrong.

For example, in the theological debate between atheists and theists, both attempt to account for the existence of the universe using similar ideas. Atheists believe that the universe is self-contained and had no preceding 'cause'. Theists believe that the universe was created by God, and that God is self-contained and has no 'cause'. Both theories contain a similar uncaused element, but, the theistic theory contains an additional assumption that the uncaused cause is a god. By employing Occam's razor, many would guess that the simpler, atheistic, theory is more likely to be correct because it contains less unanswered questions (assumptions) than the theist one.

1.6. Randomized Double-Blind Trials31

#placebo_effect #science #scientific_method

Scientific experiments that involve human subjects are influenced by psychological effects that often skew the results. People simply behave differently and report symptoms differently, if they think they are part of an experiment. If you divide people into one group who are exposed to one factor, and a different group that is not (the control group, the first group performs better no matter what the tested factor is. There are three steps to solving this problem:

  1. Blinding: If the subjects do not know who is in the control group and who is in the experimental group, then, both groups are affected by the same psychology, and the difference between them must be due to the factor being tested.32,33

  2. Randomization: When it comes to dividing subjects into groups, experimenters' subconscious biases or assumptions may lead to them selecting 'the right kind' of person for the experimental group, and this can bias the results. The solution is to randomly divide patients into groups.33

  3. Double-blinding: Experimenters tend to accidentally give away clues as to which group is which, and so double-blind trials are even better, where the experimenter and observers themselves do not know which group the subjects are in.34

The result, double-blinded randomized trials, are the standard mode of operation for all experiments that involve humans - be it drugs trials, tests on sociological effects, psychological investigations, taste-trials, or any other field of research.

"The Scientific Method: Randomized Double-Blinded Trials" by Vexen Crabtree (2018)

See: The Scientific Method: Randomized Double-Blinded Trials.

1.7. Be Open to Criticism and Make Way for New Evidence

#knowledge #science #scientific_method

The nature of the scientific method lends itself to continual improvement in knowledge35, based on a continual stream of new data. Scientists spend "a great deal of time questioning and challenging one another"36 and frequently admit when their theories have been superseded or corrected by their peers37,38. The procedures of peer review and independent verification both ensure that mistakes in theory, application or analysis of data are spotted when other scientists examine and repeat the experiment. "The essence of science is that it is self-correcting" says the eminent scientist Carl Sagan (1995)39. Yet it is still rare that new evidence completely destroys a theory. Philosopher Bertrand Russell states that "theories, if they are important, can generally be revived in a new form"40. Hence, theories undergo continual improvement.

Still perhaps it may appear better, nay to be our duty where the safety of the truth is concerned, to upset if need be even our own theories, specially as we are lovers of wisdom: for since both are dear to us, we are bound to prefer the truth.

"Ethics" by Aristotle (350BCE)41

Sometimes, individual theories do have to be abandoned. Rarely, entire scientific paradigms are questioned such as when Newtonian physics gave way to Einstein. New evidence can cause entire theoretical frameworks to be undermined, resulting in a scientific revolution. "Science involves an endless succession of long, peaceful periods [... and then periods of] scientific revolution" (Kuhn 1962). To overthrow a theory requires strong evidence and a full cycle of the scientific method. To overthrow an established theory, which is often supported by many other theories and mountains of testing and evidence, requires extraordinary evidence42.

See: Scientific Theories Must Make Way for New Evidence.

2. A History of Science

#history #science

2.1. Ionia, 6th century BCE

#ionia

Book Cover2,500 years ago, there was a glorious awakening in Ionia: on Samos and the other nearby Greek colonies that grew up among the islands and inlets of the busy eastern Aegean Sea. Suddenly there were people who believed that everything was made of atoms; that human beings and other animals had sprung from simpler forms; that diseases were not caused by demons or the gods; that the Earth was only a planet going around the Sun. And that the stars were very far away. [...]

In the 6th century B.C., in Ionia, a new concept developed, one of the great ideas of the human species. The universe is knowable, the ancient Ionians argued, because it exhibits an internal order: there are regularities in Nature that permit its secrets to be uncovered. [...] This ordered and admirable character of the universe was called Cosmos. [...]

Between 600 and 400 B.C., this great revolution in human thought began. [...] The leading figures in this revolution were men with Greek names, largely unfamiliar to us today, the truest pioneers in the development of our civilization and our humanity.

"Cosmos" by Carl Sagan (1995)43

The city of Alexandria was the greatest in the ancient world. Its famous Library of Alexandria was constructed in the third century BCE by the Greek Kings, the Ptolemys. It became a scientific research centre and publishing capital of the world. Ionians forged ahead in many arenas of knowledge. "Eratosthenes accurately calculated the size of the Earth [...], Hipparchus anticipated that the stars come into being, slowly move during the course of centuries, and eventually perish, it was he who first catalogued the positions and magnitudes of the stars to detect such changes. Euclid produced a textbook on geometry from which humans learned for twenty-three centuries"44. Such astounding wisdom backed up by studious thinking and experimentation could have launched the world into the modern era. But it didn't.

Rising superstition, the taking of slaves and the growth of monotheistic religion led to the demise of scientific enterprise. The culture changed. The last great scientist of Alexandria, Hypatia, was born in 370CE at a time when the "growing Christian Church was consolidating its power and attempting to eradicate pagan influence and culture". Cyril, the Archbishop of Alexandria, considered Hypatia to be a symbol of the learning and science which he considered to be pagan. "In the year 415, on her way to work she was set upon by a fanatical mob of Cyril's parishioners. They dragged her from her chariot, tore off her clothes, and, armed with abalone shells, flayed her flesh from her bones. Her remains were burned, her works obliterated, her name forgotten. Cyril was made a saint"44.

The last remains of the Alexandrian Library were destroyed not long after Hypatia's death. Nearly all the books and documents were completely destroyed. The Western Dark Ages had begun, and all knowledge and science was forgotten in the West for over a thousand years.

2.2. The Rise of Science From the 17th Century

During the Middle Ages, the West had again begun to contribute to the science and learning of the world. In the interim the Arabic lands to the East had thankfully translated Greek works and carried the torch of knowledge. Philosopher-scientists emerged from the West and East, and debated the finer points of epistemology. As the centuries went on, thought became freer, and as the material life improved, the seventeenth century saw the dawning of a new age of human thought: modern scientific methods were back on the menu after nearly 2000 years in hiatus.

Almost everything that distinguishes the modern world from earlier centuries is attributable to science, which achieved its most spectacular triumphs in the seventeenth century. The Italian Renaissance, though not medieval, is not modern; it is more akin to the best age of Greece. [...] The modern world, so far as mental outlook is concerned, begins in the seventeenth century. No Italian of the Renaissance would have been unintelligible to Plato or Aristotle; Luther would have horrified Thomas Aquinas, but would not have been difficult for him to understand. With the seventeenth century it is different: Plato and Aristotle, Aquinas and Occam, could not have made head nor tail of Newton. [...]

Four great men - Copernicus [1473-1543], Kepler, Galileo, and Newton - are pre-eminent in the creation of science. Of these, Copernicus belongs to the sixteenth century, but in his own time he had little influence.

"History of Western Philosophy" by Bertrand Russell (1946)45

3. The Battles Between Science and Religion

Further reading on Science and Religion:

As a scientist, I am hostile to fundamentalist religion because it [...] teaches us not to change our minds

"The God Delusion" by Prof. Richard Dawkins (2006)46

4. Open Access to Research

Secrecy can impede the progress of science, and openness is a hallmark of good science"

Prof. A. Scott
In Skeptical Inquirer (2007)47

Open Access speeds up the worldwide application of scientific research and allows theories and results to be tested, checked and analysed to scientists across the world, leading to more reliable science, data, technology for everyone. As much science is funded by government, the general populace should therefore have free access to its results.

4.1. Publishing Charges

#australia #belgium #canada #denmark #finland #germany #hungary #netherlands #portugal #sweden #UK #USA

A mass of valued research comes from university researchers that are funded by national government, costing hundreds of millions of dollars each year to support. Their results are published in peer-reviewed journals that have to be paid for; the publications are then bought by the Universities where the research is done. This is highly inefficient, and the public end up paying twice in order to read the results (when they pay the taxes that supports the research, and when they pay for the publications).

Prof. Michael Geist holds the Canada Research Chair in Internet and E-commerce Law at the University of Ottawa, and says “The model certainly proved lucrative for large publishers [but] the emergence of the internet dramatically changes the equation. Researchers are increasingly choosing to publish in freely available, open access journals posted on the internet, rather than in conventional, subscription-based publications”49.

Sweden leads the world in open access to research archives (see the chart). A Swedish project call the "Directory of Open Access Journals" links to scientific open access journals. It now lists more than 2500 worldwide, including over 127000 articles.49

Aided by the Open Journal System, a Canadian open source software project based at Simon Fraser University in British Columbia, more than 800 journals, many in the developing world, currently use the freely available OJS to bring their publications to the internet.

For those researchers committed to traditional publication, open access principles mandate that they self-archive their work by depositing an electronic copy in freely available institutional repositories shortly after publication. This approach grants the public full access to the work, while retaining the current peer-reviewed conventional publication model.

While today this self-archiving approach is typically optional, a growing number of funding agencies are moving toward a mandatory requirement. These include the National Institutes of Health in the US, the Wellcome Trust in the United Kingdom, and the Australian Research Council. Moreover, some countries are considering legislatively mandating open access.

"Push for open access to research" by Michael Geist, Prof (2007 Feb 28)49

4.2. An EU-wide Open Access Principle

Last month five leading European research institutions launched a petition that called on the European Commission to establish a new policy that would require all government-funded research to be made available to the public shortly after publication. That requirement - called an open access principle - would leverage widespread internet connectivity with low-cost electronic publication to create a freely available virtual scientific library available to the entire globe.

Despite scant media attention, word of the petition spread quickly throughout the scientific and research communities.

Within weeks, it garnered more than 20,000 signatures, including several Nobel Prize winners and 750 education, research, and cultural organisations from around the world.

In response, the European Commission committed more than $100m (£51m) towards facilitating greater open access through support for open access journals and for the building of the infrastructure needed to house institutional repositories that can store the millions of academic articles written each year.

"Push for open access to research" by Michael Geist, Prof (2007 Feb 28)49

It seems right that such a depository of publicly-funded research should be made available for free to the public that paid for it.

5. Overcoming Thinking Errors

The point of The Scientific Method is to overcome the many sources of human error that arise from flawed cognitive processes and our imperfect perceptions of reality.