The Human Truth Foundation

The Crabtree Scale of Sources
Who Can We Trust?

By Vexen Crabtree 2018

#mass_media #news #social_media #truth

The world is awash with misinformation, exaggeration, hyperbole, well-intentioned confusion, popular mistakes in thinking and delusions about facts. These filter into our mass media and social media outlets and the scale of the problem makes it difficult to select good, high-enough quality and trustworthy sources of information. Here is a scale of reliability, going from zero out of ten (unregulated and anonymous internet websites) through to regulated and quality news (5/10) and on to those peer-reviewed, well-established and independently verified outputs of academia.

(0/10) Unregulated and Anonymous Internet Websites

#internet #social_media

Websites, forums or social media outlets where anyone can post anything are very open to abuse by organized propaganda bodies (who pay people to 'troll', spread doubt, and spread false stories), and open to all manner of crazy, ill-informed, semi-informed and daft stories. Some of these become very popular and repeated in other news outlets ('going viral...'), but, popularity does not indicate better quality.

Content monitoring (regulation) often doesn't help, especially if it focuses on illegal content or on censoring adult content. Neither of these change the quantity of false or inaccurate information in articles that are written in a responsible-looking manner.

Researcher Nic Newman was one of the academics who in 2016 joined in alarm-calls about the influence of poor sources of news on social media platforms such as Facebook, which for many people provides their primary insight into the news of the world. The downside, he argues, is that "the impact of traditional fact- and reason-based-news brands is diluted [due] to a huge range of other content that includes disreputable news, partial news and fake news"1.


(1/10) Anecdotes and Personal Experience

#beliefs #news #subjectivism

Anecdotal stories based on personal experience, but without theoretical basis or experimental evidence to back them up, are often exaggerated, mis-told, taken out of content, the result of all kinds of neurological or psychology misdirections, or the result of poorly analyzed situations.


(2/10) Trashy, Popularist News Outlets

#daily_mail #mass_media #newspapers #paranoia #the_sun #uk_newspapers #western_culture

Downmarket media publications reflect - and exaggerate - many of the fears of society. News outlets have dropped most fact-checking and critical analysis steps in order to churn out news more cheaply and quicker and as a result daft and untrue stories are appearing in mainstream news2,3. There are virtually no checks or quality control mechanisms that newspapers have to adhere to, and, occasional outrages against press misbehaviour are quickly forgotten by paying customers. The purpose of it all is (1) sell more newspapers, or (2) influence public opinion. People are all too willing to believe exaggerated claims. People want their lives to be part of historical drama4. The millennium bug, worldwide pandemics, moral panics and fear that society is going wrong all betray humankind's neophobic reactions to progress and change. Newspaper editors pick on this fear and concoct alarmist stories from everyday events and statistics; for example they publish alarmist articles on dangers from mobile phone masts even though there are none, and there are almost no good-news stories about children despite massively improved circumstances5. Many editors and media owners have explained the usefulness of fear-mongering and sensationalism - it certainly sells more copy than balanced news. Fears become amplified and made more real by their appearance in headlines, creating a hysteria about a topic whereas in reality things are much more mundane and acceptable6. Professional broadcaster Fraser McAlpine says "news outlets are behaving like spoof sites, and they're making spoof sites look like sensible news" and people are finding it harder to tell the difference7. Modern newspapers and news outlets are producing low quality, misleading and untrue stories because they are driven by consumers who prefer entertainment, gloom and outrage rather than serious text of reasonable reading. Some news outlets remain high quality, with good fact-checking and little clickbait; but the modern era in which quantity-of-page-views is more important than reputation means that quality appears to be permanently lost from popular outlets. Always remember that after thousands of hyped-up press warnings, on midnight of the 31st of December 1999, nothing happened.8


(3/10) Unregulated News Outlets

#mass_media #news #newspapers

Unregulated news outlets that are perceived to have good fact-checking and quality-checking mechanisms can easily appear mainstream. But many organized lobby groups produce genuine-seeming publications (often attracting general public followers) but which routinely publish biased and skewed evidence and argumentation. The smoking lobby, petrol industries and fat/sugar food industries have been caught doing this repeatedly.


(4/10) Regulated News Outlets

#mass_media #news #newspapers

Regulated news outlets which enforce editorial policy and some fact-checking within national or international frameworks of good journalism do manage to produce a lot of good content. Many countries require mass-media journalists to be registered and the production of false material, secret public-relations material, commercially biased text and poorly-checked content can result in journalists being de-barred. The regulatory bodies should be independent of government and politics.

Warning: Assume that the quicker news is reported after an event, the poorer the analysis and fact-checking. News that is produced simultaneously by multiple outlets normally comes from news agencies who are motivated to produce content quickly rather than carefully. Often, news outlets copy each other's stories, and falsehoods can find themselves widely spread before they are correctly questioned or corrected (if ever).

(5/10) Quality News

#mass_media #newspapers #the_guardian

Some news and media outlets are known for good quality, judging from academic research into errors and public-relations material in the press9 - some pride themselves on quality. But, even so, the speed at which news is produced puts a serious limit on its reliability.

A study showed that there was a correlation between internet usage and depression and speculated Internet use might cause depression. The correlation did not show causation, you see, it was perfectly possible that if you are depressed, then you go online more (i.e., the causation could be the other way around). But The Guardian, which you would expect to have higher standards of reporting than this, stated that an increase in Internet usage led to an increase on the depression scale. [The Guardian, 1998 Aug 31, Loneliness of virtual living].

"Research Methods and Statistics in Psychology" by Hugh Coolican (2004)10


(5/10) Expert Content

#experts #thinking_errors

Expert content produced by an author who has correct qualifications. But be warned, if the material is published directly on their own websites or in outlets where there is no independent critical analysis, then, such material could be flawed. Content published in this manner should be routinely considered biased or flawed and needs commentary by other experts in the same field of expertise. Hence, on its own expert content isn't quite as good as you might like to assume.


(8/10) Peer-Reviewed Publications

#epistemology #knowledge #peer_review #science #truth

Expert content published in peer-reviewed magazines with a reputation for strong critical pre-analysis of articles, and in a publication that actively removes historical material if it is found to be flawed. Generally speaking, these are academic and scientific publications who follow The Scientific Method. But be warned not to assume that single-study results are trustworthy (statistical anomalies do occur), nor that the paper doesn't contain undetected flaws.

The "Scientific Method" is a set of steps taken to ensure that conclusions are reached sensibly, experiments designed carefully, data is interpreted in accordance with the results of tests, and that procedures can be verified independently. The system is designed to reduce as much Human error and bias as possible11. Ideas and theories must be subject to criticism, and counter-evidence must be taken into account in order to produce new and more accurate theories12. Everything should be questioned. Most people cannot "do" science and do not have the skills to analyse data in an adequate manner13. The Scientific Method is hard and demanding, with high standards of ethical conduct expected - Daniel C. Dennett wrote that "good intentions and inspiration are simply not enough" (2007)14. The effects of science can impact on all human development, changing entire societies15. Science has been responsible for a staggering increase in human knowledge, human technology and human capabilities over the last few centuries.16


(9/10) Peer-Reviewed and Well-Established

#peer_review #science #scientific_method #truth

Peer-review is an important part of the scientific method17. It is naive to believe that scientists act without passion, subconscious bias or social influences when they conduct studies - this is admitted by scientists themselves18. So scientific publications are sent to a number of recognized experts in appropriate fields for review. The publishing journal will wait for the results and feedback from those experts, and decide whether they want to publish the paper or not. Some get published straight away. Others might be sent back to their authors with the scientific concerns of the experts put forth, and the journal will wait for an edited version to be resubmitted, before (probably) sending it off for peer review again. Some studies are found to be "fatally flawed" and so never get published.

Likewise, some papers can be removed from the original publication even years after the journal was printed, such as where a study is later on found to be flawed, completely erroneous, or to be fraudulent. Sometimes, things such as undisclosed funding can cause an article to be withdrawn, such as when a scientist is secretly paid by an industry body to produce favourable "science" to support the industry in question. Such tactics have been employed by oil, tobacco, drinks, alternative therapies and lobby groups for other industries that are responding to criticism from governments and scientists by attempting to "buy" scientific credence for their activities. Being removed from previous publications is a serious indication that something was wrong with the study.

Expert content published in peer-reviewed magazines that has been debated, statistically summarized, independently confirmed, attacked and defended, and yet still remains widely accepted, is as trustworthy as any theory or idea can be. Such 'established' theories can still be debated and evidence can still accumulate for/against them, but, they've proven themselves to be worthy of being considered legitimate.

(10/10) Omniscience

#epistemology #omniscience #truth

Is anything so trustworthy that it cannot be questioned? Even if there was, then, it is still better to assume that mass-delusion, logic-errors or deeply hidden flaws could exist. For this reason, many incredibly well-established theories ('theory of gravity', to name one) are still only at 9/10 on this scale. Large-scale cultural, systemic and procedural bias can still result in peer-reviewed and established ideas being incorrect.

The perfect state of knowledge is one of omniscience, however, omniscience (being all-knowing) is an epistemologically impossible state of affairs (epistemology is the study of how things can be known).