Tag Archives: credibility

Critical Thinking 21: Credibility of Sources

Critical Thinking, Middle Way Philosophy, Objectivity training, Politics

I’ve been moved to revive my critical thinking series by the acuteness of the problems people seem to have with credibility judgements in current political debate. Russia has been implicated in the recent use of a nerve agent for attempted murder in Salisbury, England, and in the use of chemical weapons in Syria. In both cases they deny it. Most of us have no direct knowledge of these issues or situations, so we rely entirely on information about them that we get through the media. That means that we have to use credibility judgements – and we need to use them with great care. My own judgement is that the Russian government has very low credibility compared to most Western sources – but to see why you need to look at the kinds of credibility criteria that can be applied and think about each one of them, rather than jumping to conclusions that may be based on your reaction to past weaknesses in Western objectivity. I’d like to invite you to consider the account of credibility below and apply it to this example (and similar ones) for yourself.

This post connects strongly to Critical Thinking 7: Authority and Credibility, which you might also like to look at.

Credibility is an estimation of how much trust to place in a source of information – e.g. a person, an organisation, or a book. Most of the information we actually encounter that is used to support arguments has to be taken on trust, because we are not in a position to check it ourselves. For example, if I’m reading an article about the Large Hadron Collider (a famous massive physics experiment), I am entirely reliant on the physicists who are giving the information to accurately explain their evidence.

There are two extreme attitudes to credibility which would be equally unhelpful: to take everything on trust without question on the one hand, or to believe nothing on the other. If we believed nothing that anyone else told us, then we would could not make use of the vast majority of information we take for granted. For example, I have never been to Australia, so without believing other people I would have no grounds for believing that Australia exists at all. On the other hand, if we believe everything, then we become prey to unscrupulous advertisers, email hoaxes such as “phishing” for bank account details, and sincere but deluded extremists of all kinds in both religion and politics. We need a method of judging others’ credibility. In fact we have all already developed our judgements about this: we believe some people more than others. However, examining the subject carefully  may help you to refine and justify these judgements.

Credibility issues must be carefully distinguished from issues of argument. It is a way of judging the information that feeds into an argument when you have no other way of judging it – not the argument itself. So whilst deductive arguments are either valid or invalid, credibility is always a matter of degree, and judging it is an extension of inductive reasoning in relation to sources.  Credibility is just a way of judging assumptions, where those assumptions consist in claims from certain sources, and we’re not in a position to assess the evidence for those claims ourselves.

An example of a scenario needing credibility assessment
Suppose you are a teacher in a primary school on playground duty, and you hear distressed yells. You turn and see two eight-year old boys fighting. One has thumped the other, who is crying. The crying boy says he was picked on, whilst the thumping boy says the other boy hit him first. Two other boys were witnesses but they disagree about who was to blame.

Perhaps it would be quite common in such a scenario for a teacher to punish both boys due to doubts about who started it: but would this be fair? It is difficult to decide, because both boys (and their respective friends) all have a strong interest in maintaining their side of the story. The witnesses are also divided, so you can’t rely on the weight of their testimony. One possible way out would be to rely on reputations. Are either of the boys known to have lied, or to have been bullies, in the past? If one boy has a record of being involved in lots of fights in the past and the other does not, this might well sway the teacher’s judgement. But of course if this assumption is made too readily it could also reconfirm the “known trouble maker” as such, because even if he is innocent people will assume that he is guilty. Judgements about credibility are always made under uncertainty.

Factors of credibility
When judging the credibility of a person, or any other sort of human source, it is helpful to have a checklist of factors in mind. We are going to consider a list of 5 credibility factors here, which can be easily remembered using the mnemonic RAVEN.
Reputation
Ability to get information
Vested interest
Expertise
Neutrality or bias

We will now look more closely at these 5 factors.

Reputation
Reputation is what we know about a person or organisation’s track record for trustworthiness. This will often come from the assessments of others, whether they are experts or ordinary people. For example, restaurants seek to get a good reputation by being given stars in the Michelin guide. Reputation has also been democratised because it can be so easily shared on the internet, with different book suppliers being rated for reliability on Amazon or different hotels being rated by people who have stayed there on websites like Bookings.com.

Apart from an individual or organisation, you might need to consider the reputation of a newspaper, other publication, broadcaster, or website. Generally, for example, the BBC has a good reputation as an objective provider of news coverage, whereas the Sun is well known for being more interested in selling newspapers and pleasing its readers than providing objective reports. This will remain generally the case even if you feel that certain reports have tarnished the reputation of the BBC or improved that of the Sun. All credibility judgements need to be proportional, so you need to think carefully about what proportion of the BBC’s vast output is generally acknowledged as credible, rather than just about a small number of negative instances, in order to arrive at a fair judgement of reputation.

Ability to get information
This covers a range of ways that people may or may not have been able to access what they claim to know through experience: ability to observe, ability to gain access, and ability to recall. If someone claims to have observed a foul at a football game that the referee judged wrongly, their testimony is of less weight if they were five times further away from the incident than the referee was and could only see it distantly. If someone claims to have seen documents that their company or government would never have actually given them access to, this would also reduce credibility. If someone is known to have an unreliable memory, or only remembers something in a vague way, this would also affect the credibility of their claims.

The ability to observe is also relevant to the distinction (often used in history) between primary sources and secondary sources. A primary source is one which records a person’s experiences directly, but a secondary source gets the information second hand. So, for example, if an officer wrote a memoir of his experiences in the Battle of Waterloo, this would become a primary historical document in gaining information about that battle, but a historian who used that document, together with others, to write a book about the battle would be producing a secondary source. On average, primary sources tend to be more worthy of credibility in reporting an event than secondary ones, but primary sources can be unreliable (the officer might not have been in a good position to see what was happening in the whole battle, for example) and secondary sources may sometimes give a more comprehensive picture with greater expertise and neutrality (see below).

Vested interest
A vested interest is something that a person has to gain or lose from a certain outcome. For example, a salesman has a vested interest in getting you to buy his company’s double glazing, because they will give him extra commission if he sells it to you. This gives him a reason to give you a possibly misleading impression of its high quality, low price etc. Vested interests can cut both ways, though: there can be a vested interest to deceive (as in the case of a salesman), but also a vested interest to tell the truth, for example where someone’s job depends on them maintaining a good reputation for reliability. As well as an incentive for stretching the truth a little bit, a double glazing salesman also has a vested interest in keeping close enough to the truth not to be subject to legal action for grossly misrepresenting his product.

It’s important to keep vested interests in perspective, because most people have some vested interests in both directions. Nearly everyone has something to gain from getting your money or your support or even your friendship, but on the other hand they also have the incentive of maintaining a social reputation as reliable, and – if they are a professional – for maintaining their career prospects, which depend on that reputation. However, in cases like advertising or political campaigning it’s obvious that the vested interests lie strongly in one direction.

Expertise
If someone is an expert on the topic under consideration, then this normally adds substantially to their credibility, because they will know a lot more of the facts of the matter and also understand the relationship between them. We all rely on expertise constantly: the doctor, the computer technician, the academic on TV or writing a book. You can look for formal academic qualifications (BA’s, MA’s, & Ph.D.’s) as evidence of expertise, or it may just be a question of professional experience or life experience (e.g. someone has worked 20 years as a gardener, or practised meditation for 10 years, or whatever). People who hold university posts in a subject, or who have written books on it, are often the starting-point in the media when an expert is needed.

Apart from whether expertise is genuine, the other thing you might want to consider when deciding whether to trust it is whether it is relevant. Someone with a Ph.D. in physics may know a bit about biology, but not necessarily that much. The fact that someone is an Olympic gold medal winner may give them expertise in how to train in their sport, but not necessarily about, say, politics or business. ‘Celebrities’ who are largely famous for being famous, may assume expertise on subjects that they don’t actually know more than average about.

From the Middle Way point of view, it is also worth considering that expertise in modern society often results from over-specialisation that may lead people into making absolute assumptions that are specific to their highly specialised expert groups. This means that whilst highly specialised experts may be very reliable on very specific points within their expertise, the moment their judgement starts to involve synthesis or comparison with other areas it may actually become less reliable, because they may have effectively sacrificed their wider objectivity for the sake of specialisation. For example, when well-known specialised scientists start talking about ethics or religion I often have this impression – not that they are not entitled to express their views on these topics, but that their views are very narrowly based. On the other hand, there are also other people whose expertise is more broadly based.

Neutrality or bias
Finally, you can assess someone’s claims according to their overall approach to the topic and the kind of interpretation they make of it. Some people may clearly set out to be as objective as possible, whereas others adopt a deliberately biased approach in order to promote a particular point of view. Honest bias is probably better than false neutrality, but you need to be aware of the ways that the biased approach will limit people’s interpretation of the facts. For example, the comments of a politician arguing for their policies are going to be biased in favour of promoting those policies – compared to, say, a political journalist from the BBC who sets out to analyse the issue in a more objective way that explains the different views of it.

Bias should not be confused with vested interest, although they may go together in many cases. Someone can have a vested interest, yet take an objective and balanced tone when explaining the facts as they see them. On the other hand, someone without a lot of vested interests may be inspired by sympathy with one side or the other to weigh strongly into a debate: for example, the actor Joanna Lumley got involved in the campaign to give immigration rights to the UK to Nepalese Gurkha soldiers in the British army. She clearly had nothing much to gain from this herself, but nevertheless was a passionate advocate of the cause.

Conclusion

So, do you believe the Russian government? The judgement needs to be incremental and comparative. So, compare it to another source, say the British government on the Skripal Case. What are their reputations, their abilities to get information, their vested interests, expertise, and record on bias? These all need to be put together, with none of them being used as an absolute to either accept total authority or to completely dismiss.

 

For an index of all critical thinking blogs see this page.

Picture: Franco Atirador (Wikimedia Commons)

 

Critical Thinking 7: Authority and Credibility

Critical Thinking,

The Middle Way involves the avoidance of absolute authorities, but also the avoidance of the converse – dismissing all authority. In between we have the idea of justified authority that can be based on experience. Recognised fallacies in Critical Thinking, such as the irrelevant appeal to authority or the genetic fallacy, reinforce this need to avoid dogmatic extremes in thinking about authority. John_Caird_(theologian)Cognitive biases such as the authority bias also reflect psychological evidence of our tendency to uncritically obey authorities, especially reflected in the famous Milgram experiments.

The basic fallacy recognised in Critical Thinking in relation to authority is the irrelevant appeal to authority. This involves the assumption that an authority figure must be correct in whatever they claim, whether or not it is relevant to their particular expertise. Many adverts trade on this by using celebrities to endorse products that they have no particular expertise in. The following US commercial from the 1950’s is a classic example:

Nevertheless, because doctors are probably not a relevant authority to consult on which brand of cigarette to smoke, this does not imply that they do not have any authority. I am likely to rely on my doctor’s authority if he prescribes me a drug for a serious condition – not because I think him/her infallible, but because experience suggests that doctors are the best available source of information. The justification is largely second-hand, but no worse for that: the experience of most other people, that they have communicated to me, is that doctors are the best available guides to the complex field of medicine. To dismiss this authority on the basis of some mistakes made by doctors (even some serious ones) would be to make the reverse fallacy.

Another way of putting this more positively is that doctors have credibility. We are obliged to make judgements of credibility constantly, when deciding which books to read, which experts to consult, whose advice to heed etc, when (as often) we have no direct understanding of the issues. There are various criteria you can consciously apply to reflect on credibility: reputation, ability to gain information, vested interest, expertise, corroborating or conflicting information, and previously-known tendency towards bias. None of these are decisive by themselves, but they can be weighed together to try to reach a justified judgement.

It would be unjustified to give a huge weighting to one way of judging but ignore the others. For example, people often have some kind of vested interest in your acceptance of the information they offer (e.g. now, if you read this blog post, I have some vested interest in the shape of a vague hope that this will encourage you to buy one of my books in future). However, vested interest does not necessarily mean that the information should be treated with suspicion, or that the person’s motives are dominated by it, and if we dismissed everyone with a vested interest, we would never consult any experts about anything. Similarly, we often over-rate the effect of one well-known event on someone’s reputation: but if someone made a mistake or even committed a crime in the past, this might just as well be taken as a sign that they are likely to avoid that mistake in the future than that they will repeat it.

The authority bias, then, is like a single over-rated criterion of credibility. If, as in the Milgram experiments, you’re prepared to give painful electric shocks to others because a man in a white coat tells you to do so, you’re assuming that they have absolute authority because of their apparent expertise, and not considering the possibility of a problem with their moral judgement. Similarly, in a religious context, if you think that the fact that the book of Leviticus forbids homosexuality offers a relevant moral command for today, you are effectively taking the reputation of that religious text within a certain limited group and using this as your sole criterion for its credibility, whilst ignoring the lack of relevant expertise of the people who created the text, and the conflicts with other more recent sources of information.

Appeals to authority at their broadest are genetic fallacies – that is, the assumption that a claim is absolutely right because of where it comes from. This can apply to people, texts, governments, traditions, supposed intuitions and supposed observations. The basic problem is that wherever you think it has come from, you are still responsible for interpreting it and judging that it is justified and credible. If we take responsibility for our own judgements, the authority bias becomes much less likely.

Exercise

Do the following appear to be justifiable uses of authority? Why/ why not?

1. A father tells his three-year old daughter to stop slapping her sister. When she asks “Why?” he replies “Because I said so.”

2. A traditional Thai Buddhist buys a caged bird and releases it to gain merit. When questioned about this by a Western visitor, he explains that this is supported by Buddhist tradition and obeys the precepts of the Buddha.

3. A Member of Parliament votes against her conscience. When questioned by a constituent she explains that she was pressured by her party whip, and risked being de-selected by her party if she refused to vote with the party.

4. An amendment to the law in Afghanistan restores the right of a husband to use violence against his wife. The amendment is justified with reference to the Qur’an.

5. A disruptive student is told to leave the classroom by a teacher. The student refuses to leave, on the grounds that the teacher “Can’t just boss me around – I’ve got rights”.

6. A doctor gives advice on diet to an obese patient, which the patient refuses to follow. The patient points out that the doctor is overweight himself.

7. A husband had an extra-marital affair which was discovered, apologised for and forgiven a year ago. Since then his relationship with his wife has been good and trust seems to have been restored. However, now she is again suspicious about his fidelity (though the evidence is ambiguous). When the husband protests his innocence, she refuses to believe him, on the grounds that she was deceived before.