I’ve been moved to revive my critical thinking series by the acuteness of the problems people seem to have with credibility judgements in current political debate. Russia has been implicated in the recent use of a nerve agent for attempted murder in Salisbury, England, and in the use of chemical weapons in Syria. In both cases they deny it. Most of us have no direct knowledge of these issues or situations, so we rely entirely on information about them that we get through the media. That means that we have to use credibility judgements – and we need to use them with great care. My own judgement is that the Russian government has very low credibility compared to most Western sources – but to see why you need to look at the kinds of credibility criteria that can be applied and think about each one of them, rather than jumping to conclusions that may be based on your reaction to past weaknesses in Western objectivity. I’d like to invite you to consider the account of credibility below and apply it to this example (and similar ones) for yourself.

This post connects strongly to Critical Thinking 7: Authority and Credibility, which you might also like to look at.

Credibility is an estimation of how much trust to place in a source of information – e.g. a person, an organisation, or a book. Most of the information we actually encounter that is used to support arguments has to be taken on trust, because we are not in a position to check it ourselves. For example, if I’m reading an article about the Large Hadron Collider (a famous massive physics experiment), I am entirely reliant on the physicists who are giving the information to accurately explain their evidence.

There are two extreme attitudes to credibility which would be equally unhelpful: to take everything on trust without question on the one hand, or to believe nothing on the other. If we believed nothing that anyone else told us, then we would could not make use of the vast majority of information we take for granted. For example, I have never been to Australia, so without believing other people I would have no grounds for believing that Australia exists at all. On the other hand, if we believe everything, then we become prey to unscrupulous advertisers, email hoaxes such as “phishing” for bank account details, and sincere but deluded extremists of all kinds in both religion and politics. We need a method of judging others’ credibility. In fact we have all already developed our judgements about this: we believe some people more than others. However, examining the subject carefully  may help you to refine and justify these judgements.

Credibility issues must be carefully distinguished from issues of argument. It is a way of judging the information that feeds into an argument when you have no other way of judging it – not the argument itself. So whilst deductive arguments are either valid or invalid, credibility is always a matter of degree, and judging it is an extension of inductive reasoning in relation to sources.  Credibility is just a way of judging assumptions, where those assumptions consist in claims from certain sources, and we’re not in a position to assess the evidence for those claims ourselves.

An example of a scenario needing credibility assessment
Suppose you are a teacher in a primary school on playground duty, and you hear distressed yells. You turn and see two eight-year old boys fighting. One has thumped the other, who is crying. The crying boy says he was picked on, whilst the thumping boy says the other boy hit him first. Two other boys were witnesses but they disagree about who was to blame.

Perhaps it would be quite common in such a scenario for a teacher to punish both boys due to doubts about who started it: but would this be fair? It is difficult to decide, because both boys (and their respective friends) all have a strong interest in maintaining their side of the story. The witnesses are also divided, so you can’t rely on the weight of their testimony. One possible way out would be to rely on reputations. Are either of the boys known to have lied, or to have been bullies, in the past? If one boy has a record of being involved in lots of fights in the past and the other does not, this might well sway the teacher’s judgement. But of course if this assumption is made too readily it could also reconfirm the “known trouble maker” as such, because even if he is innocent people will assume that he is guilty. Judgements about credibility are always made under uncertainty.

Factors of credibility
When judging the credibility of a person, or any other sort of human source, it is helpful to have a checklist of factors in mind. We are going to consider a list of 5 credibility factors here, which can be easily remembered using the mnemonic RAVEN.
Reputation
Ability to get information
Vested interest
Expertise
Neutrality or bias

We will now look more closely at these 5 factors.

Reputation
Reputation is what we know about a person or organisation’s track record for trustworthiness. This will often come from the assessments of others, whether they are experts or ordinary people. For example, restaurants seek to get a good reputation by being given stars in the Michelin guide. Reputation has also been democratised because it can be so easily shared on the internet, with different book suppliers being rated for reliability on Amazon or different hotels being rated by people who have stayed there on websites like Bookings.com.

Apart from an individual or organisation, you might need to consider the reputation of a newspaper, other publication, broadcaster, or website. Generally, for example, the BBC has a good reputation as an objective provider of news coverage, whereas the Sun is well known for being more interested in selling newspapers and pleasing its readers than providing objective reports. This will remain generally the case even if you feel that certain reports have tarnished the reputation of the BBC or improved that of the Sun. All credibility judgements need to be proportional, so you need to think carefully about what proportion of the BBC’s vast output is generally acknowledged as credible, rather than just about a small number of negative instances, in order to arrive at a fair judgement of reputation.

Ability to get information
This covers a range of ways that people may or may not have been able to access what they claim to know through experience: ability to observe, ability to gain access, and ability to recall. If someone claims to have observed a foul at a football game that the referee judged wrongly, their testimony is of less weight if they were five times further away from the incident than the referee was and could only see it distantly. If someone claims to have seen documents that their company or government would never have actually given them access to, this would also reduce credibility. If someone is known to have an unreliable memory, or only remembers something in a vague way, this would also affect the credibility of their claims.

The ability to observe is also relevant to the distinction (often used in history) between primary sources and secondary sources. A primary source is one which records a person’s experiences directly, but a secondary source gets the information second hand. So, for example, if an officer wrote a memoir of his experiences in the Battle of Waterloo, this would become a primary historical document in gaining information about that battle, but a historian who used that document, together with others, to write a book about the battle would be producing a secondary source. On average, primary sources tend to be more worthy of credibility in reporting an event than secondary ones, but primary sources can be unreliable (the officer might not have been in a good position to see what was happening in the whole battle, for example) and secondary sources may sometimes give a more comprehensive picture with greater expertise and neutrality (see below).

Vested interest
A vested interest is something that a person has to gain or lose from a certain outcome. For example, a salesman has a vested interest in getting you to buy his company’s double glazing, because they will give him extra commission if he sells it to you. This gives him a reason to give you a possibly misleading impression of its high quality, low price etc. Vested interests can cut both ways, though: there can be a vested interest to deceive (as in the case of a salesman), but also a vested interest to tell the truth, for example where someone’s job depends on them maintaining a good reputation for reliability. As well as an incentive for stretching the truth a little bit, a double glazing salesman also has a vested interest in keeping close enough to the truth not to be subject to legal action for grossly misrepresenting his product.

It’s important to keep vested interests in perspective, because most people have some vested interests in both directions. Nearly everyone has something to gain from getting your money or your support or even your friendship, but on the other hand they also have the incentive of maintaining a social reputation as reliable, and – if they are a professional – for maintaining their career prospects, which depend on that reputation. However, in cases like advertising or political campaigning it’s obvious that the vested interests lie strongly in one direction.

Expertise
If someone is an expert on the topic under consideration, then this normally adds substantially to their credibility, because they will know a lot more of the facts of the matter and also understand the relationship between them. We all rely on expertise constantly: the doctor, the computer technician, the academic on TV or writing a book. You can look for formal academic qualifications (BA’s, MA’s, & Ph.D.’s) as evidence of expertise, or it may just be a question of professional experience or life experience (e.g. someone has worked 20 years as a gardener, or practised meditation for 10 years, or whatever). People who hold university posts in a subject, or who have written books on it, are often the starting-point in the media when an expert is needed.

Apart from whether expertise is genuine, the other thing you might want to consider when deciding whether to trust it is whether it is relevant. Someone with a Ph.D. in physics may know a bit about biology, but not necessarily that much. The fact that someone is an Olympic gold medal winner may give them expertise in how to train in their sport, but not necessarily about, say, politics or business. ‘Celebrities’ who are largely famous for being famous, may assume expertise on subjects that they don’t actually know more than average about.

From the Middle Way point of view, it is also worth considering that expertise in modern society often results from over-specialisation that may lead people into making absolute assumptions that are specific to their highly specialised expert groups. This means that whilst highly specialised experts may be very reliable on very specific points within their expertise, the moment their judgement starts to involve synthesis or comparison with other areas it may actually become less reliable, because they may have effectively sacrificed their wider objectivity for the sake of specialisation. For example, when well-known specialised scientists start talking about ethics or religion I often have this impression – not that they are not entitled to express their views on these topics, but that their views are very narrowly based. On the other hand, there are also other people whose expertise is more broadly based.

Neutrality or bias
Finally, you can assess someone’s claims according to their overall approach to the topic and the kind of interpretation they make of it. Some people may clearly set out to be as objective as possible, whereas others adopt a deliberately biased approach in order to promote a particular point of view. Honest bias is probably better than false neutrality, but you need to be aware of the ways that the biased approach will limit people’s interpretation of the facts. For example, the comments of a politician arguing for their policies are going to be biased in favour of promoting those policies – compared to, say, a political journalist from the BBC who sets out to analyse the issue in a more objective way that explains the different views of it.

Bias should not be confused with vested interest, although they may go together in many cases. Someone can have a vested interest, yet take an objective and balanced tone when explaining the facts as they see them. On the other hand, someone without a lot of vested interests may be inspired by sympathy with one side or the other to weigh strongly into a debate: for example, the actor Joanna Lumley got involved in the campaign to give immigration rights to the UK to Nepalese Gurkha soldiers in the British army. She clearly had nothing much to gain from this herself, but nevertheless was a passionate advocate of the cause.

Conclusion

So, do you believe the Russian government? The judgement needs to be incremental and comparative. So, compare it to another source, say the British government on the Skripal Case. What are their reputations, their abilities to get information, their vested interests, expertise, and record on bias? These all need to be put together, with none of them being used as an absolute to either accept total authority or to completely dismiss.

For an index of all critical thinking blogs see this page.

Picture: Franco Atirador (Wikimedia Commons)