Figuring out the reliability of evidence by using Bayes Theorem

Evidence, I mean not the CIA or police kind of evidence, I mean things like observations such as, your old friend wearing a particular hat, a bicycle, a key, a car, a shoe, etc. Anything that can support a particular view of yours.

Here I am going to stick only to the evidence from observations so that it will be easier to understand this concept. (please keep in mind that evidence is not limited to observations.)

Out of the things we consider as evidence there are strong evidences and weak evidences. And knowing the difference is crucial in making right choices in everyday life.

Here is an example:

It is raining, but you saw the sun shining bright. Based on past experience you know that when the sun shines during the rain it is a sign that the rain will stop soon.

From the above situation which is the evidence?

Your observation is the evidence. The sunshine. Is this a strong evidence? If you ask an average person, they will say it is quite a reliable evidence. How reliable? Like how many percent?

Probably 70%

Let me give a contrast just showing that this is actually a weak evidence.

You went out and looked outside and saw that the clouds are gathering up, and it is windy too.

You went out and found a new evidence. The gathering clouds.

Now how strong is this evidence? Probably 90%.

Let’s take both separately and try to figure out which of these two evidences most likely can predict it will rain or not.

The amount of sunshine vs amount of clouds.

Now we can see that the sun shine is not a strong evidence. One reason that the number of clouds is a strong evidence is because of causality. Clouds directly cause rain. But the sunshine does not cause rain even though less sunshine might start rain. But this is a correlation. We see before it rains it usually gets dark, so less sun shines.

Can you say it will rain just because it’s windy and starts getting dark?

I know I did. But that was before I came to know Bayes Theorem. Let’s dive into it.

Bayes theorem forces us to ask if the view is true, what is the likelihood of the evidence to be true. Then it goes on to ask what the world was like before discovering this evidence. (Also called prior. There is a technical term for ignoring it. It’s called Base rate neglect or base rate fallacy.)

Here are two examples, one with math and the other without the math.

When I see it’s getting darker, I assume it’s going to rain.

The Bayes theorem asks if it rains how likely it gets dark? Say almost always. About 90% of the time.

Then it asks how likely it rains these days? Ok this is the rainy season so it rains 60% of the week.

Now it asks how often it gets dark?
Very often. Like 65%

Now let’s calculate: (I applied the formula.)
90% x 60% / 65% = 83%

Based on the evidence there is 83% of the chance it will rain.

Let’s take another example, this time without numbers and formulas so that we can decide immediately.

As sam was walking he saw smoke rising from a cottage afar. He thinks there might be a fire ��.

Let’s examine. There are three things we need to figure out. When the evidence is true how likely the belief is true? In this case when there is fire how likely we find smoke? Here we are talking about accidental fires. Accidental fires almost always have smoke. So smoke: very likely.

Next part is how likely there are accidental fires? Very less.

Finally how likely are there smokes: Quite lot. From waste burning, to cooking, to power house, there are a lot of smokes without accidental fires.

In this situation sam does not need to worry too much. Though it is wise to look for some more evidence.

Let’s go to the first part of the Bayes theorem. The question, if the belief/view is true how likely the evidence is true? This question is the most important part in determining the quality of your evidence. We are questioning the evidence. Example:

When you see cloud, you want to know how likely it is going to rain. Here you ask if it rains how likely you will see clouds?

The answer is always. So this is a strong evidence.

When you see fire and you want to know if there is an accidental fire happening. You ask how likely you see fire when there is accidental fire?

The answer is always. So this is also a strong evidence.

Let’s check a weak evidence.

You went to fishing and saw five birds flying over one area. How likely there are fish in that spot? Again you ask, if there are fish how likely you will see five birds?

Not much. So this is a weak evidence.

By putting your evidence under this question, you are checking the weight of the evidence. This is what makes Bayes Theorem so useful.

Original: 10-05-2022
Revision: 1, 2