Are you a developer?

Join my startup: Elody

Work together with other developers and build a Gestalt AI.

Elody can make your programs accessible to

non-technical users.

# Probabilities, uncertainty and learning

Probability values are deceptive. Many people feel intuitively that a raw probability value 'is just math' and does not capture all there is to say on a subject. This sentiment is correct: By giving only a probability value, a lot of relevant information can be glossed over. What is the missing information that is not captured by the probability number, and how can it be quantified?

## An example problem

Imagine that a person comes up to you with a coin and asks you what the probability is that it will come up heads when he flips it.

In scenario one, you answer naively: 50%.

In scenario two, you take the question seriously and investigate in depth: You test the coin and find out that it is balanced and has not been tampered with. You investigate the person who will flip the coin and find no evidence that they are a conman who would know how to manipulate a coin flip. You investigate why you are being asked this question, and find that the person is conducting a study on psychology to test basic math competencies, and this is only a test question. He therefore has no incentive to falsify the results of the coin flip.

After the thorough investigation in scenario two, you answer confidently: 50%.

You will note that the answer was the same in both scenarios. From the point of view of a third person who only hears your response, the two scenarios are completely identical. This is terribly misleading.

Imagine that before giving your answer, you hear a rumor that there is a conman going around with a loaded coin that always ends up tails.

It's not like there are many people going around asking about coinflips, so the chances are high that this person is that conman. In scenario one, this information should cause you to change your answer and give a number somewhat lower than 50%, depending on how likely you think it is that the person in front of you is the conman. In scenario two however, the rumor does not influence your assessment at all. After all, you already investigated the person and verified that he isn't a conman.

In both scenarios, the same probability value is given initially, and there is no way for a third person to differentiate the two scenarios. But after the same additional information is given, the probability in one of the two scenarios changes, and the other does not.

This means that there is hidden information here that the probability value is not accounting for.

## The general case

Unfortunately, there is no easy fix for this problem in the general case.

A probability value is based on a calculation that can depend on any number of factors in any number of combinations. There are formulas that are very stable under changing inputs. There are formulas where altering a single input by a minuscule amount changes the output completely. And there are formulas where the inputs are such abstract things that even quantifying them at all is nigh impossible. After all, how do you quantify "You hear a rumor that a conman is going around with a loaded coin"?

Due to this complexity, it is impossible to quantify everything that has gone into a probability value.

However, there is still a way to deal with this problem that is better than nothing:

In addition to giving a probability value, you also give a (non-exhaustive) list of additional sources of relevant evidence along with an estimate of how likely further investigation in those areas would be to affect the probability value. For example:

- Investigating the coin has an x% chance of determining that the probability it will land on heads differs from 50% by more than 0.1%.
- In the scenario 1, x is the probability that the coin is counterfeit. In scenario 2 x is virtually 0 because it has already been tested and proven to be genuine.
- What is the probability that the scenario is counterfeit? This is actually another very complex question of its own. If you want to be very thorough, you could not just give a probability here but recurse and give additional sources of relevant evidence for this probability value as well.

- Investigating the motivation and sleight-of-hand skills of the person asking the question has a y% chance of determining that the probability it will land on heads differs from 50% by more than 0.1%.

I believe that it would be a worthy endeveaour to try to develop a standardized way to present such additional sources of relevant evidence. This would allow scientists to make more accurate, less misleading statements about probabilities and allow them to succinctly summarize which areas of further research are most likely to yield fruits.