Saturday, July 18, 2015

The fallacy of 'billions of billions', or: Why popular arguments that aliens must exist are bogus.

The universe is an awfully big place.  Granted, most of it is empty space.  But within that empty space, there are trillions of stars.  Maybe more.  At least some of those stars have planets around them, and some of those planets are the kind that could give rise to life.  That's still hundreds of billions (at least) of planets that can support life out there.  And so, the popular argument goes, even if the odds of life arising on another planet are very small, there are so many planets that it is bound to happen.  Thus, there almost certainly exist extraterrestrial life forms.  It isn't a matter of if, but of when we find them.

Here's a video of Dr. Carl Sagan presenting a more sophisticated version of this (with actual numbers) to estimate the number of inhabited planets in our galaxy.  Or try this worksheet on the Drake Equation on the BBC website.

It's a common argument.  And it sounds pretty convincing.  If you keep trying over and over, even though something is unlikely, eventually you will succeed.

It's common and convincing, but it's also fallacious.  Here's the problem: How many times do we have to try before we're guaranteed to succeed?

The mathematical answer is infinitely many times.

But that's to guarantee we succeed, with 100% probability.  So a better question might be: what happens to the probability of success as we keep trying?

Let the probability of a success be very low, set to $10^{-X}$, where $X$ is some large number.  This makes $10^{-X}$ a very small number.  Then let the number trials be $10^{Y}$, where $Y$ is some large number.  This makes $10^{Y}$ a very large number.  Now we define a quantity $P_0$, which is the probability of never succeeding, even after $10^Y$ trials.  (If you can't see my math, check your browser's plugin settings)

Assuming whether we succeed or not on a given trial is a simple coin flip with probability $10^{-X}$ of success, then the probability of failure in a single trial is $(1-10^{-X})$.  The probability of never succeeding after $10^{Y}$ trials is just the product
$$P_0 = \left(1-\frac{1}{10^{X}}\right)^{10^Y}.$$

We have said that $X$ is large.  Maybe you remember from algebra learning the formula for continuously compounded interest, where you ended up with an exponential, like so:
$$e^{x} = \lim_{n\rightarrow \infty} \left(1 + \frac{x}{n}\right)^n.$$

Well, in our case, if $X$ is large, then $10^{X}$  is really large, and
$$\left(1 + \frac{-1}{10^{X}}\right)^{10^X} \approx e^{-1} \approx 0.36788$$

If we re-write our expression for $P_0$, then, we find
$$P_0 = \left(1-\frac{1}{10^{X}}\right)^{10^{X + Y-X}} = \left[\left(1+\frac{-1}{10^{X}} \right)^{10^X}\right]^{10^{Y-X}} \approx = \left[e^{-1}\right]^{10^{Y-X}} = e^{-10^{Y-X}}.$$

Now, $e^{-1} \approx 0.36788$ is less than one, so squaring it or tripling it will make it even smaller.  However, taking the square root of it will make it larger.  The resolution comes down to: how does $X$ compare to $Y$?

Consider a simple case, where $X=Y$.  Then $10^{Y-X} = 10^0 = 1.$  So $P_0=e^{-1} = 0.36788.$  That is, there is only about a 37% chance of there being no successes, or in other words, there is a 63% chance of a success happening at some point.  It's not a guarantee, but it's more likely than not.

Now suppose that $Y = X+1$.  This means that we do ten times as many trials as our inverse probability; if the probability is a 1/10, do 100 trials, if the probability is 1/2, do 20 trials, etc.  Then $10^{Y-X} = 10^{1}= 10$, so $P = e^{-10} = 0.0000454$.  That is, the probability of success is 99.995%.  As we increase $Y$, this probability gets even closer to 100%.  Success is all-but guaranteed.

However, now suppose that $Y = X-1$.  This means that we only do a tenth as many as the inverse probability; if the probability is 1/10, do 1 trial.  If the probability is 1/20, do 2 trials, etc.  Then $10^{Y-X} = 10^{-1} = 0.1$, so $P_0 = e^{-0.1} = 0.9048.$  That is, the probability of success is down to a measly 9.516%.  As we increase $X$, this number gets even closer to 0%.

As we can see, our confidence of success depends drastically on the value of $Y-X$.  Even slight differences here can mean huge changes in the probability of success, $P_{\geq1} = 1-P_0$.

Simple graph showing the steep rise from 0 to 1 in the probability of success.

What this comes down to is whether $X$ is greater than or less than $Y$.  Put differently, does the probability of a single success compare to the number of trials?  Or put in terms of aliens, is the number of planets out there that can give rise to life close to the inverse of the probability of life actually arising?

And the answer is: no one knows!

We do not know how many planets there are.  If we estimate this as $N_p = 10^Y$, then $Y$ might be off by 2 or 3 in either direction.  There might be a thousand times as many as we think now, or there might be only a hundredth of our current guess.  As we just saw, for a fixed $X$, changing $Y$ even by 1 can drastically affect our confidence of extraterrestrial life existing.

Way more crucially, we have no idea how likely it is for life to occur on a planet that can give rise to life.  Think about this.  We have only ever observed life arising on a planet once.  This means that we don't have a very good definition of a planet where life can arise (see above), but it also means that we have a single data point upon which to base a probability.  If I were a pollster, and I went out on the street and asked a single person who they were voting for, and from that concluded that 54\% of voters supported the candidate, you would rightly question my methodology.  If we estimate this probability of life arising on a planet as $p_L = 10^{-X}$, then we don't even know what $X$ is.

Since we do not know what $X$ is, then we don't know what $P_{\geq 1}$ is.  This is a simple model, but it makes its point: Even small differences between $Y$ and $X$ can lead to very different predictions.

Consider again what it would mean for $Y=X-1$.  Take $10^Y$ to be the total number of planets what ever exist or will exist in our universe's lifetime.  Take $10^{-X}$ to be the probability of life ever arising on any given planet in our universe other than our own.  Then if $Y = X-1$, as above, we have $P_{\geq1} \approx$ 10% as the probability of extraterrestrial life ever arising in this universe.  This means that we'd need roughly 10 other universes just like our own before we can be back at roughly 63% probability of life arising again.

The popular statement that the universe is so big that there must be life in it somewhere is a false one.  The universe is quite big, but the probability of life arising can also be so small as to negate this bigness, and we have no way to know if this is the case or not.

The universe can still be as big as it is, and yet still not be big enough for life to arise anywhere else within it.