Okay, Kodi, Why is Bayesian Statistics Awesome?

I've heard you extol it's wonders on multiple occasions, and you've got me curious.

EDIT: Probably should be 'are,' but there's no editing titles.
---
Fame is but a slow decay.
-Theodore Tilton

(Nope, you got it right the first time. "Statistics" is singular.)

Basically, what's cool about it is its comprehensive approach to dealing with uncertainty. You can build into your model whatever sources of uncertainty you think are worth modeling, whether that uncertainty arises from a process you think is inherently probabilistic or from mere ignorance. Once you've fit the model, you can estimate any quantity you can express in terms of the model parameters, observed data, and predicted data. Importantly, these estimates aren't just point estimates, like "on average, people will pay $200 more for a red car"; they're full probability distributions themselves, and can be summarized with statements like "with 95% probability, people will pay from $20 to $400 more for a red car". You can even frame your research question in terms of these estimates, allowing you to calculate the probability that your hypothesis is true, a feat totally outside the realm of null-hypothesis significance testing.

Bayesian statistics has been described as an extension of logic to the probabilistic realm. I like to think of it as using mathematics to decide what to believe. For, indeed, the Bayesian notion of probability can be interpreted as degree of belief.

Here are the slides from a presentation I did a few weeks ago, in case they're enlightening, although I didn't really write them to be comprehensible on their own:

http://arfer.net/x/bayes.pdf

---
"I'm sure everyone on this list will be glad to know I don't plan to reproduce myself." -Richard Stallman

The statistics themselves are plural. The name of the field of study, Statistics, is singular. "My statistics tell me I'm right," vs. "Statistics is an awesome field of study."

Not to mention that Bayesian Statistics can be applied to quantum mechanics, and quantum mechanics is flipping crazy and awesome.
---
"I always wanted to be somebody, but now I realize I should have been more specific."

It can? That surprises me. I was talking to my physics-geek friend the other day about the difference between frequentist and Bayesian notions of probability, and he said that the frequentist idea of some processes being inherently probabilistic (as opposed to the Bayesian view that probability is just a lack of knowledge) is necessary to explain some effects in quantum physics. I still can't say I understand why, though.

---
"I'm sure everyone on this list will be glad to know I don't plan to reproduce myself." -Richard Stallman

the Bayesian view that probability is just a lack of knowledge

Really? I've heard this idea associated with Bayesian Statistics before, but didn't think it was a defining trait (which differentiates it from other kinds of probability).

I'm no expert on physics, so I've always assumed the Bayesian view.If we had enough knowledge about quantum particles, we wouldn't have to use probability to measure them. But because of the ridiculously small size of these particles, it's impossible to gather enough data to make a deterministic prediction. On top of that, Newtonian physics break down at this scale, so we don't even know what (non-probabilistic) laws/principles to apply.

Perhaps your friend is referring to interesting quantum phenomena, such as one particle being in two places at once and the observer effect. But in my mind, this is just stuff that we can't possibly know, rather than being inherently probabilistic.
---
"I always wanted to be somebody, but now I realize I should have been more specific."

the Bayesian view that probability is just a lack of knowledge

Really? I've heard this idea associated with Bayesian Statistics before, but didn't think it was a defining trait (which differentiates it from other kinds of probability).

Ah, well, I admit being a bit confused about this issue myself. I think it goes something like: the Bayesian notion of probability is without question a measure of how much you ought to believe in something, but there is controversy about how subjective it is, that is, whether it is itself a sort of belief or whether it's merely credibility. Sounds like angels dancing on the head of a pin to me.

I am certain that the mainstream view in quantum physics is that certain processes are inherently probabilistic, and that getting around the uncertainty principle is a logical, not just a physical, impossibility, but don't ask me why.

---
"I'm sure everyone on this list will be glad to know I don't plan to reproduce myself." -Richard Stallman