Bayesian Statistics

Bayesian Update Table

A Bayesian Update Table can be constructed for events with discrete probability as follows:

Hypothesis	Prior	Likelihood	Bayes numerator	Posterior
$\Large \mathcal{H}_i$	$\Large P(\mathcal{H}_i)$	$\Large P(\mathcal{D} \textbar \mathcal{H}_i)$	$\Large P(\mathcal{D} \textbar \mathcal{H}_i) P(\mathcal{H}_i)$	$\Large P(\mathcal{H}_i \textbar \mathcal{D}) = \frac{P(\mathcal{D} \textbar \mathcal{H}_i) P(\mathcal{H}_i)}{P(\mathcal{D})}$
...	...	...	...	...
Total			$\Large P(\mathcal{D}) = \sum\limits_i P(\mathcal{D} \textbar \mathcal{H}_i) P(\mathcal{H}_i)$

$\Large \mathcal{H}_i$ $\Large \mathcal{D}$ is the data.

A Bayesian Update Table can be constructed for events with continuous probability as follows:

Hypothesis	Prior	Likelihood	Bayes numerator	Posterior
$\Large \theta$	$\Large f(\theta) d\theta$	$\Large f(x \textbar \theta)$	$\Large f(x \textbar \theta) f(\theta) d\theta$	$\Large f(\theta \textbar x) = \frac{f(x \textbar \theta) f(\theta) d\theta}{f(x)}$
Total			$\Large f(x) = \int f(x \textbar \theta) f(\theta) d\theta$

$\Large \theta$ is the probability of the hypothesis.

A Bayesian Update Table can be constructed for events with Beta distribution priors as follows:

Distribution Types (Prior/Likelihood)	Hypothesis	Prior	Likelihood	Bayes numerator	Posterior
Beta/Bernoulli (on success)	$\Large \theta \in [0, 1]$	$\Large Beta(a, b) d\theta$	$\Large \theta$	$\Large \theta \times Beta(a, b) d\theta$	$\Large Beta(a + 1, b)$
Beta/Bernoulli (on failure)	$\Large \theta \in [0, 1]$	$\Large Beta(a, b) d\theta$	$\Large (1 - \theta)$	$\Large (1 - \theta) \times Beta(a, b) d\theta$	$\Large Beta(a, b + 1)$
Beta/Binomial	$\Large \theta \in [0, 1]$	$\Large Beta(a, b) d\theta$	$\Large \binom{n}{x} \theta^n (1 - \theta)^{n - x}$	$\Large \binom{n}{x} \theta^n (1 - \theta)^{n - x} \times Beta(a, b) d\theta$	$\Large Beta(a + x, b + n - x)$
Beta/Geometric	$\Large \theta \in [0, 1]$	$\Large Beta(a, b) d\theta$	$\Large \theta^x(1 - \theta)$	$\Large \theta^x(1 - \theta) \times Beta(a, b) d\theta$	$\Large Beta(a + x, b + 1)$

where the Beta distribution is given as follows:

B e t a (a, b) = \frac{(a + b - 1)!}{(a - 1)! (b - 1)!} θ^{a - 1} (1 - θ)^{b - 1}

A Bayesian Update Table can be constructed for events with normal distribution priors and likelihoods as follows:

Distribution Types (Prior/Likelihood)	Hypothesis	Prior	Likelihood	Bayes numerator	Posterior
Normal/Normal	$\Large \theta \in (-\infty, \infty)$	$\Large N(\mu_{prior}, \sigma^2_{prior}) d\theta$	$\Large N(\theta, \sigma^2)$	$\Large N(a, b) \times N(\theta, \sigma^2) d\theta$	$\Large N(\mu_{post}, \sigma^2_{post})$

$\Large \bar{x}$ $\Large n$ is the number of data points, and:

\begin{matrix} a = \frac{1}{σ_{p r i o r}^{2}}, b = \frac{n}{σ^{2}} \\ μ_{p o s t} = \frac{a μ_{p r i o r} + b \bar{x}}{a + b}, σ_{p o s t}^{2} = \frac{1}{a + b} \end{matrix}