베이지안 통계학(Bayesian Statistics)

created : 2021-10-03T10:46:55+00:00
modified : 2022-02-19T08:40:46+00:00

Rmarkdown

Historical Perspective (관점의 변화)

en

Frequentist/Classical Paradigm (빈도주의/고전적 패러다임)

en

ko

Bayesian Paradigm (베이지안주의 패러다임)

en

ko

Differences Between Frequentist and Bayesian (빈도주의와 베이지안주의 간의 차이)

en

ko

Overall Recommendation

Bayesian Approach (베이지안적 접근)

en

ko

Bayes’ Theorem

Bayesian Modeling

  1. Model specification:
    • $p(y \vert \theta)$ : likelihood function of y
    • $p(\theta)$ : prior distribution of $\theta$
  2. Performing inference:
    • $p(\theta \vert y)$ : posterior distribution of $\theta$ given y
    • $p(\theta \vert y) \propto p(y \vert \theta) p(\theta)$
    • How ?:
    • analystically-only possibile for certain models.
    • using simulation when we are not able to write down the exact form of the posterior density.
  3. Inference results:
    • ex) posterior mean : $E[\theta \vert y] = \int _{\theta} \theta p(\theta \vert y) d \theta$

Binomial Model

Binomial Model with Beta Prior

Example : Placenta Previa

Posterior Predictive Distribution

Example


Bayesian Statistics


Bayesian Paradigm

Historical Perspective

Frequentist vs. Bayesian

Frequentist/Classical Paradigm

Bayesian Paradigm

Differences Between Frequentist and Bayesian

Overall Recommendation

Probability Review

Probabilities Defined on Events

Conditional Proabilities

Independent Events

Law of Total Probability

## Bayes’ Theorem

Discrete Distributions

Bernoulli Trials

Binomial Distribution

Multinomial Distribution

Geometric Distribution

Negative Binomial Distribution

Poisson Distribution

Continuous Distributions

Uniform Distribution

Normal Distribution

Gamma Distribution

Chi-square Distribution

Beta Distribution

Bayesian Approach

Bayesian Approach

Notation

Bayes’ Theorem

Baeysian Modeling

  1. Model specification:
    • $p(y \vert \theta)$ : likelihood function of y
    • $p(\theta)$ : prior distribution of $\theta$
  2. Performing inference
    • $p(\theta \vert y)$ : posterior distribution of $\theta$ given y
    • $p(\theta \vert y) \propto p(y \vert \theta) p(\theta)$
    • How?
    • analytically-only possible for certain models.
    • using simulation when we are not able to write down the exact form of the posterior density.
  3. Inference results
    • ex) posetrior mean : $E[\theta \vert y] = \int_\theta \theta p(\theta \vert y) d \theta$

Binomial Model

Binomial Model

Binomial Model with Beta Prior

Posterior Predictive Distribution

Posterior Predictive Distribution

Poisson Model

Poisson Model

\(\begin{aligned} p(\theta \vert y) & \propto p(y \vert \theta) p(\theta) \\ & \propto e^{\sum y_i + \alpha - 1} e^{-(n + \frac{1}{\beta}) \theta} \\ & \sim Gamma(\sum y_i + \alpha, [n + \frac{1}{\beta}]^{-1}) \end{aligned}\)

Posterior Predictive Distribution of Poisson Model

Normal Model

Normal Model with Multiple Observations

Posterior Predictive Distribution

Normal Model with Known Mean and Unknown Variance

Conjugate Families

Prior Distributions

Noninformative Prior Distributions

Informative Prior Distributions

Proper / Improper Prior Distributions

Jeffreys’ Prior

Lack of Invariance

Jeffreys’ Noninformative Prior

Frequentist Criteria for Evaluating Estimators

Posterior MSE

Confidence Regions

Frequentist Confidence Interval

Bayesian Credible Interval

Highest Posterior Density (HPD) Credible Set

Hypothesis Testing

Classical P-values

Posterior Predictive P-values

Bayes Factor

Bayesian Hypothesis Testing

## Bayes Factor

Probabilities of bayes Factor

BF Strength of Evidence
< 1 Negative (supports $H_0$)
1 to 3 Barely worth mentioning
3 to 20 Positive
20 to 150 Strong
> 150 Very Strong

Multiparameter Models

Normal with Noninformative Prior

Normal Model with a Noninformative Prior Distribution

Multinomial Model

Multinomial Model for Categorical Data

Dirichlet Distribuiton

Multivariate Normal Models

Multivariate Normal Model with Known Variance

Conjugate Analysis

Posterior Marginal and Conditional Distribution of Subvectors of $\mu$

Posterior Predictive Distribution for New Data

## Noninformative Prior Density for $\mu$

\[p(\mu \vert y) \propto exp[-\frac{1}{2} (\mu - \bar y)^T n \Sigma^{-1} (\mu - \bar y)]\]

Hierarchical Models

Hierarchical Models

General Framework

  1. Likelihood function: \(y_1, ..., y_n \vert \theta_1, ..., \theta_n, \phi \sim p(y_i \vert \theta_i)\)
  2. Prior distribution: \(\theta_1, ..., \theta_n \vert \phi \sim p(\theta_i \vert \phi)\)
  3. Hyperprior distribution: \(\phi \sim p(\phi)\)

Conditional and Marginal Distributions

Posterior Summaries

# Hierarchical Binomial Model ## Hierarchical Binomial Model

Hierarchical Poisson Model

Hierarchical Poisson Model

Hierarchinal Normal Model

Monte Carlo Integration

Numerical Integration

Monte Carlo Integration

Rejection Sampling

Rejection Sampling

Remarks

Importance Sampling

Importance sampling

Remarks

Markov Chain Monte Carlo

Markov Chains