Bayesian Statistics

Objective

This class will present the main concepts of Bayesian statistics. We will mainly address parametric statistical models, starting from the specification of the prior distribution and we will then look at the inference (point estimation, credible sets and tests) and at some theoretical asymptotic properties (posterior consistency and Bernstein - Von Mises theorem).

I will also provide an introduction to more advanced topics like high-dimensional and nonparametric models. For these models we will mainly look at the specification of prior distributions. I will also mention some results about rate of convergence. Bayesian statistics is a vast field, this course will aim at providing students with the basic tools to be able to understand.

Bayesian procedures and to implement Bayesian analysis of some statistical models. In particular, at the end of the course student will be able to : — compute the posterior distribution for conjugate models, construct Bayesian point estimators and credible regions ;
— implement Bayesian hypothesis testing through the construction of Bayes factors ;
— understand the difference between the frequentist inference procedure and the Bayesian one ;
— implement on a statistical software a simple Bayesian inference procedure (MetropolisHasting and Gibbs sampling) ;
— Solve exercises related to the material seen in class. Advised courses : Simulation and Monte Carlo methods (2A).

This class will not follow any specific book. The support for the class is given by a set of slides (available on Pamplemousse) and exercises that will be provided and correct in class (most of them will not be on Pamplemousse). Participation in the course is strongly recommended.

Evaluation : There will be two homeworks and a final project to be done in groups 2 of 3 students. Please fill in this document https://docs.google.com/spreadsheets/ d/1teTDmGzyHyHQxWLmRfOhbwTMiaGAJgnuGFw_fajYSY8/edit?usp=sharing with the 3 names of students in your group as soon as possible (and before the first Assignement).

Planning

Chapter 1 : Introduction (basics of Bayesian Statistics, Exchangeability, Likelihood).
Chapter 2 : The prior distribution (subjective determination of the prior, noninformative priors, conjugate priors, hierarchical prior and empirical Bayes).
Chapter 3 : Bayesian inference (point estimation, the conjugate Gaussian model and the linear regression model, credible regions, hypothesis testing).
Chapter 4 : Asymptotic properties of parametric Bayes procedures.
Chapter 5 : Model comparison and Bayes factor.
Chapter 6 : Sparsity and high-dimensional models.
Chapter 7 : Introduction to Bayesian nonparametrics (multinomial sampling, Dirichlet distribution and Dirichlet process).
Chapter 8 : Bayesian inference for nonparametric density functions and nonparametric regression functions.

References

— “The Bayesian Choice : from Decision-Theoretic Motivations to Computational Implementation”, C. Robert, Springer-Verlag, New York (2001). 
— “A First course in Bayesian Statistical Methods”. P. Hoff, Springer.
— “Statistical Decision Theory and Bayesian Analysis”, J. Berger, Springer-Verlag, New York (1985).
— Bernardo, J. and Smith, A., Bayesian theory, John Wiley & Sons, Chichester (1993).
— “Bayesian nonparametrics”, Hjort et al. eds., Cambridge University Press, (2010).
— Ghosh, J. and R. Ramamoorthi, Bayesian Nonparametrics, Springer Verlag, Berlin (2003).
— Ghoshal, S. and A. Van der Vaart, Fundamentals of Nonparametric Bayesian Inference, Cambridge (2017). Further references will be provided during the class.