Probability and Statistics

From Ioannis Kourouklides
Jump to navigation Jump to search
Probability.gif

This page contains resources about Probability Theory and Statistics in general.

More specific information is included in each subfield.

Subfields and Concepts

See Category:Probability and Statistics for all its subfields.

Statistical Inference / Inferential Statistics

  • Frequentist Inference
    • Statistical Hypothesis Testing / Statistical Tests
      • Fisher's Null Hypothesis Testing
      • Neyman-Pearson Theory
      • Analysis of Variance (ANOVA)
      • Analysis of Covariance (ANCOVA)
      • Multivariate Analysis of Variance (MANOVA)
      • T-test
      • F-test
      • Tests of Goodness-of-Fit
    • Confidence Intervals
    • Bootstrapping
  • Bayesian Inference
  • Inductive inference
  • Empirical Inference
  • Causal Inference
  • Interval Estimation
  • Estimation Theory / Point Estimation
  • Decision Theory
    • Neyman-Pearson Theory
    • The Expected Loss Principle
    • Optimal decision rules
    • Bayesian Decision Theory / Bayesian Estimator
    • Cost function / Loss function
    • Risk function
    • Admissibility
    • Unbiasedness
    • Minimaxity
  • Algorithmic Information Theory
    • Kolmogorov Complexity / Algorithmic Complexity
    • Algorithmic Probability / Solomonoff Probability
    • Universal Search (by Levin)
    • Algorithmic Randomness (by Martin-Lof)
    • Solomonoff's Theory of Inductive Inference
    • Epicurus' Principle of Multiple Explanations
    • Occam's Razor
    • Bayes rule
    • Minimum Description Length (MDL) principle
    • Minimum Message Length (MML)
  • Model Selection and Evaluation
    • Akaike Information Criterion (AIC)
    • Bayesian Information Criterion (BIC)
    • Deviance Information Criterion (DIC)
    • Bayesian Predictive Information Criterion (BPIC)
    • Focused Information Criterion (FIC)
    • Bayesian Model Selection / Bayesian Model Comparison
      • Bayesian Model Averaging
    • Bayesian Parameter Estimation
    • Minimum Description Length (MDL) principle
    • Minimum Message Length (MML)
    • Akaike Final Prediction Error (FPE)
    • Parzen's Criterion Autoregressive Transfer Function (CAT)
    • Cross-Validation
    • Statistical Hypothesis Testing (for Multilevel Models / Nested Models only)
      • Lagrange multiplier test / Score test / Score Method
      • Likelihood-ratio test
      • Wald test

Statistical Models

  • Regression Analysis
    • Linear Regression Model
    • Simple Linear Regression
    • Multiple Linear Regression (not to be confused with Multivariate Linear Regression)
    • General Linear Model / Multivariate Linear Regression
    • Generalized Linear Model (GLM or GLIM)
    • Poisson Regression
    • Least Squares Methods
      • Ordinary Least Squares / Linear Least Squares
      • Weighted Least Squares
      • Nonlinear Least Squares
    • Logistic Regression Model / Logit Model
    • Probit Model
    • Fixed Effects Model
    • Hierarchical Linear Models / Multilevel Models / Nested Data Models
      • Random Effects Model / Variance Components Model
      • Mixed Effects Models (not to be confused with Mixture Models)
    • Nonparametric Regression Models
    • Nonlinear Regression Models
    • Robust Regression Models
    • Random sample consensus (RANSAC)
    • Regularization
      • Ridge regression / Tikhonov regularization
      • Least absolute shrinkage and selection operator (LASSO)
      • Elastic Nets
  • Probabilistic Models
  • State Space Models
    • Time Series Models

Probability Theory

  • Random Variables
    • Continuous Random Variables
      • Probability Density Function
    • Discrete Random Variables
      • Probability Mass Function
    • Jointly Distributed Random Variables
      • Joint Density Function
    • Independent Random Variables
    • Uncorrelated Random Variables
  • Moments of a distribution
    • First Moment / Mean
    • Second Moment / Variance
    • Third Moment / Skewness
    • Fourth Moment / Kurtosis
  • Probabilistic Models
  • Stochastic Convergence
  • Probability Space
  • Measure Space
  • State Space
  • Theorem of Total Probability
  • Central Limit Theorem
  • Bayesian Probability Theory
  • Frequentist Probability Theory
  • Queueing Theory
  • Martingale Theory
  • Ergodic Theory
  • Decision Theory
  • Measure Theory
  • Utility Theory

Online Courses

Video Lectures


Lecture Notes

Books

Statistical Inference and Theory of Statistics

  • Bruce, P., & Bruce, A. (2017). Practical Statistics for Data Scientists: 50 Essential Concepts. O'Reilly Media.
  • Imbens, G. W., & Rubin D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction.
  • Ross, S. M. (2014). Introduction to probability models. 11th Ed. Academic Press.
  • Smith, R. C. (2013). Uncertainty quantification: theory, implementation, and applications. SIAM.
  • Gentle, J. E. (2013). Theory of statistics. (link)
  • DeGroot, M. H., & Schervish, M. J. (2012). Probability and statistics. 4th Ed. Pearson.
  • Abu-Mostafa, Y. S., Magdon-Ismail, M., & Lin, H. T. (2012). Learning From Data. AMLBook.
  • Diez, D. M., Barr, C. D., & Cetinkaya-Rundel, M. (2012). OpenIntro Statistics. CreateSpace.
  • Ramachandran, K. M., & Tsokos, C. P. (2012). Mathematical Statistics with Applications in R. Elsevier.
  • Liero, H., & Zwanzig, S. (2012). Introduction to the theory of statistical inference. CRC Press.
  • Gentle, J. E. (2007). Matrix algebra: theory, computations, and applications in statistics. Springer Science & Business Media.
  • Rice, J. (2006). Mathematical statistics and data analysis. 3rd Ed. Duxbury Press.
  • Cox, D. R. (2006). Principles of statistical inference. Cambridge University Press.
  • Lavine, M. (2005). Introduction to Statistical Thought. Michael Lavine.
  • Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference. Cambridge University Press.
  • Lehmann, E. L., & Casella, G. (2003). Theory of point estimation. Springer.
  • Bertsekas, D. P., & Tsitsiklis, J. N. (2002). Introduction to Probability. Athena scientific.
  • Casella, G., & Berger, R. L. (2002). Statistical inference. Cengage Learning.
  • Garthwaite, P. H., Jolliffe, I. T., & Jones, B. (2002). Statistical inference. Oxford University Press.
  • Shao, J. (2000). Mathematical Statistics. Springer.
  • Mukhopadhyay, N. (2000). Probability and statistical inference. CRC Press.
  • Schervish, M. J. (1995). Theory of statistics. Springer Science & Business Media.

Regression Analysis and Generalized Linear Models

  • Harrell, F. (2015). Regression modeling strategies. 2nd Ed. Springer.
  • Kroese, D. P., & Chan, J. C. (2016). Statistical modeling and computation. Springer.
  • Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example. 5th Ed. John Wiley & Sons.
  • Goldstein, H. (2010). Multilevel statistical models. 4th Ed. John Wiley & Sons.
  • Freedman, D. A. (2009). Statistical models: theory and practice. Cambridge University Press.
  • Dobson, A. J., & Barnett, A. (2008). An introduction to generalized linear models. 3rd Ed. CRC press.
  • Davison, A. C. (2003). Statistical models. Cambridge University Press.
  • Fox, J. (2008). Applied regression analysis and generalized linear models. 2nd Ed. Sage Publications.
  • Stapleton, J. H. (2007). Models for probability and statistical inference: theory and applications. John Wiley & Sons.
  • Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
  • Faraway, J. J. (2005). Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
  • Ruppert, D., Wand, M. P., & Carroll, R. J. (2003). Semiparametric regression. Cambridge University Press.
  • Faraway, J. J. (2002). Practical regression and ANOVA using R. (link)
  • Draper, N. R., & Smith, H. (1998). Applied regression analysis. 3rd Ed. John Wiley & Sons.
  • Long, J. S., & Freese, J. (1997). Regression models for categorical dependent variables. Sage Publications.
  • McCullagh, P., & Nelder, J. A. (1989). Generalized linear models. CRC press.

Counting and Probability

  • Shu, Z. (2016). Probability and Expectation (Volume 14). World Scientific
  • Zhou, X. (2015). Counting: Math for Gifted Students. CreateSpace. 
  • Hollos, S. & Hollos, J. R. (2013). Probability Problems and Solutions. Abrazol Publishing.
  • Patrick, D. (2007). Introduction to Counting and Probability. 2nd Ed. AoPS Incorporated.
  • Patrick, D. (2007). Intermediate Counting and Probability. AoPS Incorporated.
  • Hamming, R. W. (1993). The Art of Probability for Scientists and Engineers. CRC Press.

Software

See List of Statistical packages for a complete list.

See also

Other Resources