Probability and Statistics

From Ioannis Kourouklides
Jump to: navigation, search
Probability.gif

This page contains resources about Probability Theory and Statistics in general.

More specific information is included in each subfield.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

Subfields and Concepts[edit]

See Category:Probability and Statistics for all its subfields.

Statistical Inference / Inferential Statistics[edit]

  • Frequentist Inference
    • Statistical Hypothesis Testing / Statistical Tests
      • Fisher's Null Hypothesis Testing
      • Neyman-Pearson Theory
      • Analysis of Variance (ANOVA)
      • Analysis of Covariance (ANCOVA)
      • Multivariate Analysis of Variance (MANOVA)
      • T-test
      • F-test
      • Tests of Goodness-of-Fit
    • Confidence Intervals
    • Bootstrapping
  • Bayesian Inference
    • Bayesian Testing: Bayes Factor
    • Bayesian Confidence Sets: Credible Intervals
    • Hierarchical Bayes
    • Empirical Bayes
    • Full Bayes
  • Computational Methods for Bayesian Inference (i.e. using Algorithmic Methods)
  • Inductive inference
  • Empirical Inference
  • Causal Inference
  • Interval Estimation
  • Estimation Theory / Point Estimation
  • Sufficiency, Minimality, Completeness and Variance Reduction Techniques (VRT)
    • Gauss-Markov Theorem
    • Lehmann–Scheffe Theorem
    • Factorization Theorem
    • Complete statistic
    • Minimal sufficient statistic
    • Ancillary statistic
    • Fisher information
    • Fisher information metric / Fisher–Rao metric
    • Scoring algorithm / Fisher's scoring
    • Score function
    • Cramer–Rao bound (CRB) / Cramer–Rao lower bound (CRLB)
    • Rao–Blackwell Theorem
      • Rao–Blackwellization
      • Rao–Blackwell estimator
    • Exponential family
    • Conjugate prior family
  • Decision Theory
    • Neyman-Pearson Theory
    • The Expected Loss Principle
    • Optimal decision rules
    • Bayesian Decision Theory / Bayes estimator
    • Cost function / Loss function
    • Risk function
    • Admissibility
    • Unbiasedness
    • Minimaxity
  • Algorithmic Information Theory
    • Kolmogorov Complexity / Algorithmic Complexity
    • Algorithmic Probability / Solomonoff Probability
    • Universal Search (by Levin)
    • Algorithmic Randomness (by Martin-Lof)
    • Solomonoff's Theory of Inductive Inference
    • Epicurus' Principle of Multiple Explanations
    • Occam's Razor
    • Bayes' rule
    • Minimum Description Length (MDL) principle
    • Minimum Message Length (MML)
    • Algorithmic Statistics
  • Model Selection and Evaluation
    • Akaike Information Criterion (AIC)
    • Bayesian Information Criterion (BIC)
    • Deviance Information Criterion (DIC)
    • Bayesian Predictive Information Criterion (BPIC)
    • Focused Information Criterion (FIC)
    • Minimum Description Length (MDL)
    • Minimum Message Length (MML)
    • Akaike Final Prediction Error (FPE)
    • Parzen's Criterion Autoregressive Transfer Function (CAT)
    • Bayesian Model Selection / Bayesian Model Comparison
    • Cross-Validation
    • Statistical Hypothesis Testing (for Multilevel Models / Nested Models only)
      • Lagrange multiplier test / Score test / Score Method
      • Likelihood-ratio test
      • Wald test
    • Model Evaluation Metrics (for Classification)
      • Confusion Matrix
      • Accuracy
      • F-measure / F1-score / F-score
      • Precision
      • Recall / Sensitivity / True Positive Rate
      • Specificity / True Negative Rate
      • False Positive Rate
      • False Negative Rate
    • Model Evaluation Metrics (for Regression)
      • Mean Square Error (MSE)
      • Root MSE (RMSE)
      • Mean Absolute Error (MAE)
      • R-Squared

Statistical Models[edit]

  • Regression Analysis
    • Linear Regression Model
    • Simple Linear Regression
    • Multiple Linear Regression (not to be confused with Multivariate Linear Regression)
    • General Linear Model / Multivariate Linear Model
    • Generalized Linear Model (GLM or GLIM)
    • Poisson Regression
    • Negative Binomial Regression
    • Logistic Regression Model / Logit Model
    • Multinomial Logistic Regression / Softmax Regression
    • Probit Model
    • Fixed Effects Model
    • Hierarchical Linear Models / Multilevel Models / Nested Data Models
      • Random Effects Model / Variance Components Model
      • Mixed Effects Models (not to be confused with Mixture Models)
    • Nonparametric Regression Models
    • Semi-parametric Regression Models
    • Nonlinear Regression Models
    • Robust Regression Models
    • Random sample consensus (RANSAC)
    • Least Squares Methods
      • Ordinary Least Squares / Linear Least Squares
      • Weighted Least Squares
      • Nonlinear Least Squares
      • L1-regularization / Least absolute shrinkage and selection operator (LASSO) / Laplace prior
      • L2-regularization / Ridge Regression / Tikhonov Regularization / Gaussian prior
  • Probabilistic Models
  • State Space Models
    • Time Series Models
  • Reliability Engineering / Reliability Modelling
    • Survival Analysis
    • Reliability Theory
    • Risk Assessment
    • Hazard Function

Probability Theory[edit]

  • Random Variables
    • Continuous Random Variables
      • Probability Density Function
    • Discrete Random Variables
      • Probability Mass Function
    • Jointly Distributed Random Variables
      • Joint Density Function
    • Independent Random Variables
    • Uncorrelated Random Variables
  • Moments of a distribution
    • First Moment / Mean
    • Second Moment / Variance
    • Third Moment / Skewness
    • Fourth Moment / Kurtosis
  • Probabilistic Models
  • Stochastic Convergence
  • Probability Space
  • Measure Space
  • State Space
  • Theorem of Total Probability
  • Central Limit Theorem
  • Conditional Probability
  • Bayesian Probability Theory
  • Frequentist Probability Theory
  • Queueing Theory
  • Martingale Theory
  • Ergodic Theory
  • Decision Theory
  • Measure Theory
  • Utility Theory

Online Courses[edit]

Video Lectures[edit]


Lecture Notes[edit]

Books[edit]

Statistical Inference and Theory of Statistics[edit]

  • Bruce, P., & Bruce, A. (2017). Practical Statistics for Data Scientists: 50 Essential Concepts. O'Reilly Media.
  • Imbens, G. W., & Rubin D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction.
  • Ross, S. M. (2014). Introduction to probability models. 11th Ed. Academic Press.
  • Smith, R. C. (2013). Uncertainty quantification: theory, implementation, and applications. SIAM.
  • Gentle, J. E. (2013). Theory of statistics. (link)
  • DeGroot, M. H., & Schervish, M. J. (2012). Probability and statistics. 4th Ed. Pearson.
  • Abu-Mostafa, Y. S., Magdon-Ismail, M., & Lin, H. T. (2012). Learning From Data. AMLBook.
  • Diez, D. M., Barr, C. D., & Cetinkaya-Rundel, M. (2012). OpenIntro Statistics. CreateSpace.
  • Ramachandran, K. M., & Tsokos, C. P. (2012). Mathematical Statistics with Applications in R. Elsevier.
  • Liero, H., & Zwanzig, S. (2012). Introduction to the theory of statistical inference. CRC Press.
  • Wasserman, L. (2013). All of statistics: a concise course in statistical inference. Springer Science & Business Media.
  • Gentle, J. E. (2007). Matrix algebra: theory, computations, and applications in statistics. Springer Science & Business Media.
  • Rice, J. (2006). Mathematical statistics and data analysis. 3rd Ed. Duxbury Press.
  • Cox, D. R. (2006). Principles of statistical inference. Cambridge University Press.
  • Lavine, M. (2005). Introduction to Statistical Thought. Michael Lavine.
  • Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference. Cambridge University Press.
  • Lehmann, E. L., & Casella, G. (2003). Theory of point estimation. Springer.
  • Bertsekas, D. P., & Tsitsiklis, J. N. (2002). Introduction to Probability. Athena scientific.
  • Casella, G., & Berger, R. L. (2002). Statistical inference. Cengage Learning.
  • Garthwaite, P. H., Jolliffe, I. T., & Jones, B. (2002). Statistical inference. Oxford University Press.
  • Shao, J. (2000). Mathematical Statistics. Springer.
  • Mukhopadhyay, N. (2000). Probability and statistical inference. CRC Press.
  • Schervish, M. J. (1995). Theory of statistics. Springer Science & Business Media.

Regression Analysis, Reliability and Generalized Linear Models[edit]

  • Greene, W. H. (2018). Econometric analysis. 8th Ed. Pearson.
  • Harrell, F. (2015). Regression modeling strategies. 2nd Ed. Springer.
  • Kroese, D. P., & Chan, J. C. (2016). Statistical modeling and computation. Springer.
  • Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example. 5th Ed. John Wiley & Sons.
  • Kaminskiy, M. P. (2012). Reliability models for engineers and scientists. CRC Press.
  • Goldstein, H. (2010). Multilevel statistical models. 4th Ed. John Wiley & Sons.
  • Tobias, P. A., & Trindade, D. (2011). Applied reliability. 3rd Ed. CRC Press.
  • Freedman, D. A. (2009). Statistical models: theory and practice. Cambridge University Press.
  • Dobson, A. J., & Barnett, A. (2008). An introduction to generalized linear models. 3rd Ed. CRC press.
  • Davison, A. C. (2003). Statistical models. Cambridge University Press.
  • Fox, J. (2008). Applied regression analysis and generalized linear models. 2nd Ed. Sage Publications.
  • Stapleton, J. H. (2007). Models for probability and statistical inference: theory and applications. John Wiley & Sons.
  • Li, Q., & Racine, J. S. (2007). Nonparametric Econometrics: Theory and Practice. Princeton University Press.
  • Birolini, A. (2007). Reliability engineering: theory and practice. 5th Ed. Springer.
  • Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
  • Faraway, J. J. (2005). Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
  • Rausand, M., & Arnljot, H. A. (2004). System reliability theory: models, statistical methods, and applications. John Wiley & Sons.
  • Bazovsky, I. (2004). Reliability theory and practice. Courier Corporation.
  • Ruppert, D., Wand, M. P., & Carroll, R. J. (2003). Semiparametric regression. Cambridge University Press.
  • Faraway, J. J. (2002). Practical regression and ANOVA using R. (link)
  • O'Connor, P., & Kleyner, A. (2002). Practical reliability engineering. 4th Ed. John Wiley & Sons.
  • Hayashi, F. (2000). Econometrics. Princeton University Press.
  • Elandt-Johnson, R. C., & Johnson, N. L. (1999). Survival models and data analysis. John Wiley & Sons.
  • Draper, N. R., & Smith, H. (1998). Applied regression analysis. 3rd Ed. John Wiley & Sons.
  • Long, J. S., & Freese, J. (1997). Regression models for categorical dependent variables. Sage Publications.
  • Leemis, L. M. (1995). Reliability: probabilistic models and statistical methods. Prentice Hall.
  • McCullagh, P., & Nelder, J. A. (1989). Generalized linear models. CRC press.

Counting and Probability[edit]

  • Shu, Z. (2016). Probability and Expectation (Volume 14). World Scientific
  • Zhou, X. (2015). Counting: Math for Gifted Students. CreateSpace. 
  • Hollos, S. & Hollos, J. R. (2013). Probability Problems and Solutions. Abrazol Publishing.
  • Patrick, D. (2007). Introduction to Counting and Probability. 2nd Ed. AoPS Incorporated.
  • Hamming, R. W. (1993). The Art of Probability for Scientists and Engineers. CRC Press.

Software[edit]

See List of Statistical packages for a complete list.

See also[edit]

Other Resources[edit]

Retrieved from "https://wiki.kourouklides.com/w/index.php?title=Probability_and_Statistics&oldid=1600"