# Difference between revisions of "Probability and Statistics"

More specific information is included in each subfield.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

## Subfields and Concepts

See Category:Probability and Statistics for all its subfields.

### Statistical Inference / Inferential Statistics

• Frequentist Inference
• Statistical Hypothesis Testing / Statistical Tests
• Fisher's Null Hypothesis Testing
• Neyman-Pearson Theory
• Analysis of Variance (ANOVA)
• Analysis of Covariance (ANCOVA)
• Multivariate Analysis of Variance (MANOVA)
• T-test
• F-test
• Tests of Goodness-of-Fit
• Confidence Intervals
• Bootstrapping
• Bayesian Inference
• Bayesian Testing: Bayes Factor
• Bayesian Confidence Sets: Credible Intervals
• Hierarchical Bayes
• Empirical Bayes
• Full Bayes
• Computational Methods for Bayesian Inference (i.e. using Algorithmic Methods)
• Inductive inference
• Empirical Inference
• Causal Inference
• Interval Estimation
• Estimation Theory / Point Estimation
• Sufficiency, Minimality, Completeness and Variance Reduction Techniques (VRT)
• Gauss-Markov Theorem
• Lehmann–Scheffe Theorem
• Factorization Theorem
• Complete statistic
• Minimal sufficient statistic
• Ancillary statistic
• Fisher information
• Fisher information metric / Fisher–Rao metric
• Scoring algorithm / Fisher's scoring
• Score function
• Cramer–Rao bound (CRB) / Cramer–Rao lower bound (CRLB)
• Rao–Blackwell Theorem
• Rao–Blackwellization
• Rao–Blackwell estimator
• Exponential family
• Conjugate prior family
• Decision Theory
• Neyman-Pearson Theory
• The Expected Loss Principle
• Optimal decision rules
• Bayesian Decision Theory / Bayes estimator
• Cost function / Loss function
• Risk function
• Unbiasedness
• Minimaxity
• Algorithmic Information Theory
• Kolmogorov Complexity / Algorithmic Complexity
• Algorithmic Probability / Solomonoff Probability
• Universal Search (by Levin)
• Algorithmic Randomness (by Martin-Lof)
• Solomonoff's Theory of Inductive Inference
• Epicurus' Principle of Multiple Explanations
• Occam's Razor
• Bayes' rule
• Minimum Description Length (MDL) principle
• Minimum Message Length (MML)
• Algorithmic Statistics
• Model Selection and Evaluation
• Akaike Information Criterion (AIC)
• Bayesian Information Criterion (BIC)
• Deviance Information Criterion (DIC)
• Bayesian Predictive Information Criterion (BPIC)
• Focused Information Criterion (FIC)
• Minimum Description Length (MDL)
• Minimum Message Length (MML)
• Akaike Final Prediction Error (FPE)
• Parzen's Criterion Autoregressive Transfer Function (CAT)
• Bayesian Model Selection / Bayesian Model Comparison
• Cross-Validation
• Statistical Hypothesis Testing (for Multilevel Models / Nested Models only)
• Lagrange multiplier test / Score test / Score Method
• Likelihood-ratio test
• Wald test
• Model Evaluation Metrics (for Classification)
• Confusion Matrix
• Accuracy
• F-measure / F1-score / F-score
• Precision
• Recall / Sensitivity / True Positive Rate
• Specificity / True Negative Rate
• False Positive Rate
• False Negative Rate
• Model Evaluation Metrics (for Regression)
• Mean Square Error (MSE)
• Root MSE (RMSE)
• Mean Absolute Error (MAE)
• R-Squared

### Statistical Models

• Regression Analysis
• Linear Regression Model
• Simple Linear Regression
• Multiple Linear Regression (not to be confused with Multivariate Linear Regression)
• General Linear Model / Multivariate Linear Model
• Generalized Linear Model (GLM or GLIM)
• Poisson Regression
• Negative Binomial Regression
• Logistic Regression Model / Logit Model
• Multinomial Logistic Regression / Softmax Regression
• Probit Model
• Fixed Effects Model
• Hierarchical Linear Models / Multilevel Models / Nested Data Models
• Random Effects Model / Variance Components Model
• Mixed Effects Models (not to be confused with Mixture Models)
• Nonparametric Regression Models
• Semi-parametric Regression Models
• Nonlinear Regression Models
• Robust Regression Models
• Random sample consensus (RANSAC)
• Least Squares Methods
• Ordinary Least Squares / Linear Least Squares
• Weighted Least Squares
• Nonlinear Least Squares
• L1-regularization / Least absolute shrinkage and selection operator (LASSO) / Laplace prior
• L2-regularization / Ridge Regression / Tikhonov Regularization / Gaussian prior
• Probabilistic Models
• State Space Models
• Time Series Models
• Reliability Engineering / Reliability Modelling
• Survival Analysis
• Reliability Theory
• Risk Assessment
• Hazard Function

### Probability Theory

• Random Variables
• Continuous Random Variables
• Probability Density Function
• Discrete Random Variables
• Probability Mass Function
• Jointly Distributed Random Variables
• Joint Density Function
• Independent Random Variables
• Uncorrelated Random Variables
• Moments of a distribution
• First Moment / Mean
• Second Moment / Variance
• Third Moment / Skewness
• Fourth Moment / Kurtosis
• Probabilistic Models
• Stochastic Convergence
• Probability Space
• Measure Space
• State Space
• Theorem of Total Probability
• Central Limit Theorem
• Conditional Probability
• Bayesian Probability Theory
• Frequentist Probability Theory
• Queueing Theory
• Martingale Theory
• Ergodic Theory
• Decision Theory
• Measure Theory
• Utility Theory

## Books

### Statistical Inference and Theory of Statistics

• Bruce, P., & Bruce, A. (2017). Practical Statistics for Data Scientists: 50 Essential Concepts. O'Reilly Media.
• Imbens, G. W., & Rubin D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction.
• Ross, S. M. (2014). Introduction to probability models. 11th Ed. Academic Press.
• Smith, R. C. (2013). Uncertainty quantification: theory, implementation, and applications. SIAM.
• Gentle, J. E. (2013). Theory of statistics. (link)
• DeGroot, M. H., & Schervish, M. J. (2012). Probability and statistics. 4th Ed. Pearson.
• Abu-Mostafa, Y. S., Magdon-Ismail, M., & Lin, H. T. (2012). Learning From Data. AMLBook.
• Diez, D. M., Barr, C. D., & Cetinkaya-Rundel, M. (2012). OpenIntro Statistics. CreateSpace.
• Ramachandran, K. M., & Tsokos, C. P. (2012). Mathematical Statistics with Applications in R. Elsevier.
• Liero, H., & Zwanzig, S. (2012). Introduction to the theory of statistical inference. CRC Press.
• Wasserman, L. (2013). All of statistics: a concise course in statistical inference. Springer Science & Business Media.
• Gentle, J. E. (2007). Matrix algebra: theory, computations, and applications in statistics. Springer Science & Business Media.
• Rice, J. (2006). Mathematical statistics and data analysis. 3rd Ed. Duxbury Press.
• Cox, D. R. (2006). Principles of statistical inference. Cambridge University Press.
• Lavine, M. (2005). Introduction to Statistical Thought. Michael Lavine.
• Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference. Cambridge University Press.
• Lehmann, E. L., & Casella, G. (2003). Theory of point estimation. Springer.
• Bertsekas, D. P., & Tsitsiklis, J. N. (2002). Introduction to Probability. Athena scientific.
• Casella, G., & Berger, R. L. (2002). Statistical inference. Cengage Learning.
• Garthwaite, P. H., Jolliffe, I. T., & Jones, B. (2002). Statistical inference. Oxford University Press.
• Shao, J. (2000). Mathematical Statistics. Springer.
• Mukhopadhyay, N. (2000). Probability and statistical inference. CRC Press.
• Schervish, M. J. (1995). Theory of statistics. Springer Science & Business Media.

### Regression Analysis, Reliability and Generalized Linear Models

• Greene, W. H. (2018). Econometric analysis. 8th Ed. Pearson.
• Harrell, F. (2015). Regression modeling strategies. 2nd Ed. Springer.
• Kroese, D. P., & Chan, J. C. (2016). Statistical modeling and computation. Springer.
• Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example. 5th Ed. John Wiley & Sons.
• Kaminskiy, M. P. (2012). Reliability models for engineers and scientists. CRC Press.
• Goldstein, H. (2010). Multilevel statistical models. 4th Ed. John Wiley & Sons.
• Tobias, P. A., & Trindade, D. (2011). Applied reliability. 3rd Ed. CRC Press.
• Freedman, D. A. (2009). Statistical models: theory and practice. Cambridge University Press.
• Dobson, A. J., & Barnett, A. (2008). An introduction to generalized linear models. 3rd Ed. CRC press.
• Davison, A. C. (2003). Statistical models. Cambridge University Press.
• Fox, J. (2008). Applied regression analysis and generalized linear models. 2nd Ed. Sage Publications.
• Stapleton, J. H. (2007). Models for probability and statistical inference: theory and applications. John Wiley & Sons.
• Li, Q., & Racine, J. S. (2007). Nonparametric Econometrics: Theory and Practice. Princeton University Press.
• Birolini, A. (2007). Reliability engineering: theory and practice. 5th Ed. Springer.
• Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
• Faraway, J. J. (2005). Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
• Rausand, M., & Arnljot, H. A. (2004). System reliability theory: models, statistical methods, and applications. John Wiley & Sons.
• Bazovsky, I. (2004). Reliability theory and practice. Courier Corporation.
• Ruppert, D., Wand, M. P., & Carroll, R. J. (2003). Semiparametric regression. Cambridge University Press.
• Faraway, J. J. (2002). Practical regression and ANOVA using R. (link)
• O'Connor, P., & Kleyner, A. (2002). Practical reliability engineering. 4th Ed. John Wiley & Sons.
• Hayashi, F. (2000). Econometrics. Princeton University Press.
• Elandt-Johnson, R. C., & Johnson, N. L. (1999). Survival models and data analysis. John Wiley & Sons.
• Draper, N. R., & Smith, H. (1998). Applied regression analysis. 3rd Ed. John Wiley & Sons.
• Long, J. S., & Freese, J. (1997). Regression models for categorical dependent variables. Sage Publications.
• Leemis, L. M. (1995). Reliability: probabilistic models and statistical methods. Prentice Hall.
• McCullagh, P., & Nelder, J. A. (1989). Generalized linear models. CRC press.

### Counting and Probability

• Shu, Z. (2016). Probability and Expectation (Volume 14). World Scientific
• Zhou, X. (2015). Counting: Math for Gifted Students. CreateSpace.
• Hollos, S. & Hollos, J. R. (2013). Probability Problems and Solutions. Abrazol Publishing.
• Patrick, D. (2007). Introduction to Counting and Probability. 2nd Ed. AoPS Incorporated.
• Hamming, R. W. (1993). The Art of Probability for Scientists and Engineers. CRC Press.

## Software

See List of Statistical packages for a complete list.