Difference between revisions of "Probability and Statistics"

From Ioannis Kourouklides
Jump to navigation Jump to search
Line 99: Line 99:
 
*** Mixed Effects Models ''(not to be confused with Mixture Models)''
 
*** Mixed Effects Models ''(not to be confused with Mixture Models)''
 
** Nonparametric Regression Models
 
** Nonparametric Regression Models
 +
** Semi-parametric Regression Models
 
** Nonlinear Regression Models
 
** Nonlinear Regression Models
 
** Robust Regression Models
 
** Robust Regression Models

Revision as of 08:30, 8 May 2018

Probability.gif

This page contains resources about Probability Theory and Statistics in general.

More specific information is included in each subfield.

Subfields and Concepts

See Category:Probability and Statistics for all its subfields.

Statistical Inference / Inferential Statistics

  • Frequentist Inference
    • Statistical Hypothesis Testing / Statistical Tests
      • Fisher's Null Hypothesis Testing
      • Neyman-Pearson Theory
      • Analysis of Variance (ANOVA)
      • Analysis of Covariance (ANCOVA)
      • Multivariate Analysis of Variance (MANOVA)
      • T-test
      • F-test
      • Tests of Goodness-of-Fit
    • Confidence Intervals
    • Bootstrapping
  • Bayesian Inference
  • Inductive inference
  • Empirical Inference
  • Causal Inference
  • Interval Estimation
  • Estimation Theory / Point Estimation
  • Decision Theory
    • Neyman-Pearson Theory
    • The Expected Loss Principle
    • Optimal decision rules
    • Bayesian Decision Theory / Bayesian Estimator
    • Cost function / Loss function
    • Risk function
    • Admissibility
    • Unbiasedness
    • Minimaxity
  • Algorithmic Information Theory
    • Kolmogorov Complexity / Algorithmic Complexity
    • Algorithmic Probability / Solomonoff Probability
    • Universal Search (by Levin)
    • Algorithmic Randomness (by Martin-Lof)
    • Solomonoff's Theory of Inductive Inference
    • Epicurus' Principle of Multiple Explanations
    • Occam's Razor
    • Bayes rule
    • Minimum Description Length (MDL) principle
    • Minimum Message Length (MML)
  • Model Selection and Evaluation
    • Akaike Information Criterion (AIC)
    • Bayesian Information Criterion (BIC)
    • Deviance Information Criterion (DIC)
    • Bayesian Predictive Information Criterion (BPIC)
    • Focused Information Criterion (FIC)
    • Bayesian Model Selection / Bayesian Model Comparison
      • Bayesian Model Averaging
    • Bayesian Parameter Estimation
    • Minimum Description Length (MDL) principle
    • Minimum Message Length (MML)
    • Akaike Final Prediction Error (FPE)
    • Parzen's Criterion Autoregressive Transfer Function (CAT)
    • Cross-Validation
    • Statistical Hypothesis Testing (for Multilevel Models / Nested Models only)
      • Lagrange multiplier test / Score test / Score Method
      • Likelihood-ratio test
      • Wald test

Statistical Models

  • Regression Analysis
    • Linear Regression Model
    • Simple Linear Regression
    • Multiple Linear Regression (not to be confused with Multivariate Linear Regression)
    • General Linear Model / Multivariate Linear Regression
    • Generalized Linear Model (GLM or GLIM)
    • Poisson Regression
    • Logistic Regression Model / Logit Model
    • Probit Model
    • Fixed Effects Model
    • Hierarchical Linear Models / Multilevel Models / Nested Data Models
      • Random Effects Model / Variance Components Model
      • Mixed Effects Models (not to be confused with Mixture Models)
    • Nonparametric Regression Models
    • Semi-parametric Regression Models
    • Nonlinear Regression Models
    • Robust Regression Models
    • Random sample consensus (RANSAC)
    • Least Squares Methods
      • Ordinary Least Squares / Linear Least Squares
      • Weighted Least Squares
      • Nonlinear Least Squares
      • L1-regularization / Least absolute shrinkage and selection operator (LASSO) / Laplace prior
      • L2-regularization / Ridge Regression / Tikhonov Regularization / Gaussian prior
  • Probabilistic Models
  • State Space Models
    • Time Series Models

Probability Theory

  • Random Variables
    • Continuous Random Variables
      • Probability Density Function
    • Discrete Random Variables
      • Probability Mass Function
    • Jointly Distributed Random Variables
      • Joint Density Function
    • Independent Random Variables
    • Uncorrelated Random Variables
  • Moments of a distribution
    • First Moment / Mean
    • Second Moment / Variance
    • Third Moment / Skewness
    • Fourth Moment / Kurtosis
  • Probabilistic Models
  • Stochastic Convergence
  • Probability Space
  • Measure Space
  • State Space
  • Theorem of Total Probability
  • Central Limit Theorem
  • Bayesian Probability Theory
  • Frequentist Probability Theory
  • Queueing Theory
  • Martingale Theory
  • Ergodic Theory
  • Decision Theory
  • Measure Theory
  • Utility Theory

Online Courses

Video Lectures


Lecture Notes

Books

Statistical Inference and Theory of Statistics

  • Bruce, P., & Bruce, A. (2017). Practical Statistics for Data Scientists: 50 Essential Concepts. O'Reilly Media.
  • Imbens, G. W., & Rubin D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction.
  • Ross, S. M. (2014). Introduction to probability models. 11th Ed. Academic Press.
  • Smith, R. C. (2013). Uncertainty quantification: theory, implementation, and applications. SIAM.
  • Gentle, J. E. (2013). Theory of statistics. (link)
  • DeGroot, M. H., & Schervish, M. J. (2012). Probability and statistics. 4th Ed. Pearson.
  • Abu-Mostafa, Y. S., Magdon-Ismail, M., & Lin, H. T. (2012). Learning From Data. AMLBook.
  • Diez, D. M., Barr, C. D., & Cetinkaya-Rundel, M. (2012). OpenIntro Statistics. CreateSpace.
  • Ramachandran, K. M., & Tsokos, C. P. (2012). Mathematical Statistics with Applications in R. Elsevier.
  • Liero, H., & Zwanzig, S. (2012). Introduction to the theory of statistical inference. CRC Press.
  • Gentle, J. E. (2007). Matrix algebra: theory, computations, and applications in statistics. Springer Science & Business Media.
  • Rice, J. (2006). Mathematical statistics and data analysis. 3rd Ed. Duxbury Press.
  • Cox, D. R. (2006). Principles of statistical inference. Cambridge University Press.
  • Lavine, M. (2005). Introduction to Statistical Thought. Michael Lavine.
  • Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference. Cambridge University Press.
  • Lehmann, E. L., & Casella, G. (2003). Theory of point estimation. Springer.
  • Bertsekas, D. P., & Tsitsiklis, J. N. (2002). Introduction to Probability. Athena scientific.
  • Casella, G., & Berger, R. L. (2002). Statistical inference. Cengage Learning.
  • Garthwaite, P. H., Jolliffe, I. T., & Jones, B. (2002). Statistical inference. Oxford University Press.
  • Shao, J. (2000). Mathematical Statistics. Springer.
  • Mukhopadhyay, N. (2000). Probability and statistical inference. CRC Press.
  • Schervish, M. J. (1995). Theory of statistics. Springer Science & Business Media.

Regression Analysis and Generalized Linear Models

  • Greene, W. H. (2018). Econometric analysis. 8th Ed. Pearson.
  • Harrell, F. (2015). Regression modeling strategies. 2nd Ed. Springer.
  • Kroese, D. P., & Chan, J. C. (2016). Statistical modeling and computation. Springer.
  • Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example. 5th Ed. John Wiley & Sons.
  • Goldstein, H. (2010). Multilevel statistical models. 4th Ed. John Wiley & Sons.
  • Freedman, D. A. (2009). Statistical models: theory and practice. Cambridge University Press.
  • Dobson, A. J., & Barnett, A. (2008). An introduction to generalized linear models. 3rd Ed. CRC press.
  • Davison, A. C. (2003). Statistical models. Cambridge University Press.
  • Fox, J. (2008). Applied regression analysis and generalized linear models. 2nd Ed. Sage Publications.
  • Stapleton, J. H. (2007). Models for probability and statistical inference: theory and applications. John Wiley & Sons.
  • Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
  • Faraway, J. J. (2005). Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
  • Ruppert, D., Wand, M. P., & Carroll, R. J. (2003). Semiparametric regression. Cambridge University Press.
  • Hayashi, F. (2000). Econometrics. Princeton University Press.
  • Faraway, J. J. (2002). Practical regression and ANOVA using R. (link)
  • Draper, N. R., & Smith, H. (1998). Applied regression analysis. 3rd Ed. John Wiley & Sons.
  • Long, J. S., & Freese, J. (1997). Regression models for categorical dependent variables. Sage Publications.
  • McCullagh, P., & Nelder, J. A. (1989). Generalized linear models. CRC press.

Counting and Probability

  • Shu, Z. (2016). Probability and Expectation (Volume 14). World Scientific
  • Zhou, X. (2015). Counting: Math for Gifted Students. CreateSpace. 
  • Hollos, S. & Hollos, J. R. (2013). Probability Problems and Solutions. Abrazol Publishing.
  • Patrick, D. (2007). Introduction to Counting and Probability. 2nd Ed. AoPS Incorporated.
  • Patrick, D. (2007). Intermediate Counting and Probability. AoPS Incorporated.
  • Hamming, R. W. (1993). The Art of Probability for Scientists and Engineers. CRC Press.

Software

See List of Statistical packages for a complete list.

See also

Other Resources