Regression Modeling Strategies

With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis

Book Information and Updates

Short Courses

Full Semester Course

• Offered for the first time in the Vanderbilt University Department of Biostatistics graduate program Spring 2013 (Jan-Apr). It is taught yearly by Prof. Harrell

Discussion Board

• datamethods and here
• stats.stackexchange.com
• To be added from rmsdisc: An older discussion board for readers and the author to discuss questions, issues, controversies, and new research related to the text

Datasets

• Datasets used in the book

Quizzes

• Quizzes (with answer sheets) on concepts in the text and on prerequisites, are available to instructors by E-mailing the author

Software

• To be added from BioMod#FittingDemos: Interactive scripts demonstrating various curve fitting criteria and showing the flexibility of restricted cubic splines (see also this)
• Warren Sarle’s SAS macros and examples for bootstrapping and jackknifing. See Warren’s cautionary note on bootstrap confidence intervals, with a good example related to R^2 in multiple regression. The example shows that when the estimate of R^2 is badly biased, bootstrap confidence limits are badly displaced to the right. Included in the notes is the standard error of R^2 and information about adjusted R^2.
• The penalized package in R
• function for binary logistic model external validation

Studies of Methods Used in the Text

• Recent simulation experiments conducted by Carl Moons and Frank Harrell indicate that the performance of transcan for multiple imputation is about halfway between single conditional mean imputation and MICE (see below), consistent with the findings from Faris PD, Ghali WA, et al (2002): Multiple imputation versus data enhancement for dealing with missing data in observational health care outcome analyses. J Clin Epidemiology 55:184-191. Suboptimal performance of transcan for multiple imputation is probably due to the fact thattranscan fits the flexible additive imputation models and then draws all multiple imputations from the fitted models. A new function in the Hmisc package, aregImpute, uses the bootstrap to re-fit additive nonparametric imputation models for each of the multiple imputations. Results for aregImpute are very promising (see below).
• Validation of binary logistic models
  • Simulation studies
  • Steyerberg EW, Harrell FE, Borsboom GJJM, Eijkemans MJC, Vergouwe Y, Habbema JDF (2001): Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology 54:774-781.
  • Steyerberg EW et al (2003): Internal and external validation of predictive models: A simulation study of bias and precision in small samples. Journal of Clinical Epidemiology 56:441-447.
  • Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF (2005): Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. Journal of Clinical Epidemiology 58:475-483.
• Studying the degrees of freedom spending strategy that uses generalized Spearman rho^2, in terms of preserving type I error and sigma^2 in ordinary least squares
• Prediction Error in Cox Models Varying Number of Predictors
• Shrinkage and problems with stepwise variable selection: See Steyerberg EW, Eijkemans MJC, Harrell FE, Habbema JDF (2001): Prognostic modeling with logistic regression analysis: In search of a sensible strategy in small data sets. Medical Decision Making 21:45-56.
• Model simplification and stepwise variable selection: See Ambler G, Brady AR, Royston P (2002): Simplifying a prognostic model: a simulation study based on clinical data. Statistics in Medicine 21:3803-3822. The authors studied the performance of the model simplification strategy discussed in the book, and compared it with more traditional variable selection methods, finding that standard variable selection can work well when there is a large proportion of irrelevant variables.
• New case study on penalized maximum likelihood estimation for binary logistic modeling: Moons KGM, Donders ART, Steyerberg EW, Harrell FE (2004): Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example. J Clinical Epidemiology 57:1262-1270.
• Choosing the penalty
• Interactive demonstrations of curve fitting, effects of categorization, etc.
• Peter Ellis’ article about overanalysis of time series data

R Help

• Stackoverflow

Multiple Imputation

• Joseph Schafer’s Multiple imputation FAQ
• Napier University’s imputation page
• Online Multiple Imputation and R MICE Software by Stef van Buuren and Karin Oudshoorn
• To subscribe to the Impute E-mail discussion group led by Juned Siddique of Northwestern University, click here.
• A paper containing a good overview of multiple imputation and a comparison of some software packages is Horton NJ, Lipsitz SR, The American Statistician 55:244-254; 2001.
• An excellent recent survey of missing data methods is Schafer, JL and Graham JW, Psychological Methods 7:147-177; 2002.
• See also Analysis of biases in SPSS by Paul von Hippel, The American Statistician 58:160-164; 2004.
• Notes from Tim Hesterberg on why the response variable must be used when doing multiple imputation. Tim’s notes include code to do several simulations illustrating his points.
• Comparisons of aregImpute with other imputation algorithms
  • Moons KGM, Donders RART, Stijnen T, Harrell FE, J Clinical Epidemiology 59:1092-1101; 2006.
  • Horton NJ, Kleinman KP, The American Statistician 61:79-90; 2007.

General Statistical Information

• Biostatistics for Biomedical Research Course

• Prediction modeling by Maarten van Smeeden

• Problems with categorizing continuous variables

• Medical citations for statistical issues

• Julian Faraway’s free book Practicle Regression and Anova using R

• Open source online book by Legler and Roback on statistical modeling

• Bender and Benner’s paper Calculating ordinal regression models in SAS and S-Plus

• Stephan Rudolfer’s presentation Diagnosis of Carpal Tunnel Syndrome using Logistic Regression, an excellent presentation on various types of ordinal logistic models. Includes a nice discussion of accuracy indexes.

• John Fox’s Applications of Quantitative Models in Sociology course material, including information on polytomous logistic regression

• John Fox’s excellent article on Bootstrapping Regression Models

• Brian Ripley’s terrific presentation Selecting amongst large classes of models, containing highly useful thoughts about AIC, cross-validation, and other concepts

• Paul Allison’s excellent discussion about R^2 measures

• Lindsay Smith’s nice tutorial on principal components analysis

• Glossary of Statistical Terms

• Annotated bibliography with emphasis on predictive methods, survival analysis, logistic regression, prognosis, diagnosis, modeling strategies, model validation, practical Bayesian methods, clinical trials, graphical methods, papers for teaching statistical methods, bootstrap, etc; FE Harrell

• Miscellaneous information on methodology, much of it culled from electronic discussion groups ( .zip file)

• Bob Obenchain’s Shrinkage Regression Web Page

• Patrick Burns’ Bootstrap and resampling page

• How to read output from the rms package nomogram function

• Jan de Leeuw’s excellent working paper on splines, including monotone splines

• Comparison of pseudo-R^2

• More about R^2 measures