Generalized, linear, and mixed models

An accessible and self-contained introduction to statistical models-now in a modernized new edition

Generalized, Linear, and Mixed Models, Second Edition provides an up-to-date treatment of the essential techniques for developing and applying a wide variety of statistical models. The book presents thorough and unified coverage of the theory behind generalized, linear, and mixed models and highlights their similarities and differences in various construction, application, and computational aspects.

A clear introduction to the basic ideas of fixed effects models, random effects models, and mixed models is maintained throughout, and each chapter illustrates how these models are applicable in a wide array of contexts. In addition, a discussion of general methods for the analysis of such models is presented with an emphasis on the method of maximum likelihood for the estimation of parameters. The authors also provide comprehensive coverage of the latest statistical models for correlated, non-normally distributed data. Thoroughly updated to reflect the latest developments in the field, the Second Edition features: A new chapter that covers omitted covariates, incorrect random effects distribution, correlation of covariates and random effects, and robust variance estimation A new chapter that treats shared random effects models, latent class models, and properties of models A revised chapter on longitudinal data, which now includes a discussion of generalized linear models, modern advances in longitudinal data analysis, and the use between and within covariate decompositions Expanded coverage of marginal versus conditional models Numerous new and updated examples
With its accessible style and wealth of illustrative exercises, Generalized, Linear, and Mixed Models , Second Edition is an ideal book for courses on generalized linear and mixed models at the upper-undergraduate and beginning-graduate levels. It also serves as a valuable reference for applied statisticians, industrial practitioners, and researchers.

Author Notes

Charles E. McCulloch, PhD, is Professor and Head of the Division of Biostatistics in the School of Medicine at the University of California, San Francisco
Shayle R. Searle, PhD, is Professor Emeritus in the Department of Biological Statistics and Computational Biology at Cornell University
John M. Neuhaus, PhD, is Professor of Biostatistics in the School of Medicine at the University of California, San Francisco

Preface	p. xxi
Preface to the First Edition	p. xxiii
1 Introduction	p. 1
1.1 Models	p. 1
a Linear models (LM) and linear mixed models (LMM)	p. 1
b Generalized models (GLMs and GLMMs)	p. 2
1.2 Factors, Levels, Cells, Effects and Data	p. 2
1.3 Fixed Effects Models	p. 5
a Example 1: Placebo and a drug	p. 6
b Example 2: Comprehension of humor	p. 7
c Example 3: Four dose levels of a drug	p. 8
1.4 Random Effects Models	p. 8
a Example 4: Clinics	p. 8
b Notation	p. 9
c Example 5: Ball bearings and calipers	p. 12
1.5 Linear Mixed Models (LMMs)	p. 13
a Example 6: Medications and clinics	p. 13
b Example 7: Drying methods and fabrics	p. 13
c Example 8: Potomac River Fever	p. 14
d Regression models	p. 14
e Longitudinal data	p. 14
f Example 9: Osteoarthritis Initiative	p. 16
g Model equations	p. 16
1.6 Fixed or Random?	p. 16
a Example 10: Clinical trials	p. 17
b Making a decision	p. 17
1.7 Inference	p. 19
a Estimation	p. 20
b Testing	p. 24
c Prediction	p. 25
1.8 Computer Software	p. 25
1.9 Exercises	p. 26
2 One-Way Classifications	p. 28
2.1 Normality and Fixed Effects	p. 29
a Model	p. 29
b Estimation by ML	p. 29
c Generalized likelihood ratio test	p. 31
d Confidence intervals	p. 32
e Hypothesis tests	p. 34
2.2 Normality, Random Effects and MLE	p. 34
a Model	p. 34
b Balanced data	p. 37
c Unbalanced data	p. 42
d Bias	p. 44
e Sampling variances	p. 44
2.3 Normality, Random Effects and Reml	p. 45
a Balanced data	p. 45
b Unbalanced data	p. 48
2.4 More on Random Effects and Normality	p. 48
a Tests and confidence intervals	p. 48
b Predicting random effects	p. 49
2.5 Binary Data: Fixed Effects	p. 51
a Model equation	p. 51
b Likelihood	p. 51
c ML equations and their solutions	p. 52
d Likelihood ratio test	p. 52
e The usual chi-square test	p. 52
f Large-sample tests and confidence intervals	p. 54
g Exact tests and confidence intervals	p. 55
h Example: Snake strike data	p. 56
2.6 Binary Data: Random Effects	p. 57
a Model equation	p. 57
b Beta-binomial model	p. 57
c Logit-normal model	p. 64
d Probit-normal model	p. 68
2.7 Computing	p. 68
2.8 Exercises	p. 68
3 Single-Predictor Regression	p. 72
3.1 Introduction	p. 72
3.2 Normality: Simple Linear Regression	p. 73
a Model	p. 73
b Likelihood	p. 74
c Maximum likelihood estimators	p. 74
d Distributions of MLEs	p. 75
e Tests and confidence intervals	p. 76
f Illustration	p. 76
3.3 Normality: A Nonlinear Model	p. 77
a Model	p. 77
b Likelihood	p. 77
c Maximum likelihood estimators	p. 78
d Distributions of MLEs	p. 79
3.4 Transforming Versus Linking	p. 80
a Transforming	p. 80
b Linking	p. 80
c Comparisons	p. 80
3.5 Random Intercepts: Balanced Data	p. 81
a The model	p. 81
b Estimating [mu] and [beta]	p. 83
c Estimating variances	p. 86
d Tests of hypotheses - using LRT	p. 89
e Illustration	p. 92
f Predicting the random intercepts	p. 93
3.6 Random Intercepts: Unbalanced Data	p. 95
a The model	p. 97
b Estimating [mu] and [beta] when variances are known	p. 98
3.7 Bernoulli - Logistic Regression	p. 101
a Logistic regression model	p. 102
b Likelihood	p. 104
c ML equations	p. 104
d Large-sample tests and confidence intervals	p. 107
3.8 Bernoulli - Logistic with Random Intercepts	p. 108
a Model	p. 108
b Likelihood	p. 109
c Large-sample tests and confidence intervals	p. 110
d Prediction	p. 110
e Conditional Inference	p. 111
3.9 Exercises	p. 112
4 Linear Models (LMs)	p. 114
4.1 A General Model	p. 115
4.2 A Linear Model for Fixed Effects	p. 116
4.3 Mle Under Normality	p. 117
4.4 Sufficient Statistics	p. 118
4.5 Many Apparent Estimators	p. 119
a General result	p. 119
b Mean and variance	p. 120
c Invariance properties	p. 120
d Distributions	p. 121
4.6 Estimable Functions	p. 121
a Introduction	p. 121
b Definition	p. 122
c Properties	p. 122
d Estimation	p. 123
4.7 A Numerical Example	p. 123
4.8 Estimating Residual Variance	p. 125
a Estimation	p. 125
b Distribution of estimators	p. 126
4.9 The One- and Two-Way Classifications	p. 127
a The one-way classification	p. 127
b The two-way classification	p. 128
4.10 Testing Linear Hypotheses	p. 129
a Likelihood ratio test	p. 130
b Wald test	p. 131
4.11 t-Tests and Confidence Intervals	p. 131
4.12 Unique Estimation Using Restrictions	p. 132
4.13 Exercises	p. 134
5 Generalized Linear Models (GLMs)	p. 136
5.1 Introduction	p. 136
5.2 Structure of the Model	p. 138
a Distribution of y	p. 138
b Link function	p. 139
c Predictors	p. 139
d Linear models	p. 140
5.3 Transforming Versus Linking	p. 140
5.4 Estimation by Maximum Likelihood	p. 140
a Likelihood	p. 140
b Some useful identities	p. 141
c Likelihood equations	p. 142
d Large-sample variances	p. 144
e Solving the ML equations	p. 144
f Example: Potato flour dilutions	p. 145
5.5 Tests of Hypotheses	p. 148
a Likelihood ratio tests	p. 148
b Wald tests	p. 149
c Illustration of tests	p. 150
d Confidence intervals	p. 151
e Illustration of confidence intervals	p. 151
5.6 Maximum Quasi-Likelihood	p. 152
a Introduction	p. 152
b Definition	p. 152
5.7 Exercises	p. 156
6 Linear Mixed Models (LMMs)	p. 157
6.1 A General Model	p. 157
a Introduction	p. 157
b Basic properties	p. 158
6.2 Attributing Structure to Var(y)	p. 159
a Example	p. 159
b Taking covariances between factors as zero	p. 159
c The traditional variance components model	p. 161
d An LMM for longitudinal data	p. 163
6.3 Estimating Fixed Effects for V Known	p. 163
6.4 Estimating Fixed Effects for V Unknown	p. 165
a Estimation	p. 165
b Sampling variance	p. 165
c Bias in the variance	p. 167
d Approximate F-statistics	p. 168
6.5 Predicting Random Effects for V Known	p. 169
6.6 Predicting Random Effects for V Unknown	p. 171
a Estimation	p. 171
b Sampling variance	p. 171
c Bias in the variance	p. 172
6.7 Anova Estimation of Variance Components	p. 172
a Balanced data	p. 173
b Unbalanced data	p. 174
6.8 Maximum Likelihood (ML) Estimation	p. 174
a Estimators	p. 174
b Information matrix	p. 176
c Asymptotic sampling variances	p. 176
6.9 Restricted Maximum Likelihood (REML)	p. 177
a Estimation	p. 177
b Sampling variances	p. 178
6.10 Notes and Extensions	p. 178
a ML or REML?	p. 178
b Other methods for estimating variances	p. 179
6.11 Appendix for Chapter 6	p. 179
a Differentiating a log likelihood	p. 179
b Differentiating a generalized inverse	p. 182
c Differentiation for the variance components model	p. 183
6.12 Exercises	p. 185
7 Generalized Linear Mixed Models	p. 188
7.1 Introduction	p. 188
7.2 Structure of the Model	p. 189
a Conditional distribution of y	p. 189
7.3 Consequences of Having Random Effects	p. 190
a Marginal versus conditional distribution	p. 190
b Mean of y	p. 190
c Variances	p. 191
d Covariances and correlations	p. 192
7.4 Estimation by Maximum Likelihood	p. 193
a Likelihood	p. 193
b Likelihood equations	p. 195
7.5 Other Methods of Estimation	p. 196
a Penalized quasi-likelihood	p. 196
b Conditional likelihood	p. 198
c Simpler models	p. 203
7.6 Tests of Hypotheses	p. 204
a Likelihood ratio tests	p. 204
b Asymptotic variances	p. 204
c Wald tests	p. 204
d Score tests	p. 205
7.7 Illustration: Chestnut Leaf Blight	p. 205
a A random effects probit model	p. 206
7.8 Exercises	p. 210
8 Models for Longitudinal Data	p. 212
8.1 Introduction	p. 212
8.2 A Model for Balanced Data	p. 213
a Prescription	p. 213
b Estimating the mean	p. 213
c Estimating V[subscript 0]	p. 214
8.3 A Mixed Model Approach	p. 215
a Fixed and random effects	p. 215
b Variances	p. 215
8.4 Random Intercept and Slope Models	p. 216
a Variances	p. 217
b Within-subject correlations	p. 217
8.5 Predicting Random Effects	p. 219
a Uncorrelated subjects	p. 219
b Uncorrelated between, and within, subjects	p. 220
c Uncorrelated between, and autocorrelated within	p. 220
d Random intercepts and slopes	p. 221
8.6 Estimating Parameters	p. 221
a The general case	p. 221
b Uncorrelated subjects	p. 222
c Uncorrelated between, and autocorrelated within, subjects	p. 223
8.7 Unbalanced Data	p. 225
a Example and model	p. 225
b Uncorrelated subjects	p. 227
8.8 Models for Non-Normal Responses	p. 228
a Covariances and correlations	p. 229
b Estimation	p. 229
c Prediction of random effects	p. 229
d Binary responses, random intercepts and slopes	p. 231
8.9 A Summary of Results	p. 231
a Balanced data	p. 232
b Unbalanced data	p. 233
8.10 Appendix	p. 233
a For Section 8.4a	p. 233
b For Section 8.4b	p. 234
8.11 Exercises	p. 234
9 Marginal Models	p. 236
9.1 Introduction	p. 236
9.2 Examples of Marginal Regression Models	p. 238
9.3 Generalized Estimating Equations	p. 239
a Models with marginal and conditional interpretations	p. 244
9.4 Contrasting Marginal and Conditional Models	p. 246
9.5 Exercises	p. 247
10 Multivariate Models	p. 249
10.1 Introduction	p. 249
10.2 Multivariate Normal Outcomes	p. 250
10.3 Non-Normally Distributed Outcomes	p. 252
a A multivariate binary model	p. 252
b A binary/normal example	p. 253
c A Poisson/Normal Example	p. 257
10.4 Correlated Random Effects	p. 260
10.5 Likelihood-Based Analysis	p. 261
10.6 Example: Osteoarthritis Initiative	p. 263
10.7 Notes and Extensions	p. 264
a Missing data	p. 264
b Efficiency	p. 265
10.8 Exercises	p. 265
11 Nonlinear Models	p. 266
11.1 Introduction	p. 266
11.2 Example: Corn Photosynthesis	p. 266
11.3 Pharmacokinetic Models	p. 269
11.4 Computations for Nonlinear Mixed Models	p. 270
11.5 Exercises	p. 270
12 Departures from Assumptions	p. 271
12.1 Introduction	p. 271
12.2 Incorrect Model for Response	p. 272
a Omitted covariates	p. 272
b Misspecified link functions	p. 275
c Misclassified binary outcomes	p. 276
d Informative cluster sizes	p. 278
12.3 Incorrect Random Effects Distribution	p. 281
a Incorrect distributional family	p. 282
b Correlation of covariates and random effects	p. 290
c Covariate-dependent random effects variance	p. 293
12.4 Diagnosing Misspecification	p. 295
a Conditional likelihood methods	p. 295
b Between/within cluster covariate decompositions	p. 297
c Specification tests	p. 298
d Nonparametric maximum likelihood	p. 299
12.5 A Summary of Results	p. 300
12.6 Exercises	p. 301
13 Prediction	p. 303
13.1 Introduction	p. 303
13.2 Best Prediction (BP)	p. 304
a The best predictor	p. 304
b Mean and variance properties	p. 305
c A correlation property	p. 305
d Maximizing a mean	p. 305
e Normality	p. 306
13.3 Best Linear Prediction (BLP)	p. 306
a BLP(u)	p. 306
b Example	p. 307
c Derivation	p. 308
d Ranking	p. 309
13.4 Linear Mixed Model Prediction (BLUP)	p. 310
a BLUE(X[beta])	p. 310
b BLUP(t'X[beta] + s'u)	p. 311
c Two variances	p. 312
d Other derivations	p. 312
13.5 Required Assumptions	p. 313
13.6 Estimated Best Prediction	p. 313
13.7 Henderson's Mixed Model Equations	p. 314
a Origin	p. 314
b Solutions	p. 315
c Use in ML estimation of variance components	p. 316
13.8 Appendix	p. 317
a Verification of (13.5)	p. 317
b Verification of (13.7) and (13.8)	p. 318
13.9 Exercises	p. 318
14 Computing	p. 320
14.1 Introduction	p. 320
14.2 Computing ML Estimates for LMMs	p. 320
a The EM algorithm	p. 320
b Using E[u\|y]	p. 323
c Newton-Raphson method	p. 324
14.3 Computing ML Estimates for GLMMs	p. 326
a Numerical quadrature	p. 326
b EM algorithm	p. 331
c Markov chain Monte Carlo algorithms	p. 333
d Stochastic approximation algorithms	p. 336
e Simulated maximum likelihood	p. 337
14.4 Penalized Quasi-Likelihood and Laplace	p. 338
14.5 Iterative Bootstrap Bias Correction	p. 342
14.6 Exercises	p. 342
Appendix M Some Matrix Results	p. 344
M.1 Vectors and Matrices of Ones	p. 344
M.2 Kronecker (or Direct) Products	p. 345
M.3 A Matrix Notation in Terms of Elements	p. 346
M.4 Generalized Inverses	p. 346
a Definition	p. 346
b Generalized inverses of X'X	p. 347
c Two results involving X(X'V[superscript -1]X)[superscript -]X'V[superscript -1]	p. 348
d Solving linear equations	p. 349
e Rank results	p. 349
f Vectors orthogonal to columns of X	p. 349
g A theorem for K' with K'X being null	p. 350
M.5 Differential Calculus	p. 350
a Definition	p. 350
b Scalars	p. 350
c Vectors	p. 351
d Inner products	p. 351
e Quadratic forms	p. 351
f Inverse matrices	p. 351
g Determinants	p. 352
Appendix S Some Statistical Results	p. 353
S.1 Moments	p. 353
a Conditional moments	p. 353
b Mean of a quadratic form	p. 354
c Moment generating function	p. 354
S.2 Normal Distributions	p. 355
a Univariate	p. 355
b Multivariate	p. 355
c Quadratic forms in normal variables	p. 356
S.3 Exponential Families	p. 357
S.4 Maximum Likelihood	p. 357
a The likelihood function	p. 357
b Maximum likelihood estimation	p. 358
c Asymptotic variance-covariance matrix	p. 358
d Asymptotic distribution of MLEs	p. 359
S.5 Likelihood Ratio Tests	p. 359
S.6 MLE Under Normality	p. 360
a Estimation of [beta]	p. 360
b Estimation of variance components	p. 361
c Asymptotic variance-covariance matrix	p. 361
d Restricted maximum likelihood (REML)	p. 362
References	p. 364
Index	p. 378

Available:*

On Order

Summary

Summary

Author Notes

Table of Contents