Generalized Linear Regression

The world generates data in many shapes and forms, and most of it doesn't follow a neat bell curve. When a hospital tracks the number of patient readmissions per month, those counts are always non-negative integers. When a bank records whether a credit card transaction is fraudulent, the outcome is either yes or no. When an insurer logs claim amounts, the values are strictly positive and often heavily right-skewed. Each of these data types has its own natural statistical distribution, and fitting a model that respects that distribution leads to better predictions, more honest uncertainty estimates, and more meaningful interpretations.

GLMs matter because they let the data speak in its own language. Instead of transforming your response variable to approximate normality and then hoping the transformation doesn't distort your conclusions, GLMs work directly with the natural distribution of your outcome. A Poisson GLM for count data, for example, automatically ensures that predicted counts are non-negative. A logistic regression GLM for binary outcomes automatically constrains predictions to the valid probability range between 0 and 1. This isn't just mathematical elegance; it has real consequences for the quality and trustworthiness of your analysis.

The practical reach of GLMs is enormous. Consider a few concrete examples of where they appear in everyday analytical work:

• Healthcare: Modeling the probability of disease diagnosis based on patient risk factors using logistic regression
• Insurance: Predicting claim frequency with Poisson regression and claim severity with gamma regression
• Marketing: Estimating customer churn probability or the number of purchases a customer will make in a given period
• Ecology: Counting species occurrences across different habitats and environmental conditions
• Finance: Modeling default risk, credit scoring, and the distribution of financial returns

Beyond their practical applications, GLMs also serve as a conceptual bridge to more advanced modeling techniques. Understanding GLMs deeply makes it much easier to learn generalized additive models, mixed-effects models, survival analysis, and even certain classes of neural networks. The ideas of link functions, likelihood-based estimation, and deviance-based model comparison appear throughout modern statistical modeling. Investing time in GLMs pays dividends far beyond the models themselves.

There's also a compelling argument for GLMs from the perspective of scientific integrity. When you use a model that matches the true data-generating process, your standard errors are more accurate, your confidence intervals are better calibrated, and your hypothesis tests are more reliable. Using ordinary linear regression on binary data, for instance, can produce standard errors that are systematically wrong, leading to false confidence in your conclusions.^[2] GLMs help you avoid these pitfalls by building the right statistical assumptions into the model from the start.

Finally, GLMs are widely supported in every major statistical computing environment. Whether you work in R, Python, SAS, Stata, or Julia, you have access to well-tested, efficient implementations of GLMs. The concepts you learn in this book translate directly into code you can run on your own data, making this knowledge immediately actionable rather than purely theoretical.

Linear regression is one of the most powerful and widely used tools in statistics, and for good reason. When its assumptions hold, it provides unbiased estimates, interpretable coefficients, and well-understood uncertainty quantification. The problem is that its assumptions are quite specific, and real-world data frequently violates them in ways that matter. Understanding exactly where and why linear regression breaks down is the clearest motivation for learning GLMs.

The most fundamental assumption of linear regression is that the response variable follows a normal distribution around the predicted mean, with constant variance across all levels of the predictors. This assumption is called homoscedasticity, and it's violated in many common situations. When you're modeling count data, the variance tends to increase with the mean. When you're modeling proportions, the variance is highest near 0.5 and shrinks toward zero as the proportion approaches 0 or 1. These patterns are not bugs in your data; they are natural features of the underlying distributions, and linear regression has no way to accommodate them.

A second critical limitation is that linear regression places no constraints on the range of predicted values. The linear predictor can produce any real number, from negative infinity to positive infinity. This is perfectly appropriate when your outcome is a continuous measurement that can take any value. But consider what happens when you apply this to a binary outcome coded as 0 and 1. The model will happily predict values like 1.4 or -0.2, which are meaningless as probabilities. Analysts sometimes try to fix this by capping predictions at 0 and 1, but this is an ad hoc patch that doesn't address the underlying statistical problem.

A classic demonstration of this failure: fit a linear probability model to a dataset where a binary outcome depends on a single continuous predictor. Plot the fitted line. For low values of the predictor, the predicted probability drops below zero. For high values, it rises above one. The model is technically fitting a line, but the line is describing something that cannot exist in reality.

Data transformations are often proposed as a solution to these problems. Taking the logarithm of a skewed response variable, for example, can sometimes make it more normally distributed. But transformations come with serious drawbacks. They change the scale of interpretation, making coefficients harder to explain. They can introduce new violations of assumptions. And they don't always work: you can't log-transform a binary variable, and transforming count data with many zeros is problematic. GLMs sidestep these issues entirely by modeling the response on its natural scale while using a link function to connect it to the linear predictor.

There's also the issue of what statisticians call the mean-variance relationship. In ordinary linear regression, the variance of the response is assumed to be constant and independent of the mean. In many real distributions, this is simply not true. For Poisson-distributed count data, the variance equals the mean. For binomial data, the variance is a specific function of the mean. When you ignore these relationships and use linear regression anyway, your standard errors are wrong, and your inference is unreliable. GLMs explicitly model the mean-variance relationship as part of the distributional assumption, giving you correct standard errors automatically.

It's worth being clear that linear regression is not a bad model. It's an excellent model for the problems it was designed to solve. The issue arises when analysts apply it to problems outside its domain of validity, either out of habit, convenience, or unfamiliarity with the alternatives. The goal of this book is not to replace linear regression in your toolkit but to expand that toolkit so you can choose the right model for each problem you face.

Here is a summary of the key situations where linear regression is likely to fail and what GLM alternative is typically more appropriate:

• Binary outcomes (yes/no, success/failure): Use logistic regression with a binomial distribution and logit link
• Count data (number of events): Use Poisson regression with a Poisson distribution and log link
• Proportions between 0 and 1: Use binomial regression or beta regression depending on the data structure
• Positive continuous data with right skew: Use gamma regression with a gamma distribution and log or inverse link
• Highly right-skewed positive data: Consider inverse Gaussian regression for extreme skewness

This book is organized to build your understanding progressively, starting with the conceptual and mathematical foundations and moving toward practical application and advanced topics. Each chapter builds on the previous ones, so working through the material in order will give you the strongest foundation. That said, if you already have solid knowledge of a particular area, the chapters are written to be reasonably self-contained, and you can navigate to the topics most relevant to your current needs.

Chapter 1: Foundations of Generalized Linear Models starts with a concise review of linear regression to establish a shared baseline, then introduces the three core components of every GLM: the random component, the systematic component, and the link function. It also introduces the exponential family of distributions, which provides the mathematical foundation for why GLMs work across so many different data types. This chapter is essential reading if you want to understand the "why" behind the techniques, not just the "how."

Chapters 2 and 3 go deeper into two of the three GLM components. Chapter 2 covers the random component, walking through the key distributions you'll use in practice: normal, binomial, Poisson, and gamma. Chapter 3 covers link functions in detail, explaining the identity, logit, and log links, and giving you a framework for choosing the right link function for your problem. Together, these two chapters give you the vocabulary and intuition to design appropriate models for a wide range of data types.

Chapter 4 covers model fitting through maximum likelihood estimation. This is where you'll learn how GLMs are actually computed, including the iterative algorithms that find the best-fitting parameters. Understanding this material helps you troubleshoot convergence problems and interpret software output more confidently. It also deepens your appreciation for why GLMs behave differently from linear regression in terms of computation and inference.

Chapters 5 and 6 are the practical heart of the book. Chapter 5 covers logistic regression in depth, including binary logistic regression, odds ratio interpretation, and multinomial logistic regression for outcomes with more than two categories. Chapter 6 covers Poisson regression for count data, including the important topic of overdispersion and how to handle it with negative binomial regression. Both chapters include detailed worked examples and real-world case studies.

Chapter 7 broadens the picture by covering GLM models beyond logistic and Poisson regression. You'll learn about gamma regression for positive continuous data, inverse Gaussian regression for extreme skewness, and quasi-likelihood methods for situations where no standard distribution fits well. This chapter also provides a practical guide to selecting the right GLM for your specific data and research question.

Chapters 8 and 9 shift focus from model building to model evaluation and communication. Chapter 8 covers diagnostics specific to GLMs, including deviance residuals, influence measures, and goodness-of-fit tests. Chapter 9 addresses the often-overlooked challenge of interpreting and communicating GLM results, including how to translate coefficients on the link function scale into meaningful, accessible statements for non-technical audiences.

Chapter 10 introduces advanced extensions of the GLM framework, including regularization techniques like ridge and lasso regression, generalized linear mixed models for clustered or repeated-measures data, and Bayesian approaches to GLM. These topics point toward the frontier of modern statistical modeling and give you a sense of where to go after mastering the core material.

This book is written for intermediate learners who already have a working knowledge of basic statistics and linear regression. You should be comfortable with concepts like regression coefficients, p-values, confidence intervals, and residuals. You don't need to be a mathematician, but you should be willing to engage with equations when they appear. Every mathematical expression in this book is accompanied by a plain-language explanation, so the goal is always understanding, not symbol manipulation for its own sake.

The most effective way to use this book is to read actively, not passively. When you encounter a worked example, try to work through it yourself before reading the solution. When you see a formula, ask yourself what it's saying in words before moving on. When a new concept is introduced, think about how it connects to something you already know. This kind of active engagement dramatically improves retention and builds the kind of flexible understanding that lets you apply concepts to new problems.

Code examples throughout the book are written in a way that emphasizes the statistical concepts rather than the syntax of any particular language. The logic of fitting a GLM, extracting coefficients, and checking diagnostics is essentially the same whether you're working in R or Python. Where specific syntax is shown, it is clearly labeled and explained. If you're working in a different environment, the conceptual steps will translate directly even if the exact commands differ.

Here are some practical recommendations for getting the most out of each chapter:

1. Read the chapter introduction first to understand what problem the chapter is solving and why it matters before diving into the details.
2. Work through the examples with your own software environment open, reproducing the results and experimenting with variations.
3. Apply each technique to your own data as soon as possible after learning it. Real data always raises questions that textbook examples don't, and those questions deepen your understanding.
4. Review the key terms at the end of each chapter and make sure you can explain each one in plain language without looking at the definition.
5. Don't skip the diagnostics chapters. Many analysts learn to fit models but neglect to check whether those models are actually appropriate. Chapters 8 and 9 are where good modeling practice is built.

A note on the mathematical level: this book aims to be rigorous without being unnecessarily abstract. Derivations are included when they build genuine intuition, and skipped or summarized when they would distract from the main point. If you find a particular mathematical section challenging, it's usually safe to focus on the result and the interpretation rather than every step of the derivation. You can always return to the details later once you've seen how the result is used in practice.

The glossary at the back of the book is a useful reference when you encounter a term you're not sure about. The further reading section points you toward the primary literature and advanced textbooks for each major topic. These resources are particularly valuable if you want to go deeper into any specific area, whether that's the theory of exponential families, the computational details of maximum likelihood estimation, or the latest developments in Bayesian GLMs.

One of the most common mistakes learners make with GLMs is treating them as a collection of separate techniques rather than as a unified framework. Logistic regression, Poisson regression, and gamma regression are not three different tools; they are three instances of the same underlying structure with different distributional and link function choices. Keeping this unity in mind as you work through the book will help you build a coherent mental model that makes it much easier to learn new GLM variants and to troubleshoot problems when they arise.

Finally, approach this material with patience and curiosity. GLMs are genuinely powerful, and mastering them opens up a much wider range of analytical problems than linear regression alone can handle. The investment you make in understanding the foundations will pay off every time you encounter a new dataset with a non-normal response variable and know exactly how to approach it. That moment of recognition, when you look at a problem and immediately see which GLM is the right tool, is what this book is designed to help you reach.