Partial Differential Equations Unleashed

Partial differential equations are the language in which the physical world writes its rules. The way heat spreads through a material, the way sound waves travel through air, the way fluids flow around an obstacle, all of these phenomena are described by PDEs. When machine learning models are trained on data generated by physical processes, they are implicitly trying to learn the solutions to these equations. Making that connection explicit is what physics-informed machine learning is all about.

Consider diffusion models, which have become the dominant architecture for image generation. Systems like Stable Diffusion and DALL-E 2 work by gradually adding noise to an image and then learning to reverse that process. The forward noising process is mathematically equivalent to solving the heat equation forward in time. The reverse denoising process corresponds to solving a reverse-time stochastic differential equation.^[3] Engineers who understand this connection can reason about why certain noise schedules work better than others, how to design more efficient samplers, and where the model is likely to fail. Engineers who do not understand it are left guessing.

The same pattern appears across many areas of modern AI. Here are some concrete examples of where PDE knowledge directly improves your work as an ML practitioner:

• Generative models: Score-based diffusion models are grounded in the Fokker-Planck equation and reverse-time SDEs. Understanding these connections helps you design better noise schedules and sampling algorithms.
• Neural operators: Fourier Neural Operators and DeepONets learn mappings between function spaces, a concept that only makes sense if you understand what a PDE solution operator is.
• Scientific surrogate models: In aerospace, climate modeling, and drug discovery, neural networks are used to replace expensive PDE solvers. Building reliable surrogates requires understanding what the solver is actually computing.
• Physics-constrained optimization: Training neural networks to satisfy physical constraints, such as conservation of mass or energy, requires encoding those constraints as differential equations in the loss function.
• Inverse problems: Inferring material properties from sensor data, reconstructing fluid velocity fields from pressure measurements, or identifying parameters in biological models all require solving PDEs in reverse.

Beyond these specific applications, PDE literacy gives you a different way of thinking about neural networks themselves. A deep residual network, for example, can be interpreted as a discretization of an ordinary differential equation.^[4] This perspective, developed in the Neural ODE literature, opened up entirely new architectures and training strategies. The mathematical lens of differential equations reveals structure that the purely statistical lens misses.

There is also a practical argument about data efficiency. Physics-informed models can achieve accurate predictions with far less training data than purely data-driven models, because they use physical laws to constrain the solution space. In domains where data is expensive or scarce, such as medical imaging, materials science, or climate simulation, this advantage is decisive. A model that knows the heat equation does not need to learn it from scratch from millions of examples.

The research community has recognized this. Venues like NeurIPS, ICML, and ICLR now regularly feature papers on physics-informed learning, neural operators, and PDE-based generative models. The Journal of Computational Physics and the SIAM Journal on Scientific Computing are publishing more machine learning papers than ever before. If you want to contribute to this space, you need to speak both languages fluently. This book teaches you how.

It is worth being honest about the challenges too. PDEs are genuinely difficult. The theory involves functional analysis, measure theory, and numerical analysis, all of which take time to absorb. The implementations require careful attention to boundary conditions, discretization errors, and numerical stability. But the difficulty is not a reason to avoid the subject. It is a reason to approach it systematically, with good examples and clear explanations, which is exactly what this book aims to provide.

This book is organized as a progressive journey. Each chapter builds on the previous one, and the mathematical complexity increases gradually. The early chapters establish the vocabulary and intuition you need to make sense of the later material. By the time you reach operator learning and inverse problems, you will have enough background to engage with the ideas at a research level rather than just a tutorial level.

Chapter 1: PDE Fundamentals for Machine Learning starts from the ground up. It introduces what PDEs are, how they differ from ordinary differential equations, and why the classification into elliptic, parabolic, and hyperbolic types matters for both numerical methods and machine learning applications. It also covers boundary and initial conditions in practical terms, because getting these right is one of the most common sources of error in physics-informed neural network implementations.

Chapter 2: Physics-Informed Neural Networks (PINNs) is where theory meets implementation. You will learn how to encode a PDE as a loss function, design network architectures that respect physical constraints, and implement a working PINN in Python using PyTorch. The chapter includes case studies on the Burgers equation and the Navier-Stokes equations, with full code examples and discussion of common failure modes.

Chapter 3: Classical Numerical PDE Methods covers finite difference methods, finite element analysis, and spectral methods. This chapter serves two purposes. First, it gives you the classical tools that practitioners have used for decades, which remain important for validation and benchmarking. Second, it provides the conceptual foundation for understanding why neural network solvers behave the way they do. Understanding discretization error, for example, helps you interpret the convergence behavior of PINNs.

The middle section of the book moves into more advanced territory:

• Chapter 4 covers operator learning frameworks, including Fourier Neural Operators and graph neural network approaches. This is one of the most active research areas in scientific machine learning, and the chapter includes a full implementation of an FNO for the Navier-Stokes equations.
• Chapter 5 connects PDE theory to diffusion models and generative AI. If you have ever wondered why the math behind Stable Diffusion looks like it belongs in a physics textbook, this chapter answers that question in detail.
• Chapter 6 addresses inverse problems, covering how to formulate them, how to solve them with neural networks, and how to quantify uncertainty in the solutions. This is critical for any application where you are trying to infer physical parameters from data.

The final section focuses on advanced techniques and research skills. Chapter 7 covers automatic differentiation and variational calculus, showing how these two perspectives on optimization are deeply connected. Chapter 8 addresses scalability through domain decomposition and convergence analysis. Chapter 9 presents extended case studies in wave modeling, fluid dynamics, and the Burgers equation. Chapter 10 is dedicated to research publication, covering how to design experiments, write papers, and benchmark your methods against classical solvers.

The appendix provides a concise review of the mathematical prerequisites, including linear algebra, multivariable calculus, and the Python libraries you will use throughout the book. If you find yourself uncertain about a concept in the main chapters, the appendix is a good place to check your foundations before continuing.

The overall arc of the book moves from understanding to implementation to research contribution. By the end, you should be able to read a paper on physics-informed machine learning, understand the mathematical claims being made, reproduce the key experiments, and identify directions for original follow-up work. That is the level of fluency this book is designed to build.

Getting the most out of this book requires active engagement, not passive reading. The material is dense enough that simply reading through the chapters without working through the examples will leave you with a superficial understanding that fades quickly. The recommendations below are based on how people actually learn technical mathematics and how they successfully transition from understanding concepts to implementing them in code.

Work through the code examples yourself. Every implementation in this book is designed to be run, modified, and broken. When you encounter a Python example, do not just read it. Type it out, run it, change a parameter, and observe what happens. When a PINN fails to converge, try to diagnose why before reading the explanation. This kind of active debugging builds intuition that no amount of reading can replace.

The recommended software environment for this book is as follows:

• Python 3.10 or later for all implementations
• PyTorch 2.0 or later for neural network implementations and automatic differentiation
• NumPy and SciPy for numerical methods and classical PDE solvers
• Matplotlib for visualization of PDE solutions and training dynamics
• FEniCS or FEniCSx for finite element method examples (optional but recommended for Chapter 3)
• JAX for some advanced automatic differentiation examples in Chapter 7

All code examples in the book are self-contained and include the necessary imports. You do not need to install every library before starting. Install what you need as you reach each chapter. A complete setup guide is provided in the appendix under the Python Tools section.

Do not skip the mathematical derivations. It is tempting, especially if your background is more engineering than mathematics, to skip over the derivations and jump straight to the code. Resist this temptation. The derivations are where the intuition lives. When you understand why the PINN loss function takes the form it does, you are much better equipped to modify it for a new problem. When you understand the derivation of the Fourier Neural Operator, you can reason about when it will and will not work. The code is the destination, but the math is the map.

Here is a suggested approach for working through each chapter:

1. Read the chapter introduction and the first subchapter without stopping to implement anything. Get the big picture first.
2. Go back to the beginning and work through each section with a notebook open. Implement the examples as you read them.
3. At the end of each chapter, close the book and try to reproduce the main implementation from memory. Use the book only to check your work, not to guide it.
4. Identify one concept from the chapter that you do not fully understand and spend extra time on it before moving on. Gaps in understanding compound as the book progresses.

If you are using this book as part of an MSc program or research group, the chapter structure maps naturally onto a semester-long course. Chapters 1 through 3 cover the foundational material that should be completed in the first four to five weeks. Chapters 4 through 6 form the core of the course and are suitable for weeks six through ten. Chapters 7 through 10 are advanced material appropriate for the final weeks, with Chapter 10 serving as a guide for thesis and paper writing.

For self-study, a realistic pace is one chapter per week if you are working through the material seriously. Some chapters, particularly Chapter 2 on PINNs and Chapter 4 on operator learning, may take longer because the implementations are more involved. Do not rush. Depth of understanding matters more than speed of coverage.

A useful benchmark for each chapter: if you can explain the main idea to a colleague who has not read the chapter, implement the core example without looking at the book, and identify one open research question raised by the material, you have understood it well enough to move on.

The research publication chapter at the end of the book is worth reading early, even before you have mastered all the technical content. Understanding what a strong research paper looks like will help you read the literature more critically and will shape how you think about the experiments you run as you work through the earlier chapters. Research skills and technical skills develop in parallel, not in sequence.

Finally, engage with the broader community. The physics-informed machine learning field moves quickly, and the best way to stay current is to follow the key research groups, read preprints on arXiv, and participate in communities like the CRUNCH group at Brown University or the open-source projects around DeepXDE and neuraloperator.^[5] The concepts in this book are a foundation, not a ceiling. The field is young enough that a motivated MSc student or engineer can make genuine contributions, and the tools you build here are the starting point for that work.