An Intro Course to Scientific Machine Learning

Name: Scientific Machine Learning
Author: Cheng (Jesse) Jing

Six lectures covering the full spectrum of Scientific Machine Learning — from physics-constrained training to structure-preserving architectures, from operator learning to generative uncertainty quantification, and from world models to production-grade engineering.

6 Lectures·6 Code Demos·BibTeX Citations·Slide Decks

Physics-Informed Neural Networks

Embedding physical laws directly into the loss function

Classical numerical solvers discretize space and time on a mesh. PINNs take a fundamentally different approach: they parameterize the solution as a neural network and enforce the governing PDE, initial conditions, and boundary conditions as soft constraints in the training loss. This lecture derives the PINN loss from first principles, explains why automatic differentiation enables exact PDE residuals without a mesh, and demonstrates the method on the viscous Burgers equation.

PDEAutogradCollocationInverse Problems

Benchmark

Burgers equation (ν = 0.01/π), 5,081 parameters

Result

~5% relative L² error, zero labeled solution data

View lecture →

Neural Operators

Learning mappings between infinite-dimensional function spaces

A PINN solves one PDE instance. A neural operator learns the entire solution operator — the mapping from initial/boundary conditions to solutions — across a family of PDEs. This lecture introduces the universal approximation theorem for operators (Chen & Chen, 1995), derives the DeepONet Branch-Trunk architecture, and develops the Fourier Neural Operator (FNO) as a spectral integral kernel method that is resolution-invariant by construction.

Operator LearningDeepONetFNOSpectral Methods

Benchmark

Antiderivative operator, 2,000 training samples

Result

~2% relative L² error, zero-shot super-resolution

Coming soon

World Models & JEPA

Latent predictive representations for physical systems

This lecture takes a sharp turn toward the broader question of how intelligent systems model the world. LeCun's Joint Embedding Predictive Architecture (JEPA) argues that prediction should occur in latent space, not pixel space — a principle that resonates deeply with SciML, where we seek compact representations of physical dynamics. The lecture covers the four prediction architectures, the collapse prevention mechanism (EMA target encoder, stop-gradient), and the I-JEPA and V-JEPA instantiations.

World ModelsJEPASelf-Supervised LearningLatent Space

Benchmark

Van der Pol oscillator, latent representation learning

Result

Smooth parameter manifold in PCA embedding space

Coming soon

Generative Modeling & UQ

Probabilistic forecasting for chaotic physical systems

The future of a chaotic system is not a single path — it is a distribution over paths. This lecture addresses the stochasticity of the real world, which is crucial for world models where deterministic prediction is insufficient. We derive score-based diffusion models from the VP-SDE framework, show how denoising score matching provides a tractable training objective, and apply the method to probabilistic forecasting of the Lorenz-96 atmospheric model. The lecture connects to DeepMind's GenCast, which outperforms ECMWF ENS on 97.2% of forecasting targets.

Diffusion ModelsScore MatchingUQLorenz-96GenCast

Benchmark

Lorenz-96 (40-dimensional), probabilistic forecasting

Result

Lower CRPS and better spread-skill than MC Dropout baseline

Coming soon

Neural ODEs & Geometric Mechanics

Structure-preserving deep learning for Hamiltonian systems

Physical systems obey conservation laws — energy, momentum, symplecticity. A standard neural network trained on trajectory data will violate these laws during long-term rollout, leading to unphysical energy drift. This lecture covers the adjoint method for Neural ODEs, Hamiltonian mechanics in phase space, and three structure-preserving architectures: Hamiltonian Neural Networks (HNN), Lagrangian Neural Networks (LNN), and SymODEN. The key insight is that embedding the scalar Hamiltonian as the network output — rather than the vector field — guarantees symplectic dynamics by construction.

Neural ODEHamiltonianSymODENConservation LawsSymplectic

Benchmark

Nonlinear pendulum, 800-epoch training

Result

12× less energy drift vs. baseline; SymODEN recovers true V(q) and M(q)

Coming soon

Engineering & Performance of SciML

From theory to production: profiling, precision, and kernel optimization

A theoretically elegant model is useless if it cannot run at scale. This capstone lecture bridges the gap between research prototypes and production deployments. We cover TFLOP counting and the Roofline model for each SciML method, the precision pitfalls that are easy to overlook (PINNs require float64 due to the autograd signal chain), the ML infrastructure stack (PyTorch vs. JAX/XLA, distributed training strategies), and how to write specialized Triton kernels to fuse the FFT-GEMM-iFFT pipeline in FNO for a 2.5× speedup.

TFLOPsRoofline ModelTritonJAXfloat64Distributed Training

Benchmark

A100 GPU profiling of all five SciML methods

Result

PINNs: memory-bound; FNO: bandwidth-limited; NODE: compute-bound

Coming soon

Course Progression

#	Topic	Key Method	Core Idea	Benchmark
L1	Physics-Informed NNs	PINNs	Physics in the loss	Burgers eq.
L2	Neural Operators	FNO / DeepONet	Resolution-invariant mappings	Antiderivative op.
L3	World Models	JEPA	Latent predictive representations	Van der Pol
L4	Generative UQ	VP-SDE Diffusion	Stochastic futures	Lorenz-96
L5	Geometric Mechanics	SymODEN / HNN	Conservation by construction	Pendulum
L6	Engineering	Triton / JAX	Making it fast on silicon	A100 Roofline

Acknowledgments

ASU Research Computing — Sol Supercomputer

The authors acknowledge Research Computing at Arizona State University for providing HPC resources that have contributed to the research results reported in this course.

When acknowledging work done on the Sol supercomputer, please cite the peer-reviewed paper ( doi:10.1145/3569951.3597573): Jennewein, Douglas M. et al. “The Sol Supercomputer at Arizona State University.” In Practice and Experience in Advanced Research Computing (pp. 296–301). Association for Computing Machinery, 2023.

BibTeX — HPC:ASU23

@inproceedings{HPC:ASU23,
  title     = "{The Sol Supercomputer at Arizona State University}",
  doi       = {10.1145/3569951.3597573},
  year      = {2023},
  author    = {
    Jennewein, Douglas M. and
    Lee, Johnathan and
    Kurtz, Chris and
    Dizon, Will and
    Shaeffer, Ian and
    Chapman, Alan and
    Chiquete, Alejandro and
    Burks, Josh and
    Carlson, Amber and
    Mason, Natalie and
    Kobwala, Arhat and
    Jagadeesan, Thirugnanam and
    Barghav, Praful and
    Battelle, Torey and
    Belshe, Rebecca and
    McCaffrey, Debra and
    Brazil, Marisa and
    Inumella, Chaitanya and
    Kuznia, Kirby and
    Buzinski, Jade and
    Dudley, Sean and
    Shah, Dhruvil and
    Speyer, Gil and
    Yalim, Jason
  },
  isbn      = {9781450399852},
  month     = {Jul},
  pages     = {296--301},
  booktitle = {Practice and Experience in Advanced Research Computing},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  numpages  = {6},
  location  = {Portland, OR, USA},
  series    = {PEARC '23},
}

Cite As

If you use this course material in your research or teaching, please cite:

Cite this course

@misc{jing2025sciml,
  title        = {An Intro Course to Scientific Machine Learning},
  author       = {Jing, Cheng},
  year         = {2025},
  note         = {An Intro Course to Scientific Machine Learning, Arizona State University},
  url          = {https://jessecj.me/course},
  howpublished = {\url{https://jessecj.me/course}}
}