PPDL: A New Deep Learning Language for Probabilistic Programs

Probabilistic programming has long been a powerful tool for building models that reason under uncertainty, but it has struggled to keep pace with the deep learning revolution. Traditional probabilistic programming languages (PPLs) like Stan or Pyro are designed around structured, often manually specified models, and integrating neural networks — which excel at learning representations from raw data — has been awkward and computationally expensive.

A new paper on arXiv introduces PPDL (Probabilistic Programming with Deep Learning), a language designed to bridge this gap. PPDL extends the probabilistic programming paradigm to include deep neural networks as first-class citizens, allowing developers to define models where a neural network can serve as a learned prior, a likelihood function, or a variational approximation.

This is a significant step forward for practitioners who want to combine the flexibility of deep learning with the principled uncertainty quantification of Bayesian methods. In traditional PPLs, mixing a neural network into a generative model often requires manually writing custom inference code or resorting to black-box methods that are slow or approximate. PPDL aims to make this natural and automatic.

Why PPDL matters

The core innovation is in how PPDL handles inference. The paper demonstrates that models defined in PPDL can be trained using a combination of variational inference and stochastic gradient descent, leveraging the same autodifferentiation machinery that powers modern deep learning frameworks. This means that large-scale Bayesian deep learning — for example, a neural network with uncertain weights — becomes as straightforward as training a standard deep network.

The authors also show that PPDL provides a unified syntax and semantics for expressing a wide range of models, from Bayesian neural networks to hierarchical models with deep features. This could lower the barrier for researchers and engineers who want to adopt probabilistic methods but find existing PPLs too limited or too difficult to integrate with their deep learning stack.

A practical look

While the paper is primarily a language design and proof-of-concept, it includes examples that illustrate how to write PPDL programs. For instance, a Bayesian neural network for regression can be specified in just a few lines:

@ppdl.model
def bayesian_nn(x):
    w = ppdl.sample(ppdl.Normal(0, 1), shape=(10, 1))
    b = ppdl.sample(ppdl.Normal(0, 1))
    return t.nn.linear(x, w, b)

This is a simplified illustrative snippet, but it conveys the idea: you define random variables and use them inside a neural network forward pass, and PPDL handles the inference automatically.

The authors released the implementation, which is built on top of PyTorch, making it accessible to a wide audience of developers already familiar with that ecosystem.

What's next

PPDL is still an early-stage research language, but it adds momentum to a growing trend: making probabilistic programming more expressive and scalable by embracing deep learning rather than treating it as an afterthought. As AI tools continue to evolve, the ability to quantify uncertainty in neural networks could become a standard feature rather than a separate discipline.

For developers, this means that the next generation of AI frameworks may not just train models — they'll let you ask those models what they don't know.