University of Illinois Urbana-Champaign

On representations of differential equations for state space statistical forecasting

Naumer, Helmuth J

Content Files
NAUMER-DISSERTATION-2024.pdf

Permalink

https://hdl.handle.net/2142/124378

Description

Title
On representations of differential equations for state space statistical forecasting
Author(s)
Naumer, Helmuth J
Issue Date
2024-04-24
Director of Research (if dissertation) or Advisor (if thesis)
Kamalabadi, Farzad
Doctoral Committee Chair(s)
Kamalabadi, Farzad
Committee Member(s)
  • Raginsky, Maxim
  • Srikant, Rayadurgam
  • Banerjee, Arindam
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Statistical inference
  • Estimation theory
  • Dynamic systems
  • Differential equations
Abstract
The key to understanding the statistical limitations of a problem is often in finding the right representation of the space. Under sub-optimal representations, guarantees on the performance of an estimator can be too loose or even seemingly contradictory due to the inherently local nature of frequentist statistics. This work investigates several interconnected problems in forecasting dynamical systems: seeking to use different parameterizations of the system to provide strong statistical guarantees on the underlying families of probability distributions and to complete a detailed analysis of specific estimators applied to the problem. We begin this work with a focus on linear space-invariant spatiotemporal processes. When observations are made at discrete time steps, we identify statistical guarantees around a grid-based approximation of an inherently continuous system defined by a partial differential equation (PDE). In particular, by applying a Fourier transform to the grid approximation, we provide guarantees on the performance of the Kalman filter under mismatched model assumptions, as well as the least-squares estimator of the underlying state-transition operator. Many generalizations of the core analysis are outlined throughout the chapter, including an extension to piecewise constant dynamics. The second focus of this work is on deterministic nonlinear ordinary differential equations (ODEs) under noisy observations. Assuming Lipschitz continuity of the differential equation, we demonstrate that the set of trajectories of a system form a manifold where each trajectory corresponds to a single point. This realization motivates a collection of different perspectives on the forecasting problem based on properties of different time horizons. By considering the family of observations to be parameterized by the state of the system, we prove the often contradictory behavior of the infinitely large Cramér–Rao lower bound in the neighborhood of repelling points, as well as the dimensionality reduction inherent in convergence to low-dimensional attractors. We furthermore demonstrate two concrete algorithms enabled by these observations: multiple shrinkage estimators to stabilize error in the neighborhood of the unstable equilibria and a sequential optimal experimental design policy for infinite-horizon forecasting. Finally, we conclude with an investigation into the prediction of the formation of shocks in PDEs based on noisy measurements of boundary conditions. As shocks, or discontinuities in the solutions of PDEs, are particularly poorly behaved mathematical structures, we introduce a sequence of relaxations of the prediction problem. By instead looking at the variation within epsilon-balls, we propose a Monte Carlo method to quantify the probability of observing a discontinuity. Under a conjecture on the behavior of shocks, the proposed algorithm converges to a function analogous to an arrival rate in a point process. The behavior is verified in simulation using Burgers' Equation. This dissertation creates a solid foundation for the future study of statistical forecasting with differential equation governed systems. This work provides an analysis of various representations of the observation process in state-space models, as well as a methodology to construct new estimation techniques and bounds. The proposed methods emphasize the geometry of the space of solutions and thus are equally applicable to Bayesian and Frequentist methods in statistics.
Graduation Semester
2024-05
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/124378
Copyright and License Information
Copyright 2024 Helmuth Naumer

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois