On representations of differential equations for state space statistical forecasting

Naumer, Helmuth J

Permalink

https://hdl.handle.net/2142/124378 Copy

Description

Title

On representations of differential equations for state space statistical forecasting

Author(s)

Naumer, Helmuth J

Issue Date

2024-04-24

Director of Research (if dissertation) or Advisor (if thesis)

Kamalabadi, Farzad

Doctoral Committee Chair(s)

Kamalabadi, Farzad

Committee Member(s)

• Raginsky, Maxim
• Srikant, Rayadurgam
• Banerjee, Arindam

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Statistical inference
• Estimation theory
• Dynamic systems
• Differential equations

Abstract

The key to understanding the statistical limitations of a problem is often in finding the right representation of the space. Under sub-optimal representations, guarantees on the performance of an estimator can be too loose or even seemingly contradictory due to the inherently local nature of frequentist statistics. This work investigates several interconnected problems in forecasting dynamical systems: seeking to use different parameterizations of the system to provide strong statistical guarantees on the underlying families of probability distributions and to complete a detailed analysis of specific estimators applied to the problem. We begin this work with a focus on linear space-invariant spatiotemporal processes. When observations are made at discrete time steps, we identify statistical guarantees around a grid-based approximation of an inherently continuous system defined by a partial differential equation (PDE). In particular, by applying a Fourier transform to the grid approximation, we provide guarantees on the performance of the Kalman filter under mismatched model assumptions, as well as the least-squares estimator of the underlying state-transition operator. Many generalizations of the core analysis are outlined throughout the chapter, including an extension to piecewise constant dynamics. The second focus of this work is on deterministic nonlinear ordinary differential equations (ODEs) under noisy observations. Assuming Lipschitz continuity of the differential equation, we demonstrate that the set of trajectories of a system form a manifold where each trajectory corresponds to a single point. This realization motivates a collection of different perspectives on the forecasting problem based on properties of different time horizons. By considering the family of observations to be parameterized by the state of the system, we prove the often contradictory behavior of the infinitely large Cramér–Rao lower bound in the neighborhood of repelling points, as well as the dimensionality reduction inherent in convergence to low-dimensional attractors. We furthermore demonstrate two concrete algorithms enabled by these observations: multiple shrinkage estimators to stabilize error in the neighborhood of the unstable equilibria and a sequential optimal experimental design policy for infinite-horizon forecasting. Finally, we conclude with an investigation into the prediction of the formation of shocks in PDEs based on noisy measurements of boundary conditions. As shocks, or discontinuities in the solutions of PDEs, are particularly poorly behaved mathematical structures, we introduce a sequence of relaxations of the prediction problem. By instead looking at the variation within epsilon-balls, we propose a Monte Carlo method to quantify the probability of observing a discontinuity. Under a conjecture on the behavior of shocks, the proposed algorithm converges to a function analogous to an arrival rate in a point process. The behavior is verified in simulation using Burgers' Equation. This dissertation creates a solid foundation for the future study of statistical forecasting with differential equation governed systems. This work provides an analysis of various representations of the observation process in state-space models, as well as a methodology to construct new estimation techniques and bounds. The proposed methods emphasize the geometry of the space of solutions and thus are equally applicable to Bayesian and Frequentist methods in statistics.

Graduation Semester

2024-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2024 Helmuth Naumer

Owning Collections