Authors:

Gabriel Silva

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

The Kalman filter is one of the most influential algorithms for recursive estimation in noisy systems, with applications spanning aerospace navigation to financial forecasting. At its core, it is a state-space framework grounded in Bayesian inference that produces both point estimates of latent states and measures of uncertainty. While financial markets present an appealing domain for Kalman filtering, practical applications remain limited because the standard filter assumes Gaussian disturbances and fixed noise covariances; assumptions frequently violated by regime shifts, volatility clustering, and heavy-tailed return distributions. A central limitation is the specification of the process-noise covariance matrix Q,, which governs how uncertainty enters the state dynamics. Typically treated as fixed or estimated via maximum likelihood over rolling windows, Q, adapts slowly to structural market changes, leading to persistent forecast errors during regime transitions. This paper proposes a lightweight scaling mechanism that dynamically adjusts Q, using a dynamic factor g, derived from the normalized innovation squared (NIS). Unlike Mehra's classical covariance moment-matching approach, which relies on iterative optimization, our method directly scales the covariance matrix in the correct direction using recent innovation statistics to respond rapidly to regime changes. We evaluate this approach within a two-factor state-space framework inspired by the Schwartz—Smith commodity pricing model, comparing four specifications: a baseline Kalman filter, the proposed g-scaling model, Mehra's method, and a combined specification. Results demonstrate that the proposed g-scaling mechanism yields an 18% increase in both RMSE and MAE efficiency relative to Mehra's specification, with approximately half of the gain arising independently of Mehra's structure, confirming that the two methods capture complementary sources of misspecification. Distributional testing via the Kolmogorov—Smimov test shows that the baseline Kalman filter fails chi-squared alignment, while all augmented models pass. The combined model reduces tail misspecification by 66% relative to the standard filter and improves Mehra's tail calibration by 20%, achieving the closest empirical coverage to the theoretical 95% threshold.

Source:

Loyola University Chicago

Topics:

No topics listed

Co-authors:

Gabriel Silva