Andrew C Harvey, Siem Jan Koopman and J Penzer

Published March 1997

---

Many series are subject to data irregularities such as missing values, outliers, structural breaks and irregular spacing. Data can also be messy, and hence difficult to handle by standard procedures, when they are intrinsically non-Gaussian or contain complicated periodic patterns because they are observed on an hourly or weekly basis. This paper presents a unified approach to the analysis of messy data. The technical treatment is based on state space methods. These methods can be applied to any linear model including those from the autoregressive integrated moving average class. However, the ease of interpretation of structural time series models, together with the associated information produced by the Kalman filter and smoother, makes them a more natural vehicle for dealing with messy data. Structural time series models can also be formulated in continuous time thereby allowing for a general treatment of irregularly spaced observations. The periodic patterns associated with hourly or weekly data can be dealt with effectively using time-varying splines while the statistical analysis of non-Gaussian models is now feasible because of recent developments in simulation techniques.