Statistical monitoring of spatio-temporal processes
A04 develops methodology for online monitoring in spatio-temporal models and timely detection of structural alterations. It develops robust procedures, considers tail behavior in form of, e.g., extreme quantiles and investigates high-dimensional scenarios with many variables. The long-term goal is the development of robust monitoring tools for various classes of nonlinear spatio-temporal processes and for high-dimensional mixed-frequency data.
Project Leaders
Prof. Dr. Roland Fried
Department of Statistics - Chair of Mathematical Statistics and Applications in Science
TU Dortmund University
Prof. Dr. Vasyl Golosnoy
Faculty of Management and Economics - Chair of Statistics and Econometrics
Ruhr University Bochum
Summary
We develop novel methods for online monitoring of the validity and the stability of complex spatio-temporal models (STMs) with a particular focus on applications in energy and transport. Online monitoring is conducted using statistical decision rules (so-called control charts). These charts evaluate at every new time point whether the model is still adequate or not by comparing a control statistic to critical limits, which need to be chosen such that certain performance criteria are met concerning both the frequency of false alarms in the former and the detection delay in the latter case. By designing monitoring tools for STMs we confront the challenges of finding appropriate sparse statistical representations for these complex modeling scenarios and deriving their statistical properties. We consider STMs in various settings where we (a) incorporate autoregressive features for both space (e.g., the network structure) and time, (b) develop procedures that are robust with respect to restrictive distributional assumptions, (c) consider quantiles and the tail behaviour of spatial data, and (d) treat multivariate and high-dimensional settings and apply appropriate dimension reduction techniques for making STMs empirically tractable. Our methods will enable a timely detection of changes in STM parameters and more generally of model deficiencies. This is of essential importance for using such models in applications, in particular for adequate prediction.
Abbas, S. and R. Fried (2020). Robust control charts for the mean of a locally linear time series. Journal of Statistical Computation and Simulation 90, 2741–2765. doi: 10.1080/00949655.2020.1788562.
Adrian, T. and M. K. Brunnermeier (2016). CoVaR. American Economic Review 106, 1705–1741. doi: 10.1257/aer.20120555.
Aeberhard, W. H., E. Cantoni, G. Marra, and R. Radice (2021). Robust fitting for generalized additive models for location, scale and shape. Statistics and Computing 31. doi: 10.1007/s11222-020-09979-x.
Alfaro, J.-L. and J.-F. Ortega (2012). Robust Hotelling’s T 2 control charts under non-normality: the case of t -Student distribution. Journal of Statistical Computation and Simulation 82, 1437–1447. doi: 10.1080/00949655.2011.580746.
Alwan, L. C. and H. V. Roberts (1988). Time-series modeling for statistical process control. Journal of Business & Economic Statistics 6, 87–95. doi: 10.1080/07350015.1988.10509640.
Andreou, E., P. Gagliardini, E. Ghysels, and M. Rubin (2020). Mixed-frequency macro-finance factor models: Theory and applications. Journal of Financial Econometrics 18, 585–628. doi: 10.1093/jjfinec/nbaa015.
Antolín-Díaz, J. and J. F. Rubio-Ramírez (2018). Narrative Sign Restrictions for SVARs. American Economic Review 108, 2802–2829. doi: 10.1257/aer.20161852.
Atienza, O. O., L. C. Tang, and B. W. Ang (1998). A SPC procedure for detecting level shifts of autocorrelated processes. Journal of Quality Technology 30, 340–351. doi: 10.1080/00224065.1998.11979870.
Bai, P., A. Safikhani, and G. Michailidis (2023). Multiple change point detection in reduced rank high dimensional vector autoregressive models. Journal of the American Statistical Association, 1–17. doi: 10.1080/01621459.2022.2079514.
Bersimis, S., A. Sgora, and S. Psarakis (2018). The application of multivariate statistical process monitoring in non-industrial processes. Quality Technology & Quantitative Management 15, 526–549. doi: 10.1080/16843703.2016.1226711.
Binder, N., H. Dette, J. Franz, D. Zöller, et al. (2022a). Data Mining in Urology: Understanding Real-world Treatment Pathways for Lower Urinary Tract Systems via Exploration of Big Data. European urology focus. doi: 10.1016/j.euf.2022.03.019.
Binder, N., T. A. Gerds, and P. K. Andersen (2014). Pseudo-observations for competing risks with covariate dependent censoring. Lifetime data analysis 20, 303–315. doi: 10.1007/s10985-013-9247-7.
Binder, N., A.-S. Herrnböck, and M. Schumacher (2017). Estimating hazard ratios in cohort data with missing disease information due to death. Biometrical Journal 59, 251–269. doi: 10.1002/bimj.201500167.
Binder, N., K. Möllenhoff, A. Sigle, and H. Dette (2022b). Similarity of competing risks models with constant intensities in an application to clinical healthcare pathways involving prostate cancer surgery. Statistics in Medicine 41, 3804–3819. doi: 10.1002/sim.9481.
Bock, C., F.-X. Aubet, J. Gasthaus, A. Kan, et al. (2022). Online time series anomaly detection with state space Gaussian processes.
Bodnar, O. and W. Schmid (2007). Surveillance of the mean behavior of multivariate time series. Statistica Neerlandica 61, 383–406. doi: 10.1111/j.1467-9574.2007.00365.x.
Bodnar, O. and W. Schmid (2011). CUSUM charts for monitoring the mean of a multivariate Gaussian process. Journal of Statistical Planning and Inference 141, 2055–2070. doi: 10.1016/j.jspi.2010.12.020.
Borowski, M., D. Busse, and R. Fried (2015). Robust online-surveillance of trend-coherence in multivariate data streams: the similar trend monitoring (STM) procedure. Statistics and Computing 25, 913–928. doi: 10.1007/s11222-014-9462-4.
Bretz, F., K. Möllenhoff, H. Dette, W. Liu, et al. (2018). Assessing the similarity of dose response and target doses in two non-overlapping subgroups. Statistics in Medicine 37, 722–738. doi: 10.1002/sim.7546.
Bücher, A., J. Lilienthal, P. Kinsvater, and R. Fried (2021). Penalized quasi-maximum likelihood estimation for extreme value models with application to flood frequency analysis. Extremes 24, 325–348. doi: 10.1007/s10687-020-00379-y.
Calvet, L. E., V. Czellar, and E. Ronchetti (2015). Robust Filtering. Journal of the American Statistical Association 110, 1591–1606. doi: 10.1080/01621459.2014.983520.
Chen, C. Y.-H., W. K. Härdle, and Y. Okhrin (2019). Tail event driven networks of SIFIs. Journal of Econometrics 208, 282–298. doi: 10.1016/j.jeconom.2018.09.016.
Chen, C. Y.-H., Y. Okhrin, and T. Wang (2022). Monitoring Network Changes in Social Media. Journal of Business & Economic Statistics, 1–16. doi: 10.1080/07350015.2021.2016425.
Chen, C. and L.-M. Liu (1993). Joint estimation of model parameters and outlier effects in time series. Journal of the American Statistical Association 88, 284–297. doi: 10.2307/2290724.
Chenouri, S., S. H. Steiner, and A. M. Variyath (2009). A Multivariate Robust Control Chart for Individual Observations. Journal of Quality Technology 41, 259–271. doi: 10.1080/00224065.2009.11917781.
Croux, C., S. Gelper, and K. Mahieu (2010). Robust exponential smoothing of multivariate time series. Computational Statistics & Data Analysis 54, 2999–3006. doi: 10.1016/j.csda.2009.05.003.
Dette, H., K. Möllenhoff, S. Volgushev, and F. Bretz (2018). Equivalence of regression curves. Journal of the American Statistical Association 113, 711–729. doi: 10.1080/01621459.2017.1281813.
Ding, X., J. Xie, and X. Yan (2021). Model averaging for multiple quantile regression with covariates missing at random. Journal of Statistical Computation and Simulation 91, 2249–2275. doi: 10.1080/00949655.2021.1890733.
Fan, J., A. Furger, and D. Xiu (2016). Incorporating Global Industrial Classification Standard into Portfolio Allocation: A Simple Factor-Based Large Covariance Matrix Estimator with High Frequency Data. SSRN Electronic Journal. doi: 10.2139/ssrn.2548613.
Flossdorf, J., R. Fried, and C. Jentsch (2023). Online monitoring of dynamic networks using flexible multivariate control charts. Social Network Analysis and Mining 13, 489–502. doi: 10.1007/s13278-023-01091-y.
Flossdorf, J. and C. Jentsch (2021). Change detection in dynamic networks using network characteristics. IEEE Transactions on Signal and Information Processing over Networks 7, 451–464. doi: 10.1109/TSIPN.2021.3094900.
Fokianos, K. and R. Fried (2010). Interventions in INGARCH processes. Journal of Time Series Analysis 31, 210–225. doi: 10.1111/j.1467-9892.2010.00657.x.
Galeano, P., D. Peña, and R. S. Tsay (2006). Outlier detection in multivariate time series by projection pursuit. Journal of the American Statistical Association 101, 654–669. doi: 10.1198/016214505000001131.
Garthoff, R. and P. Otto (2017). Control charts for multivariate spatial autoregressive models. AStA Advances in Statistical Analysis 101, 67–94. doi: 10.1007/s10182-016-0276-x.
Garthoff, R., I. Okhrin, and W. Schmid (2015). Control charts for multivariate nonlinear time series. REVSTAT Statistical Journal 13, 131–144. doi: 10.57805/revstat.v13i2.168.
Golosnoy, V. (2018). Sequential monitoring of portfolio betas. Statistical Papers 59, 663–684. doi: 10.1007/s00362-016-0783-6.
Golosnoy, V., B. Hildebrandt, S. Köhler, W. Schmid, et al. (2023). Control charts for measurement error models. AStA Advances in Statistical Analysis. doi: 10.1007/s10182-022-00462-8.
Golosnoy, V., S. Köhler, W. Schmid, and M. Seifert (2021). Testing for parameter changes in linear state space models. Applied Stochastic Models in Business and Industry 37, 1060–1079. doi: 10.1002/asmb.2636.
Golosnoy, V. and M. Seifert (2021). Monitoring mean changes in persistent multivariate time series. Statistics 55, 475–488. doi: 10.1080/02331888.2021.1949013.
Golosnoy, V., B. Gribisch, and R. Liesenfeld (2015). Intra-daily volatility spillovers in international stock markets. Journal of International Money and Finance 53, 95–114. doi: 10.1016/j.jimonfin.2015.01.002.
Golosnoy, V. and J. Roestel (2019). REAL-TIME MONITORING OF THE US INFLATION EXPECTATION PROCESS. Macroeconomic Dynamics 23, 2221–2249. doi: 10.1017/S1365100517000670.
Golosnoy, V., W. Schmid, M. I. Seifert, and T. Lazariv (2020). Statistical inferences for realized portfolio weights. Econometrics and Statistics 14, 49–62. doi: 10.1016/j.ecosta.2018.08.003.
Gösmann, J., C. Stoehr, J. Heiny, and H. Dette (2022). Sequential change point detection in high dimensional time series. Electronic Journal of Statistics 16. doi: 10.1214/22-EJS2027.
Guljanov, G., W. Mutschler, and M. Trede (2023). Pruned skewed Kalman filter and smoother: With applications to the yield curve and asymmetric monetary policy shocks. Working Paper, 475–488. url: https://ideas.repec.org/p/cqe/wpaper/10122.html.
Hafner, C. M., H. Herwartz, and S. Maxand (2022). Identification of structural multivariate GARCH models. Journal of Econometrics 227, 212–227. doi: 10.1016/j.jeconom.2020.07.019.
Herwartz, H., A. Lange, and S. Maxand (2022). Data-driven identification in SVARs—When and how can statistical characteristics be used to unravel causal relationships? Economic Inquiry 60, 668–693. doi: 10.1111/ecin.13035.
Hoga, Y. and M. Demetrescu (2023). Monitoring value-at-risk and expected shortfall forecasts. Management Science 69, 2954–2971. doi: 10.1287/mnsc.2022.4460.
Hoga, Y. and D. Wied (2017). Sequential monitoring of the tail behavior of dependent data. Journal of Statistical Planning and Inference 182, 29–49. doi: 10.1016/j.jspi.2016.08.010.
Huber, P. J. (1985). Projection pursuit. The Annals of Statistics 13, 435–475. doi: 10.1214/aos/1176349519.
Jarrett, J. E. and X. Pan (2007). The quality control chart for monitoring multivariate autocorrelated processes. Computational Statistics & Data Analysis 51, 3862–3870. doi: 10.1016/j.csda.2006.01.020.
Jentsch, C. and K. G. Lunsford (2019). The dynamic effects of personal and corporate income tax changes in the United States: Comment. American Economic Review 109, 2655–2678. doi: 10.1257/aer.20162011.
Jentsch, C. and K. G. Lunsford (2022). Asymptotically Valid Bootstrap Inference for Proxy SVARs. Journal of Business & Economic Statistics 40, 1876–1891. doi: 10.1080/07350015.2021.1990770.
Jeske, D. R., N. T. Stevens, A. G. Tartakovsky, and J. D. Wilson (2018). Statistical methods for network surveillance. Applied Stochastic Models in Business and Industry 34, 425–445. doi: 10.1002/asmb.2326.
Kapoor, M., H. H. Kelejian, and I. R. Prucha (2007). Panel data models with spatially correlated error components. Journal of Econometrics 140, 97–130. doi: 10.1016/j.jeconom.2006.09.004.
Katzfuss, M. (2017). A multi-resolution approximation for massive spatial datasets. Journal of the American Statistical Association 112, 201–214. doi: 10.1080/01621459.2015.1123632.
Keweloh, S. (2021). A Generalized Method of Moments Estimator for Structural Vector Autoregressions Based on Higher Moments. Journal of Business & Economic Statistics 39, 772–782. doi: 10.1080/07350015.2020.1730858.
Kilian, L. and H. Lütkepohl (2017). Structural vector autoregressive analysis. Cambridge University Press. doi: 10.1017/9781108164818.
Kinsvater, P. and R. Fried (2017). Conditional heavy-tail behavior with applications to precipitation and river flow extremes. Stochastic Environmental Research and Risk Assessment 31, 1155–1169. doi: 10.1007/s00477-016-1345-0.
Klüppelberg, C. and M. I. Seifert (2019). Financial risk measures for a network of individual agents holding portfolios of light-tailed objects. Finance and Stochastics 23, 795–826. doi: 10.1007/s00780-019-00401-7.
Koenker, R., Z. Xiao, J. Fan, Y. Fan, et al. (2006). Quantile autoregression [with comments, rejoinder]. Journal of the American Statistical Association 101.475, 980–1006.
Kohl, T., A. Sigle, T. Kuru, J. Salem, et al. (2022). Comprehensive analysis of complications after transperineal prostate biopsy without antibiotic prophylaxis: Results of a multicenter trial with 30 days’ follow-up. Prostate Cancer and Prostatic Diseases 25, 264–268. doi: 10.1038/s41391-021-00423-3.
Koop, G. and D. Korobilis (2013). Large time-varying parameter VARs. Journal of Econometrics 177, 185–198. doi: 10.1016/j.jeconom.2013.04.007.
Lazariv, T. and W. Schmid (2019). Surveillance of non-stationary processes. AStA Advances in Statistical Analysis 103, 305–331. doi: 10.1007/s10182-018-00330-4.
Lee, L.-F. and J. Yu (2015). Spatial panel data models. The Oxford Handbook of Panel Data. Ed. by B. H. Baltagi. Oxford University Press, 363–401. doi: 10.1093/oxfordhb/9780199940042.001.0001.
Leeuwen, P. van, H. Künsch, L. Nerger, R. Potthast, et al. (2019). Particle filters for high-dimensional geoscience applications: A review. Q J R Meteorol Soc. 145, 2335–2365. doi: 10.1002/qj.3551.
LeSage, J. and R. K. Pace (2009). Introduction to Spatial Econometrics.
Lilienthal, J., R. Fried, and A. Schumann (2018). Homogeneity testing for skewed and cross-correlated data in regional flood frequency analysis. Journal of Hydrology 556, 557–571. doi: 10.1016/j.jhydrol.2017.10.056.
Lilienthal, J., L. Zanger, A. Bücher, and R. Fried (2022). A note on statistical tests for homogeneities in multivariate extreme value models for block maxima. Environmetrics 33. doi: 10.1002/env.2746.
Lu, X. and L. Su (2015). Jackknife model averaging for quantile regressions. Journal of Econometrics 188, 40–58. doi: 10.1016/j.jeconom.2014.11.005.
Maleki, F., S. Mehri, A. Aghaie, and H. Shahriari (2020). Robust T2 control chart using median-based estimators. Quality and Reliability Engineering Journal 36, 2187–2201. doi: 10.1002/qre.2691.
Malinovskaya, A. and P. Otto (2021). Online network monitoring. Statistical Methods and Applications 30, 1337–1364. doi: 10.1007/s10260-021-00589-z.
Malinovskaya, A., P. Otto, and T. Peters (2022). Statistical learning for change point and anomaly detection in graphs. Artificial Intelligence, Big Data and Data Science in Statistics: Challenges and Solutions in Environmetrics, the Natural Sciences and Technology. Ed. by A. Steland and K.-L. Tsui. Cham: Springer International Publishing, 85–109. doi: 10.1007/978-3-031-07155-3_4.
Mertens, K. and M. O. Ravn (2013). The Dynamic Effects of Personal and Corporate Income Tax Changes in the United States. American Economic Review 103, 1212–1247. doi: 10.1257/aer.103.4.1212.
Möllenhoff, K., F. Bretz, and H. Dette (2020). Equivalence of regression curves sharing common parameters. Biometrics 76, 518–529. doi: 10.1111/biom.13149.
Möllenhoff, K., H. Dette, E. Kotzagiorgis, S. Volgushev, et al. (2018). Regulatory assessment of drug dissolution profiles comparability via maximum deviation. Statistics in medicine 37, 2968–2981. doi: 10.1002/sim.7689.
Möllenhoff, K., F. Loingeville, J. Bertrand, T. T. Nguyen, et al. (2022). Efficient model-based bioequivalence testing. Biostatistics 23, 314–327. doi: 10.1093/biostatistics/kxaa026.
Nunkesser, R., R. Fried, K. Schettlinger, and U. Gather (2009). Online analysis of time series by the Qn estimator. Computational Statistics & Data Analysis 53, 2354–2362. doi: 10.1016/j.csda.2008.02.027.
Otto, P., W. Schmid, and R. Garthoff (2018). Generalised spatial and spatiotemporal autoregressive conditional heteroscedasticity. Spatial Statistics 26, 125–145. doi: 10.1016/j.spasta.2018.07.005.
Perreault, S., T. Duchesne, and J. G. Neslehova (2019). Detection of block-exchangeable structure in large-scale correlation matrices. Journal of Multivariate Analysis 169, 400–422. doi: 10.1016/j.jmva.2018.10.009.
Pfeffermann, D. and J. Allon (1989). Multivariate exponential smoothing: Method and practice. International Journal of Forecasting 5, 83–98. doi: 10.1016/0169-2070(89)90066-6.
Qiu, P. (2018). Some perspectives on nonparametric statistical process control. Journal of Quality Technology 50, 49–65. doi: 10.1080/00224065.2018.1404315.
Qiu, P. and K. Yang (2023). Spatio-temporal process monitoring using exponentially weighted spatial LASSO. Journal of Quality Technology 55, 163–180. doi: 10.1080/00224065.2022.2081104.
Rigby, R. A. and D. M. Stasinopoulos (2005). Generalized additive models for location, scale and shape. Journal of the Royal Statistical Society. Series C (Applied Statistics) 54, 507–554. doi: 10.1111/j.1467-9876.2005.00510.x.
Ruckdeschel, P., B. Spangl, and D. Pupashenko (2014). Robust Kalman tracking and smoothing with propagating and non-propagating outliers. Statistical Papers 55, 93–123. doi: 10.1007/s00362-012-0496-4.
Rügamer, D., P. F. M. Baumann, T. Kneib, and T. Hothorn (2023). Probabilistic time series forecasts with autoregressive transformation models. Statistics and Computing 33, 37. doi: 10.1007/s11222-023-10212-8.
Salibian-Barrera, M., S. Van Aelst, and G. Willems (2008). Fast and robust bootstrap. Statistical Methods and Applications 17, 41–71. doi: 10.1007/s10260-007-0048-6.
Salibian-Barrera, M. and R. H. Zamar (2002). Bootrapping robust estimates of regression. The Annals of Statistics 30, 556–582. doi: 10.1214/aos/1021379865.
Schmidt, S. K., M. Wornowizki, R. Fried, and H. Dehling (2021). An asymptotic test for constancy of the variance under short-range dependence. The Annals of Statistics 49. doi: 10.1214/21-AOS2092.
Schwierz, T., S. Raczka, D. Hilbrich, and C. Rehtanz (2022). Development of a measurement-based algorithm for topology detection in distribution grids. CIRED Porto Workshop 2022: E-Mobility and Power Distribution Systems. doi: 10.1049/icp.2022.0712.
Smith, A. F. M. and M. West (1983). Monitoring renal transplants: An application of the multiprocess Kalman filter. Biometrics 39, 867–878. doi: 10.2307/2531322.
Tsay, R. S., D. Peña, and A. E. Pankratz (2000). Outliers in multivariate time series. Biometrika 87, 789–804. doi: 10.1093/biomet/87.4.789.
Wang, M., X. Zhang, A. T. K. Wan, K. You, et al. (2023). Jackknife model averaging for high-dimensional quantile regression. Biometrics 79, 178–189. doi: 10.1111/biom.13574.
Wang, M. and G. Zou (2019). An outlier-robust model averaging approach by Mallows-type criterion.
Xie, X. and P. Qiu (2023). A general framework for robust monitoring of multivariate correlated processes. Technometrics. doi: 10.1080/00401706.2023.2224411.
Xu, R., J. Wu, X. Yue, and Y. Li (2023). Online Structural Change-Point Detection of High-dimensional Streaming Data via Dynamic Sparse Subspace Learning. Technometrics 65, 19–32. doi: 10.1080/00401706.2022.2046171.
Zhu, X., W. Wang, H. Wang, and W. K. Härdle (2019). Network quantile autoregression. Journal of Econometrics 212, 345–358. doi: 10.1016/j.jeconom.2019.04.034.
Ziel, F. (2022). M5 competition uncertainty: Overdispersion, distributional forecasting, GAMLSS, and beyond. International Journal of Forecasting 38, 1546–1554. doi: 10.1016/j.ijforecast.2021.09.008.