Comparative analysis of autoregressive approaches and recurrent neural networks for modeling and forecasting nonlinear nonstationary processes

Authors

  • Oleg Belas Institute of Special Communication and Information Security of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, http://orcid.org/0000-0002-1595-3029
  • Petro Bidiuk Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, http://orcid.org/0000-0002-7421-3565
  • Andrii Belas Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, http://orcid.org/0000-0001-7883-2489

DOI:

https://doi.org/10.20535/2411-1031.2019.7.1.184395

Keywords:

Mathematical modeling, signal processing, nonstationary processes, autoregressive models, neural networks, recurrent neural networks

Abstract

Nonlinear nonstationary processes presented in the form of time series can describe the dynamics of processes in both technical and economic systems. Forecasting of such processes has numerous applications in power engineering, network systems, trade, and investment activities. However, there is no single approach to modeling and predicting such processes currently. This paper considers the most commonly used approaches. They are considered to be effective in working with data presented in the form of sequences: autoregressive models and recurrent neural networks. Classical regression approaches predict a target variable by a linear combination of past values of this variable. Therefore, they are quite simply used both from the theoretical and computational point of view due to the simple structure. However, this approach is limited to the complexity of taking into account a large number of external factors due to the problem of multicollinearity, as well as their possible nonlinear influence. Neural networks learn from experience and adapt to a changing environment that is modeled. Neural technologies are used for nonlinear modeling, resistant to information noise and capable of generalization based on historical data. The use of neural networks allows obtaining accurate and adequate models, even with a qualitative analysis of the interconnections factors that influence the result of forecasting. Therefore, recurrent neural networks are used to work with sequences. This allows solving the problem of modeling taking into account the nonlinear or combined effects of external factors. However, the application of this approach is limited to large computational costs. In addition, this approach can’t be applied to very long sequences. This is a problem for solving modern problems using large amount of data. From the analysis, it follows from the necessity of developing a new, effective from a computational point of view approach to modeling large sequences taking into account the nonlinear or combined effects of external factors

Author Biographies

Oleg Belas, Institute of Special Communication and Information Security of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,

doctor of technical sciences, professor, 
professor at the special telecommunications 
systems academic department

Petro Bidiuk, Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,

doctor of technical sciences, professor,
professor at the mathematical methods
for system analysis academic department

Andrii Belas, Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,

Ph.D. student

References

P. I. Bidyuk, V. D. Romanenko, and O. L. Timoschuk, Analysis of time series. Kyiv, Ukraine: Polytechnic, 2010.

R. S. Tsay, Analysis of Financial Time Series. New Jersey, USA: Wiley, 2010.

R. Hyndman, G. Athanasopoulos, Forecasting: Principles and Practice. Melbourne, Australia: OTexts, 2013.

Shumway R., Stoffer D., Time Series Analysis and Its Applications. New York, USA: Springer, 2011.

S. Hochreiter, and J. Schmidhuber, “Long Short-Term Memory”, Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.

F. A. Gers, D. Eck, and J.Schmidhuber, “Applying LSTM to Time Series Predictable Through Time-Window Approaches”, in Proc. of International Conference on Artificial Neural Networks, Vienna, 2001, pp. 669-676. doi: 10.1007/3-540-44668-0_93.

S. Hochreiter, Y. Bengio, and J. Schmidhuber, “Gradient flow in recurrent nets: the difficulty of learning long-term dependencies”. [Online]. Available: http://www.bioinf.jku.at/ publications/older/ch7.pdf . Accessed on: Dec. 12, 2018.

“Understanding LSTM Networks”. [Online]. Available: http://colah.github.io/ posts/2015-08-Understanding-LSTMs/. Accessed on: March 9, 2018.

F. Chollet, and J. Allaire, Deep Learning with R. New York, USA: Manning, 2018.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, USA: MIT Press, 2016.

Published

2019-06-30

How to Cite

Belas, O., Bidiuk, P., & Belas, A. (2019). Comparative analysis of autoregressive approaches and recurrent neural networks for modeling and forecasting nonlinear nonstationary processes. Information Technology and Security, 7(1), 91–99. https://doi.org/10.20535/2411-1031.2019.7.1.184395

Issue

Section

TELECOMMUNICATION SYSTEMS AND NETWORKS