Comparative analysis of autoregressive approaches and recurrent neural networks for modeling and forecasting nonlinear nonstationary processes

Authors

  • Oleg Belas Institute of Special Communication and Information Security of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,, Ukraine https://orcid.org/0000-0002-1595-3029
  • Petro Bidiuk Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,, Ukraine https://orcid.org/0000-0002-7421-3565
  • Andrii Belas Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,, Ukraine https://orcid.org/0000-0001-7883-2489

DOI:

https://doi.org/10.20535/2411-1031.2019.7.1.184395

Keywords:

Mathematical modeling, signal processing, nonstationary processes, autoregressive models, neural networks, recurrent neural networks

Abstract

Nonlinear nonstationary processes presented in the form of time series can describe the dynamics of processes in both technical and economic systems. Forecasting of such processes has numerous applications in power engineering, network systems, trade, and investment activities. However, there is no single approach to modeling and predicting such processes currently. This paper considers the most commonly used approaches. They are considered to be effective in working with data presented in the form of sequences: autoregressive models and recurrent neural networks. Classical regression approaches predict a target variable by a linear combination of past values of this variable. Therefore, they are quite simply used both from the theoretical and computational point of view due to the simple structure. However, this approach is limited to the complexity of taking into account a large number of external factors due to the problem of multicollinearity, as well as their possible nonlinear influence. Neural networks learn from experience and adapt to a changing environment that is modeled. Neural technologies are used for nonlinear modeling, resistant to information noise and capable of generalization based on historical data. The use of neural networks allows obtaining accurate and adequate models, even with a qualitative analysis of the interconnections factors that influence the result of forecasting. Therefore, recurrent neural networks are used to work with sequences. This allows solving the problem of modeling taking into account the nonlinear or combined effects of external factors. However, the application of this approach is limited to large computational costs. In addition, this approach can’t be applied to very long sequences. This is a problem for solving modern problems using large amount of data. From the analysis, it follows from the necessity of developing a new, effective from a computational point of view approach to modeling large sequences taking into account the nonlinear or combined effects of external factors

Author Biographies

Oleg Belas, Institute of Special Communication and Information Security of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,

doctor of technical sciences, professor, 
professor at the special telecommunications 
systems academic department

Petro Bidiuk, Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,

doctor of technical sciences, professor,
professor at the mathematical methods
for system analysis academic department

Andrii Belas, Institute for applied system analysis of National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv,

Ph.D. student

References

P. I. Bidyuk, V. D. Romanenko, and O. L. Timoschuk, Analysis of time series. Kyiv, Ukraine: Polytechnic, 2010.

R. S. Tsay, Analysis of Financial Time Series. New Jersey, USA: Wiley, 2010.

R. Hyndman, G. Athanasopoulos, Forecasting: Principles and Practice. Melbourne, Australia: OTexts, 2013.

Shumway R., Stoffer D., Time Series Analysis and Its Applications. New York, USA: Springer, 2011.

S. Hochreiter, and J. Schmidhuber, “Long Short-Term Memory”, Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.

F. A. Gers, D. Eck, and J.Schmidhuber, “Applying LSTM to Time Series Predictable Through Time-Window Approaches”, in Proc. of International Conference on Artificial Neural Networks, Vienna, 2001, pp. 669-676. doi: 10.1007/3-540-44668-0_93.

S. Hochreiter, Y. Bengio, and J. Schmidhuber, “Gradient flow in recurrent nets: the difficulty of learning long-term dependencies”. [Online]. Available: http://www.bioinf.jku.at/ publications/older/ch7.pdf . Accessed on: Dec. 12, 2018.

“Understanding LSTM Networks”. [Online]. Available: http://colah.github.io/ posts/2015-08-Understanding-LSTMs/. Accessed on: March 9, 2018.

F. Chollet, and J. Allaire, Deep Learning with R. New York, USA: Manning, 2018.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, USA: MIT Press, 2016.

Downloads

Published

2019-06-30

How to Cite

Belas, O., Bidiuk, P., & Belas, A. (2019). Comparative analysis of autoregressive approaches and recurrent neural networks for modeling and forecasting nonlinear nonstationary processes. Collection "Information Technology and Security", 7(1), 91–99. https://doi.org/10.20535/2411-1031.2019.7.1.184395

Issue

Vol. 7 No. 1 (2019)

Section

ELECTRONIC COMMUNICATION SYSTEMS AND NETWORKS

License

Copyright (c) 2020 Collection "Information technology and security"

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The authors that are published in this collection, agree to the following terms:

  1. The authors reserve the right to authorship of their work and pass the collection right of first publication this work is licensed under the Creative Commons Attribution License, which allows others to freely distribute the published work with the obligatory reference to the authors of the original work and the first publication of the work in this collection.
     
  2. The authors have the right to conclude an agreement on exclusive distribution of the work in the form in which it was published this anthology (for example, to place the work in a digital repository institution or to publish in the structure of the monograph), provided that references to the first publication of the work in this collection.
     
  3. Policy of the journal allows and encourages the placement of authors on the Internet (for example, in storage facilities or on personal web sites) the manuscript of the work, prior to the submission of the manuscript to the editor, and during its editorial processing, as it contributes to productive scientific discussion and positive effect on the efficiency and dynamics of citations of published work (see The Effect of Open Access).