Scientific journal paper Q1
Deep learning in time series forecasting with transformer models and RNNs
Dos Santos, Rogerio Pereira (Dos Santos, R. P. ); Matos-Carvalho, João P. (Matos-Carvalho, J.); Valderi Leithardt (Leithardt, V.);
Journal Title
PeerJ Computer Science
Year (definitive publication)
2025
Language
English
Country
United States of America
More Information
Web of Science®

Times Cited: 0

(Last checked: 2025-12-03 11:07)

View record in Web of Science®

Scopus

This publication is not indexed in Scopus

Google Scholar

Times Cited: 1

(Last checked: 2025-11-30 01:51)

View record in Google Scholar

This publication is not indexed in Overton

Abstract
Given the increasing need for accurate weather forecasts, the use of neural networks, especially transformer and recurrent neural networks (RNNs), has been highlighted for their ability to capture complex patterns in time series. This study examined 14 neural network models applied to forecast weather variables, evaluated using metrics such as median absolute error (MedianAbsE), mean absolute error (MeanAbsE), maximum absolute error (MaxAbsE), root mean squared percent error (RMSPE), and root mean square error (RMSE). Transformer-based models such as Informer, iTransformer, Former, and patch time series transformer (PatchTST) stood out for their accuracy in capturing long-term patterns, with Informer showing the best performance. In contrast, RNN models such as auto-temporal convolutional networks (TCN) and bidirectional TCN (BiTCN) were better suited to short-term forecasting, despite being more prone to significant errors. Using iTransformer it was possible to achieve a MedianAbsE of 1.21, MeanAbsE of 1.24, MaxAbsE of 2.86, RMSPE de 0.66, and RMSE de 1.43. This study demonstrates the potential of neural networks, especially transformers, to improve accuracy, providing a practical and theoretical basis for selecting the most suitable models for predictive applications.
Acknowledgements
--
Keywords
Deep learning,Neural networks,Recurrent neural networks (RNNs),Transformer models,Predictive applications,Accuracy in forecasting
  • Computer and Information Sciences - Natural Sciences
  • Other Engineering and Technology Sciences - Engineering and Technology
  • Electrical Engineering, Electronic Engineering, Information Engineering - Engineering and Technology
Funding Records
Funding Reference Funding Entity
101183162 European Union’s Horizon-MSCA-2023-SE-01 Programme
UID/00408/2025 Fundação para a Ciência e a Tecnologia

With the objective to increase the research activity directed towards the achievement of the United Nations 2030 Sustainable Development Goals, the possibility of associating scientific publications with the Sustainable Development Goals is now available in Ciência_Iscte. These are the Sustainable Development Goals identified by the author(s) for this publication. For more detailed information on the Sustainable Development Goals, click here.