An Investigation of The Contribution of Attention-Based Hybrid Deep Learning Models to Prediction Errors in Cryptocurrency Markets
DOI:
https://doi.org/10.5281/zenodo.20800529Keywords:
Cryptocurrency, Deep Learning, LSTM, GRU, Attention Mechanism, Price ForecastingAbstract
This study comparatively analyzes deep learning architectures for predicting the daily closing prices of Ethereum, Solana, and BNB cryptocurrencies. Given the high volatility, non-linear price movements, and noise-intensive data structures of cryptocurrency markets, accurate price modeling holds significant importance for both academic research and investment decisions. The models examined include LSTM, GRU, CNN, CNN-LSTM, CNN-GRU, Attention-LSTM, and Attention-GRU. The dataset covers the period from April 10, 2020, to February 12, 2026, at daily frequency, split into 70% training and 30% testing. Hyperparameter optimization was conducted across varying window lengths, epochs, batch sizes, neuron counts, and dropout rates. Model performance was evaluated using MAE, MSE, MAPE, and R² metrics. Results indicate that GRU-based models consistently yield lower error values and more stable performance on cryptocurrency time series. The hybrid CNN-GRU architecture produces competitive results on certain series; however, attention mechanism-integrated models generally increase error values. These findings suggest that model complexity does not guarantee superior prediction accuracy in financial time series, and that compact architectures may offer stronger generalization in highly volatile data environments. The study also addresses a methodological gap by benchmarking multiple deep learning models under identical datasets and experimental conditions.
References
Albayrak, E., and Saran, N. (2023). Stock Price Forecasting Using Statistical and Deep Learning Models. Journal of Computer Science and Engineering, 16(2), 161-169. https://doi.org/10.54525/tbbmd.1031017
Aruwaji, A. M., & Swanepoel, M. (July 2025). Finformer: An Attention-Based Deep Learning Model for Cryptocurrency and Stock Price Forecasting. In 2025 Conference on Information Communications Technology and Society (ICTAS) (pp. 1-6). IEEE.
Ataei, S., Ataei, S. T., Omidmand, P., Karahroodi, H. H., & Nikzat, P. (2025). Applications of Deep Learning to Cryptocurrency Trading: A Systematic Analysis. Soft Computing: Fusion with Applications, 2(4), 255-268.
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Baur, D. G., Hong, K., & Lee, A. D. (2018). Bitcoin: Medium of exchange or speculative asset?. Journal of International Financial Markets, Institutions and Money, 54, 177-189.
Borovykh, A., Bohte, S., & Oosterlee, C. W. (2017). Conditional time series forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691.
Gao, M. (2025). Cryptocurrency Price Prediction Based on the CNN-BiLSTM-AM Model.
Gautam, M. (2025). Crypto Price Prediction Using LSTM+XGBoost. arXiv preprint arXiv:2506.22055.
Goodfellow, I., Bengio, Y., Courville, A., & Bengio, Y. (2016). Deep Learning (Vol. 1, No. 2, pp. 1–800). Cambridge: MIT Press.
Katsiampa, P. (2019). An empirical investigation of volatility dynamics in the cryptocurrency market. Research in International Business and Finance, 50, 322–335.
Kumar, S., Kour, V., Kumar, P., et al. GRU-LSTM with attention-based forecasting for enhanced air quality. Nat Hazards 121, 15925–15947 (2025). https://doi.org/10.1007/s11069-025-07408-8
Lahuddin, H., Muliawan, M. R. A., Takemoto, K., Darwis, H., Jabir, S. R., & Adawiyah, R. (September 2025). Cryptocurrency Prices Forecasting Using LSTM, CNN, Transformer, TCN, and Hybrid Model: A Deep Learning Approach. In 2025 9th International Conference on Electrical, Electronics, and Information Engineering (ICEEIE) (pp. 1–6). IEEE.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
Lee, M. C. (2024). Attention-Based Deep Learning Models for Cryptocurrency Price Prediction: A Comparative Analysis with Technical Indicators.
Luong, M. T., Pham, H., & Manning, C. D. (September 2015). Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 1412–1421).
Mahdi, E., Martin-Barreiro, C., & Cabezas, X. (2025). A novel hybrid approach using an attention-based transformer+GRU model for predicting cryptocurrency prices. Mathematics, 13(9), 1484.
McNally, S., Roche, J., & Caton, S. (March 2018). Predicting the price of Bitcoin using machine learning. In Proceedings of the 2018 26th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP) (pp. 339–343). IEEE.
Priadinata, I. P. B., Sudipa, I. G. I., Meinarni, N. P. S., Radhitya, I. M. L., & Supartha, I. K. D. G. (2025). Comparative analysis of LSTM, GRU, and Bi-LSTM deep learning models for time series cryptocurrency price forecasting. Sinkron: Journal of Computer Science Research, 9(3), 1024–1035.
Rezaei, H., Faaljou, H., and Mansourfar, G. (2021). Stock price prediction using deep learning and frequency decomposition. Expert Systems with Applications, 169, 114332. https://doi.org/10.1016/j.eswa.2020.114332
Saqware, G. J., & B, I. (2024, March). Hybrid deep learning model integrating attention mechanism for the accurate prediction and forecasting of the cryptocurrency market. In Operations research forum (Vol. 5, No. 1, p. 19). Cham: Springer International Publishing.
Saracık, Ö., & İncekırık, A. (2023). Stock Price Forecasting with Deep Learning Techniques. Alphanumeric Journal, 11(2), 137-156. https://doi.org/10.17093/alphanumeric.1357466
Seabe, P. L., Moutsinga, C. R. B., & Pindza, E. (2025). Sentiment-driven cryptocurrency forecasting: analyzing LSTM, GRU, Bi-LSTM, and temporal attention model (TAM). Social Network Analysis and Mining, 15(1), 52.
Shi, Z., Hu, Y., Mo, G., & Wu, J. (2022). An attention-based CNN-LSTM and XGBoost hybrid model for stock price prediction. arXiv preprint arXiv:2204.02623.
Sun, X. (2024). Application of Attention-Based LSTM Hybrid Models for Stock Price Prediction. Advances in Economics, Management and Political Sciences, 104, 77-91.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Wei, Z., Wang, L., Liu, Z., & Zhou, Y. (2026). Prediction of extraction uranium concentration and optimization of injection acid concentration using a CNN–GRU–attention model in in-situ uranium leaching. Journal of Analytical Science and Technology, 17(1), 14.
Wen, X., & Li, W. (2023). Time series prediction based on an LSTM-attention-LSTM model. IEEE Access, 11, 48322–48331.
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., ... & Bengio, Y. (June 2015). Show, attend, and tell: Neural image caption generation with visual attention. In International Conference on Machine Learning (pp. 2048–2057). PMLR.
Yahoo Finance. (2025). Historical daily closing prices for BNB-USD, ETH-USD, SOL-USD [Data set]. https://finance.yahoo.com/ Accessed on (02/14/2026)
Yousufi, M. L., & İncekırık, A. (2024). An analysis of Ethereum price prediction in the digital economy using deep learning techniques. In GENÇLERLE 360° 10th International Student Congress / The Circular Economy in a Globalizing World – Recycling / Circular Economy in the Globalizing World (Vol. 1, No. 1, pp. 58–66). Manisa Celal Bayar University. https://genclerle360.mcbu.edu.tr/wp-content/uploads/sites/11/2025/01/01.08.-Onuncu-bildiri-kitabi.pdf
Zaremba, W., Sutskever, I., & Vinyals, O. (2014). Recurrent neural network regularization. arXiv preprint arXiv:1409.2329.
Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1253.
Zhang, X., Dou, Y., Mao, J., Liu, W., & Han, H. (2023). Carbon Price Forecasting Approach Based on Multi-Scale Decomposition and Transfer Learning. Journal of Beijing Institute of Technology, 32(2), 242-255.
Zhao, X., Liu, J., Wang, Y., & Wang, J. (2026). CryptoMamba-SSM: Linear-Complexity State-Space Models for Cryptocurrency Volatility Prediction. IEEE Open Journal of the Computer Society, 7, 226–243.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 International Journal of Contemporary Economics and Administrative Sciences
This work is licensed under a Creative Commons Attribution 4.0 International License.
The Author(s) must make formal transfer of copyright for each article prior to publication in the International Journal of Contemporary Economics and Administrative Sciences. Such transfer enables the Journal to defend itself against plagiarism and other forms of copyright infringement. Your cooperation is appreciated. You agree that copyright of your article to be published in the International Journal of Contemporary Economics and Administrative Sciences is hereby transferred, throughout the World and for the full term and all extensions and renewals thereof, to International Journal of Contemporary Economics and Administrative Sciences.
The Author(s) reserve(s): (a) the trademark rights and patent rights, if any, and (b) the right to use all or part of the information contained in this article in future, non-commercial works of the Author's own, or, if the article is a "work-for-hire" and made within the scope of the Author's employment, the employer may use all or part of the information contained in this article for intra-company use, provided the usual acknowledgements are given regarding copyright notice and reference to the original publication.
The Author(s) warrant(s) that the article is Author's original work, and has not been published before. If excerpts from copyrighted works are included, the Author will obtain written permission from the copyright owners and shall credit the sources in the article. The author also warrants that the article contains no libelous or unlawful statements, and does not infringe on the rights of others. If the article was prepared jointly with other Author(s), the Author agrees to inform the co-Author(s) of the terms of the copyright transfer and to sign on their behalf; or in the case of a "work-for-hire" the employer or an authorized representative of the employer.
The journal is registered with the ISSN : 1925-4423.
IJCEAS is licensed under a Creative Commons Attribution 4.0 International License.
This license lets others distribute, remix, tweak, and build upon your work, even commercially, as long as they credit you for the original creation. This is the most accommodating of licenses offered. Recommended for maximum dissemination and use of licensed materials.
