DOI: 10.17587/prin.16.612-621

Scaled Triple VAW Algorithm for Online Multi-Kernel Linear Regression

O. V. Gurtovaya, Postgraduate Student, imedashvili@sfedu.ru, Institute of Mathematics, Mechanics and Computer Sciences of the Southern Federal University, Rostov-on-Don, 344090, Russian Federation

Corresponding author: Olga V. Gurtovaya, Postgraduate Student, Institute of Mathematics, Mechanics and Computer Sciences of the Southern Federal University, Rostov-on-Don, 344090, Russian Federation, E-mail: imedashvili@sfedu.ru

This article introduces S-VAW2, an innovative three-level Vovk-Azoury-Warmuth (VAW2) algorithm specifically engineered for online regression tasks. Building upon the existing two-level VAW2 algorithm, S-VAW2 significantly enhances its capabilities by integrating multi-kernel learning techniques with advanced data scaling strategies. This combination enables S-VAW2 to exhibit superior performance in dynamic data environments, where algorithms must adapt to continuously changing data streams without the need for full model retraining. Online learning algorithms are crucial for applications that demand real-time data processing and adaptive responses to continuous changes. The primary objective of this work is to conduct a comprehensive comparative analysis between S-VAW2 and AutoGluon-Tabular, a state-of-the-art automated machine learning (AutoML) system. AutoGluon is renowned for its ability to automate the entire machine learning pipeline, including data preprocessing, feature selection, and algo­rithm optimization, while delivering robust predictive performance across various data types, especially tabular data. The study evaluates the performance of S-VAW2 against AutoGluon-Tabular using 13 diverse tabular datasets. The experimental results conclusively demonstrate that S-VAW 2 effectively adapts to data heterogeneity and achieves competitive outcomes when compared to cutting-edge AutoML solutions. This highlights S-VAW2's significant potential for robust and efficient online regression in practical scenarios, offering a valuable alternative to existing methodolo­gies, especially when adapting to data drift.

Keywords: Vovk—Azoury—Warmuth algorithm, online multi-kernel learning, RKHS, random Fourier features, regret bounds, linear regression, online regression, predictions with expert advice

pp. 612—621

For citation:
Gutovaya O. V. Scaled Triple VAW Algorithm for Online Multi-Kernel Linear Regression, Programmnaya Ingeneria, 2025, vol. 16, no. 12, pp. 612—621. DOI: 10.17857/prin.612-621. (in Russian).

References:

1. Rokhlin D. B., Gurtovaya O. V. Random feature-based double Vovk-Azoury-Warmuth algorithm for online multi-kernel learning. arXiv preprint, 2025. DOI: 10.48550/arXiv.2503.20087.
2. Rahimi A., Recht B. Uniform approximation of functions with random bases, 2008 46th Annual Allerton Conference on Communication, Control, and Computing, 2008, pp. 555—561. DOI: 10.1109/ALLERTON.2008.4797607.
3. Erickson N., Mueller J., Shirkov A. et al. AutoGluon-Tabular: Robust and accurate AutoML for structured data. arXiv preprint, 2020. DOI: 10.48550/arXiv.2003.06505.
4. Cesa-Bianchi N., Lugosi G. Prediction, Learning, and Games, Cambridge, Cambridge University Press, 2006, 409 p.
5. Vovk V. Online regression competitive with reproducing kernel Hilbert spaces, International Conference on Theory and Applications of Models of Computation, 2006, pp. 452—463.
6. Li L., Jamieson K., DeSalvo G. et al. Hyperband: A novel banditbased approach to hyperparameter optimization, Journal of Machine Learning Research, 2018, vol. 18, article 185. DOI: 10.48550/arXiv.1603.06560.
7. Ke G., Meng Q., Finley T. et al. LightGBM: A Highly Efficient Gradient Boosting Decision Tree, Advances in Neural Information Processing Systems, 2017, vol. 30, рр. 3146—3154.
8. Prokhorenkova L., Gusev G., Vorobev A., Dorogush A. V., Gulin A. CatBoost: Unbiased boosting with categorical features, Advances in Neural Information Processing Systems, 2018, vol. 31. DOI: 10.48550/arXiv.1706.09516.
9. Chen T., Guestrin C. XGBoost: A scalable tree boosting system, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 785—794. DOI: 10.48550/arXiv.1603.02754.
10. Breiman L. Random forests, Machine Learning, 2001, vol. 45, pp. 5—32. DOI: 10.1023/A:1010933404324.
11. Geurts P., Ernst D., Wehenkel L. Extremely randomized trees, Machine Learning, 2006, vol. 63, pp. 3—42. DOI: 10.1007/s10994-006-6226-1.
12. Ghari P. M., Shen Y. Graph-aided online multi-kernel learning, Journal of Machine Learning Research, 2023, vol. 24, article 21. DOI: 10.48550/arXiv.2102.04690.