Live Session
Thursday Posters
Late Breaking Results
Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets
Lukas Wegmeth (University of Siegen), Tobias Vente (University of Siegen) and Joeran Beel (University of Siegen)
Abstract
The recommender systems algorithm selection problem for ranking prediction on implicit feedback datasets is under-explored. Traditional approaches in recommender systems algorithm selection focus predominantly on rating prediction on explicit feedback datasets, leaving a research gap for ranking prediction on implicit feedback datasets. Algorithm selection is a critical challenge for nearly every practitioner in recommender systems. In this work, we take the first steps toward addressing this research gap. We employ meta-learning to solve the algorithm selection problem and explore the quality of tried-and-tested dataset meta-features. Additionally, we investigate the performance of automated machine-learning meta-models. The machine-learning community uses these techniques to address the algorithm selection problem. However, they have not yet been explored for solving the recommender systems algorithm selection problem.We evaluate the NDCG@10 of 24 recommender systems algorithms, each with two hyperparameter configurations, on 72 recommender systems datasets. We train four optimized machine-learning meta-models and one automated machine-learning meta-model with three different settings on the resulting meta-dataset. Our results show that the predictions of all tested meta-models exhibit a median Spearman correlation ranging from 0.857 to 0.918 with the ground truth. We show that the median Spearman correlation between meta-model predictions and the ground truth increases by an average of 0.124 when the meta-model is optimized to predict the ranking of algorithms instead of their performance. Furthermore, in terms of predicting the best algorithm for an unknown dataset, we demonstrate that the best optimized traditional meta-model, e.g., XGBoost, achieves a recall of 48.6%, outperforming the best tested automated machine learning meta-model, e.g., AutoGluon, which achieves a recall of 47.2%.