An Overview of the Algorithm Selection Problem

Salisu Mamman Abdulrahman, Alhassan Adamu, Yazid Ado Ibrahim, Akilu Rilwan Muhammad


Users of machine learning algorithms need methods that can help them to identify algorithm or their combinations (workflows) that achieve the potentially best performance. Selecting the best algorithm to solve a given problem has been the subject of many studies over the past four decades. This survey presents an overview of the contributions made in the area of algorithm selection problems. We present different methods for solving the algorithm selection problem identifying some of the future research challenges in this domain.


Machine Learning; Algorithm selection; Workflows.

Full Text:



Rice, J. R. (1976). The algorithm selection problem. Advances in computers, 15, 65-118.

Brazdil, P., Carrier, C. G., Soares, C., & Vilalta, R. (2008). Metalearning: Applications to data mining. Springer Science & Business Media.

Smith-Miles, K. A. (2009). Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Computing Surveys (CSUR), 41(1), 6.

Michie, D., Spiegelhalter, D. J., & Taylor, C. C. (1994). Machine learning, neural and statistical classification.

Brazdil, P. B., Soares, C., & Da Costa, J. P. (2003). Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Machine Learning, 50(3), 251-277.

Vilalta, R., & Drissi, Y. (2002). A perspective view and survey of meta-learning. Artificial Intelligence Review, 18(2), 77-95.

Lemke, C., Budka, M., & Gabrys, B. (2015). Metalearning: a survey of trends and technologies. Artificial intelligence review, 44(1), 117.

Vanschoren, J. (2010). Understanding machine learning performance with experiment databases. lirias. kuleuven. be, no. May.

Prudêncio, R. B., & Ludermir, T. B. (2004). Meta-learning approaches to selecting time series models. Neurocomputing, 61, 121-137.

Wang, X., Smith-Miles, K., & Hyndman, R. (2009). Rule induction for forecasting method selection: Meta-learning the characteristics of univariate time series. Neurocomputing, 72(10), 2581-2594.

Lemke, C., & Gabrys, B. (2010). Meta-learning for time series forecasting and forecast combination. Neurocomputing, 73(10), 2006-2016.

Soares, C., & Brazdil, P. B. (2006, April). Selecting parameters of SVM using meta-learning and kernel matrix-based meta-features. In Proceedings of the 2006 ACM symposium on Applied computing (pp. 564-568). ACM.

Aha, D. W. (1992). Tolerating noisy, irrelevant and novel attributes in instance-based learning algorithms. International Journal of Man-Machine Studies, 36(2), 267-287.

King, R. D., Feng, C., & Sutherland, A. (1995). Statlog: comparison of classification algorithms on large real-world problems. Applied Artificial Intelligence an International Journal, 9(3), 289-333.

Giraud-Carrier, C. (2005, December). The data mining advisor: meta-learning at the service of practitioners. In Machine Learning and Applications, 2005. Proceedings. Fourth International Conference on (pp. 7-pp). IEEE.

Bernstein, A., Provost, F., & Hill, S. (2005). Toward intelligent assistance for a data mining process: An ontology-based approach for cost-sensitive classification. IEEE Transactions on knowledge and data engineering, 17(4), 503-518., Retrieved on 5th January 2017.

Kietz, J., Serban, F., Bernstein, A., & Fischer, S. (2009, September). Towards cooperative planning of data mining workflows. In Proceedings of the Third Generation Data Mining Workshop at the 2009 European Conference on Machine Learning (ECML 2009) (pp. 1-12).

Yang, G., Kifer, M., Zhao, C., & Chowdhary, V. (2005). FLORA-2: User’s manual. Version 0.94 (Narumigata). April, 30.

Hilario, M., Kalousis, A., Nguyen, P., & Woznica, A. (2009, September). A data mining ontology for algorithm selection and meta-mining. In Proceedings of the ECML/PKDD09 Workshop on 3rd generation Data Mining (SoKD-09) (pp. 76-87).

Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., & Euler, T. (2006, August). Yale: Rapid prototyping for complex data mining tasks. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 935-940). ACM.

Morik, K., & Scholz, M. (2004). The miningmart approach to knowledge discovery in databases. Intelligent technologies for information analysis, 47-65.

Kalousis, A. (2002). Algorithm selection via meta-learning. University of Geneva, Genebra.

Todorovski, L., Blockeel, H., & Dzeroski, S. (2002, August). Ranking with predictive clustering trees. In European Conference on Machine Learning (pp. 444-455). Springer, Berlin, Heidelberg.

Bensusan, H., & Kalousis, A. (2001). Estimating the predictive accuracy of a classifier. Machine Learning: ECML 2001, 25-36.

Castellano, G., Castiello, C., Fanelli, A. M., & Mencar, C. (2005). Knowledge discovery by a neuro-fuzzy modeling framework. Fuzzy sets and Systems, 149(1), 187-207.

Botía, J., Gómez-Skarmeta, A., Valdés, M., & Padilla, A. (2001). Metala: A meta-learning architecture. Computational Intelligence. Theory and Applications, 688-698.

Hernansaenz, J. M., Botía, J. A., & Skarmeta, A. F. (2016). METALA: a J2EE technology based framework for web mining. Revista Colombiana de Computación-RCC, 5(1).

Grabczewski, K., & Jankowski, N. (2007, March). Versatile and efficient meta-learning architecture: Knowledge representation and management in computational intelligence. In Computational Intelligence and Data Mining, 2007. CIDM 2007. IEEE Symposium on (pp. 51-58). IEEE.

Kotthoff, L., Thornton, C., Hoos, H. H., Hutter, F., & Leyton-Brown, K. (2016). Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA. Journal of Machine Learning Research, 17, 1-5.

Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013, August). Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 847-855). ACM.

Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011). Sequential Model-Based Optimization for General Algorithm Configuration. LION, 5, 507-523.

Bergstra, J. S., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems (pp. 2546-2554).

Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Fréchette, A., ... & Vanschoren, J. (2016). Aslib: A benchmark library for algorithm selection. Artificial Intelligence, 237, 41-58.

Lindauer, M., Hoos, H. H., Hutter, F., & Schaub, T. (2015). Autofolio: An automatically configured algorithm selector. Journal of Artificial Intelligence Research, 53, 745-778.

Kotthoff, L. (2013). LLAMA: leveraging learning to automatically manage algorithms. arXiv preprint arXiv:1306.1031.

Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., & Hutter, F. (2015). Efficient and robust automated machine learning. In Advances in Neural Information Processing Systems (pp. 2962-2970).

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., & Vanderplas, J. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(Oct), 2825-2830.


  • There are currently no refbacks.





About IJC | Privacy PolicyTerms & Conditions | Contact Us | DisclaimerFAQs 

IJC is published by (GSSRR).