2015

• Klein, A. and Bartels, S. and Falkner, S. and Hennig, P. and Hutter, F. (pdf)(bib)
  Towards efficient Bayesian Optimization for Big Data
  In: NIPS 2015 Bayesian Optimization Workshop
  
• Feurer, M. and Klein, A. and Eggensperger, K. and Springenberg, J. and Blum, M. and Hutter, F. (preprint)(published)(supplementary)(bib)
  Efficient and Robust Automated Machine Leraning
  In: Advances in Neural Information Processing Systems 28
  
• Falkner, S. and Lindauer, M. and Hutter, F. (pdf)(bib)
  SpySMAC: Automated Configuration and Performance Analysis of SAT Solvers
  In: Proceedings of the International Conference on Satisfiability Solving (SAT'15)
  
• Lindauer, M. and Hoos, H. and Hutter, F. and Schaub, T. (pdf)(bib)
  AutoFolio: An automatically configured Algorithm Selector
  In: Journal of Artificial Intelligence 53 (2015): 745-778
  
• Vallati, Mauro and Hutter, Frank and Chrpa, Lukáš and McCluskey, T.L. (pdf)(bib)
  On the Effective Configuration of Planning Domain Models
  In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI)
  
• Frank Hutter and Lin Xu and Holger H. Hoos and Kevin Leyton-Brown (pdf)(bib)
  Algorithm runtime prediction: Methods & evaluation (extended abstract)
  In: Proceedings of the Journal Track of the 24th International Joint Conference on Artificial Intelligence (IJCAI)
  
• Tobias Domhan and Jost Tobias Springenberg and Frank Hutter (pdf)(bib)
  Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves
  In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI)
  
• Hutter, F. and Lücke, J. and Schmidt-Thieme, L. (published)(bib)
  Beyond Manual Tuning of Hyperparameters
  In: Künstliche Intelligenz 0. (2015): 1-9
  
• Feurer, M. and Klein, A. and Eggensperger, K. and Springenberg, J. and Blum, M. and Hutter, F. (pdf)(poster)(slides)(bib)
  Methods for Improving Bayesian Optimization for AutoML
  In: ICML 2015 AutoML Workshop
  
• Hutter, F. and Lindauer, M. and Balint, A. and Bayless, S. and Hoos, H. and Leyton-Brown, K. (arXiv)(bib)
  The Configurable SAT Solver Challenge (CSSC)
  In: CoRR abs/1505.01221 (2015)
  To appear in the AI Journal (competition track).
  
• Bischl, B. and Kerschke, P. and Kotthoff, L. and Lindauer, M. and Malitsky, Y. and Frechétte, A. and Hoos, H. and Hutter, F. and Leyton-Brown, K. and Tierney, K. and Vanschoren, J. (pdf)(bib)
  ASlib: A Benchmark Library for Algorithm Selection
  In: Arxiv abs/1506.02465 (2015)
  
• Marius Lindauer and Holger H. Hoos and Frank Hutter and Torsten Schaub (pdf)(bib)
  AutoFolio: Algorithm Configuration for Algorithm Selection
  In: Proceedings of the Twenty-Ninth AAAI Workshops on Artificial Intelligence
  
• Jendrik Seipp, Silvan Sievers, Malte Helmert and Frank Hutter. (pdf)(bib)
  Automatic Configuration of Sequential Planning Portfolios
  In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
  
• Katharina Eggensperger and Frank Hutter and Holger H. Hoos and Kevin Leyton-Brown (pdf)(poster)(bib)
  Efficient Benchmarking of Hyperparameter Optimizers via Surrogates
  In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
  
• Matthias Feurer and Tobias Springenberg and Frank Hutter (pdf)(supplementary)(poster)(bib)
  Initializing Bayesian Hyperparameter Optimization via Meta-Learning
  In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
  
• Marius Lindauer and Holger H. Hoos and and Frank Hutter (pdf)(bib)
  From Sequential Algorithm Selection to Parallel Portfolio Selection
  In: Proceedings of the International Conference on Learning and Intelligent Optimization (LION'15)
  

2014

• Seipp, Jendrick and Sievers, Silvan and Hutter, Frank (pdf)(bib)
  Fast Downward SMAC
  Planner abstract, IPC 2014 Planning and Learning Track.
  Best basic solver award, and third place in the categories overall best quality and best learner.
  
• Seipp, Jendrick and Sievers, Silvan and Hutter, Frank (pdf)(bib)
  Fast Downward Cedalion
  Planner abstract, IPC 2014 Planning and Learning Track.
  Best learner award, and second place in the category overall best quality at the IPC 2014 Planning and Learning Track. Also achieved the highest coverage in the IPC 2014 sequential agile planning track.
  
• Katharina Eggensperger and Frank Hutter and Holger H. Hoos and Kevin Leyton-Brown (pdf)(slides)(bib)
  Surrogate Benchmarks for Hyperparameter Optimization
  In: ECAI workshop on Metalearning and Algorithm Selection (MetaSel)
  Superseeded by the AAAI15 paper Efficient Benchmarking of Hyperparameter Optimizers via Surrogates
  
• Matthias Feurer and Tobias Springenberg and Frank Hutter (pdf)(slides)(bib)
  Using Meta-Learning to Initialize Bayesian Optimization of Hyperparameters
  In: ECAI workshop on Metalearning and Algorithm Selection (MetaSel)
  Superseeded by the AAAI15 paper Initializing Bayesian Hyperparameter Optimization via Meta-Learning
  
• Chris Fawcett, Mauro Vallati, Frank Hutter, Jörg Hoffmann, Holger Hoos and Kevin Leyton-Brown (pdf)(bib)
  Improved Features for Runtime Prediction of Domain-Independent Planners
  In: Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS 2014)
  
• Frank Hutter and Holger Hoos and Kevin Leyton-Brown (pdf)(longversion)(bib)
  An Efficient Approach for Assessing Hyperparameter Importance
  In: Proceedings of International Conference on Machine Learning 2014 (ICML 2014)
  
• Domhan, Tobias and Springenberg, Tobias and Hutter, Frank (pdf)(poster)(bib)
  Extrapolating Learning Curves of Deep Neural Networks
  In: ICML 2014 AutoML Workshop
  
• Leyton-Brown, Kevin and Hoos, Holger and Hutter, Frank and Xu, Lin (preprint)(published)(bib)
  Understanding the Empirical Hardness of NP-complete Problems
  In: Communications of the Association for Computing Machinery (CACM) 57.5 (2014): 98--107
  
• Daniel Geschwender, Frank Hutter, Lars Kotthoff, Yuri Malitsky, Holger Hoos and Kevin Leyton-Brown (pdf)(slides)(bib)
  Algorithm Configuration in the Cloud: A Feasibility Study
  In: Proceedings of the Learning and Intelligent OptimizatioN Conference (LION 8)
  
• Frank Hutter, Manuel López-Ibáñez, Chris Fawcett, Marius Lindauer, Holger Hoos, Kevin Leyton-Brown and Thomas Stützle (pdf)(slides)(bib)
  AClib: a Benchmark Library for Algorithm Configuration
  In: Proceedings of the Learning and Intelligent OptimizatioN Conference (LION 8)
  
• Frank Hutter and Lin Xu and Holger H. Hoos and Kevin Leyton-Brown (arXiv)(published)(bib)
  Algorithm runtime prediction: Methods & evaluation
  In: Artificial Intelligence 206.0 (2014): 79--111
  The data and source code for this paper are available from our Empirical Performance Models project page.
  

2013

• Swersky, Kevin and Duvenaud, David and Snoek, Jasper and Hutter, Frank and Osborne, Michael (pdf)(bib)
  Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces
  In: NIPS workshop on Bayesian Optimization in Theory and Practice
  
• Katharina Eggensperger and Matthias Feurer and Frank Hutter and James Bergstra and Jasper Snoek and Holger Hoos and Kevin Leyton-Brown (pdf)(poster)(bib)
  Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters
  In: NIPS workshop on Bayesian Optimization in Theory and Practice
  Software and benchmarks are available from our HPOlib website.
  
• Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin (pdf)(poster)(bib)
  An Efficient Approach for Assessing Parameter Importance in Bayesian Optimization
  In: NIPS workshop on Bayesian Optimization in Theory and Practice
  
• Ziyu Wang and Masrour Zoghi and Frank Hutter and David Matheson and Nando de Freitas (arXiv)(pdf)(bib)
  Bayesian Optimization in High Dimensions via Random Embeddings
  In: Proceedings of the 23rd international joint conference on Artificial Intelligence (IJCAI)
  Distinguished paper award.
  
• Chris Thornton and Frank Hutter and Holger H. Hoos and Kevin Leyton-Brown (pdf)(bib)
  Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms
  In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'13)
  The software is available from our Auto-WEKA page.
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. (pdf)(bib)
  An Evaluation of Sequential Model-Based Optimization for Expensive Blackbox Functions
  In: Proceedings of GECCO-13 Workshop on Blackbox Optimization Benchmarking (BBOB'13)
  Software and data are available from the SMAC page.
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. (pdf)(bib)
  Identifying Key Algorithm Parameters and Instance Features using Forward Selection
  In: Proceedings of the 7th International Conference on Learning and Optimization (LION-7)
  The data and source code for this paper are available from our Empirical Performance Models project page.
  

2012

• Lin Xu and Frank Hutter and Jonatahn Shen and Holger Hoos and Kevin Leyton-Brown (pdf)(bib)
  SATzilla2012: Improved Algorithm Selection Based on Cost-sensitive Classification Models
  In: Proceedings of SAT Challenge 2012: Solver and Benchmark Descriptions
  Solver description for the 2012 SAT challenge. SATzilla2012 won 3 out of the 4 categories for which it was eligible, and placed 2nd in the remaining one. Details: it won the sequential portfolio track, was the best solver for 2 of the 3 main sequential categories (Application and Hard Combinatorial), and 2nd in the sequential Random Category (beaten only by a new non-portfolio solver, CCASAT). See the SATzilla project page for details on SATzilla and source code.
  
• Xu, Lin and Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin (pdf)(bib)
  Evaluating Component Solver Contributions to Portfolio-Based Algorithm Selectors
  In: International Conference on Theory and Applications of Satisfiability Testing (SAT'12)
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. (pdf)(pptx)(bib)
  Parallel Algorithm Configuration
  In: Proceedings of the Learning and Intelligent OptimizatioN Conference LION 6
  

2011

• Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin (pdf)(bib)
  Bayesian Optimization With Censored Response Data
  In: NIPS workshop on Bayesian Optimization, Sequential Experimental Design, and Bandits
  Published online. There is also a new, extended arXiv version.
  
• Xu, Lin and Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin (pdf)(bib)
  Hydra-MIP: Automated Algorithm Configuration and Selection for Mixed Integer Programming
  In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI)
  
• Xu, Lin and Hutter, Frank and Hoos, Holger and Leyton-Brown, Kevin (pdf)(bib)
  Detailed SATzilla Results from the Data Analysis Track of the 2011 SAT Competition
  SAT 2011 Competition, Data Analysis Track, 2011.
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. (pdf)(slides)(bib)
  Sequential Model-Based Optimization for General Algorithm Configuration
  In: Proceedings of the conference on Learning and Intelligent OptimizatioN (LION 5)
  Best paper award (second prize).
  SMAC, ROAR, and the instances used are available from the Automated Algorithm Configuration project page.
  An extended version with additional details is available as UBC tech report TR-2010-10. (pdf)(bib)
  

2010

• Hutter, F. and Bartz-Beielstein, T. and Hoos, H.H. and Leyton-Brown, K. and Murphy, K.P. (pdf)(bib)
  Sequential Model-Based Parameter Optimisation: an Experimental Investigation of Automated and Interactive Approaches
  In: Empirical Methods for the Analysis of Optimization Algorithms
  
• Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin (pdf)(bib)
  Tradeoffs in the Empirical Evaluation of Competing Algorithm Designs
  In: Annals of Mathematics and Artificial Intelligenc (AMAI), Special Issue on Learning and Intelligent Optimization 60.1 (2010): 65--89
  The data from this paper, as well as the empirical analysis tools we introduced are available from the Automated Algorithm Configuration project page.
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. (pdf)(slides)(bib)
  Automated Configuration of Mixed Integer Programming Solvers
  In: Proceedings of the Conference on Integration of Artificial Intelligence and Operations Research techniques in Constraint Programming (CPAIOR)
  Our webpage on Automated Configuration of MIP solvers also gives the parameter files for CPLEX, Gurobi, and lpsolve.
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. and Murphy, K. P. (pdf)(slides)(bib)
  Time-Bounded Sequential Parameter Optimization
  In: Proceedings of the conference on Learning and Intelligent OptimizatioN (LION 4)
  Runner-up for the best paper award
  

2009

• Frank Hutter (pdf)(slides)(bib)
  Automated Configuration of Algorithms for Solving Hard Computational Problems
  PhD thesis, University of British Columbia, Department of Computer Science, Vancouver, Canada
  There are also slides from invited presentation at Canadian AI grad student symposium.
  2010 CAIAC Doctoral Dissertation Award for the best thesis in Artificial Intelligence at a Canadian University completed in 2009.
  See the Automated Algorithm Configuration project page for a lot of experimental data (target algorithms, parameters, benchmark instances, and configuration proceduers).
  
• Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin and Stützle, Thomas (pdf)(bib)
  ParamILS: An Automatic Algorithm Configuration Framework
  In: Journal of Artificial Intelligence Research 36 (2009): 267--306
  See the ParamILS project page for a lot of experimental data for this paper (target algorithms, parameters, resulting parameter configurations).
  There's also a quick start guide available to help you apply ParamILS for tuning your own algorithms.
  There's also an older tech report about ParamILS, including additional material (pdf)(bib).
  
• Hutter, F. and de Oca, M. A. Montes (pdf)(bib)
  SLS-DS 2009: Doctoral Symposium on Engineering Stochastic Local Search Algorithms
  IRIDIA, Université Libre de Bruxelles, Brussels, Belgium
  
• Hutter, F. and Hoos, H. H. and Leyton-Brown, K. and Murphy, K. P. (pdf)(bib)
  An Experimental Investigation of Model-Based Parameter Optimisation: SPO and Beyond
  In: Proceedings of the 11th annual conference on Genetic and evolutionary computation (GECCO '09)
  
• Lin Xu and Frank Hutter and Holger Hoos and Kevin Leyton-Brown (pdf)(bib)
  SATzilla2009: an Automatic Algorithm Portfolio for SAT
  Solver description, SAT competition 2009
  Solver description for the 2009 SAT competition. SATzilla2009 won 3 gold and 2 silver medals in that competition. See the SATzilla project page for details and source code.
  

2008

• Xu, Lin and Hutter, Frank and Hoos, Holger H. and Leyton-Brown, Kevin (pdf)(bib)
  SATzilla: Portfolio-based Algorithm Selection for SAT
  In: Journal of Artificial Intelligence Research 32 (2008): 565--606
  2010 IJCAI/JAIR Best Paper Prize for the period 2005-2009.
  See the SATzilla project page for details and source code.
  

2007

• Hutter, Frank and Babic, Domagoj and Hoos, Holger H. and Hu, Alan J. (pdf)(bib)
  Boosting Verification by Automatic Tuning of Decision Proceedingsdures
  In: Proceedings of Formal Methods in Computer Aided Design (FMCAD'07)
  With the tuning discussed in this paper Domagoj's solver Spear won the QF_BV (Quantifier-Free Bit Vector) category of the 2007 Satisfiability Modulo Theories Competition.
  
• Hutter, Frank (pdf)(bib)
  On the Potential of Automatic Algorithm Configuration
  In: Proceedings of the Doctoral Symposium on Engineering Stochastic Local Search Algorithms (SLS-DS).
  Best poster award (voted by the attendees of SLS 07).
  
• Lin Xu and Frank Hutter and Holger H. Hoos and Kevin Leyton-Brown (pdf)(bib)
  SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT
  In: Principles and Practice of Constraint Programming (CP'07)
  SATzilla won 3 gold medals, 1 silver and 1 bronze in the 2007 SAT competition! It is available for download from the SATzilla website.
  
• Frank Hutter and Holger H. Hoos and Thomas Stützle (pdf)(slides)(bib)
  Automatic Algorithm Configuration based on Local Search
  In: Proceedings of the Twenty-Second Conference on Artifical Intelligence (AAAI '07)
  The ParamILS algorithm introduced in this paper is available for download from the . There's also a quick start guide available to help you apply it for tuning your own algorithms.
  
• Tompkins, Dave and Hutter, Frank and and Hoos, Holger H. (pdf)(bib)
  Scaling and Probabilistic Smoothing (SAPS)
  Solver description from the SAT competition 2007
  SAPS is unchanged from last year, but I got a tenfold speedup by automated parameter tuning (using the techniques from the AAAI-07 ParamILS paper)
  
• Xu, Lin and Hutter, Frank and Hoos, Holger H. and Leyton-brown, Kevin (pdf)(bib)
  SATzilla2007: a new & improved algorithm portfolio for SAT
  Solver description from the SAT competition 2007
  In a nutshell, SATzilla predicts the runtime of each solver in the portfolio and picks the most promising one.
  SATzilla2007 won 3 gold medals, 1 silver and 1 bronze! See the SAT competition webpage for details.
  
• Domagoj Babić and Frank Hutter (pdf)(bib)
  SPEAR Theorem Prover
  Solver description from the SAT competition 2007
  SPEAR is a new tree search algorithm with 25 free parameters. I tuned it (using the techniques from the AAAI-07 paper on ParamILS), getting a 30% speedup; for software verification, my parameter settings beat the default by a factor of 50!
  

2006

• Frank Hutter and Youssef Hamadi and Holger H. Hoos and Kevin Leyton-Brown (pdf)(ppt)(bib)
  Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms
  In: Principles and Practice of Constraint Programming (CP'06)
  All our experimental data for this paper, as well as our Matlab code, is available on the Empirical Hardness Models project page.
  
• Hutter, Frank and Ruml, Wheeler (bib)
  Learning for Search: Papers from the 2006 AAAI Workshop
  Proceedings of the AAAI 06 workshop on Learning for Search
  
• Hutter, Frank (bib)
  Automated Algorithm Configuration Based on Machine Learning
  PhD proposal, Department of Computer Science, University of British Columbia
  

2005

• Hutter, Frank and Hamadi, Youssef (pdf)(bib)
  Parameter Adjustment Based on Performance Prediction: Towards an Instance-Aware Problem Solver
  Techreport MSR-TR-2005-125, Microsoft Research
  Slides from a talk I gave at the Cork Constraint Computation Centre (4C)
  Slides from a talk I gave in the Lab for Computational Intelligence at UBC
  
• Hutter, Frank and Hoos, Holger H. and Stützle, Thomas (pdf)(ppt)(bib)
  Efficient Stochastic Local Search for MPE Solving
  In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI-05)
  My solver GLS+ and the test instances we used are available on our MPE page. The solver can read general factor graphs, i.e. Bayes nets (in BNT format), MRFs, CRFs, etc. There's also a nice Matlab interface.
  

2004

• Hutter, Frank (pdf)(bib)
  Stochastic Local Search for Solving the Most Probable Explanation Problem in Bayesian Networks
  MSc thesis, Darmstadt University of Technoloy
  Supervisor: Thomas Stützle, Cosupervisor: Holger Hoos; My solver GLS+ and most of the test instances I used are available on our MPE page.
  
• Hutter, Frank and Ng, Brenda and Dearden, Richard (pdf)(bib)
  Incremental Thin Junction Trees for Dynamic Bayesian Networks
  Techreport TR-AIDA-04-01, Intellectics Group, Darmstadt University of Technology
  
• de Freitas, Nando and Dearden, Richard and Hutter, Frank and Morales-Menendez, Ruben and Mutch, Jim and Poole, David (pdf)(bib)
  Diagnosis by a Waiter and a Mars Explorer
  In: Proceedings of the IEEE 92.4 (2004): 139-144
  Check out my GPF webpage for the particle filtering code used for the rover examples.
  
• Dearden, Richard and Willeke, Thomas and Hutter, Frank and Simmons, Reid and Verma, Vandi and Thrun, Sebastian (pdf)(bib)
  Real-time Fault Detection and Situational Awareness for Rovers: Report on the Mars Technology Program Task
  In: In Proceedings of IEEE Aerospace Conference, 2004
  Check out my GPF webpage for the particle filtering code.
  
• Andronescu, M. and Fejes, A. P. and Hutter, F. and Hoos, H. H. and Condon, A. (pdf)(bib)
  A new algorithm for RNA secondary structure design
  In: Journal of Molecular Biology 336.3 (2004): 607--624
  Check out the free RNA Designer Software at http://www.rnasoft.ca/
  

2003

• Hutter, Frank and Dearden, Richard (pdf)(bib)
  The Gaussian Particle Filter for Diagnosis of Non-Linear Systems
  In: Proceedings of the 14th International Conference on Principles of Diagnosis(DX03)
  Check out my GPF webpage for the Gaussian particle filtering code.
  
• Hutter, Frank and Dearden, Richard (pdf)(bib)
  Efficient On-line Fault Diagnosis for Non-Linear Systems
  In: Seventh International Symposium on Artificial Intelligence and Robotics in Space (i-SAIRAS-03)
  Check out my GPF webpage for the Gaussian particle filtering code.
  

2002

• Frank Hutter and Dave A.D. Tompkins and Holger H. Hoos (pdf)(bib)
  Scaling and Probabilistic Smoothing: Efficient Dynamic Local Search for SAT
  In: Principles and Practice of Constraint Programming - CP 2002
  Check out the DLS for SAT webpage, maintained by Dave.
  
• Andronescu, M. and Fejes, A. P. and Hutter, F. and Hoos, H. H. and Condon, A. (bib)
  A New SLS Algorithm for RNA Secondary Structure Design
  Techreport TR-2002-10, Department of Computer Science, University of British Columbia
  Available as a postscript file.
  Check out the free RNA Designer Software at http://www.rnasoft.ca/