Submit problems and corrections on GitHub with pull requests / issues, or by email: koen.van.der.blom@cwi.nl
| name | textual description | suite/generator/single | objectives | dimensionality | variable type | constraints | dynamic | noise | multi-fidelity | source (real-world/artificial) | reference | implementation |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BBOB | suite | 1 | scalable | continuous | no | no | no | no | https://doi.org/10.1080/10556788.2020.1808977 | https://github.com/numbbo/coco | ||
| BBOB-biobj | suite | 2 | 2-40 | continuous | no | no | no | no | https://doi.org/10.48550/arXiv.1604.00359 | https://github.com/numbbo/coco | ||
| BBOB-noisy | suite | 1 | scalable | continuous | no | no | yes | no | https://hal.inria.fr/inria-00369466 | https://web.archive.org/web/20210416065610/https://coco.gforge.inria.fr/doku.php?id=downloads | ||
| BBOB-largescale | suite | 1 | 20-640 | continuous | no | no | no | no | https://doi.org/10.48550/arXiv.1903.06396 | https://github.com/numbbo/coco | ||
| BBOB-mixint | suite | 1 | 5-160 | integer;continuous;mixed | no | no | no | no | https://doi.org/10.1145/3321707.3321868 | https://github.com/numbbo/coco | ||
| BBOB-biobj-mixint | suite | 2 | 5-160 | integer;continuous;mixed | no | no | no | no | https://doi.org/10.1145/3321707.3321868 | https://github.com/numbbo/coco | ||
| BBOB-constrained | suite | 1 | 2-40 | continuous | yes | no | no | no | http://numbbo.github.io/coco-doc/bbob-constrained/ | https://github.com/numbbo/coco | ||
| MOrepo | suite | 2 | ? | combinatorial | ? | ? | ? | no | https://github.com/MCDMSociety/MOrepo | |||
| ZDT | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1162/106365600568202 | https://github.com/anyoptimization/pymoo | ||
| DTLZ | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/CEC.2002.1007032 | https://pymoo.org/problems/many/dtlz.html | ||
| WFG | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2005.861417 | https://pymoo.org/problems/many/wfg.html | ||
| CDMP | suite | 2+ | scalable | continuous | yes | ? | ? | no | https://doi.org/10.1145/3321707.3321878 | ? | ||
| SDP | suite | 2+ | scalable | continuous | no | yes | ? | no | https://doi.org/10.1109/TCYB.2019.2896021 | ? | ||
| MaOP | suite | 2+ | scalable | continuous | no | no | ? | no | https://doi.org/10.1016/j.swevo.2019.02.003 | ? | ||
| BP | suite | 2+ | scalable | continuous | no | no | ? | no | https://doi.org/10.1109/CEC.2019.8790277 | ? | ||
| GPD | generator | 2+ | scalable | continuous | optional | no | optional | no | https://doi.org/10.1016/j.asoc.2020.106139 | ? | ||
| ETMOF | suite | 2-50 | 25-10000 | continuous | no | yes | no | no | https://doi.org/10.48550/arXiv.2110.08033 | https://github.com/songbai-liu/etmo | ||
| MMOPP | suite | 2-7 | ? | ? | yes | no | no | no | http://www5.zzu.edu.cn/system/_content/download.jsp?urltype=news.DownloadAttachUrl&owner=1327567121&wbfileid=4764412 | http://www5.zzu.edu.cn/ecilab/info/1036/1251.htm | ||
| CFD | expensive evaluations 30s-15m | suite | 1-2 | scalable | ? | yes | no | no | no | real world | https://doi.org/10.1007/978-3-319-99259-4_24 | https://bitbucket.org/arahat/cfd-test-problem-suite |
| GBEA | expensive evaluations 5s-35s | suite | 1-2 | scalable | continuous | no | no | yes | no | real world | https://doi.org/10.1145/3321707.3321805 | ? |
| Car structure | 54 constraints | suite | 2 | 144-222 | discrete | yes | no | no | no | real world | https://doi.org/10.1145/3205651.3205702 | http://ladse.eng.isas.jaxa.jp/benchmark/ |
| EMO2017 | suite | 2 | 4-24 | continuous | no | no | no | no | real world | https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/ | https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/downloads/realworld-problems-bbcomp-EMO-2017.zip | |
| JSEC2019 | expensive evaluations 3s; 22 constraints | single | 1-5 | 32 | continuous | yes | no | no | no | real world | http://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html | http://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html |
| RE | suite | 2-9 | 2-7 | continuous;integer;mixed | no | no | no | no | real world like | https://doi.org/10.1016/j.asoc.2020.106078 | https://github.com/ryojitanabe/reproblems | |
| CRE | suite | 2-5 | 3-7 | continuous;integer;mixed | yes | no | no | no | real world like | https://doi.org/10.1016/j.asoc.2020.106078 | https://github.com/ryojitanabe/reproblems | |
| Radar waveform | single | 9 | 4-12 | integer | yes | no | no | no | real world | https://doi.org/10.1007/978-3-540-70928-2_53 | http://code.evanhughes.org/ | |
| MF2 | suite | 1 | 1-n | continuous | no | no | no | yes | https://doi.org/10.21105/joss.02049 | https://github.com/sjvrijn/mf2 | ||
| AMVOP | suite | 1 | scalable | mixed continuous+ordinal+categorical+both | no | no | no | no | https://doi.org/10.1109/TEVC.2013.2281531 | ? | ||
| RWMVOP | suite | 1 | scalable | continuous;mixed continuous+ordinal+categorical+both | yes | no | no | no | real world | https://doi.org/10.1109/TEVC.2013.2281531 | ? | |
| SBOX-COST | problems from BBOB but allows instances with the optimum close to the boundary | suite | 1 | scalable | continuous | no | no | no | no | https://doi.org/10.48550/arXiv.2305.12221 | https://github.com/IOHprofiler/IOHexperimenter/ | |
| ρMNK-Landscapes | tunable variable and objective dimensions; tunable multimodality and correlation between objectives | generator | scalable | scalable | binary | no | no | no | no | https://doi.org/10.1016/j.ejor.2012.12.019 | https://gitlab.com/aliefooghe/mocobench/ | |
| mUBQP | tunable variable and objective dimensions; tunable density and correlation between objectives | generator | scalable | scalable | binary | no | no | no | no | https://doi.org/10.1016/j.asoc.2013.11.008 | https://gitlab.com/aliefooghe/mocobench/ | |
| ρmTSP | tunable variable and objective dimensions; tunable instance type (euclidian/random); tunable correlation between objectives | generator | scalable | scalable | permutations | no (apart from being permutations) | no | no | no | https://doi.org/10.1007/978-3-319-45823-6_40 | https://gitlab.com/aliefooghe/mocobench/ | |
| CEC2015-DMOO | suite | 2-3 | ? | continuous | ? | yes | no | no | Benchmark Functions for CEC 2015 Special Session and Competition on Dynamic Multi-objective Optimization | |||
| Ealain | Real-world-like, easily extensible to increase complexity | generator | 1+ | scalable | continuous,binary,integer | optional | optional | no | optional | Real-world-like | https://doi.org/10.1145/3638530.3654299 | https://github.com/qrenau/Ealain |
| MA-BBOB | Generator that creates affine combinations of BBOB functions | generator | 1 | scalable | continuous | no | no | no | no | artificial | https://doi.org/10.1145/3673908 | https://github.com/IOHprofiler/IOHexperimenter/blob/master/example/Competitions/MA-BBOB/Example_MABBOB.ipynb |
| MPM2 | nonlinear nonseparable nonsymmetric; scalable in terms of time to evaluate the objective function | generator | 1 | scalable | continuous | no | no | no | no | https://ls11-www.cs.tu-dortmund.de/_media/techreports/tr15-01.pdf | https://github.com/jakobbossek/smoof/blob/master/inst/mpm2.py | |
| Convex DTLZ2 | Variant of DTLZ2 with a convex Pareto front (instead of concave) | single | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2013.2281535 | ? | |
| Inverted DTLZ1 | Variant of DTLZ1 with an inverted Pareto front | single | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2013.2281534 | ? | |
| Minus DTLZ | Variant of DTLZ that minimises the inverse of the base DTLZ functions | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2016.2587749 | ? | |
| Minus WFG | Variant of WFG that minimises the inverse of the base WFG functions | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2016.2587749 | ? | |
| L1-ZDT | Variant of ZDT with linkages between variables within one of two groups but not between variables in a different group; Linear recombination operators can potentially take advantage of the problem structure | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L2-ZDT | Variant of ZDT with linkages between all variables; Linear recombination operators can potentially take advantage of the problem structure | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L3-ZDT | Variant of L2-ZDT using a mapping to prevent linear recombination operators from potentially taking advantage of the problem structure | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L2-DTLZ | Variant of DTLZ2 and DTLZ3 with linkages between all variables; Linear recombination operators can potentially take advantage of the problem structure | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L3-DTLZ | Variant of L2-DTLZ using a mapping to prevent linear recombination operators from potentially taking advantage of the problem structure | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| CEC2018 DT - CEC2018 Competition on Dynamic Multiobjective Optimisation | 14 problems. Time-dependent: Pareto front/Pareto set geometry; irregular Pareto front shapes; variable-linkage; number of disconnected Pareto front segments; etc. | suite | 2 or 3 | scalable? | ? | no | yes | no | no | artificial | https://www.academia.edu/download/94499025/TR-CEC2018-DMOP-Competition.pdf | https://pymoo.org/problems/dynamic/df.html |
| MODAct - multiobjective design of actuators | Realistic Constrained Multi-Objective Optimization Benchmark Problems from Design. Need the https://github.com/epfl-lamd/modact package installed; evaluation times around 20ms | suite | 2 3 4 or 5 | 20 | mixed; integer and continuous | yes | no | no | no | real-world | https://doi.org/10.1109/TEVC.2020.3020046 | https://pymoo.org/problems/constrained/modact.html |
| IOHClustering | Set of benchmark problems from clustering: optimization task is selecting cluster centers for a given set of data, with the number of clusters defining problem dimensionality. Includes both a suite and a generator. Based on ML clustering datasets | suite; generator | 1 | scalable | continuous | no | no | no | no | artificial, but based on real data | https://arxiv.org/pdf/2505.09233 | https://github.com/IOHprofiler/IOHClustering |
| GNBG-II | Generalized Numerical Benchmark Generator (version 2). Also in IOH https://github.com/IOHprofiler/IOHGNBG | suite; generator | 1 | scalable | continuous | no | no | no | no | artificial | https://dl.acm.org/doi/pdf/10.1145/3712255.3734271 | https://github.com/rohitsalgotra/GNBG-II |
| GNBG | Generalized Numerical Benchmark Generator | suite; generator | 1 | scalable | continuous | no | no | no | no | artificial | https://arxiv.org/abs/2312.07083 | https://github.com/Danial-Yazdani/GNBG-Generator |
| DynamicBinVal | Four versions of the dynamic binary value problem | suite | 1 | scalable | binary | no | yes | no | no | artificial | https://arxiv.org/pdf/2404.15837 | https://github.com/IOHprofiler/IOHexperimenter |
| PBO | Suite of 25 binary optimization problems | suite | 1 | scalable | binary | no | no | no | no | artificial | https://dl.acm.org/doi/pdf/10.1145/3319619.3326810 | https://github.com/IOHprofiler/IOHexperimenter |
| W-model | Tunable generator for binary optimization based on several difficulty features | generator | 1 | scalable | binary | no | no | no | no | artificial | https://dl.acm.org/doi/abs/10.1145/3205651.3208240?casa_token=S4U_Pi9f6MwAAAAA:U9ztNTPwmupT8K3GamWZfBL7-8fqjxPtr_kprv51vdwA-REsp0EyOFGa99BtbANb0XbqyrVg795hIw | https://github.com/thomasWeise/BBDOB_W_Model |
| Submodular Optimitzation | set of graph-based submodular optimization problems from 4 problem types | suite | 1 | scalable | binary | no | no | no | no | artificial | https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10254181 | https://github.com/IOHprofiler/IOHexperimenter |
| CEC2013 | suite used for cec2013 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimenter | suite | 1 | scalable | continuous | no | no | no | no | artificial | https://peerj.com/articles/cs-2671/CEC2013.pdf | https://github.com/P-N-Suganthan/CEC2013 |
| CEC2022 | suite used for cec2022 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimenter | suite | 1 | scalable | continuous | no | no | no | no | artificial | https://github.com/P-N-Suganthan/2022-SO-BO/blob/main/CEC2022%20TR.pdf | https://github.com/P-N-Suganthan/2022-SO-BO |
| Onemax+Sphere / Zeromax+Sphere | single | 2 | scalable | binary and continuous;mixed; | no | no | no | no | artificial | https://doi.org/10.1145/3449726.3459521 | None | |
| Onemax+Sphere / DeceptiveTrap+RotatedEllipsoid | single | 2 | scalable | binary and continuous;mixed; | no | no | no | no | artificial | https://doi.org/10.1145/3449726.3459521 | None | |
| InverseDeceptiveTrap+RotatedEllipsoid / DeceptiveTrap+RotatedEllipsoid | single | 2 | scalable | binary and continuous;mixed; | no | no | no | no | artificial | https://doi.org/10.1145/3449726.3459521 | None | |
| name | textual description | suite/generator/single | objectives | dimensionality | variable type | constraints | dynamic | noise | multi-fidelity | source (real-world/artificial) | reference | implementation |