Submit problems and corrections on GitHub with pull requests / issues, or by email: koen.van.der.blom@cwi.nl

name suite/generator/single objectives dimensionality variable type constraints dynamic noise multimodal multi-fidelity other info reference implementation
BBOB suite 1 scalable continuous no no no yes no https://doi.org/10.1080/10556788.2020.1808977 https://github.com/numbbo/coco
BBOB-biobj suite 2 2-40 continuous no no no yes no https://doi.org/10.48550/arXiv.1604.00359 https://github.com/numbbo/coco
BBOB-noisy suite 1 scalable continuous no no yes yes no https://hal.inria.fr/inria-00369466 https://web.archive.org/web/20210416065610/https://coco.gforge.inria.fr/doku.php?id=downloads
BBOB-largescale suite 1 20-640 continuous no no no yes no https://doi.org/10.48550/arXiv.1903.06396 https://github.com/numbbo/coco
BBOB-mixint suite 1 5-160 integer;continuous;mixed no no no yes no https://doi.org/10.1145/3321707.3321868 https://github.com/numbbo/coco
BBOB-biobj-mixint suite 2 5-160 integer;continuous;mixed no no no yes no https://doi.org/10.1145/3321707.3321868 https://github.com/numbbo/coco
BBOB-constrained suite 1 2-40 continuous yes no no yes no http://numbbo.github.io/coco-doc/bbob-constrained/ https://github.com/numbbo/coco
MOrepo suite 2 ? combinatorial ? ? ? ? no https://github.com/MCDMSociety/MOrepo
ZDT suite 2 scalable continuous;binary no ? ? ? no https://doi.org/10.1162/106365600568202 https://github.com/anyoptimization/pymoo
DTLZ suite 2+ scalable continuous no ? ? ? no https://doi.org/10.1109/CEC.2002.1007032 https://pymoo.org/problems/many/dtlz.html
WFG suite 2+ scalable continuous no ? ? ? no https://doi.org/10.1109/TEVC.2005.861417 https://pymoo.org/problems/many/wfg.html
CDMP suite 2+ scalable continuous yes ? ? ? no https://doi.org/10.1145/3321707.3321878 ?
SDP suite 2+ scalable continuous no yes ? ? no https://doi.org/10.1109/TCYB.2019.2896021 ?
MaOP suite 2+ scalable continuous no no ? ? no https://doi.org/10.1016/j.swevo.2019.02.003 ?
BP suite 2+ scalable continuous no no ? ? no https://doi.org/10.1109/CEC.2019.8790277 ?
GPD generator 2+ scalable continuous optional no optional ? no https://doi.org/10.1016/j.asoc.2020.106139 ?
ETMOF suite 2-50 25-10000 continuous no yes no ? no https://doi.org/10.48550/arXiv.2110.08033 https://github.com/songbai-liu/etmo
MMOPP suite 2-7 ? ? yes no no yes no http://www5.zzu.edu.cn/system/_content/download.jsp?urltype=news.DownloadAttachUrl&owner=1327567121&wbfileid=4764412 http://www5.zzu.edu.cn/ecilab/info/1036/1251.htm
CFD suite 1-2 scalable ? yes no no ? no expensive evaluations 30s-15m; real world https://doi.org/10.1007/978-3-319-99259-4_24 https://bitbucket.org/arahat/cfd-test-problem-suite
GBEA suite 1-2 scalable continuous no no yes ? no expensive evaluations 5s-35s; real world https://doi.org/10.1145/3321707.3321805 ?
Car structure suite 2 144-222 discrete yes no no ? no real world; 54 constraints https://doi.org/10.1145/3205651.3205702 http://ladse.eng.isas.jaxa.jp/benchmark/
EMO2017 suite 2 4-24 continuous no no no ? no real world https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/ https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/downloads/realworld-problems-bbcomp-EMO-2017.zip
JSEC2019 single 1-5 32 continuous yes no no ? no expensive evaluations 3s; real world; 22 constraints http://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html http://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html
RE suite 2-9 2-7 continuous;integer;mixed no no no ? no real world like https://doi.org/10.1016/j.asoc.2020.106078 https://github.com/ryojitanabe/reproblems
CRE suite 2-5 3-7 continuous;integer;mixed yes no no ? no real world like https://doi.org/10.1016/j.asoc.2020.106078 https://github.com/ryojitanabe/reproblems
Radar waveform single 9 4-12 integer yes no no ? no real world https://doi.org/10.1007/978-3-540-70928-2_53 http://code.evanhughes.org/
MF2 suite 1 1-n continuous no no no ? yes https://doi.org/10.21105/joss.02049 https://github.com/sjvrijn/mf2
AMVOP suite 1 scalable mixed continuous+ordinal+categorical+both no no no yes no https://doi.org/10.1109/TEVC.2013.2281531 ?
RWMVOP suite 1 scalable continuous;mixed continuous+ordinal+categorical+both yes no no ? no real world https://doi.org/10.1109/TEVC.2013.2281531 ?
SBOX-COST suite 1 scalable continuous no no no yes no problems from BBOB but allows instances with the optimum close to the boundary https://doi.org/10.48550/arXiv.2305.12221 https://github.com/IOHprofiler/IOHexperimenter/
ρMNK-Landscapes generator scalable scalable binary no no no yes no tunable variable and objective dimensions; tunable multimodality and correlation between objectives https://doi.org/10.1016/j.ejor.2012.12.019 https://gitlab.com/aliefooghe/mocobench/
mUBQP generator scalable scalable binary no no no yes (quadratic) no tunable variable and objective dimensions; tunable density and correlation between objectives https://doi.org/10.1016/j.asoc.2013.11.008 https://gitlab.com/aliefooghe/mocobench/
ρmTSP generator scalable scalable permutations no (apart from being permutations) no no yes (quadratic) no tunable variable and objective dimensions; tunable instance type (euclidian/random); tunable correlation between objectives https://doi.org/10.1007/978-3-319-45823-6_40 https://gitlab.com/aliefooghe/mocobench/
CEC2015-DMOO suite 2-3 ? continuous ? yes no ? no Benchmark Functions for CEC 2015 Special Session and Competition on Dynamic Multi-objective Optimization
Ealain generator 1+ scalable continuous,binary,integer optional optional no ? optional Real-world-like, easily extensible to increase complexity https://doi.org/10.1145/3638530.3654299 https://github.com/qrenau/Ealain
name suite/generator/single objectives dimensionality variable type constraints dynamic noise multimodal multi-fidelity other info reference implementation