REVIEW 5 cited by
IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
IOHprofiler is a new tool for analyzing and comparing iterative optimization heuristics. Given as input algorithms and problems written in C or Python, it provides as output a statistical evaluation of the algorithms' performance by means of the distribution on the fixed-target running time and the fixed-budget function values. In addition, IOHprofiler also allows to track the evolution of algorithm parameters, making our tool particularly useful for the analysis, comparison, and design of (self-)adaptive algorithms. IOHprofiler is a ready-to-use software. It consists of two parts: an experimental part, which generates the running time data, and a post-processing part, which produces the summarizing comparisons and statistical evaluations. The experimental part is build on the COCO software, which has been adjusted to cope with optimization problems that are formulated as functions $f:\mathcal{S}^n \to \R$ with $\mathcal{S}$ being a discrete alphabet of integers. The post-processing part is our own work. It can be used as a stand-alone tool for the evaluation of running time data of arbitrary benchmark problems. It accepts as input files not only the output files of IOHprofiler, but also original COCO data files. The post-processing tool is designed for an interactive evaluation, allowing the user to chose the ranges and the precision of the displayed data according to his/her needs. IOHprofiler is available on GitHub at \url{https://github.com/IOHprofiler}.
Forward citations
Cited by 5 Pith papers
-
Lost in Vagueness: Towards Context-Sensitive Standards for Robustness Assessment under the EU AI Act
Because robustness under the EU AI Act is context-sensitive, the paper argues, horizontal standards must be complemented by domain-specific specifications and a dynamic repository of assessment practices.
-
Bencher: Simple and Reproducible Benchmarking for Black-Box Optimization
Bencher isolates each benchmark in a virtual environment and exposes a unified RPC interface, supporting a large set of benchmarks for black-box optimization evaluation.
-
Evolutionary Computation and Large Language Models: A Survey of Methods, Synergies, and Applications
A survey that maps bidirectional synergies between evolutionary computation and large language models and proposes a taxonomy plus research gaps.
-
Evaluating Real-World Generalizability of Algorithm Selection Models
Cross-benchmark study tests transfer of AS models between BBOB/CEC suites and robotics/UAV problems, identifying generalization failures in realistic settings.
-
Time-Fair Benchmarking for Metaheuristics: A Restart-Fair Protocol for Fixed-Time Comparisons
A fixed-time, restart-fair benchmarking protocol for metaheuristics, using ERT and time-based performance profiles, is proposed without empirical validation.
Discussion (0). Continue with ORCID to comment.