I think it would be nice to have benchmark configuration files. My SLURM scripts have become a bit messy because we have 6 different benchmark runs, each with different instances, rounding funcs, stopping criteria, etc.
Instead, it would be convenient to have these benchmark configurations in a TOML file, so that I could run benchmarks like this:
poetry run bench solve --bench_loc cvrp.toml
The TOML configuration files could follow the args interface like this:
instances = ["instances/CVRP/*.vrp"]
round_func = "round"
max_runtime = 2.4
per_client = true
It should be possible to specify other args combined with the config:
poetry run bench solve --bench_loc cvrp.toml --seed 1 --sol_dir out/cvrp/
An error should be raised when an argument is both passed and in the configuration file.
Example TOML file:
instances = [
"instances/CVRP/X-n101-k25.vrp",
"instances/CVRP/X-n1001-k43.vrp"
]
solutions = [
"instances/CVRP/X-n101-k25.sol",
"instances/CVRP/X-n1001-k43.sol"
]
seed = 42
num_procs = 4
stats_dir = "stats"
sol_dir = "solutions"
round_func = "dimacs"
config_loc = "configs/params.toml"
max_runtime = 60.0
max_iterations = 25000
no_improvement = 5000
per_client = false
display = false
I think it would be nice to have benchmark configuration files. My SLURM scripts have become a bit messy because we have 6 different benchmark runs, each with different instances, rounding funcs, stopping criteria, etc.
Instead, it would be convenient to have these benchmark configurations in a TOML file, so that I could run benchmarks like this:
The TOML configuration files could follow the args interface like this:
It should be possible to specify other args combined with the config:
An error should be raised when an argument is both passed and in the configuration file.
Example TOML file:
bench_config_locconfig_loctosolve_params_loc