Energy optimization is gaining real momentum due to growing environmental consciousness and increasing pressure on energy infrastructure. Several factors contribute, including rising electricity demand from electric vehicles (EVs), expanding data center capacity, and the growing integration of renewable energy sources, which together are increasing system complexity, energy consumption, and strain on power grids.
Mathematical optimization plays a key role in solving the underlying business problems.
Key results
- Default MOSEK is, on average (using shifted geometric mean), about 2.5-4 times faster than the selected default open-source solver.
- Running MOSEK’s interior-point method in a multi-threaded setting yields speed-ups ranging from 15% (2 threads) to 37% (8 threads) compared to single-threaded performance.
- When solving large energy models, memory may be of concern. Reducing from 8 threads to 2 (or 1) threads helps reduce peak memory consumption by 20-40%.
- If you are mostly interested in the solution value and don’t require a basic solution, turning basis identification (a.k.a. crossover) off yields, on average, 10% runtime improvement.
Methodology and computational setup
Methodology:
- We use LP instances from https://openenergybenchmark.org/ available on the GitHub page of Open Energy Transition (“OET”).
- We focus on the 70 medium (M)- sized instances and omit large (L) instances due to the high runtime of open-source solvers. Reporting on too many unsolved large instances is not meaningful. We also decided to limit our runs to 1 hour, which is reasonable for the M instances.
- We compare MOSEK against itself to provide insights into its workings and to demonstrate the computational trade-offs involved when running with different settings.
- We further compare MOSEK version 11.2 with the open-source solver HiGHS in version 1.14. This is to provide readers with an anchor for the publicly available benchmarks
- For performance comparison, we use the shifted geometric mean (GMST):
, where s=10, to follow the standard mean as used by Mittelmann (https://plato.asu.edu/ftp/shgeom.html) and OET (https://openenergybenchmark.org/methodology#ranking-solvers ), and calculate the ratios between the shifted geometric means in column GMRST.
- A solver “wins” (#WINs) when it is not more than 5% slower than the fastest.
Computational set-up:
- Machine set-up: We use a single machine of type Supermicro AS-1115HS-TNR-EU – AMD EPYC 9135, 16 cores, 3.65 GHz, 377 GB DDR5-6400 ECC RDIMM memory
- We use MOSEK default options. No tuning of parameters.
- We use a time limit of 1h per instance.
- We limit each run to 8 threads; any variations for specific experiments are noted below.
- We executed the solves sequentially, so purely system processes may interfere, which we expect not to accumulate to more than 1% in measurement.
MOSEK versus open-source
For comparison with the results on openenergybenchmarks.com, we include runtime results for HiGHS, an open-source solver. MOSEK is run with the interior-point method, while we tested two runs for HiGHS: the dual simplex method (“DSIM”) and the new multi-threaded, factorization-based interior-point solver HiPO (“HIPO”).
From the table and graphics below, we can see that for small instances, HiGHS performs reasonably well, whereas with increasing complexity, using a commercial solver such as MOSEK quickly makes a significant difference. Observe that 18 instances could not be solved by HiGHS within the time limit, such that the shifted geometric mean is only calculated over 52 of 70 instances. With respect to the shifted geometric mean, HiGHS with HiPO is about 2.5 times (about 4 times with dual simplex) slower than MOSEK. The performance profiles reveal the distance between HiGHS and MOSEK, and further show that HiGHS’ HIPO is usually stronger than HiGHS’ dual simplex on this set of instances. We kept both HiGHS runs mainly because for about 20% of instances, their dual simplex was faster than using HiPO.
Looking at all 68 instances that HiGHS with HiPO could solve within the time limit, the ratio GMRST is about 2.9.
Note that on our fast machine, several runtimes fall into the 5-20-second bucket, and the selected 10-second shift affects the overall ratio. E.g., when setting the shift to s=1 in the shifted geometric mean or similarly when using a less powerful machine (which we initially did), the ratio GMRST between HiGHS and MOSEK increases. When you read about any performance ratios in public benchmarks , it is important to review the instance sets and the calculation logic.
Limited overhead to retrieve basic solutions with MOSEK!
When running primal or dual simplex, you get a basic solution, whereas with an interior-point solver, you need to perform basis identification (a.k.a. crossover) to obtain one. The first question to ask yourself is, do you need a basic solution?
You need it, for example, to perform some types of sensitivity analysis (changes in right-hand side or cost coefficients before basis changes,...), to solve mixed-integer optimization problems, or to obtain a sparse solution. If you are mainly interested in the objective value and need to save time, this step is best skipped, as it is often a computational bottleneck when solving large LPs. Here, we ran an experiment to investigate the impact of the basis identification step on the overall runtime.
Our tests reveal that, on average, only 10% of the time is spent on basis identification, demonstrating its limited impact on overall solving time for MOSEK. In our experience, several solvers tend to spend a lot of time on basis identification (a.k.a. crossover), whereas MOSEK already tends to provide numerically favorable solutions, so retrieving a basic solution does not take long; see the paper by E.D. Andersen “On exploiting problem structure in a basis identification procedure for linear programming” in INFORMS Journal on Computing (https://pubsonline.informs.org/doi/10.1287/mnsc.42.12.1719 ) for more details.
Yet, there are a few outliers in which, for one instance, basis identification took more than 90% of the total runtime. Also, there is one instance where the interior-point method, which ran without basis identification, did not converge and was excluded. So, while it is negligible on average, there are outliers where it may make sense to turn it off.
Threads
Some users have powerful machines and want to run large models one after another, while others may want to solve many problems in parallel. We compare MOSEK running on 1 (“_T1”), 2, 4, and 8 (“_T8”) threads.
The overall runtime improvement on 8 threads is about 37% compared to a single thread. Observe that one instance could not be solved with too few threads; hence, shifted geometric means are calculated over 69 of 70 instances.
Memory comparison
Based on the results above and after consultation with energy modeling experts, we created a separate benchmark set containing the realistic PyPSA electricity models from the OET benchmark set to measure memory consumption. Memory measurements in a multi-threaded setup are non-deterministic and should only be used as indicative. From the data, we see that switching from 1 thread (M_DEB1) to 2 threads (M_DEB2) increases the memory by less than 10%, whereas switching to 8 threads (M_DEB) increases it by up to 40%.
Hence, if you are solving large instances and your machine has limited memory, it may be worth running with fewer threads.
Summary
This study provides an indication of MOSEK's strengths for linear optimization problems, in this case from energy applications, and identifies which user controls are worth investigating. We hope this study provides a few anchors to look into and to investigate for your own assessment!
Clearly, results are tied to the instances and computational setup chosen. We always recommend that you check the performance for your own set of instances, and in the environment you are running them - don’t use stronger or much weaker machines for the evaluation; test on machines comparable to your production environment.
Fair benchmarking is challenging - any benchmark requires trade-offs between, for example, representativeness, computational effort, and the practical constraints of available time and resources. We therefore chose to focus on medium-sized instances from the OET benchmark set. These instances are sufficiently challenging to produce meaningful runtime comparisons, yet small enough that open-source solvers can solve a substantial fraction of the test set without excessive timeouts within the imposed time limit. While no benchmark can perfectly capture all real-world scenarios, we believe this selection provides a balanced and informative basis for comparison.
We encourage you to evaluate MOSEK on the instances that matter to you, with your chosen time limits, be it 10 minutes or 24 hours, and on the machines available to you!
Acknowledgement
We are grateful to the Open Energy Transition for making the Open Energy Benchmark publicly available! You can simply run your own benchmarks on your hardware using their scripts (https://github.com/open-energy-transition/solver-benchmark).
