Add same-machine linopy vs polar-high build+IO benchmark#765
Closed
MaykThewessen wants to merge 1 commit into
Closed
Add same-machine linopy vs polar-high build+IO benchmark#765MaykThewessen wants to merge 1 commit into
MaykThewessen wants to merge 1 commit into
Conversation
Self-contained harness comparing linopy (io_api lp/direct) against polar-high (regular and save_memory) on the identical dense LP, one machine, one HiGHS build. Addresses the cross-hardware ambiguity in PyPSA#740: the original numbers were measured on different machines so only ratios were comparable. Result: io_api="direct" is fastest on build+IO (reverses the lp-path ranking); polar-high retains a lower peak RSS. The modelling-layer gap is a memory gap, not a speed gap.
Collaborator
|
I think polars-high is not mature enough to add it to our benchmark. But tis PR is highly apreciated as an artifact and measurement. |
Member
|
Note also that polars-high is more or less a highs only framework. For all other solvers it builds the problem and communicates to highs then dumps it into a MPS and then loads this MPS in the other solver. |
Collaborator
|
thanks @MaykThewessen , given the context and the separate benchmark refactor by @FBumann, I would close this pr, but let's reuse it later if if the topic rises again |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #740. The cross-tool numbers there were measured on different hardware, so only ratios were comparable. This adds a self-contained
benchmark/io_api_compare/harness that runs all four columns through one process-per-cell harness, one HiGHS build, one machine:linopy-lp(build ->.lpfile -> HiGHS reparse)linopy-direct(build -> in-memory highspy load)polar(polar-high regular)polar-sm(polar-highsave_memory=True)Same dense
basic_modelLP, HiGHS time-limited to isolate the modelling + IO path, peak RSS via/usr/bin/time -l.Results (macOS arm64, single thread, HiGHS 1.14.0, linopy 0.7.0, polar-high 2.4.5)
At N=3000 (~18M variables):
linopy-directlinopy-lpRanking stable across N=500..3000.
Takeaways
io_api="direct"reverses the time ranking vslp: it's the fastest of the four on build+IO, ~1.6x faster than polar-high regular on the same machine.polar-high(Polars-backed LP/MIP eDSL on HiGHS) — thoughts? #740 surfaced is a memory gap, not a speed gap.Plot (
tool_compare.svg/.png) and raw CSV are included. Happy to adjust scope, wire it into the Snakemake flow, or drop the committed artifacts if you'd rather keepbenchmark/script-only.🤖 Generated with Claude Code