CI Performance test:`HiGHS` solver pathologies

[luke-kiernan]

The performance comparison shows weird behavior that can't be attributed to noise. Evidence:

• Performance comparison on 2 stacked PRs showed 30% improvement in solve time against main, consistent across the two stacked PRs. Yet I confirmed that the PRs didn't change the objective function nor the number of constraints.
• The next day, the same performance comparison runs for hours, both locally and on CI.

I had Claude code run the UC problem in the performance script, with the only difference being the HiGHS version and the threadcount. Results:

edit: misunderstanding in back-and-forth with claude code. Doesn't look like it's simply due to threadcount/version. Continuing to look into it. But the below items still seem like reasonable courses of action

Remedies:

• Change the perf environment to use a fixed HiGHS minor version, instead of going with the most recent 1.0+ release
• Separately record and report HiGHS solve time
• Use single-threaded so it's deterministic
• Increase MIP gap: under 1.23.0 multi-threaded, 1% (current) stalls but 2% solves quickly

[jd-lara]

@luke-kiernan we don't run HiGHS multi threaded in the performance test

[luke-kiernan]

My bad: evidently I was profiling the same problem as the performance script, but not with the same settings.

I still think it'd be a good idea to report the solver time separately, pin a specific HiGHS version, or otherwise make performance test results more consistent. Not just here, but elsewhere too: see this PNM issue. Otherwise we gradually start to ignore them, and things slip through the cracks.