What's the typical adaptive_rho_interval value

[studymeow]

Thank you for making this wonderful tool! I am trying to use it get a consistent and reproducible output. I found setting adaptive_rho_interval to an integer served that purpose. I am not sure what is the typical value for adaptive_rho_interval? I understand that it will be different for each problem but I am trying to get a ballpark idea. Is this value the higher the better?how do we select it?

Thank you!

[goulart-paul]

The reason that the behaviour is not reproducible by default is that the rho update (and the consequent matrix refactorisation) is determined based on the initial factorisation time and the time taken for subsequent iterations. Since this time may vary slightly (e.g. if there are other unrelated processes running), you get slightly different behaviour between solves even on the same problem.

To avoid this you can set adaptive_rho_fraction to zero, and then set adaptive_rho_interval to some non-zero value. This triggers a rho update / refactorization (or at least a check to see if a rho update will help) at a deterministic iteration number, and the behaviour of the solver will then be completely deterministic.

The best choice depends on the problem, but I would suggest that something like 25-50 is reasonable. If you make it too small then sometimes the solver's idea of the best choice of rho will not have stabilised, which might give you erratic behaviour or lead to excessive rho updates. Making it too large means that you potentially waste a lot of time initially with a poor choice of rho before the first update is triggered.

You could also run the solver with the default choice of adaptive_rho_fraction and see what iteration count the solver is on before the first factorisation. Then set a value near that as the adaptive_rho_interval as a starting point. A strategy like this might make sense if your problem has extreme sparsity or density.

[studymeow]

Thank you so much for your detailed reply. This is really helpful. Regarding the last strategy(You could also run the solver with the default choice of adaptive_rho_fraction and see what iteration count the solver is on before the first factorisation), I am not exactly sure where should I find this information, pasted below: ( I only saw the iteration count interval is around 200 highlighted in bold

problem:
settings: linear system solver = qdldl,
eps_abs = 1.0e-04, eps_rel = 1.0e-04,
eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,
rho = 1.00e-01 (adaptive),
sigma = 1.00e-06, alpha = 1.60, max_iter = 1000000
check_termination: on (interval 25),
scaling: on, scaled_termination: off
warm start: on, polish: off, time_limit: off
iter objective pri res dua res rho time
1 -4.6532e+03 8.80e+00 4.27e+03 1.00e-01 1.40e-01s
200 -2.0503e+00 4.25e-04 2.61e+00 5.01e-01 4.31e-01s
400 -3.8533e-01 1.17e-04 2.39e+00 5.01e-01 6.93e-01s
600 -3.2582e-01 8.59e-05 1.69e+00 5.01e-01 9.56e-01s
800 -2.4474e-01 2.85e-05 1.24e+00 5.01e-01 1.39e+00s
850 -2.5198e-01 2.58e-05 7.94e-01 5.01e-01 1.47e+00s

Thank you again!

[goulart-paul]

The second to last entry in each row of the numerical output is the current value of rho. You can see that it changed from 1e-1 to 5e-1 some time in the first 200 iterations.

That's not particularly helpful to you though, and it seems that the print interval is defined by a #define and is not configurable through the settings (it is here).

A hackish way around this in the short term is to just manually set that to something small (e.g. 1 or 5) and then rebuild. That's not a particularly satisfying answer though. I think it should probably either be configurable or the solver should always write a summary line for those iterations when the rho value has updated. I will open an issue about this now.

[studymeow]

Thank you, I have tried to set the seed to reproduce the results, however, still can't reproduce. Would setting the adaptive tho interval be the only way to ensure the same results?

[goulart-paul]

To clarify - the solver does not used random number generation at any point, so setting seed values is not relevant. The issue is that by default (i.e. when adaptive_rho_fraction is non-zero), the solver tries to choose a "smart" update interval based on the duration of the initial factorization that it performs and the subsequent length of each iteration. Since your are running on a platform where those execution times are not deterministic, e.g. because there are other process running which makes the execution time a little bit variable, you won't get exactly the same update interval.

Setting "adaptive_rho_interval" non-zero will instead just fix a number of iterations between updates. In that case the solver will do exactly the same thing every time.