Benchmarking time | Differentiate an LP

[akshay326]

Tried solving, differentiating and benchmarking a simple LP of 20 variables with equality and inequality constraints.

DiffOpt solution - https://colab.research.google.com/drive/1GWR45sbss5UhJvKomoDy4Zot9d0-046J#scrollTo=SvVLd_EFz5a1
cvxpylayer solution - https://colab.research.google.com/drive/1yvQPpwXs1ngPWt9-SzBOxhm2HjwjV1Qa#scrollTo=Ged5IdWhxPsQ

for 50 evaluations
diffopt takes ~0.7ms
cvxpylayer takes ~246ms

[odow]

So diffopt is faster? Is this a problem?

I don't have permission to access the colab instances.

[blegat]

I think @akshay326 meant 0.7s for diffopt

[akshay326]

No, DiffOpt is faster (sorry just changed the access open to everyone)
This is not an issue per se, just some update.

[odow]

You should double check that the benchmark is timing the same thing in each package. If the times are that different, then perhaps DiffOpt is just returning something from a cache, or cvxpylater is doing a whole lot of unrelated work.

Perhaps it's more fair to measure the total time from setup to returning the derivative?

[odow]

I'm going to close this in favor of consolidating our benchmarking discussion in #11. I'll add a link there back to the colab notebooks for people to dig into in future.