Fusion and the art of harvesting potatoes

Week 42 in Denmark was "kartoffelferie" - fall school vacation originally designed to allow children help with the potato harvest on the farm. The vacation is still present even though kids no longer collect potatoes, except for...

... Lærke. Lærke has her own little potato field arranged in an $n\times n$ grid. For some squares of the grid she knows the exact number of potatoes that can be harvested from that square, between $0$ and $3$. She wants to design a round trip around her little field so that each of those squares will be passed by just the right number of times to pick the potatoes, one on each pass. Other squares can be passed by any number of times. A picture with a sample layout and its solution is worth a thousand words:

Now Lærke notices that she can easily write a mixed-integer model of her problem. She will associate a binary variable with each edge of the grid, to indicate if it appears in the tour, and then needs to ensure that:

• (1) every vertex of the grid meets either 0 or 2 of the tour edges,
• (2) every square with $i$ potatoes has exactly $i$ tour edges on its boundary.

That will not necessarily give Lærke one closed tour, but possibly a disjoint union of cycles. A subtour elimination scheme similar to the one used for TSP could be built on top to get a single tour. Luckily, Lærke is not so worried about that - her basket is quite small so she is happy to make a few trips anyway. What she is much more interested in is that each of the constraints (1) and (2) can be implemented basically as a one-liner in Fusion by appropriate stacking and slicing and that she can see it all and try it out live here:

https://solve.mosek.com/fusion.html?ex=slitherlink

Finally, as much as Lærke is just our imaginary MIP user in the food industry, the puzzle actually exists and is known as Slitherlink.

Bon appétit!

(images by GLmathgrant, under CC BY-SA 3.0, from Wikipedia)

CVXPY 1.1.6

The latest release 1.1.6 of CVXPY includes a completely rewritten MOSEK interface. It will, in particular, reformulate conic and linear problems in a different way than until now before passing it to the solver, so you may experience different behavior, especially in numerically challenging cases.

Here is roughly what you should expect. If you are modeling:

• a mixed-integer problem: no change.
• a sparse LMI: the new version should be much more efficient, both in terms of CVXPY modeling and solution time (the problem is dualized before calling MOSEK).
• all other linear and conic problems: the problem will be dualized before calling the solver. Ideally you should see no difference or perhaps an improvement, but it can go either way, especially if the dualized version is more challenging for any reason. In many cases MOSEK will internally choose the correct form to solve. In the remaining cases it may be necessary to explicitly force the solver to dualize.
  
  If you were already tuning some MOSEK parameters then you should reevaluate the settings. In case you were already forcing primal/dual form with iparam.intpnt_solve_form then most likely you should change to the opposite.
  
  Finally, if you are studying the solver log output, keep in mind that it is operating with the dual of your CVXPY formulation (DUAL_INFEASIBLE means your problem is primal infeasible, minimization becomes maximization etc.). 

See also a note in CVXPY docs.

Sharpe ratio - derivation and Fusion model

We are being frequently asked about the Sharpe ratio, its formulation in the conic framework, and implementation in Fusion. This involves a class of problems with an objective of the type $$\mathrm{maximize}_x\quad \frac{r^Tx-r_f}{\|Fx\|_2}$$ i.e. an affine function over a 2-norm, where $r^Tx-r_f>0$, $x\in\mathbb{R}^n$. In practical portfolio optimization $r$ would be the vector of expected returns, $r_f$ is the risk-free return rate, $x$ is the vector of asset allocations and $\|Fx\|_2 = \sqrt{x^T\Sigma x}$ is the risk associated with the covariance matrix $\Sigma$ (formulating the risk term as a 2-norm is standard and we don't go into details, see here or here). Typically there will be various constraints on $x$, for example $$\mathbb{1}^Tx=1,\ x\geq 0$$ would correspond to a fully invested portfolio with no short-selling.

Let us explain step by step how one can derive a conic formulation of (1) suitable for a solver like MOSEK. We present it in detail so that the reader can apply it almost verbatim to more complicated models of this kind. First, note that if we could fix $$r^Tx-r_f=\mathrm{const}$$ then the objective would be equivalent to minimizing $\|Fx\|_2$, that is a standard second-order cone problem. However, we don't know in advance what "const" should be. (In fact solving the problem for all values of "const" corresponds to computing the efficient frontier.) Therefore, for reasons which will become clear in a moment, we denote the "const" value by $1/z$, where $z\geq0$ is a new scalar variable, i.e. $$r^Tx-r_f=1/z$$ that is $$zr^Tx-r_fz=1.$$ Denoting $y=zx$ ($y$ is now a new vector variable) the last equation becomes $$r^Ty-r_fz=1.$$ Now $x=y/z$ and the objective function becomes $$\frac{r^Tx-r_f}{\|Fx\|_2} = \frac{1/z}{\|F\frac{y}{z}\|_2} = \frac{1}{\|Fy\|_2}$$ hence we can write the original problem as $$\begin{array}{rl}\mathrm{minimize}&\|Fy\|_2\\ \mathrm{s.t.} & r^Ty-r_fz=1,\\ & z\geq 0.\end{array}$$ Note that the new problem involves variables $y$ and $z$, but $x$ has been eliminated. Any additional constraints must also be reformulated by substituting $x=y/z$, for example $\mathbb{1}^Tx=1,\ x\geq 0$ becomes $$\mathbb{1}^Ty=z,\ y\geq 0$$ and in fact any other linear constraint $Ax=b$ becomes $$Ay=bz.$$ A solution $(y,z)$ to the reformulation gives a solution $x=y/z$ to the original problem (assuming that the problem has a feasible point with $r^Tx-r_f>0$, so that the reformulation has a solution with $z>0$).

Certain other types of constraints can also be carried through the reformulation. For example a cardinality constraint on $x$ (at most $k$ entries in $x$ are nonzero) can be imposed on $y$ using the same mixed-integer model. Another quadratic bound of the form $\|Hx\|_2\leq 1$ will become $\|Hy\|_2\leq z$ using $x=y/z$.

A sample Fusion implementation can be found here:

https://solve.mosek.com/fusion.html?ex=sharpe

Sven Wiese at CO@Work

Our very own Sven Wiese talks about "Convex Optimization via Cones and MOSEK 9" at CO@Work. See the talk at:

https://youtu.be/cGpWM2mIpC4

PDF of the slides:

https://docs.mosek.com/slides/2020/cowork/wiese_mosek.pdf

Full program:

http://co-at-work.zib.de/#schedule

CO@Work 2020

From our colleagues organizing the Combinatorial Optimization at Work 2020 summer school CO@Work 2020 here is a short advert of this very interesting event:



In its sixth instantiation, CO@Work will be a full online event, taking place from September 14 to 26, 2020.

This summer school addresses master students (in their final year), PhD students, post-docs and everyone else interested in combinatorial optimization and mathematical programming in industrial applications. Lectures will be held by around 30 experts from all over the world, including leading researchers from TU Berlin, FU Berlin, Polytechnique Montréal, RWTH Aachen, University of Southern California, University of Edinburgh, TU Darmstadt, University of Exeter as well as developers and managers of FICO, Google, Amazon, SAP, Siemens, IBM, SAS, Gurobi, Mosek, GAMS, Litic, and many more speakers.

We arranged a setup that allows us to cover all time zones: pre-recorded lectures plus two live Q&A sessions and two hands-on exercise sessions per day (11 hours apart).

You can find more information and a registration link here: http://co-at-work.zib.de/

Registration is for free, and as a TU Berlin course, it gives 10 ECTS credit points.

To upgrade or not to upgrade that is the question

A customer complained yesterday that Mosek version 7.1 did not run on a recent AMD CPU. Perhaps not that surprising since the last version 7.1 release was in 2016 and any case that version is no longer supported. 

Now we tried to run Mosek version 8.1 and 9.2 on the public available benchmark problem pds-90 on a recent Dell server with a

AMD EPYC 7402P 24-Core Processor

CPU. Version 8.1 took 169 seconds whereas 9.2 took 67 seconds. The reason for the huge difference is that Mosek normally employs the Intel MKL Blas library to perform dense matrix operations quickly. However, it performs poorly on recent AMD CPUs. Therefore starting in version 9 we switch to the Blis library when running on AMD CPUs. That switch provides the big boost.

So from this we can learn that upgrading to a newer Mosek version can provide a huge performance boost. Particularly when Mosek is employed on brand new hardware.

Now the customer was not too keen to upgrade from version 7.1 to version 9.2. Supposedly fearing compability issues which is understandable. However, if he had upgraded to version 8.1 earlier then the upgrade to version 9.2 would have been smaller. Therefore, by not upgrading you of course save some hassle but you pay the price the date you are forced upgrade. That date might be the date when your old Mosek version performs poorly on your brand new hardware.

Therefore, we recommend not to fall too much behind with upgrading. In particular if for instance version 9 is the current major version then you should at least be using a version 8.

n-queens in Fusion

In a nice post on LinkedIn by Alireza Soroudi the author presents a short GAMS integer model for the classical n-queens puzzle: place the maximal number of non-attacking queens on an $n\times n$ chessboard.

We couldn't resist writing it up in Python Fusion as well.

And here is the solution we get for $n=8$:

[[_ ♕ _ _ _ _ _ _]

 [_ _ _ _ _ ♕ _ _]

 [♕ _ _ _ _ _ _ _]

 [_ _ _ _ _ _ ♕ _]

 [_ _ _ ♕ _ _ _ _]

 [_ _ _ _ _ _ _ ♕]

 [_ _ ♕ _ _ _ _ _]

  [_ _ _ _ ♕ _ _ _]]