Monday, June 30, 2014

The licensing system overhead


MOSEK employs a license system to make sure users pay for the software so the developers can get payed.  The way the licensing system works is that a valid license token must be obtained either from a file, or from a token server, i.e. a service/daemon running on a computer on the network.
Clearly, checking out a license token costs some time, but for large scale optimization problems the cost is negligible. However in an application where a large number of small problems are solved in a short amount of time, such checkout overhead can be significant. Here will we try quantify the overhead.

Licensing system at glance

First, let's summarize how MOSEK interacts with the license system:
  1. A license is checked out the first time MOSEK tries to optimize a problem.
  2. The check out involves either reading a file or querying a token server on the network.
  3. By default MOSEK employs license caching, i.e. the token is stored in the so called MOSEK environment. A environment can be reused for all the optimizations.
  4. Therefore, the license is released when the MOSEK environment is destroyed and not when a single optimization terminates.
The license check out overhead is split among the first optimization and the environment termination. Many users, for sake of a cleaner and simpler code keep creating and destroying environments at each optimization. This approach also allows to keep licenses free as much as possible. But if you were to solve thousands of small problems, you might experience a performance penalty.

The test

So the question arises: how much overhead the licensing system introduce?

To answer this question we will test MOSEK using the Python Optimizer API. We will call the solver with an empty problem twice, and measure the time spent for the first and the second call. The test is then repeated 10000 times. We use the standard timeit  Python package. The code for the tests is the following:

We execute the test both in the case of a machine running a token server locally, and in the case of a license file stored locally. The latter corresponds to the case of using a trial license for instance. We obtain the following results:

  1. The first line shows the time spent for creating the environment, the task and then running the solver. The license is release as the environment is destroyed.
  2. The second line reports the time when the environment is created only once and kept the same, while a new task is allocated for each problem.
  3. The last line shows the case in which we only create the environment and the task only once.

It can be seen:
  • There is no time significant difference between using a license file or a token server in an ideal case as the one we are using.
  • If the environment is not reused then average time is 8 milliseconds whereas if it is then the average time is $0.1$ milliseconds. 
  • It takes around $8$ microseconds to check out a license.


Therefore, if an optimization problem requires less than say $0.1$ second to solve then license check out time is going to be significant if license caching is disabled, i.e. the MOSEK environment is created and destroyed at each solver run.