ALGENCAN: connected

So ALGENCAN have been connected and tested with all types of constraints.
However, 2nd derivatives are not implemented in openopt at all, so you can't pass the funcs (like d2f, d2c, d2h, d2L) to ALGENCAN from openopt framework for now.

ALGENCAN news

I continue trying to connect ALGENCAN to openopt.
Yestorday I informed ALGENCAN developers about minor bug in toyprob.py, and they had fix that one.

One more bug, more serious, is that they continue using obsolete Numeric module in one of files (at least, in pywrapper.c) instead of numpy, despite we agreed long tima ago they will use numpy everywhere. I informed them today and I hope they will fix it soon.
As for me I have temporary solution - replacing x by asfarray(x) in all python funcs related to ALGENCAN. So it works now, at least for UC problems, I will check constrained today. However, there are still lots of problems, and 2-nd order problems (that use d2f, d2c, d2h) are still unavailable in openopt (at all, not only for ALGENCAN).

2 more steps

2 more steps were implemented in optimizers module:
- Goldfeld
- Goldstein-Price.
Matthieu and I have some difficalties in other 2-nd order steps related to negative curvature, because available Python libraries (including numpy) hasn't modified Cholesky factorization, that is required. As we agreed no other than Python code is included to openopt, it means no fortran, etc, only python and numpy, so those fortran-written routines are inappropriate. The proposed article contains 2-pages algorithm w/o vectorization, so Python implementation will be too slow and would take much time for implementation, unittests, bugfixes.
So we decided to switch to other tasks.

Barzilai-Borwein algorithms

both Barzilai-Borwein algs have been written and tested.
Matthieu informed me he intends to rename some variables, especially containing word "step" because it yields a big ambiguity (alpha_step or x_step, and some more). previously direction also had "step" in variable name.

However, I have some problems with default sigma1, sigma2, alpha1, alpha2, rho, sigma values (from non-monotone Barzilai-Borwein alg). My initial guesses were too bad, as I noticed. Currently it works so-so - but maybe only for my tests.

UC solvers

Cubic Interpolation Search has been connected to Matthieu's code.
BarzilaiBorwein alg is written but not connected yet, I still need some more digging in Matthieu's code to connect it correctly.

l_bfgs_b from scipy.optimize have been connected to openopt

One more box-bounded NLP solver have been connected to openopt as scipy_lbfgsb.
Is solves the example from previous post much faster: ~0.1 sec.

I tried to connect cobyla, but some problems have been encountered. The main one is: all constraints should be passed as sequence of funcs. As far as I learned, even linear constraints Ax<=b or x>=lb should be separated into funcs, each one returning single value. At least, current python-cobyla binding requires this approach. Of course, I can implement the trick, but it can slow calculations very much; maybe, some changes in Python-cobyla binding will turn out to be much more effective solution. But I have no time for now, I'm busy with other things.

One more example have been added to /examples directory: nlp1.py. It uses lincher, but any other solver, capable of handling all constraints, can be use instead.

tnc have been connected to openopt

(tnc is scipy.optimize NLP solver for box-bounded problems, requires scipy installed)

example of tnc usage:
----------------------------
from scikits.openopt import NLP
from numpy import *

N = 15

ff = lambda x: ((x-15)**4).sum() + 64*((x-1)**2).sum()
p = NLP(ff, cos(arange(N)))
p.lb = arange(N)
p.ub = 5 + arange(N)
r = p.solve('scipy_tnc')
----------------------------
So it should yield something like
istop: 4 (|| F[k] - F[k-1] || < funtol)
Solver: Time elapsed = 0.12 CPU Time Elapsed = 0.12
objFunValue: 10370.1398294 (feasible)

r.xf should be equal to
array([ 5. , 6. , 7. , 8. ,
8.72421634, 8.72421837, 8.72421093, 8.72420843,
8.72420744, 9. , 10. , 11. ,
12. , 13. , 14. ])


if N = 150

lincher:
itn 37 : Fk= 887272370.281 maxResidual= 5.24576648786e-09
Solver: Time elapsed = 4.16 CPU Time Elapsed = 4.01
objFunValue: 887272370.281 (feasible)

scipy_tnc:
istop: 4 (|| F[k] - F[k-1] || < funtol)
Solver: Time elapsed = 3.02 CPU Time Elapsed = 2.93
objFunValue: 887273034.135 (feasible)
maxResidual = 0

So, despite tnc has a little bit worst objFun value, it solve the example much quicker and is more suitable for box-bounded problems - of course, because it's intended for that ones and only for that ones, while lincher handles all constraints.
On the other hand, I'm not fond of CVXOPT QP solver, using better one could significantly increase lincher speed.

However, this is the only one example I had tested.
Maybe tomorrow I'll connect and publish lbfgsb and cobyla results.

----------------------------

+ today some examples to lincher have been written, I'll commit them tomorrow after minor changes.