NLSP: Non-Linear Solve Problem (new OO class)

New class has been added to OO:
NLSP: Non-Linear Solve Problem
currently the single solver for the class is scipy.optimize.fsolve as scipy_fsolve.
I intended to connect one more solver from Trilinos NOX package, but this will not be done, at least in nearest future, because of lack of documentation (see discussion of the problem here)

Auto check: can constraints be handled?

Now OpenOpt will automatically check can solver handle the problem constraints (c(x)<=0, h(x)=0, lb<=x<=ub, Ax <= b, Aeq x = beq) or no.
Error message will be shown, for example

"Error! the solver scipy_lbfgsb cannot handle 'c' constraints"

scipy.optimize.fmin_bfgs connected as scipy_bfgs

Another one solver connected to OpenOpt:
scipy.optimize.fmin_bfgs as "scipy_bfgs"
(requires scipy)

fmin_cobyla: connected

constrained NLP solver fmin_cobyla from scipy.optimize have been connected to openopt as scipy_cobyla.
Please remeber: the solver doesn't use user-provided gradient info.

ALGENCAN have migrated from numeric to numpy

Today I was informed that ALGENCAN have finally migrated from using numeric to numpy.

Those who has encountered the error message with ALGENCAN-openopt connection:

struct has no "ndim" attribute

should re-install ALGENCAN according to new instructions
(you can get it here)

minor updates in lincher

I have committed some changes that allow NLP solver lincher (Python-written, BSD license) to solve problems where no other constraints than box-bound ones are present. Also, lincher doesn't require any QP solver for the case (currently the only one connected is CVXOPT one). However, specialized solvers (connected to openopt) like scipy_lbfgsb or tnc or ALGENCAN are usually much more faster.

examples of openopt graphics output

Here are some pictures automatically generated by openopt (matplotlib should be installed) via setting p.plot = 1. Please note, that for some solvers with constrained problems they can turn out to be uninformative - for example objFun can go down while max constraint violation go up , or wise versa, so in future, would openopt get further development, one of features that should be implemented will be adding 2nd subplot - max constraint value. Also note, that some solvers like ALGENCAN has no cross-iter output function (at least for now) that could be easily connected to openopt's one, so their pictures looks like 2 points (start and finish), connected by single line.



Other possible directions of openopt development could be: connecting solvers of NLPy package, implementing 2nd derivatives, automatic scaling, using patterns of sparsity, connecting well-known NLP IPOPT solver, connecting some global solvers and/or writing openopt own ones, writing a good QP solver (that one is required by lincher).
However, since Google Summer of Code program is finished, it requires new sponsor(s), w/o this one openopt development will be very limited.

ALGENCAN: connected

So ALGENCAN have been connected and tested with all types of constraints.
However, 2nd derivatives are not implemented in openopt at all, so you can't pass the funcs (like d2f, d2c, d2h, d2L) to ALGENCAN from openopt framework for now.