goldenSection vs scipy.optimize fminbound

For some numerical experiments (intended by our chief Petro I. Stetsyuk and me) with r-algorithm (that is implemented in OO ralg solver) a good line-search optimizer is required.

As you know scipy.optimize has single-variable box-bounded optimization routine "fminbound" (connected to OO as scipy_fminbound). Unfortunately, it has 2 drawbacks:
- it uses mix of golden section and quadratic spline approximation, and latter works very bad for non-smooth and/or noisy funcs, that we deal with
- sometimes solution returned is out of lb-ub bounds.

So I have implemented pure golden section algorithm into OO solver "goldenSection" (nothing special, just several lines of code).

"noise" parameter for noisy functions

1. For NSP problems a new parameter has been implemented: noise (currently default value is 0, mb it will be changed in future). Currently ralg can handle the param for objFunc only, not for non-lin constraints c and h.
Also, the parameter can be used with nssolve and nsmm.

examples/nssolveVSfsolve_1.py has been updated (add noise=1e-8 to prob definition).

2. Some more changes to ralg.

some changes to NLP/NSP solver ralg

some changes for ralg have been committed, they affect both trajectory and stop criteria (for constrained problems).

Scaling

Possibility to use scale factor(s) is undoubtedly a "MUST HAVE" feature for any advisable optimization solver or framework.

Since now scaling is available in OO (update from svn or tarball is required), and example of scale factor usage (p.scale, "scaleFactor" is too long to type) has been committed to OO Doc page.

Note that using of p.diffInt as vector is also allowed.
However, using scale is more preferred: it affects xtol for those solvers who use native OO stop criteria (ralg, lincher, nssolve, nsmm, mb some else).

MMP solver nsmm: constraints have been implemented

constraints handling for MMP (minimax problem) solver nsmm has been implemented.
Here's text and graphical output of the updated mmp example

starting solver nsmm (license: BSD)  with problem  unnamed
Niter      ObjFun      log10(maxResidual)
  0         6.47e+03   1.9
  10        6.45e+03   -1.1
  20        6.46e+03   -3.6
  30        6.46e+03   -5.8
nsmm has finished solving the problem unnamed
istop:  3 (|| X[k] - X[k-1] || < xtol)
Solver:   Time Elapsed = 0.26  CPU Time Elapsed = 0.21
Plotting: Time Elapsed = 3.56  CPU Time Elapsed = 2.99
objFunValue: 6461.0983572120513 (feasible, max constraint =  9.66019e-07)

bugfix for some scipy NLP solvers (unconstrained)

Another one bugfix have been committed (appered due to some recent svn changes for scipy_cg, scipy_ncg, scipy_powell, scipy_bfgs, scipy_fminbound).
Thanks to Nils Wagner for reporting the issue.
That one was due to definition of exception "isSolved" migrated from BaseProblem.py to ooMisc.py

[Linux] ALGENCAN bug, maybe due to latest glibc

Some days ago I have accepted a proposition for glibc recent version installation from Ubuntu software updates channel, and I guess it's the matter for the bug (while starting ALGENCAN):

StdErr: *** glibc detected *** /usr/bin/python: free(): invalid pointer:
0x00000000008362f0 ***

bugfix for "incorrect func index type" (some OS/Arch)

Some users informed me that some OS/Arch raises the error "incorrect func index type".
As far as we understand for now, it's due to strange nmpy.argmax() return type (numpy.int32 with type id 1810419808, that differs from 1810419616). For more info see discuss in scipy-dev mail lists: url1, url2 .
So some changes committed (also available from OO install page -> openopt.tar.gz) allows to avoid the bug.

how to change solver default parameters

A new OO doc page entry has been committed:
how to change solver default parameters (via kwargs for p.solve()).

AFAIK it's especially important for global solvers, like GA-based "galileo". However, currently only 3 parameters are available for the solver modification:
population (default 15)
crossoverRate (default 1.0, it means always do crossover)
mutationRate (default 0.05, it means "not very often")
There are some more parameters available for modification, but I lack free time and experience in global optimization.

Old way p.solverParameters.(solvername) = {...} has been removed.

solver settings as kwargs for p.solve()

An openopt user asked: could he modify GLP solver galileo parameters.
Currently it's possible to access some ralg parameters, via for example
p=NSP(...)
p.solverParameters.ralg={'h0':0.1, 'alp':3} #I suspect it's too long to type, isn't it?

(h0 is initial step length approximation, default 1, alp is space dilation parameter, default 2 (fixed for Python implementation, more advanced handling with non-fixed value in fortran version))

For modifying other solvers parameters some minor changes should be done.
Also, I had already decided how to use kwargs (and maybe some args in future, I didn't decided yet which ones) in solve(). I think it will be something like
r = p.solve('galileo', crossoverRate=0.80, mutationRate=0.15)
So, if a kwargs parameter is in dir(solverName.solverName) (i.e. is a solver parameter, like in the ralg case above:
>>>import scikits.openopt, ralg
>>>dir(ralg.ralg)
['__alg__', '__authors__', '__constraintsThatCannotBeHandled__', '__decodeIterFcnArgs__', '__doc__', '__economyMult__', '__expectedArgs__', '__homepage__', '__info__', '__init__', '__isIterPointAlwaysFeasible__', '__iterfcnConnected__', '__license__', '__module__', '__name__', '__solver__', 'alp', 'getRalgDirection', 'getRalgDirection2', 'h0', 'hmult', 'nh', 'q1', 'q2']
), then it will change the value of the solver setting, elseware if it's in dir(p) (i.e. is prob parameter, like maxIter=15) then it will change the prob field value, elseware a error message will be raised.
I intend to implement the changes during this week.