the blog is moved to new location

update your rss feeds if you want to get the posts furthermore.
New location:
http://forum.openopt.org/viewforum.php?id=19

OpenOpt (and my) Future

My interim in icyb.kiev.ua optimization dept is up to be finished.

Today I have spoken with my chiefs from about my further career.
They have informed me: because of financial crisis the situation is undefined.

Still I have some chances. Moreover, there are some chances I will be permitted to spent some work time for further openopt development.
However, it is provided some conditions to be satisfied.

Some days ago I had been forced to move my code out of scikits framework, my chiefs want me to host it inside Ukraine under our control (mb you know I had been refused to obtain veto rule for openopt code changes within scikits framework; I understand scipy community - they can't take taht something is hosted inside their server and is out of control, but I understand my chiefs position as well).

I have contacted uafoss.org (FOSS Ukraine dept), and they gladly allowed me to host it here.
Full transportation (svn repository, doc, wiki etc) will be finished within several days. But numerical optimization forum already works, you are welcome: http://forum.openopt.org. I had noticed openopt.com and openopt.net are already taken by someone, so it's definitely correct time to go.

Thanks to Michailo Danilenko and Volodimir M. Lisivka from linux.org.ua (LOU) community.
Special thanks to Wadim V. Mashkov and Michael Shigorin from linux.kiev.ua community.

OpenOpt release 0.21

Hi all,
I'm glad to inform you about new OpenOpt release: v 0.21.

Changes since previous release 0.19 (June 15, 2008):

• major changes for NLP / NSP solver ralg
• new converter: qp2nlp
• new converter: lp2nlp
• new converter: nllsp2nlp
• new converter: nlsp2nlp
• new converter: minimax to NLP
• Introducing OpenOpt GUI
• PSwarm 1.3 has been connected instead of v 1.1. It has much more parameters to modify and ability to use user-supplied callback function.
• Lots of changes related to oofun: add oovar and oolin, now fixed oovars can be correctly handled
• enhanced iterfcn connection for scipy fmin_cobyla
• some doc updates for result structure
• new openopt API func: oosolver
• some code cleanup and bugfixes
• some other minor changes
  

Backward incompatibility:

• instead of "from scikits.openopt" now you should use "from openopt import ..."
• LSP has been renamed to NLLSP
• oofun with ordinary variables (x) support had been ceased (it's hard to keep maintaining), use oovars instead.

Until OpenOpt subversion repository will be finally moved to new host, you can download v 0.21 from here

Welcome to http://forum.openopt.org - new forum about numerical optimization and related free and open source software.

For assist with new host and forum
Thanks to Michailo Danilenko and Volodimir M. Lisivka from linux.org.ua (aka LOU) community.
Special thanks to Wadim V. Mashkov and Michael Shigorin from linux.kiev.ua community.

Regards, Dmitrey.

Some changes for oofun

Some changes for oofun:

• oolin now can handle matrices: oolin(C) creates oofun that returns numpy.dot(C, inp_array); oolin(C, d) yields oofun that returns numpy.dot(C, x) + d. See updated oolin example.
  
• add fixed variables handling for oovars. Now you can declare v.fixed = True or just v = oovar(..., fixed = True). So all oofuns that recursively depend on fixed oovars only will be calculated only once, and derivatives will be all-zero (hence no calculations will be required for those parts of code that depend on fixed oofuns only). In future it will be good to have inner fixed coords for oovars, like this: v.fixed = [0, 1, 15] (i.e. positions of fixed coords inside the oovar). See the fixed oovars example.
  

However, let me remember you that currently oolin(anything) is considered as non-linear code, i.e. it goes to non-linear constraints and objective function only. In future it should be implemented oolin(some_oovars) -> general linear constraints (Ax <= b, Aeq x = beq). As for lb <= x <= ub, they are handled correctly from oovar fields.

some changes

I have committed some changes to OO Kernel, most important is a fix reducing time for connecting oovars to prob instance (the recursive function throw all oofuncs took too much time previously).

Ironclad v0.7 released (NumPy on IronPython)

IronClad developers have announced release v 0.7.
I guess it makes possible to use OO and some solvers (like ralg) from IronPython.

CorePy: Assembly Programming in Python

I've got to know about BSD-licensed v 1.0 release of CorePy - "a Python package for developing assembly-level applications on x86, Cell BE and PowerPC processors".

I guess it would be useful for those objective or non-linear functions that are required to be evaluated sufficiently faster than pure Python-coded.

Of course, using C, C++, Fortran code via Cython, f2py, ctypes, SWIG, Pyrex etc could yield some speedup as well.

new openopt API func: oosolver

I have committed entry for oosolver (that has been recently implemented) into OO Doc page.

changes for ralg linear constraints

I have committed some changes for general linear constraints (A x <= b, Aeq x = beq) handling by NLP/NSP ralg solver.

The changes are essential for len(b) >> 1 or len(beq) >> 1 only.

bugfix for nonlinear group + changes for ralg

• I have found and fixed serious bug for non-linear problems group (NLP, NSP etc). Sometimes it has been triggered with some constrained problems and those solvers who can use splitting (ralg, algencan). Still algencan doesn't work essentially better (for those examples I had tried).

• Some changes for ralg have been committed (to decrease non-linear inequality constraints evaluation number)

new converter: minimax to NLP

I have committed the converter along with usage example. Like MATLAB's fminimax and lots of similar MMP solvers, it works via solving NLP

t -> min

subjected to
t >= f0(x)
t >= f1(x)
...
t >= fk(x)

Let me note that the NLP problem obtained is always constrained (in addition to constraints lb, ub, A, Aeq, c, h from original mmp we get new non-linear inequality constraints written above).

some doc updates for result structure

I have updated the doc page ResultStruct with description of

• negative values of r.evals['df'], r.evals['dc'], r.evals['dh'] (it means they have been obtained via finite-difference approximation; in the case we take into account for r.evals['f'] all f calls - both from objFunc and from finite-difference derivatives approximation)
  
• r.iterValues.rt, r.iterValues.ri (type and index of max residual).

enhanced iterfcn connection for scipy fmin_cobyla

I have connected changes related to handling of scipy_cobyla iterfcn, so now instead of direct line (as before) it looks like this:

Of course, adequate text output is provided as well:

solver: scipy_cobyla   problem: unnamed   goal: minimum
iter    objFunVal    log10(maxResidual)
 0  6.115e+01               2.13
10  2.015e+01              -2.82
20  2.029e+01              -6.46
30  2.030e+01              -7.60
40  2.032e+01              -8.70
50  2.032e+01             -10.30
60  2.033e+01              -9.78
70  2.033e+01             -13.41
80  2.033e+01             -15.58
90  2.033e+01             -12.50
96  2.033e+01             -21.03
istop:  1000
Solver:   Time Elapsed = 0.72     CPU Time Elapsed = 0.69
Plotting: Time Elapsed = 6.72     CPU Time Elapsed = 5.31
objFunValue: 20.329368 (feasible, max constraint =  9.3314e-22)

Also, fEnough, maxTime, maxCPUTime and some other stop criteria work (for scipy_cobyla).

Initially users had to connect iterfcn by themselves (to df, dc, dh etc), then it had been done automatically by default to df, now (with latest changes) it is for f and df only and automatically.

Now NLP instance has parameter f_iter (default is max(nVars,4)), so when number of called objective function exceeds p.f_iter (for scipy_cobyla and other solvers w/o iterfcn connected and gradient using), OO iterfcn function is called (and hence user-supplied callback functions(s) if any are declared).

Also, for those solvers who hasn't native connection to OO iterfcn and use derivatives (algencan, ipopt, scipy_slsqp, some unconstrained and box-bounded ones) there is parameter df_iter, default True (use iterfcn each call for df); if positive integer s(s>1, 1 is same to default True) - use iterfcn each s-th objective function gradient call.

Mb I'll change "f_iter" and "df_iter" to more appropriate field names till next OO release.

Any Toronto openopt users?

hi all,
if anyone from Toronto is OO user (I had noticed some in webcounters) it would be nice would you mention OO in (imaginary) "Million Dollar Python Project" (link) by PyGTA (Toronto Python User's Group on Tuesday)

OpenOpt in Debian packages

Yaroslav Halchenko from PyMVPA project (that uses OpenOpt) has organized python-scikits-openopt deb package and put it into Debian Linux repository.

However, I don't know how stable it is (I haven't tried/tested it yet and will hardly do in nearest future, currently I'm busy because of many other urgent things to be done, my dept chiefs require that ones).

some changes & bugfixes

I have committed

• some changes for ralg
• some bugfixes for oofun-oovar
• some other changes
  

Major changes for ralg

I have committed some major changes for NLP/NSP solver ralg.

A couple of ideas have been stolen from SolvOpt (made by our dept worker Alex Kuntsevich during his voyage to Austria, in collaboration with Franz Kappel), that still remains to be most famous Naum Z. Shor r-algorithm implementation with Fortran90, C and MATLAB API.

However, the implementation is more heuristic than original Naum Z. Shor r-algorithm; since convergence of latter hasn't been proved yet even for smooth convex functions, proving convergence for SolvOpt's implementation will hardly be ever possible (especially because of those heuristics like backward line search approach, see SolvOpt's paper for algorithm details).

So my dept chief is not fond of SolvOpt heuristics (he consider they reduce % of problems that can be solved, because SolvOpt can stop further from optimum, then primal Naum Z. Shor r-algorithm), still SolvOpt remains rather popular in our optimization dept.

Currently I have taken 2 ideas from SolvOpt: modifications for initial step size and step multiplier for forward line search. As for backward search, I found SolvOpt's approach very unclear and currently my own way still remains in OO ralg implementation. Handling of constraints is also performed in a different way.

Also, some code cleanup for OO kernel have been committed.

PSwarm 1.3 has been connected

I have connected new PSwarm version: 1.3.

As PSwarm developer Ismael Vaz promised, now Python API has much more available parameters to modify (see updated GLP page for details) and outputfcn connected (that improves text and graphics output), so now user-provided callback function(s) can be handled by the solver (also, openopt GUI p.manage() function works better, see previous post for description).

Introducing OpenOpt GUI

I'm glad to introduce initial version of OpenOpt GUI.

I had some months of GUI experience in PyGTK and tcl/tk so I have some experience in GUI development; on the other hand I know very well what's the deuce it is: you have to try all possible combinations of buttons (and the dummy task is NP-Hard of course) to ensure all works ok, to enable/disable buttons in correct order, to story related variables etc.

For OO I decided to use Tkinter, because it doesn't require any additional soft installation (it is included in Python distribution).

I had took a look at AMPL and GAMS GUIs before starting my own one, and of course I have no intention to duplicate Python IDEs.

Running OO GUI is performed via

r = p.manage()

instead of p.solve(), or via

from scikits.openopt import manage
r = manage(p, solver, ...)

Let me also note: manage() can handle named argument start = {False}/True/0/1, that means start w/o waiting for user-pressed "Run".

Currently there are only 3 buttons: "Run/Pause", "Exit" and "Enough".
As you probably know lots of solvers have troubles with stop criteria. Especially it's relevant to NSP solvers, where calculating derivatives only for to check stop criteria isn't a good idea. So pressing "Enough" button yields triggering of stop criterion, like this:
------------
solver: ralg problem: GUI_example goal: minimum
iter objFunVal log10(maxResidual)
...
102 5.542e+01 -6.10
istop: 88 (button Enough has been pressed)
Solver: Time Elapsed = 1.19 CPU Time Elapsed = 1.12
Plotting: Time Elapsed = 6.86 CPU Time Elapsed = 5.97
objFunValue: 55.423444 (feasible, max constraint = 7.98936e-07)

Let me also note that

• pressing "Exit" before "Enough" and before solver finish will return None, so there will be no fields r.ff, r.xf etc.
• for some IDEs pressing "Exit" doesn't close matplotlib window (if you are using p.plot=1). You should either wait for newer matplotlib version (they intend to fix it) or try to fix it by yourself via choosing correct Agg, see here for details
• OO Doc webpage entry related to OO GUI will be committed later
• you could play with the GUI example by yourself

bugfix for ooIterPrint.py in Python 2.4

I have been informed of Python 2.4 issue
File
"/usr/lib/python2.4/site-packages/scikits/openopt/Kernel/ooIterPrint.py",
line 6
'isFeasible': lambda p: ('+' if p.rk < p.contol else
'-')
^
SyntaxError: invalid syntax

I have committed workaround for Python 2.4. Still I guess it's better to use Python 2.5, as it is mentioned in OO install webpage.