About prob structure redefinition

I have noticed 2 lines of code sent by an openopt user:

#unfortunately, as openopt is now, this needs to be defined again
#each time it is solved
It is followed by function for prob redefinition:
def setupF2dProblem(init_point, maxIterates):
...
return prob

Let me explain the situation once again. Maybe, you have already noticed the error messages when trying to use prob instance one more time: r=p.solve(...). This problem is due to Python issue 1515: deepcopy doesn't copy instance methods (url)
prob instance contains lots of function handlers, referring to each other, and while running p.solve(...) they seriously changes to other values, and some new fields (function handlers, flags True/False, some new values - Python lists, arrays etc) appear. It's impossible for me to remove all the changes done and successfully keep cleaning the prob instance of all that stuff after solving finish. It may cause inpredictable bugs, appearing once time-to-time, and hunting for this kind is very difficult.
So I decided to wait until Python developers will fix the bug. They informed me of the url for issue 1515 during Python bug day, when I have committed the bugreport (some weeks later pastebin.com will remove the code). So, they recommended me temporary solution - to add the line

d[types.MethodType] = _deepcopy_atomic

to copy.py file (like it's mentioned in the url provided), but modifying Python core sources is inappropriate solution for the case - I can't demand the one from each OO user (moreover, not all of them has write access to Python core files). I haven't tried it by myself as well, so I don't know does it helps or no.
So, it would be nice to increase the bug severity from "normal" (as it is assigned for now) to something bigger, maybe Python developers would increase their efforts to fix the one. Let me also attach in comment the bugreport, since it will disappear soon from pastebin.com.

New OO class: GLP (global problem)

New OO class have been created: GLP (global problem)
Currently single GLP solver is "galileo" (by Donald Goodman, alg: Genetic Algorithm; license: GPL)
I have included galileo code to OO. For now it can solve only continuous problems with finite box-bounds. I intend to add handling all-discrete problems as well (AFAIK this solver can't handle mixed discrete-continuous).

For GLP default p.plotOnlyCurrentMinimum = True, while for all other classes default value is False.
Currently stop criteria for GLP are maxIter, maxFunEvals, maxTime, maxCPUTime, fEnough. AFAIK xtol and ftol are inappropriate stop criteria for GLP class. I intend to add something like maxNonSuccess = 15 (number of iterations when better value haven't been obtained).
Here's graphical output for the example.

SAGE online Calc: very convenient

Today I have created account (available for free) for SAGE Notebook (online calculator) and found the one very convenient. Here's snapshot from Mozilla browser:

Some major changes to ralg

Some major changes to ralg have been made. Almost all tests works better now.
Also, since now ralg will take into account do you define problem as NLP or NSP (and hence behaviour and trajectory will be different).

bugfix related to "ratAll undefined"

some days ago I forget to commit some changes to svn
if you had encountered "ratAll undefined" update svn (or try using updated openopt.tar.gz file)

some more changes to ralg

some more changes to enhance solution precision.

Some changes in ralg

Some changes have been committed to NLP/NSP solver ralg (first of all to improve solution precision). They are similar to Fortran ralg version, however, equality constraints handling of the latter is still better than Python one.

Some changes in graphics

Since matplotlib has some problems with axhline+autoscaling, especially while using 2 subplots, I have made some changes.
So now examples/nlp_bench_1.py works w/o backward xlim modifications that have been occurred earlier.
Let me also remember you: using initial approximations for xlim, ylim can be very useful.

New OO class: MMP (mini-max problem)

New OO class has been created: MMP (mini-max problem).

Currently single OO solver available for MMP is nsmm ("non-somooth mini-max", unconstrained, can handle 1st derivatives, non-smooth, ill-conditioned and noisy funcs)
It defines function
F(x) = max_i {f[i](x)}
and solves NSP
F(x) -> min
using solver ralg.

It's very far from specialized solvers (like MATLAB fminimax), but it's better than having nothing at all.
This solver is intended to be enhanced in future (first of all - add constraints).
Here's example/mmp_1.py graphical output:

OpenOpt TODO list

hi OpenOpt users,
as you probably know OpenOpt still lacks finance support, so we should take care of OO further evolution by ourselves. Of course, I had applied for money support (and will apply furthermore), but some Ukrainian organizations answered they are not interested in Python scientific software, some (foreign ones) explained me I'm required to be their country's resident, and several (like Enthought) hadn't answered at all (OK, I don't mind); AFAIK PSF doesn't accept propositions (at least for last several years), so GSoC2007 (that had finished) donation remains being single one for now, and I'm not sure I'll be able to participate in GSoC2008 because maybe I'll already finished being student for the date required (April 11 or kind of).

Unfortunately, OpenOpt for Python is not related neither to my work nor to my graduation work (latter is MATLAB code for Electro-Energetic Systems State Estimation problem), so I can't pay OpenOpt too much time, at least for now.

So, here's list of OpenOpt TODO tasks, some of the ones can be done by yourselves for to speed up OpenOpt development.
Making donations would also speed up making this and others tasks done.
Regards, Dmitrey.

New doc page entry: description of output struct

New entry has been added to OpenOpt doc webpage: description of output structure r (r = p.solve(...)) fields

Link

Changes to variable names, iterFcn and user-defined callback

Some changes:

• "nIter", "r.nEvals" have been renamed to "iter", "r.evals"
• r.evals now has "iter" key, so it looks like
  

>>> dir(r)
['__doc__', '__module__', 'advanced', 'elapsed', 'evals', 'ff', 'isFeasible', 'istop', 'iterValues', 'msg', 'rf', 'solverInfo', 'stopcase', 'xf']
>>> r.evals
{'c': 285, 'dh': 215, 'f': 285, 'df': 215, 'h': 285, 'dc': 208, 'iter': 248}
>>> r.elapsed
{'plot_time': 3.71, 'solver_cputime': 0.5, 'solver_time': 0.54, 'plot_cputime': 3.44}
>>> r.msg
'|| gradient F(X[k]) || < gradtol'
>>> r.solverInfo
{'alg': 'Augmented Lagrangian Multipliers', 'homepage': 'http://www.ime.usp.br/~egbirgin/tango/', 'license': 'GPL', 'authors': 'J. M. Martinez martinezimecc [at] gmail.com, Ernesto G. Birgin egbirgin [at] ime.usp.br, Jan Marcel Paiva Gentil jgmarcel [at] ime.usp.br'}

• new field p.rk: current max residual have been added (can be used in user-defined callback function(s))
• bugfix related to NLSP and LSP classes ("struct has no classIterFuncs field")
• For solvers w/o native iterfcn (iter-to-iter callback function) using p.connectIterFcn('df') is no longer required (it will be done automatically, however, user can connect it to other func).
  
• now all stop cases except ftol and xtol can be handled in artificial connected iterfcn (like ALGENCAN, scipy_slsqp) - for example, it can be MaxCPUTime, maxTime, fEnough, maxFunEvals, maxIter

Example of user-provided callback function(s) have been updated according to latest changes.

Jan 2008: Python continue to grow (TIOBE)

Today TIOBE index have been updated for Jan 2008.
Python language continue to grow:

6

8

Python

5.538%

+2.04%

A

6th position this month, 8th - the one year ago, 5.538 - this month (gained 2.04 during year 2007)

Now, what about other free script languages?
Ruby goes a little bit down

11

10

Ruby

2.345%

-0.17%

A

However, Lua quickly grows up:

16

46

Lua

0.579%

+0.48%

A--

Fortunately, Python scientific software is much more mature than same for Lua, at least for now.
BTW I remember reading a message about Lua-Python 2-way bridge (free software).
Note that Ruby and Lua were created after Python so some Python drawbacks were taken into account (same to Python vs Perl and PHP).

PHP is growing up but the language is so inconvenient (as well as Perl) that no doubts it will be suppressed.

4

5

PHP

9.195%

+1.25%

A

7

6

Perl

5.247%

-0.99%

A

Java still remains 1st, C and C++ go down:

1	1		Java	20.849%	+1.69%	A
2	2		C	13.916%	-1.89%	A

5

3

C++

8.730%

-1.70%

A

Also, Fortran and MATLAB should be mentioned:

24

Fortran

0.305%

26

MATLAB

0.241%

However, noone knows what scientific language will suppress all others - will it be Sun Fortress (also known as Fortran successor, however, similar name is almost the single similar feature), or IBM X10 (parallel calculation libraries for Java), or Cray Chapel (C-like, something similar to D or Pyrex languages) or MS F#, or maybe another one, unknown for now.

Or, maybe, it will be Python language? :)

On the other hand, usage of Python as scientific language may become deprecated, like Ada language had been suppressed by C.

I write a little programs time-to-time using the Fortress language, btw I had wrote ralg solver using the one. Fortress FAQ says "Fortress is general language, not only mathematical one". But the language requires strict type classification.
For example:
a1 = array[\RR64\](3,4).fill(1.5)
b := a1 + a1 (or b = a1 + a1)
yields error in Fortress ("unbound variable", it makes me remember my times of OCAML programming). But it works perfectly in Python or MATLAB.
Almost all langs quickly growing up in TIOBE index (Python, Ruby, Lua etc) are free of strict type classification - users prefer to type as little code as possible, some percents of speed down are less important.

One more serious Fortress drawback is loops parallel by default:
for i <- 0#n, j <- seq(0#n)
means parallel by i. So they provoke users to use parallel loops where sequential are much more safe. I don't want to get no sleep suspecting that someone from developers team turn out to be too lazy to write "seq()" everywhere where is should be done, and then aircraft crashing etc.
For "general purpose language" sequential loops should be default ones. I had proposed using
for i <= 0#n, j <- 0#n instead, but no answer have been obtained - have it been accepted or rejected.

Fortress developers promise to yield stable compiler in 2009-2010 or so. And maybe Python, F# or other languages will have much more stronger positions (as scientific languages) for the date.

experiment: google search for optimization solvers

I had noticed some people come to openopt webpages from google search for something like "free optimization solver", especially "free NLP solver" (since NL problems are most common in numerical optimization). So I have done some google searches and interesting results have been obtained:

• google "free optimization solvers" yields OpenOpt for Python 7th position from 81700:

• google "NLP solver" yields 4th position from 67300 (after gams.com, colan.org, optcontrolcentre.com):
• google "free NLP solver" ... drumroll... 1st position from 5330!

Be sure, I didn't use any hacks to enhance openopt positions in web search engines, as well as any other dishonest tricks. To prevent some side effects (I constantly had been redirected to google.com.ua), I did this search via connecting to a US proxy.

user-defined callback functions

I have implemented user-defined callback functions; an example is provided in OO Doc page.

However, it works for ralg, lincher and some other solvers with good connection between OO Kernel and solver callback funcs, and it's not tested for artificial connections like p.connectIterFcn('df') for ALGENCAN and scipy_slsqp.

I guess it will work (as text or maybe user-defined graphic output), however, I have turned user-defined stop off, to prevent some negative undesired consequences.