[scilab-Users] using optim function in different versions of Scilab

[Michaël Baudin]

Dear Ivan,

Thank you for providing that bug report.
You probably have the same problem that was
associated with bug 3985:
http://bugzilla.scilab.org/show_bug.cgi?id=3985

I confirm that the results are not the same, on Windows, with
Scilab 4.1.2 and Scilab 5.1.1. (see in PS my results).

The rand function produces the same sequence of random
numbers everytime Scilab is started. This is because the
seed of the algorithm is constant and hard coded in the source
code so that the sequence has the
same initial point everytime Scilab is started.
But it is complicated to tests scripts which are based on
randomized inputs.
The situation is made more simple when constant matrices
are given in the test scripts (even if these matrices have been
computed from the rand function): think about it the next time
you have these kind of problems, it will make your debug process
easier. Even if you are performing a probability analysis of course,
separating the sources of problems is often a good idea.

I tried various version of your script (see in attachment).
* The first thing that I tried was to initialize the seed of the random
number generator to the same zero value: I got the same results on both 
versions of Scilab.
* I also tried to set constant random matrices instead of using
the rand function, which makes the script easier to debug.
Again, I found the same results with both versions of Scilab.
* I analysed the numerical issues of this unconstrained optimization
problem. From the test-modified3.sce script, you will see that I
computed the Hessian matrix at optimum with the following commands

[J,H] = derivative (fun,xopt,H_form="blockmat")
spec(H)
cond(H)

The condition number of the Hessian matrix is approximately
5 so that the conditionning of the problem is not very high:
the problem is well conditionned.
All the eigenvalues are positive, which prooves that the
Hessian matrix is positive definite at optimum.
The gradient is approximately 1.d-9., which is probably the best you
can get with an order 2 formula. The norm of the gradient has
reduced from 6 orders of magnitude, since the gradient at x0 is
approximately 1.e-3.

* The final experiment I made was to increase the order of the
numerical difference used in derivative (see in test-order4).
The results on both Scilab version are the same.

My conclusion is that the numerical derivatives with an order 2 formula
are generating differences in the behaviour of the algorithm.
These differences are fixed if an order 4 formula is used.
The rand function cannot be the cause of the differences, since the 
sequence is
always the same.
The cause cannot be the extended precision issue, which is
creating differences between Linux and Windows of Scilab
4.1.2 to Scilab 5.x: here the problem appears on Windows, with
different versions of Scilab.
The cause cannot be a modification of the source code of Scilab,
since none of the functions which are implied in this case have been
modified between Scilab 4.1.2 and Scilab 5.1.1: some interfaces
and architecture have changed, but not the numerical methods
connected in Scilab.
The cause may be the compilation process of Scilab, which may
cause differences in the order of the execution of the instructions.

Notice that, if you perform complicated statistics computations,
I recommend that you use the grand instead of rand.
The rand function should behave smoothly if the central
behaviour is to analyse.
But, if you analyse rare events, the grand random number generators are
far better, mainly because they are associated with much larger cycles
(their statistical properties is also better).

Best regards,

Michaël Baudin

PS
Windows, Scilab 5.1.1:

BEFORE: 0.061004
AFTER: -0.345703
BEFORE: -0.011493
AFTER: -0.476168
BEFORE: -0.045253
AFTER: -0.350020
BEFORE: -0.028446
AFTER: -0.436120
BEFORE: 0.020816
AFTER: -0.116581
BEFORE: -0.051276
AFTER: -0.532219
BEFORE: -0.007288
AFTER: -0.149870
BEFORE: 0.076331
AFTER: -0.340303
BEFORE: 0.016214
AFTER: -0.676543

Windows, Scilab 4.1.2

BEFORE: -0.004118
AFTER: -0.013940
BEFORE: 0.031497
AFTER: -0.031452
BEFORE: 0.006074
AFTER: -0.044554
BEFORE: -0.004575
AFTER: -0.132000
BEFORE: 0.058924
AFTER: -0.268444
BEFORE: 0.008280
AFTER: -0.066517
BEFORE: -0.008389
AFTER: -0.341465
BEFORE: 0.068112
AFTER: -1.044189
BEFORE: -0.059928
AFTER: -0.290209

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test-order4.sce
URL: <https://lists.scilab.org/pipermail/users/attachments/20090625/42a1105c/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test-modified3.sce
URL: <https://lists.scilab.org/pipermail/users/attachments/20090625/42a1105c/attachment-0001.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test-modified.sce
URL: <https://lists.scilab.org/pipermail/users/attachments/20090625/42a1105c/attachment-0002.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test-modified2.sce
URL: <https://lists.scilab.org/pipermail/users/attachments/20090625/42a1105c/attachment-0003.ksh>