[scilab-Users] Probably a stupid question but...

[Sébastien Bihorel]

See below

On Wed, Oct 14, 2009 at 4:01 AM, Yann Collette <yann.collette at scilab.org>wrote:

> Sébastien Bihorel a écrit :
>
>> Hi Yann,
>>
>> The installation worked just fine. I thank you again for your
>> explanations.
>>
>> Now, I would like to ask you a couple of more theoretical questions on the
>> algorithm implemented in the nm_init, step_nelder_mead or optim_nelder_mead.
>> I am not a mathematician and don't know much about the theories, and the
>> nuts and bolts of the Nelder-Mead algorithm (this is probably a mistake!).
>> My only concern, until recently, was only to know how to use it in Matlab
>> using the fminsearch function... This function is nice for lay users because
>> one doesn't have to care about dealing with simplex objects.
>> The simplex toolbox seem to require such objects to be created and passed
>> from one iteration to the other. From your documentation and examples, it is
>> my understanding that the simplex contain realizations of the objective
>> function at different places of the search space. Am I correct here?
>>
>>  There are 2 versions of the simplex in this toolbox:
> - optim_nelder_mead: this function takes as argument an objective function
> and some other parameters. All the computation are performed inside
> optim_nelder_mead. So, you just have to start optim_nelder_mead and get the
> result
> - steip_nelder_mead: it's the same method as above, the big difference is
> that step_nelder_mead doesn't perform any objective function computation.
> Each time step_nelder_mead needs an objective function value, it stops, give
> back hand to the user who have to compute the needed objective function
> computation and then restart step_nelder_mead by giving the preceding state
> of step_nelder_mead and the requested objective function values.


That's where I am confused. Based on my understanding on the Matlab
fminsearch function, the documentation of optim_nelder_mead, and the side
reply of Jerome Picaud, I thought that the algorithm was performing X
evaluations of the objective function at each iteration, then decide where
'to go' next based on these X evaluations. In the step by step, the
objective function seems to be only evaluated once at each step (see extract
of your help example). Am I correct?

while eval_Func<MaxEvalFunc
  [x_next, data_next, eval_Func, f_hist, x_hist] =
step_nelder_mead(f_current, x_next, data_next, 'run');

  f_current = rosenbrock(x_next);

// <- single objective function evaluation
  printf('step_nelder_mead - Iteration %d: f = %f\n', eval_Func, f_current);

end

Therefore, I don't really understand how this could fit into the paradigm:
one iteration = one choice among multiple function evaluations. Could you
comment on that?

Alternatively, if I would want to use the 'optim_nelder_mead' way (I did not
test it yet, but it seem to fit better to the above paradigm), how could I
do the following things:
 - provide input arguments to the objective function in addition to the
simplex of parameters values to be optimzed
 - ask optim_nelder_mead to call a certain function at the end of each
iteration and pass arguments to this function?



>
>  Another thing I am not clear about is whether or not the dimensions of the
>> simplex should be adapted to a specific problem. In the examples you
>> provide, the simplex f_init and f_current have a 3x1 dimension for
>> optimizing 2 parameters. Is that 3x1 dimension independent of the number of
>> parameters to optimize? Let's say my objective function is a 'maximum
>> likelihood function' of observations (n states observed at t times) given a
>> model dependent on m parameters and a residual variability dependent on v
>> parameters, what should be the size of the simplex and how should it be
>> calculated?
>>
>>  Normally, the X0 parameter is an initial simplex. You can compute it
> yourself or compute it using nm_init. So, the x0 dimensions are n x n+1 (the
> documentation is wrong here. I will correct this ASAP).
>
> For your problem, if you have n states at t times, you must transform you m
> x t matrix into a [mxt,1] vector and then use nm_init to compute a
> [mxt,mxt+1] initial simplex.
>
>
OK, this is not quite the answer I expected :) I thought it would the
initial simplex would be a [m+v,m+v+1] simplex, given that the objective
function ML is to be minimized not the actual model. ML is briefly defined
in my framework as:

 ML=sum(0.5.*((((Fpred-ydata).^2)./(Weight))+log(Weight)+log(2*pi)))

where ydata is a vectorized [nxt,1] array of observations, Fpred is
vectorized [nxt,1] array of model predictions given the current [m+v,1]
array of parameters to be optimized and Weight is a vectorized [nxt,1] array
of residual variabilities calculated given the current [m+v,1] array of
parameter to be optimized. Obviously, the structural model predictions Fpred
and the residual variabilities Weight need to be calculated prior to the
evaluation of ML.

YC
>
>
>
>
> --
> -----------------------------
> Yann COLLETTE
> Software Development Engineer
> -----------------------------
> The Scilab Consortium
> Digiteo Foundation
> Domaine de Voluceau
> Rocquencourt - B.P. 105
> 78153 Le Chesnay France
> Phone: +33.1.39.63.57.82
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20091014/1a6a1c58/attachment.htm>