See below<br><br><div class="gmail_quote">On Wed, Oct 14, 2009 at 4:01 AM, Yann Collette <span dir="ltr"><<a href="mailto:yann.collette@scilab.org">yann.collette@scilab.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Sébastien Bihorel a écrit :<div class="im"><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Yann,<br>
<br>
The installation worked just fine. I thank you again for your explanations.<br>
<br>
Now, I would like to ask you a couple of more theoretical questions on the algorithm implemented in the nm_init, step_nelder_mead or optim_nelder_mead. I am not a mathematician and don't know much about the theories, and the nuts and bolts of the Nelder-Mead algorithm (this is probably a mistake!). My only concern, until recently, was only to know how to use it in Matlab using the fminsearch function... This function is nice for lay users because one doesn't have to care about dealing with simplex objects.<br>
The simplex toolbox seem to require such objects to be created and passed from one iteration to the other. From your documentation and examples, it is my understanding that the simplex contain realizations of the objective function at different places of the search space. Am I correct here?<br>
<br>
</blockquote></div>
There are 2 versions of the simplex in this toolbox:<br>
- optim_nelder_mead: this function takes as argument an objective function and some other parameters. All the computation are performed inside optim_nelder_mead. So, you just have to start optim_nelder_mead and get the result<br>
- steip_nelder_mead: it's the same method as above, the big difference is that step_nelder_mead doesn't perform any objective function computation. Each time step_nelder_mead needs an objective function value, it stops, give back hand to the user who have to compute the needed objective function computation and then restart step_nelder_mead by giving the preceding state of step_nelder_mead and the requested objective function values.</blockquote>
<div><br>That's where I am confused. Based on my understanding on the Matlab fminsearch function, the documentation of optim_nelder_mead, and the side reply of Jerome Picaud, I thought that the algorithm was performing X evaluations of the objective function at each iteration, then decide where 'to go' next based on these X evaluations. In the step by step, the objective function seems to be only evaluated once at each step (see extract of your help example). Am I correct?<br>
<br><pre class="programlisting"><font style="font-family: tahoma,sans-serif;" size="2">while eval_Func<MaxEvalFunc<br> [x_next, data_next, eval_Func, f_hist, x_hist] = step_nelder_mead(f_current, x_next, data_next, 'run');<br>
f_current = rosenbrock(x_next); // <- single objective function evaluation<br> printf('step_nelder_mead - Iteration %d: f = %f\n', eval_Func, f_current);<br>
end</font><br></pre>Therefore, I don't really understand how this could fit into the paradigm: one iteration = one choice among multiple function evaluations. Could you comment on that?<br><br>Alternatively, if I would want to use the 'optim_nelder_mead' way (I did not test it yet, but it seem to fit better to the above paradigm), how could I do the following things:<br>
- provide input arguments to the objective function in addition to the simplex of parameters values to be optimzed<br> - ask optim_nelder_mead to call a certain function at the end of each iteration and pass arguments to this function?<br>
<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im"><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Another thing I am not clear about is whether or not the dimensions of the simplex should be adapted to a specific problem. In the examples you provide, the simplex f_init and f_current have a 3x1 dimension for optimizing 2 parameters. Is that 3x1 dimension independent of the number of parameters to optimize? Let's say my objective function is a 'maximum likelihood function' of observations (n states observed at t times) given a model dependent on m parameters and a residual variability dependent on v parameters, what should be the size of the simplex and how should it be calculated?<br>
<br>
</blockquote></div>
Normally, the X0 parameter is an initial simplex. You can compute it yourself or compute it using nm_init. So, the x0 dimensions are n x n+1 (the documentation is wrong here. I will correct this ASAP).<br>
<br>
For your problem, if you have n states at t times, you must transform you m x t matrix into a [mxt,1] vector and then use nm_init to compute a [mxt,mxt+1] initial simplex.<br><font color="#888888">
<br></font></blockquote><div><br>OK, this is not quite the answer I expected :) I thought it would the initial simplex would be a [m+v,m+v+1] simplex, given that the objective function ML is to be minimized not the actual model. ML is briefly defined in my framework as:<br>
<br> ML=sum(0.5.*((((Fpred-ydata).^2)./(Weight))+log(Weight)+log(2*pi)))<br><br>where ydata is a vectorized [nxt,1] array of observations, Fpred is vectorized [nxt,1] array of model predictions given the current [m+v,1] array of parameters to be optimized and Weight is a vectorized [nxt,1] array of residual variabilities calculated given the current [m+v,1] array of parameter to be optimized. Obviously, the structural model predictions Fpred and the residual variabilities Weight need to be calculated prior to the evaluation of ML.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font color="#888888">
YC</font><div><div></div><div class="h5"><br>
<br>
<br>
<br>
-- <br>
-----------------------------<br>
Yann COLLETTE<br>
Software Development Engineer<br>
-----------------------------<br>
The Scilab Consortium<br>
Digiteo Foundation<br>
Domaine de Voluceau<br>
Rocquencourt - B.P. 105<br>
78153 Le Chesnay France<br>
Phone: +33.1.39.63.57.82<br>
<br>
</div></div></blockquote></div><br>