<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 10pt; font-family: Verdana,Geneva,sans-serif'>
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">Hi all<br /> <br /> Well, from a "macro" point of view, gradient based methods (order 1 method) should be more efficient than Nelder-Mead one (order 0) especially for 1D topic, shouldn't be?<br /> <br /> In the case here bellow, when I use 'Optim' it is not the case in comparison to 'NelderMead'... I don'y know what I'm doing wrong but I decide to frop off 'optim' in favor tp Nelder-Mead - nevertheless the later method requires much more iterations :-(<br /> <br /> I'll be interested by some feedback on such topic, either in the mailing list (or in MP in order to not pollute it).<br /> <br /> Paul<br /> </div>
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">########################################"</div>
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">'optim'</div>
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">########################################</div>
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">
<pre><span>mode</span><span>(</span><span>0</span><span>)</span>
<span>function</span> <span>f</span><span>=</span><span>lineaire</span><span>(</span><span>x</span><span>, </span><span>a2</span><span>, </span><span>b2</span><span>)</span>
<span>f</span> <span>=</span> <span>a2</span><span>*</span><span>x</span><span>+</span><span>b2</span><span>;</span>
<span>endfunction</span>
<span>function</span> <span>g</span><span>=</span><span>racine</span><span>(</span><span>x</span><span>, </span><span>a1</span><span>, </span><span>b1</span><span>)</span>
<span>g</span> <span>=</span> <span>sqrt</span><span>(</span><span>a1</span><span>*</span><span>x</span><span>)</span> <span>+</span> <span>b1</span><span>;</span>
<span>endfunction</span>
<span>function</span> <span>f</span><span>=</span><span>target</span><span>(</span><span>x</span><span>)</span>
<span>val_lin</span> <span>=</span> <span>lineaire</span><span>(</span><span>x</span><span>,</span><span>1</span><span>,</span><span>2</span><span>)</span><span>;</span>
<span>val_rac</span> <span>=</span> <span>racine</span><span>(</span><span>x</span><span>,</span><span>10</span><span>,</span><span>6</span><span>)</span><span>;</span>
<span>f</span> <span>=</span> <span>abs</span><span>(</span><span>val_lin</span> <span>-</span> <span>val_rac</span><span>)</span><span>;</span>
<span>endfunction</span>
<span>// Cost function : </span>
<span>function</span> <span>[</span><span>f</span><span>, </span><span>g</span><span>, </span><span>ind</span><span>]</span><span>=</span><span>cost</span><span>(</span><span>x</span><span>, </span><span>ind</span><span>)</span>
<span>f</span> <span>=</span> <span>target</span><span>(</span><span>x</span><span>)</span><span>;</span>
<span>// g = numderivative(target, x.</span><span>'</span><span>); </span>
<span>// g = numderivative(target, x.</span><span>'</span><span>,0.1,order = 2); </span>
<span>g</span> <span>=</span> <span>numderivative</span><span>(</span><span>target</span><span>,</span> <span>x</span><span>.'</span><span>,</span><span>order</span> <span>=</span> <span>2</span><span>)</span><span>;</span> <span>// better but why ??? h automatic works better ???</span>
<span>// g = numderivative(target, x.</span><span>'</span><span>,order = 4); </span>
<span>// g = numderivative(target, x.</span><span>'</span><span>,0.1, order = 4);</span>
<span>endfunction</span>
<span>// optimisation avec optim</span>
<span>initial_parameters</span> <span>=</span> <span>[</span><span>50</span><span>]</span>
<span>lower_bounds</span> <span>=</span> <span>[</span><span>0</span><span>]</span><span>;</span>
<span>upper_bounds</span> <span>=</span> <span>[</span><span>1000</span><span>]</span><span>;</span>
<span>nocf</span> <span>=</span> <span>1000</span><span>;</span> <span>// number of call of f</span>
<span>niter</span> <span>=</span> <span>1000</span><span>;</span> <span>// number of iterations</span>
<span>[</span><span>fopt</span><span>,</span> <span>xopt</span><span>,</span> <span>gopt</span><span>,</span> <span>work</span><span>,</span> <span>iters</span><span>,</span> <span>evals</span><span>,</span> <span>err</span><span>]</span> <span>=</span> <span>optim</span><span>(</span><span>cost</span><span>,</span><span>'</span><span>b</span><span>'</span><span>,</span><span>lower_bounds</span><span>,</span><span>upper_bounds</span><span>,</span><span>initial_parameters</span><span>,</span><span>'</span><span>qn</span><span>'</span><span>,</span><span>'</span><span>ar</span><span>'</span><span>,</span><span>nocf</span><span>,</span><span>niter</span><span>,</span><span>imp</span><span>=</span><span>3</span><span>)</span><span>;</span>
<span>xopt</span>
<span>fopt</span>
<span>err</span>
<span>// traçage courbes</span>
<span>x</span> <span>=</span> <span>linspace</span><span>(</span><span>0</span><span>,</span><span>50</span><span>,</span><span>1000</span><span>)</span><span>'</span><span>;</span>
<span>plot_raci</span> <span>=</span> <span>racine</span><span>(</span><span>x</span><span>,</span><span>10</span><span>,</span><span>6</span><span>)</span><span>;</span>
<span>plot_lin</span> <span>=</span> <span>lineaire</span><span>(</span><span>x</span><span>,</span><span>1</span><span>,</span><span>2</span><span>)</span><span>;</span>
<span>scf</span><span>(</span><span>1</span><span>)</span><span>;</span>
<span>drawlater</span><span>(</span><span>)</span><span>;</span>
<span>xgrid</span><span>(</span><span>3</span><span>)</span><span>;</span>
<span>f</span> <span>=</span> <span>gcf</span><span>(</span><span>)</span><span>;</span>
<span>//f </span>
<span>f</span><span>.</span><span>figure_size</span> <span>=</span> <span>[</span><span>1000</span><span>,</span> <span>1000</span><span>]</span><span>;</span>
<span>f</span><span>.</span><span>background</span> <span>=</span> <span>color</span><span>(</span><span>255</span><span>,</span><span>255</span><span>,</span><span>255</span><span>)</span><span>;</span>
<span>a</span> <span>=</span> <span>gca</span><span>(</span><span>)</span><span>;</span>
<span>//a </span>
<span>a</span><span>.</span><span>font_size</span> <span>=</span> <span>2</span><span>;</span>
<span>a</span><span>.</span><span>x_label</span><span>.</span><span>text</span> <span>=</span> <span>"</span><span>X axis</span><span>"</span> <span>;</span>
<span>a</span><span>.</span><span>x_location</span><span>=</span><span>"</span><span>bottom</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>x_label</span><span>.</span><span>font_angle</span><span>=</span><span>0</span><span>;</span>
<span>a</span><span>.</span><span>x_label</span><span>.</span><span>font_size</span> <span>=</span> <span>4</span><span>;</span>
<span>a</span><span>.</span><span>y_label</span><span>.</span><span>text</span> <span>=</span> <span>"</span><span>Y axis</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>y_location</span><span>=</span><span>"</span><span>left</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>y_label</span><span>.</span><span>font_angle</span><span>=</span><span>-</span><span>90</span><span>;</span>
<span>a</span><span>.</span><span>Y_label</span><span>.</span><span>font_size</span> <span>=</span> <span>4</span><span>;</span>
<span>a</span><span>.</span><span>title</span><span>.</span><span>text</span> <span>=</span> <span>"</span><span>Title</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>title</span><span>.</span><span>font_size</span> <span>=</span> <span>5</span><span>;</span>
<span>a</span><span>.</span><span>line_style</span> <span>=</span> <span>1</span><span>;</span>
<span>// début des courbes</span>
<span>plot</span><span>(</span><span>x</span><span>,</span><span>plot_lin</span><span>)</span><span>;</span>
<span>e1</span> <span>=</span> <span>gce</span><span>(</span><span>)</span><span>;</span>
<span>p1</span> <span>=</span> <span>e1</span><span>.</span><span>children</span><span>;</span>
<span>p1</span><span>.</span><span>thickness</span> <span>=</span> <span>1</span><span>;</span>
<span>p1</span><span>.</span><span>line_style</span> <span>=</span> <span>1</span><span>;</span>
<span>p1</span><span>.</span><span>foreground</span> <span>=</span> <span>3</span><span>;</span>
<span>plot</span><span>(</span><span>x</span><span>,</span><span>plot_raci</span><span>)</span><span>;</span>
<span>e2</span> <span>=</span> <span>gce</span><span>(</span><span>)</span><span>;</span>
<span>p2</span> <span>=</span> <span>e2</span><span>.</span><span>children</span><span>;</span>
<span>p2</span><span>.</span><span>thickness</span> <span>=</span> <span>1</span><span>;</span>
<span>p2</span><span>.</span><span>line_style</span> <span>=</span> <span>1</span><span>;</span>
<span>p2</span><span>.</span><span>foreground</span> <span>=</span> <span>2</span><span>;</span>
<span>drawnow</span><span>(</span><span>)</span><span>;<br /><br /></span>#######################################<br />Nelder Mead<br />#######################################</pre>
<pre><span>mode</span><span>(</span><span>0</span><span>)</span>
<span>function</span> <span>f</span><span>=</span><span>lineaire</span><span>(</span><span>x</span><span>, </span><span>a2</span><span>, </span><span>b2</span><span>)</span>
<span>f</span> <span>=</span> <span>a2</span><span>*</span><span>x</span><span>+</span><span>b2</span><span>;</span>
<span>endfunction</span>
<span>function</span> <span>g</span><span>=</span><span>racine</span><span>(</span><span>x</span><span>, </span><span>a1</span><span>, </span><span>b1</span><span>)</span>
<span>g</span> <span>=</span> <span>sqrt</span><span>(</span><span>a1</span><span>*</span><span>x</span><span>)</span> <span>+</span> <span>b1</span><span>;</span>
<span>endfunction</span>
<span>function</span> <span>[</span><span>f</span><span>, </span><span>index</span><span>]</span><span>=</span><span>target</span><span>(</span><span>x</span><span>, </span><span>index</span><span>)</span>
<span>val_lin</span> <span>=</span> <span>lineaire</span><span>(</span><span>x</span><span>,</span><span>1</span><span>,</span><span>2</span><span>)</span><span>;</span>
<span>val_rac</span> <span>=</span> <span>racine</span><span>(</span><span>x</span><span>,</span><span>10</span><span>,</span><span>6</span><span>)</span><span>;</span>
<span>f</span> <span>=</span> <span>abs</span><span>(</span><span>val_lin</span> <span>-</span> <span>val_rac</span><span>)</span><span>;</span>
<span>endfunction</span>
<span>// optimisation avec optim</span>
<span>initial_parameters</span> <span>=</span> <span>[</span><span>50</span><span>]</span>
<span>lower_bounds</span> <span>=</span> <span>[</span><span>0</span><span>]</span><span>;</span>
<span>upper_bounds</span> <span>=</span> <span>[</span><span>1000</span><span>]</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_new</span> <span>(</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-numberofvariables</span><span>"</span><span>,</span><span>1</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-function</span><span>"</span><span>,</span><span>target</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-x0</span><span>"</span><span>,</span><span>initial_parameters</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-maxiter</span><span>"</span><span>,</span><span>1000</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-maxfunevals</span><span>"</span><span>,</span><span>1000</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-tolfunrelative</span><span>"</span><span>,</span><span>10</span><span>*</span><span>%eps</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-tolxrelative</span><span>"</span><span>,</span><span>10</span><span>*</span><span>%eps</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-method</span><span>"</span><span>,</span><span>"</span><span>box</span><span>"</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-boundsmin</span><span>"</span><span>,</span><span>lower_bounds</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_configure</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-boundsmax</span><span>"</span><span>,</span> <span>upper_bounds</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_search</span><span>(</span><span>nm</span><span>)</span><span>;</span>
<span>nm</span> <span>=</span> <span>neldermead_restart</span><span>(</span><span>nm</span><span>)</span><span>;</span>
<span>xopt</span> <span>=</span> <span>neldermead_get</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-xopt</span><span>"</span><span>)</span>
<span>fopt</span> <span>=</span> <span>neldermead_get</span><span>(</span><span>nm</span><span>,</span><span>"</span><span>-fopt</span><span>"</span><span>)</span>
<span>nm</span> <span>=</span> <span>neldermead_destroy</span><span>(</span><span>nm</span><span>)</span><span>;</span>
<span>// traçage courbes</span>
<span>x</span> <span>=</span> <span>linspace</span><span>(</span><span>0</span><span>,</span><span>50</span><span>,</span><span>1000</span><span>)</span><span>'</span><span>;</span>
<span>a1</span> <span>=</span> <span>10</span><span>;</span> <span>b1</span> <span>=</span> <span>6</span><span>;</span> <span>// b1 </span><span>></span><span> b2 here</span>
<span>a2</span> <span>=</span> <span>1</span><span>;</span> <span>b2</span> <span>=</span> <span>2</span><span>;</span>
<span>plot_raci</span> <span>=</span> <span>racine</span><span>(</span><span>x</span><span>,</span><span>a1</span><span>,</span><span>b1</span><span>)</span><span>;</span>
<span>plot_lin</span> <span>=</span> <span>lineaire</span><span>(</span><span>x</span><span>,</span><span>a2</span><span>,</span><span>b2</span><span>)</span><span>;</span>
<span>scf</span><span>(</span><span>1</span><span>)</span><span>;</span>
<span>drawlater</span><span>(</span><span>)</span><span>;</span>
<span>xgrid</span><span>(</span><span>3</span><span>)</span><span>;</span>
<span>f</span> <span>=</span> <span>gcf</span><span>(</span><span>)</span><span>;</span>
<span>//f </span>
<span>f</span><span>.</span><span>figure_size</span> <span>=</span> <span>[</span><span>1000</span><span>,</span> <span>1000</span><span>]</span><span>;</span>
<span>f</span><span>.</span><span>background</span> <span>=</span> <span>color</span><span>(</span><span>255</span><span>,</span><span>255</span><span>,</span><span>255</span><span>)</span><span>;</span>
<span>a</span> <span>=</span> <span>gca</span><span>(</span><span>)</span><span>;</span>
<span>//a </span>
<span>a</span><span>.</span><span>font_size</span> <span>=</span> <span>2</span><span>;</span>
<span>a</span><span>.</span><span>x_label</span><span>.</span><span>text</span> <span>=</span> <span>"</span><span>X axis</span><span>"</span> <span>;</span>
<span>a</span><span>.</span><span>x_location</span><span>=</span><span>"</span><span>bottom</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>x_label</span><span>.</span><span>font_angle</span><span>=</span><span>0</span><span>;</span>
<span>a</span><span>.</span><span>x_label</span><span>.</span><span>font_size</span> <span>=</span> <span>4</span><span>;</span>
<span>a</span><span>.</span><span>y_label</span><span>.</span><span>text</span> <span>=</span> <span>"</span><span>Y axis</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>y_location</span><span>=</span><span>"</span><span>left</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>y_label</span><span>.</span><span>font_angle</span><span>=</span><span>-</span><span>90</span><span>;</span>
<span>a</span><span>.</span><span>Y_label</span><span>.</span><span>font_size</span> <span>=</span> <span>4</span><span>;</span>
<span>a</span><span>.</span><span>title</span><span>.</span><span>text</span> <span>=</span> <span>"</span><span>Title</span><span>"</span><span>;</span>
<span>a</span><span>.</span><span>title</span><span>.</span><span>font_size</span> <span>=</span> <span>5</span><span>;</span>
<span>a</span><span>.</span><span>line_style</span> <span>=</span> <span>1</span><span>;</span>
<span>// début des courbes</span>
<span>plot</span><span>(</span><span>x</span><span>,</span><span>plot_lin</span><span>)</span><span>;</span>
<span>e1</span> <span>=</span> <span>gce</span><span>(</span><span>)</span><span>;</span>
<span>p1</span> <span>=</span> <span>e1</span><span>.</span><span>children</span><span>;</span>
<span>p1</span><span>.</span><span>thickness</span> <span>=</span> <span>1</span><span>;</span>
<span>p1</span><span>.</span><span>line_style</span> <span>=</span> <span>1</span><span>;</span>
<span>p1</span><span>.</span><span>foreground</span> <span>=</span> <span>3</span><span>;</span>
<span>plot</span><span>(</span><span>x</span><span>,</span><span>plot_raci</span><span>)</span><span>;</span>
<span>e2</span> <span>=</span> <span>gce</span><span>(</span><span>)</span><span>;</span>
<span>p2</span> <span>=</span> <span>e2</span><span>.</span><span>children</span><span>;</span>
<span>p2</span><span>.</span><span>thickness</span> <span>=</span> <span>1</span><span>;</span>
<span>p2</span><span>.</span><span>line_style</span> <span>=</span> <span>1</span><span>;</span>
<span>p2</span><span>.</span><span>foreground</span> <span>=</span> <span>2</span><span>;</span>
<span>drawnow</span><span>(</span><span>)</span><span>;</span></pre>
</div>
</body></html>